Encoding / decoding video picture data

Abstract

The invention relates to a method for predicting a block of a video picture based on bidirectional motion compensation. Two reference blocks in a reference picture in two different lists of reference pictures are determined for motion compensation. The method determines whether a sample of a reference block is outside a boundary of a reference picture, and in this case, determines a non-border-crossing rectangular block comprising at least one sample within the boundary of the reference picture, and replacing the at least one sample outside the non-border rectangular block with a corresponding sample of other reference blocks within other reference picture boundaries.

Description

[0001] Cross-reference to related applications

[0002] This disclosure claims the priority and benefit of European Patent Application No. 22306541.8, filed on October 12, 2022, the entire content of which is incorporated herein by reference. Technical field

[0003] This disclosure generally relates to video picture encoding / decoding. In particular, but not limited thereto, the technical field of this disclosure relates to prediction of blocks of video pictures based on bidirectional motion compensation. Background art

[0004] This section is intended to introduce the reader to various aspects of the field that may be related to aspects of at least one embodiment of the disclosure described and / or claimed below. This discussion is considered to help provide background information to the reader to facilitate a better understanding of the various aspects of the disclosure. Accordingly, it should be understood that these statements are to be read in this light and not as an admission of related art.

[0005] In state-of-the-art video compression systems, such as HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en) or VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), low-level and high-level picture partitions are provided to divide a video picture into picture regions, so-called coding tree units (CTUs), the size of which can typically be between 16x16 and 64x64 pixels for HEVC and between 32x32, 64x64 or 128x128 pixels for VVC.

[0006] The CTU partitioning of a video picture forms a grid consisting of CTUs of a fixed size, i.e., a CTU grid, the upper and left boundaries of which spatially coincide with the top and left boundaries of the video picture. The CTU grid represents the spatial partitioning of the video picture.

[0007] In VVC and HEVC, the CTU sizes (CTU width and CTU height) of all CTUs in the CTU grid are equal to the same default CTU size (default CTU width CTU DW and default CTU height CTU DH). For example, the default CTU size (default CTU height, default CTU width) can be equal to 128 (CTU DW = CTU DH = 128). The default CTU size (height, width) is encoded into the bitstream, e.g., at the sequence level in the Sequence Parameter Set (SPS).

[0008] The spatial position of a CTU in the CTU grid is determined according to the CTU address ctuAddr, which defines the spatial position of the upper left corner of the CTU relative to the origin. As Figure 1 shown, the CTU address can define the spatial position starting from the upper left corner of a higher-level spatial structure S that contains the CTU.

[0009] Each CTU is associated with a codec tree to determine the tree partitioning of the CTU.

[0010] As Figure 1 illustrated, in HEVC, the codec tree is a quadtree partitioning of the CTU, where each leaf is called a coding unit (CU). The spatial position of a CU in the video picture is defined by the CU index cuIdx, which indicates the spatial position starting from the upper left corner of the CTU. A CU is spatially partitioned into one or more prediction units (PUs). The spatial position of a PU in the video picture VP is defined by the PU index puIdx, which defines the spatial position starting from the upper left corner of the CTU, and the spatial position of the elements of the partitioned PU is defined by the PU partition index puPartIdx, which defines the spatial position starting from the upper left corner of the PU. Each PU is assigned some intra or inter prediction data.

[0011] The coding mode, intra or inter, is assigned at the CU level. This means that although the prediction parameters vary for each PU, the same intra / inter coding mode is assigned to each PU of the CU.

[0012] According to a quadtree called the transform tree, a CU can also be spatially partitioned into one or more transform units (TUs). A transform unit is a leaf of the transform tree. The spatial position of a TU in the video picture is defined by the TU index tuIdx, which defines the spatial position starting from the upper left corner of the CU. Each TU is assigned some transform parameters. The transform type is assigned at the TU level, and a 2D separate transform is performed at the TU level during the encoding or decoding of the picture block.

[0013] Figure 2The existing PU partition types in HEVC are illustrated. They include square partitions (2N×2N and N×N), which are the only partitions used in both intra- and inter-prediction CUs, symmetric non-square partitions (2N×N, N×2N, only used in inter-prediction CUs), and asymmetric partitions (only used in inter-prediction CUs). For example, PU type 2NxnU represents an asymmetric horizontal partition of the PU, where the smaller partition is at the top of the PU. According to another example, PU type 2NxnL represents an asymmetric horizontal partition of the PU, where the smaller partition is at the top of the PU.

[0014] As Figure 3 illustrated, in VVC, the codec tree starts from the root node (i.e., CTU). Next, the quadtree (or quad-tree) split divides the root node into 4 nodes, corresponding to 4 sub-blocks of equal size (solid lines). Next, the quadtree (or quad-tree) leaves can be further partitioned by the so-called multi-type tree, which involves binary or ternary splitting according to Figure 4 one of the 4 splitting patterns illustrated. These splitting types are vertical and horizontal binary splitting patterns (denoted as SBTV and SBTH) and vertical and horizontal ternary splitting patterns SPTTV and STTH.

[0015] In the case of a joint codec tree where the luminance and chrominance components are shared, the leaves of the codec tree of the CTU are CUs.

[0016] Contrary to HEVC, in VVC, in most cases, CUs, PUs, and TUs have equal sizes, which means that, except in some specific codec modes, the codec units are generally not partitioned into PUs or TUs.

[0017] Figure 5 and Figure 6 provides an overview of video encoding / decoding methods used, for example, in current video standard compression systems such as HEVC or VVC.

[0018] Figure 5 FIG. shows a schematic block diagram of the steps of a method 100 for encoding a video picture VP according to the related art.

[0019] In step 110, the video picture VP is partitioned into sampling blocks and the partition information data is signaled into the bitstream. Each block includes samples of one component of the video picture VP. Thus, these blocks include the samples defining each component of the video picture VP.

[0020] For example, in HEVC, a picture is divided into coding tree units (CTUs). Each CTU can be further subdivided using quadtree partitioning, where each leaf of the quadtree represents a coding unit (CU). Then, the partition information data can include data describing the CTU and the quadtree subdivision of each CTU.

[0021] Thus, each sample block (referred to simply as a block) can be a CU (if the CU includes a single PU) or a PU of a CU.

[0022] Each block is encoded along a block coding loop using an intra or inter prediction mode. The block coding loop includes steps 120 to 180.

[0023] Intra prediction (step 120) uses intra prediction data. Intra prediction includes predicting the current block by means of an intra prediction block based on the coded, decoded, and reconstructed samples that are located around the current block, typically at the top and left of the current block. Intra prediction is performed in the spatial domain.

[0024] In the inter prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for a reference block that is a good predictor of the current block in one or more reference pictures used for predictive coding of the current video picture. In uni-directional motion estimation / compensation, the candidate reference block belongs to a single reference picture of a reference picture list denoted as L0 or L1, and in bi-directional motion estimation / compensation, the candidate reference block is derived from the reference blocks of reference picture list L0 and the reference blocks of reference picture list L1.

[0025] For example, a good predictor of the current block is a candidate reference block that is similar to the current block. It can also correspond to a reference block that provides a good trade-off between its similarity to the current block and the rate cost of the motion information indicating its use for temporal prediction of the current block.

[0026] The output of the motion estimation step 130 is inter - prediction data, which includes motion information associated with the current block and other information for obtaining the same predicted block on the encoding / decoding side. Generally, the motion information includes one motion vector and one reference picture index for uni - directional estimation / compensation and two motion vectors and two reference picture indices for bi - directional estimation / compensation. Next, motion compensation (step 135) obtains the predicted block by means of the (one or more) motion vectors and (one or more) reference picture indices determined by the motion estimation step 130. Basically, the reference block belonging to the selected reference picture and pointed to by the motion vector can be used as the predicted block for the current block. In addition, since the motion vectors are expressed as fractions of integer pixel positions (which is called sub - pixel - accuracy motion vector representation), motion compensation generally involves some resampled spatial interpolation of the reference picture to calculate the predicted block.

[0027] The prediction information data is signaled into the bitstream. The prediction information can include the prediction mode, (intra or inter or skip), intra / inter - prediction data, and any other information for obtaining the same predicted block on the decoding side.

[0028] Considering the encoding of the computed prediction residual block (e.g., by subtracting the candidate predicted block from the current block) and the signaling of the prediction information data required to determine the candidate predicted block on the decoding side, method 100 selects a prediction mode (intra or inter - prediction mode) by optimizing the rate - distortion trade - off.

[0029] Generally, the best prediction mode is given as the prediction mode of the best encoding / decoding mode p* of the current block given by:

[0030]

[0031] where P is the set of all candidate encoding / decoding modes of the current block, p represents a candidate encoding / decoding mode in this set, and RD cost (p) is the rate - distortion cost of the candidate encoding / decoding mode p, usually expressed as:

[0032] RD cost(p) = D(p)+λ.R(p)

[0033] D(p) is the distortion between the current block and the reconstructed block obtained after encoding / decoding the current block with the candidate encoding / decoding mode p, R(p) is the rate cost associated with encoding / decoding the current block with the encoding / decoding mode p, and λ is the Lagrange parameter representing the rate constraint for encoding / decoding the current block and is usually calculated according to the quantization parameter used for encoding the current block.

[0034] The current block is typically encoded by a prediction residual block PR. More precisely, for example, the prediction residual block PR is calculated by subtracting the best prediction block from the current block. Then, the prediction residual block PR is transformed (step 140) by using, for example, a DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform) type transform or any other suitable transform, and the obtained transformed coefficient block is quantized (step 150).

[0035] In a variant, method 100 may also skip the transform step 140 according to the so-called transform-skip codec mode and directly apply quantization (step 150) to the prediction residual block PR.

[0036] The quantized transform coefficient block (or quantized prediction residual block) is entropy encoded into the bitstream (step 160).

[0037] Next, as part of the encoding loop, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inversely transformed (180) (or not), resulting in a decoded prediction residual block. Then, the decoded prediction residual block and the prediction block are combined (usually by summing), thus providing a reconstructed block.

[0038] Other information data may also be entropy encoded in step 160 to encode the current block of the video picture VP.

[0039] A loop filter (step 190) may be applied to the reconstructed picture (including the reconstructed blocks) to reduce compression artifacts. The loop filter may be applied after all blocks have been reconstructed. For example, they include a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter.

[0040] The reconstructed block or the filtered reconstructed block forms a reference picture, which may be stored in a decoded picture buffer (DPB) so that it can be used as a reference picture for encoding the next current block of the video picture VP or as a reference picture for encoding the next video picture to be encoded.

[0041] Figure 6 A schematic block diagram showing the steps of a method 200 for decoding a video picture VP according to the related art is shown.

[0042] In step 210, partition information data, prediction information data, and a quantized transform coefficient block (or quantized residual block) are obtained by entropy decoding the bitstream of the encoded video picture data. For example, this bitstream has been generated according to method 100.

[0043] Other information data may also be entropy decoded to decode the current block of the video picture VP from the bitstream.

[0044] In step 220, the reconstructed video picture is partitioned into current blocks based on the partitioning information.

[0045] Each current block is reconstructed along the block decoding loop using an intra or inter prediction mode. The block decoding loop includes steps 220 to 270.

[0046] Each current block is entropy decoded from the bitstream. Each decoded current block is a quantized transform coefficient block or a quantized prediction residual block.

[0047] In step 230, the current block is dequantized and possibly inverse transformed (step 240) to obtain a decoded prediction residual block.

[0048] On the other hand, the current block is predicted using the prediction information data. The predicted block is obtained by its intra prediction (step 250) or its motion compensated temporal prediction (step 260). The prediction process performed on the decoding side is exactly the same as the prediction process on the encoding side.

[0049] Next, the decoded prediction residual block and the predicted block are combined (usually by summing), which provides the reconstructed block.

[0050] In step 270, the loop filter can be applied to the reconstructed video picture (including the reconstructed block or the filtered reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference picture, which can be stored in the decoded picture buffer (DPB), as discussed above ( Figure 5 ).

[0051] In VVC, the motion information is stored in each video picture in 4×4 blocks. This means that once the reference picture is stored in the decoded picture buffer (DPB, Figure 5 or Figure 6 ), the motion vectors and reference picture indices for temporal prediction of video picture blocks are stored on a 4×4 block basis. They can be used as temporal prediction for the motion information for encoding / decoding subsequent inter predicted video pictures.

[0052] In VVC, the temporal prediction of the current block may be based on a reference block of the reference picture, which may overlap with the boundary of the reference picture. For example, Figure 7 shows an example of the prediction of a current block based on a first reference block of reference picture list L0 and a second reference block of reference picture list L1 based on bidirectional motion compensation. The first (respectively, second) reference block is pointed to by the first (respectively, second) motion vector. In Figure 7In the example, the first reference block is partially outside the boundary of the reference picture in the reference picture list L0, and the second reference block is inside the boundary of the reference picture in the reference picture list L1. More generally, one of the two reference blocks or both reference blocks of the two reference pictures may be partially or entirely outside the reference picture boundary. Note that the reference picture boundary is usually extended by filling the area.

[0053] To handle this situation, a method for processing out-of-bounds reference blocks based on motion compensation is described in section 2.1.7.5 of the JVET proposal JVET-Y0125 ("AHG12: Enhanced bi-directional motion compensation", Yi-Wen Chen, Che-Wei Kuo, Ning Yan, Wei Chen, Xiaoyu Xiu, Xianglin Wang, Joint Video Exploration Team (JVET) meeting 25, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, by teleconference, January 12 - 21, 2022, JVET-Y0125 document).

[0054] Basically, to avoid the prediction (motion-compensated prediction) block b being at least partially outside the reference picture boundary, it is not very effective. When combining more than one motion-compensated prediction block, the OOB (out-of-bounds) samples of the reference block b are discarded and only the non-OOB (inside the boundary) samples of the reference block b are used to generate the final prediction block. More precisely, let (Pos_x i,j , Pos_y i,j ) be the Cartesian coordinates of the sampling position (i, j) within the current block. Let be the Cartesian coordinates of the motion vector (MV) associated with the sampling position (i, j) within the current block. (LX indicates whether the reference picture belongs to the reference picture list L0 (x = 0) or to the reference picture list L1 (x = 1). Let Pos LeftBdry , Pos Rightjdry , Pos TopBdry and Pos BottomBdry be the Cartesian coordinates of the 4 reference picture boundaries.

[0055] The OOB criterion calculation for the predicted sample Lx at the sampling position (i, j) of the reference block P is given by the following equation (1):

[0056]

[0057] or

[0058]

[0059] Among them, according to the internal representation precision of the MV (motion vector) of the codec (for VVC, 1 / 16 pixel), half_sample represents half of the distance between two adjacent luma samples (samples of the luma component of the video picture).

[0060] Table 1 provides an adaptive averaging of reference sampling performed during bidirectional motion compensation-based prediction based on the OOB criterion given by Equation (1).

[0061] Table 1

[0062]

[0063] Where P Lx (i,j) is the sampled value of the reference block in the reference picture list Lx at the sampling position (i,j), and P final (i,j) is the sampled value of the final bidirectional motion compensation-based prediction block at the sampling position (i,j).

[0064] In other words, if the bidirectional motion compensation-based prediction block has an OOB reference block in one direction, the block part evaluated as OOB is changed to the part of the reference block in the opposite direction (or the block part evaluated as OOB is replaced with the part of the reference block in the opposite direction).

[0065] Figure 8 Schematically shows a block diagram of the steps of method 300 for predicting a block based on adaptive bidirectional motion compensation in the case of an OOB predictor according to the JVET proposal JVET-Y0125.

[0066] The input of method 300 is the block B to be predicted based on bidirectional motion compensation and two associated motion vectors for the reference picture lists L0 and L1 respectively and

[0067] In step 301, method 300 uses the motion vector to determine the first reference block P L0 .

[0068] In step 302, method 300 checks whether the first reference block P L0 reaches the block OOB criterion, that is, whether the first reference block P L0 is an OOB block. Then, method 300 assigns a value to the boolean BlockOOB L0 . When the first reference block P L0 is OOB, the boolean value BlockOOBL0 is true, otherwise the boolean value BlockOOB L0 is false.

[0069] In step 303, method 300 uses a motion vector to determine a second reference block P in a reference picture of the reference picture list L1 L1 .

[0070] In step 304, method 300 checks whether the second reference block P L1 meets the block OOB criterion, that is, whether the second block P L1 is an OOB block. Then method 300 assigns a value to the boolean value BlockOOB L1 . When the second reference block P L1 is OOB, the boolean value BlockOOB L1 is true, otherwise the boolean value BlockOOB L1 is false.

[0071] In one embodiment of steps 302 and 303, the block OOB criterion is evaluated as follows.

[0072] Let MVF be the number of bits representing the fractional part of the motion vector sub-pixel. In other words, the value D = 2 MVF represents the distance between two adjacent full-pixel positions in the reference picture.

[0073] Evaluate the block OOB criterion according to Table 2 and assign the boolean value BlockOOB Lx (x = 0,1).

[0074] Table 2

[0075]

[0076]

[0077] where picWidth, picHeight are the number of columns and rows of the video picture, block X , block Y is the position of block B, block W , block H is the size of block B.

[0078] In step 305, method 300 checks whether the boolean value BlockOOB L0 is false, and whether the boolean value BlockOOB L1 is false.

[0079] If the boolean value BlockOOB L0 is false and the boolean value BlockOOBL1 is false, i.e., if the first and second reference blocks P L0 and P L1 are non - OOB, then, in step 306, the final bi - directional prediction block P is given by averaging the first and second uni - directional blocks derived by uni - directional motion compensation using the motion vector and the associated reference pictures in reference picture lists L0 and L1. final . Basically, each sample P final (i,j) of the final motion - compensated - based prediction block is equal to the average of the corresponding samples P L0 (i,j) and P L1 (i,j) of the first and second uni - directional prediction blocks:

[0080]

[0081] If at least one of the boolean value BlockOOB L0 and the boolean value BlockOOB L1 is true, then step 305 is followed by steps 307 - 314.

[0082] In step 307, the current position (i,j) of block B is considered.

[0083] In step 308, method 300 checks whether the current sample P L0 of the first reference block P L0 (i,j) meets the sample OOB criterion, i.e., whether the sample is OOB. Then, method 300 assigns a value to the boolean value OOB L0 . When the sample P L0 (i,k) is OOB, the boolean value OOB L0 is true, otherwise the boolean value OOB L0 is false.

[0084] In step 309, method 300 checks whether the current sample P L1 of the second reference block P L1 (i,j) meets the sample OOB criterion, i.e., whether the sample is OOB. Then, method 300 assigns a value to the boolean value OOB L1 . When the sample P L1 (i,j) is OOB, the boolean value OOB L1 is true, otherwise the boolean value OOB L1 is false.

[0085] Evaluate the sample OOB criterion according to Table 3 and assign the boolean value OOB Lz (z = 0,1).

[0086] Table 3

[0087]

[0088]

[0089] In step 310, method 300 checks whether the boolean value OOB L0 is true and whether the boolean value OOB L1 is false.

[0090] If the boolean value OOB L0 is true and the boolean value OOB L1 is false, then, in step 311, the sample P final of the final bidirectional motion compensation-based prediction block P final (i,j) is set to be equal to the sample P L1 of the second reference block P L1 (i,j).

[0091] Otherwise, in step 312, method 300 checks whether the boolean value OOB L0 is false and whether the boolean value OOB L1 is true.

[0092] If the boolean value OOB L0 is false and the boolean value OOB L1 is true, then, in step 313, the sample P final of the final bidirectional motion compensation-based prediction block P final (i,j) is set to be equal to the sample P L0 of the first reference block P L0 (i,j).

[0093] Otherwise, after step 312 are steps 306 and 314.

[0094] After steps 311 and 313 are step 314.

[0095] In step 314, method 300 checks whether all sample positions (i,j) of block B have been checked. If not, after step 314 is step 307, and a new current position (i,j) of block B is considered.

[0096] In method 300, for each reference block of block B in reference picture lists L0 and L1, a first block-level test checks whether the reference block overlaps with the boundary of the reference picture to which it belongs. If so, the relevant reference block is called OOB (out-of-bounds). If at the block level, neither of the two reference blocks is non-OOB, a bidirectional prediction for normal block-level motion compensation applied to block B in the predicted video picture is generated by averaging the first and second unidirectional prediction blocks derived by unidirectional motion compensation using the motion vector and the associated reference pictures in reference picture lists L0 and L1, i.e., the final bidirectional prediction block of block B.

[0097] Otherwise, the final bidirectional prediction block is derived as follows. For each sampling position in the block, the sampling OOB criteria in Table 3 are applied. If the sampling of the reference block is OOB for one prediction direction (i.e., the corresponding sampling in the corresponding reference picture exceeds the reference picture boundary) and not OOB for the other prediction direction, the sampling of the reference block in the other prediction direction is used as the final sampled prediction value, and averaging is not applied to the sampling position. Otherwise, the reference samples are averaged for the sampling position.

[0098] Thus, in method 300, a per-sampling OOB decision is involved to determine how to calculate the predicted samples at each position of block B.

[0099] This constitutes a drawback of method 300 because it leads to an increase in the complexity of the prediction of blocks in video pictures based on bidirectional motion compensation.

[0100] The technical problem solved by the present invention is how to solve the complexity problem of method 300, in particular how to avoid the per-sampling OOB criteria.

[0101] At least one embodiment of the present disclosure is designed in view of the above situation. Summary of the Invention

[0102] The following section presents a brief overview of at least one embodiment to provide a basic understanding of some aspects of the present disclosure. This overview is not an exhaustive overview of the embodiments. Its purpose is not to identify the key or core elements of the embodiments. The following overview only presents some aspects of at least one embodiment in a simplified form as a prelude to the more detailed description provided elsewhere in this document.

[0103] According to a first aspect of the present disclosure, there is provided a method for predicting a block in a video picture based on bidirectional motion compensation, wherein the method includes:

[0104] - determining a first reference block as a block of a first reference picture in a first reference picture list, the first reference block being pointed to by a first motion vector starting from the block in the video picture;

[0105] - Determine a second reference block as a block of a second reference picture in a second reference picture list, the second reference block being pointed to by a second motion vector starting from a block of a video picture;

[0106] - Check whether at least a part of the samples of at least one of the first or second reference blocks is at least partially outside the boundary of the first or second reference picture;

[0107] Wherein the method further includes:

[0108] - If at least one sample of the first reference block is outside the boundary of the first reference picture, determine a first non-overlapping rectangular block, the first non-overlapping rectangular block including at least one sample of the first reference block within the boundary of the first reference picture;

[0109] - If at least one sample of the second reference block is at least partially outside the boundary of the second reference picture, determine a second non-overlapping rectangular block, the second non-overlapping rectangular block including at least one sample of the second reference block within the boundary of the second reference picture; and

[0110] - When the first reference block is at least partially outside the first non-overlapping rectangular block, derive a bidirectional motion compensation-based prediction block by replacing at least one sample of the first reference block with at least one co-located sample of the second reference block, and when the second reference block is at least partially outside the second non-overlapping rectangular block, derive a bidirectional motion compensation-based prediction block by replacing at least one sample of the second reference block with at least one co-located sample of the first reference block.

[0111] In one embodiment, the first and second reference blocks are split into sub-blocks, and for each current sub-block position, the method further includes:

[0112] - If a first sub-block at a current sub-block position in the first reference block is at least partially outside the first non-overlapping rectangular block and a second sub-block is within the second non-overlapping rectangular block, derive a bidirectional motion compensation-based prediction sub-block as the second sub-block;

[0113] - If a first sub-block at a current sub-block position in the first reference block is within the first non-overlapping rectangular block and a second sub-block is at least partially outside the second non-overlapping rectangular block, derive a bidirectional motion compensation-based prediction sub-block as the first sub-block;

[0114] - Otherwise, derive a bidirectional motion compensation-based prediction sub-block as the average of the samples of the first sub-block and the samples of the second sub-block.

[0115] In one embodiment, if the first sub-block is completely within the first non-overlapping rectangular block and the second sub-block is completely within the second non-overlapping rectangular block, the method further includes deriving a bidirectional motion compensation-based prediction sub-block as the average of the samples of the first sub-block and the samples of the second sub-block.

[0116] In one embodiment, the method further comprises:

[0117] - If at least one sample of the first reference block is outside the first non-overflow rectangular block, or at least one sample of the second reference block is outside the second non-overflow rectangular block, then

[0118] - When the at least one sample is within the first non-overflow rectangular block, copy at least one sample of the first reference block into a first intermediate block, and copy at least one sample of the second reference block into at least one sample position of the first intermediate block, where at least one sample position of the first intermediate block corresponds to at least one sample position of the first reference block that is outside the first non-overflow rectangular block;

[0119] - When the at least one sample is within the second non-overflow rectangular block, copy at least one sample of the second reference block (P L1 ) into a second intermediate block, and copy at least one sample of the first reference block into at least one sample position of the second intermediate block, where at least one sample position of the second intermediate block corresponds to at least one sample position of the second reference block that is outside the second non-overflow rectangular block;

[0120] - Derive a bidirectional motion compensation-based prediction block by averaging the samples of the first and second intermediate blocks.

[0121] In one embodiment, the samples of the first and second intermediate blocks or the samples of the first and second sub-blocks are averaged using a weighting factor.

[0122] In one embodiment, the first and second non-overflow rectangular blocks are determined according to the motion vectors associated with the first and second reference blocks and the size of the block of the video picture.

[0123] In one embodiment, the first and second non-overflow rectangular blocks are defined by the positioning of the upper left corner of the rectangle and the number of columns and rows.

[0124] In one embodiment, the positioning of the upper left corner of the rectangle is related to the upper left corner of the block of the video picture.

[0125] According to a second aspect of the present disclosure, there is provided a method of encoding a block of a video picture into a bitstream of encoded video picture data, the method comprising performing temporal prediction on the block of the video picture according to the method of the first aspect.

[0126] According to a third aspect of the present disclosure, there is provided a method of decoding a block of a video picture from a bitstream of encoded video picture data, the method comprising performing temporal prediction on the block of the video picture according to the method of the first aspect.

[0127] According to a fourth aspect of the present disclosure, there is provided an apparatus including means for performing one of the methods according to the first, second, and / or third aspects of the present disclosure.

[0128] According to a fifth aspect of the present disclosure, there is provided a computer program product including instructions that, when executed by one or more processors, cause the one or more processors to perform the methods according to the first, second, and / or third aspects of the present disclosure.

[0129] According to a sixth aspect of the present disclosure, there is provided a non-transitory storage medium carrying instructions for a program code for performing the methods according to the first, second, and / or third aspects of the present disclosure.

[0130] According to a seventh aspect of the present disclosure, there is provided an electronic device including: a processor; and a memory for storing instructions executable by the processor. The processor is configured to perform one of the methods according to the first, second, and / or third aspects of the present disclosure.

[0131] The specific nature of at least one embodiment, as well as other objects, advantages, features, and uses of the at least one embodiment, will become apparent from the following description of examples in conjunction with the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0132] Now, examples will be described by way of reference to the drawings showing embodiments of the present disclosure, in which:

[0133] Figure 1 An example of a coding tree unit according to HEVC is shown;

[0134] Figure 2 An example of partitioning a coding unit into prediction units according to HEVC is shown;

[0135] Figure 3 An example of CTU partitioning according to VVC is shown;

[0136] Figure 4 An example of split modes supported in multi-type tree partitioning according to VVC is shown;

[0137] Figure 5 A schematic block diagram showing steps of a method 100 for encoding a video picture VP according to the related art is shown;

[0138] Figure 6 A schematic block diagram showing steps of a method 200 for decoding a video picture VP according to the related art is shown;

[0139] Figure 7 An example of prediction based on bidirectional motion compensation according to the related art is shown;

[0140] Figure 8 FIG. 0 schematically shows a block diagram of steps of method 300 for predicting a block based on adaptive bidirectional motion compensation in the case of one OOB predictor according to JVET proposal JVET - Y0125;

[0141] Figure 9 FIG. 1 schematically shows a block diagram of steps of method 400 for predicting a block based on adaptive bidirectional motion compensation according to an embodiment;

[0142] Figure 10 FIG. 2 shows an example of prediction based on bidirectional motion compensation according to an embodiment;

[0143] Figure 11 FIG. 3 schematically shows a block diagram of steps of method 500 for predicting a block based on adaptive bidirectional motion compensation according to an embodiment;

[0144] Figure 12 FIG. 4 shows an example of a result of prediction based on bidirectional motion compensation according to method 500;

[0145] Figure 13 FIG. 5 shows an example of a result of prediction based on bidirectional motion compensation according to method 500; and

[0146] Figure 14 FIG. 6 is a schematic block diagram showing an example of a system implementing various aspects and embodiments.

[0147] Like or identical elements are referenced by the same reference numerals. DETAILED DESCRIPTION

[0148] At least one of the embodiments will be described more fully hereinafter with reference to the accompanying drawings, in which examples of at least one of the embodiments are depicted. However, the embodiments may be implemented in many alternative forms and should not be construed as limited to the examples set forth herein. Thus, it should be understood that the present invention is not intended to limit the embodiments to the particular forms disclosed. On the contrary, the present disclosure is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure.

[0149] At least one of the aspects generally relates to encoding and decoding of video pictures, another aspect generally relates to transmitting the provided or encoded bitstream, and another aspect relates to receiving / accessing the decoded bitstream.

[0150] At least one of the embodiments is described for encoding / decoding one video picture, but is extended to encoding / decoding multiple video pictures (a sequence of pictures) since each video picture is encoded / decoded sequentially as described below.

[0151] In addition, at least one embodiment is not limited to the MPEG standard, such as AVC (ISO / IEC 14496-10 Advanced Video Coding for generic audio-visual services, ITU-T Recommendation H.264, https: / / www.itu.int / rec / T-REC-H.264-202108-P / en )、EVC(ISO / IEC 23094-1 Essential video coding)、HEVC(ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en 、VVC(ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en But it can also be applied to other standards and recommendations, such as AV1 (AOMedia Video 1, http: / / aomedia.org / av1 / specification / ). At least one embodiment can be applied to existing or future developments and extensions of any such standards and recommendations. Unless otherwise specified or technically excluded, the aspects described in this disclosure can be used alone or in combination.

[0152] A pixel corresponds to the smallest display unit on the screen, which can consist of one or more light sources (1 for a monochrome screen, or 3 or more for a color screen).

[0153] A video picture, also known as a frame or picture frame, includes at least one component (also known as a picture component or channel) determined by a specific picture / video format, which specifies all information related to pixel values and all information that can be used by the display unit and / or any other device for displaying and / or decoding video picture data related to the said video picture.

[0154] A video picture includes at least one component typically represented in the form of a sampling array.

[0155] A monochrome video picture includes a single component, while a color video picture can include three components.

[0156] For example, when the picture / video format is the well-known (Y, Cb, Cr) format, a color video picture can include one luminance (or brightness) component and two chrominance components, or when the picture / video format is the well-known (R, G, B) format, a color video picture can include three color components (one for red, one for green, and one for blue).

[0157] Each component of the video picture may include a number of samples relative to the number of pixels of the screen on which the video picture is to be displayed. In a variant, the number of samples included in a component may be a multiple (or fraction) of the number of samples included in another component of the same video picture.

[0158] For example, in the case where the video format includes one luminance component and two chrominance components (such as the (Y, Cb, Cr) format), depending on the color format considered, the chrominance components may include half the number of samples in width and / or height relative to the luminance component.

[0159] A sample is the smallest visual information unit of a component that makes up a video picture. The sample value may be, for example, a luminance or chrominance value, or a color value in the (R, G, B) format.

[0160] A pixel value is the value of a pixel of the screen. For a monochrome video picture, the pixel value may be represented by one sample, while for a color video picture, the pixel value may be represented by multiple co - located samples. Co - located samples associated with a pixel refer to samples corresponding to the location of the pixel in the screen.

[0161] A video picture is generally regarded as a set of pixel values, with each pixel represented by at least one sample.

[0162] A block of a video picture is a set of samples of a component of the video picture. When the picture / video format is the well - known (Y, Cb, Cr) format, a block of at least one luminance sample or a block of at least one chrominance sample may be considered, or when the picture / video format is the well - known (R, G, B) format, a block of at least one color sample may be considered.

[0163] At least one embodiment is not limited to a specific picture / video format.

[0164] Generally, the present disclosure relates to a method for predicting blocks of a video picture based on bidirectional motion compensation. The method determines a first reference block as a block of a first reference picture in a first reference picture list, the first reference block being pointed to by a first motion vector starting from the block of the video picture, and determines a second reference block as a block of a second reference picture in a second reference picture list, the second reference block being pointed to by a second motion vector starting from the block of the video picture. Next, the method checks whether at least one sample of at least one of the first or second reference blocks is at least partially outside the boundary of the first or second reference picture. If at least one sample of the first reference block is outside the boundary of the first reference picture, the method further determines a first non-overlapping rectangular block that includes at least one sample of the first reference block within the boundary of the first reference picture. If at least one sample of the second reference block is at least partially outside the boundary of the second reference picture, the method further determines a second non-overlapping rectangular block that includes at least one sample of the second reference block within the boundary of the second reference picture. The method further: when the first reference block is at least partially outside the first non-overlapping rectangular block, replaces at least one sample of the first reference block with at least one co-located sample of the second reference block, and when the second reference block is at least partially outside the second non-overlapping rectangular block, replaces at least one sample of the second reference block with at least one co-located sample of the first reference block, to derive a prediction block based on bidirectional motion compensation.

[0165] The present invention determines the non-overlapping rectangular block from the sampling positions of the reference blocks, rather than processing the per-sample OOB criterion as in the prior art. This significantly reduces the complexity of the method compared to method 300. In fact, instead of evaluating the block OOB criterion and the sample OOB criterion for each sampling position in the block (method 300), driving the OOB decision by the non-overlapping block significantly reduces the complexity.

[0166] Figure 9 A block diagram schematically shows the steps of a method 400 for predicting a block based on adaptive bidirectional motion compensation according to an embodiment.

[0167] In this embodiment, the first reference block P L0 and the second reference block P L1 are split into sub-blocks.

[0168] For example, each reference block is uniformly split into 4×4 sub-blocks.

[0169] This embodiment is advantageous because driving the OOB decision for each sub-block in each CU significantly reduces the complexity of the OOB decision.

[0170] For example, when 4×4 sub-blocks can be obtained for the first reference block and the second reference block, compared to the per-sample evaluation of the OOB criterion, the number of OOB criterion evaluations is divided by 16.

[0171] If the boolean value BlockOOB L0 is true and / or the boolean value BlockOOB L1 is true (step 305), that is, if the first reference block P L0 is at least partially outside the boundary of the first reference picture, and / or if the second reference block P L1 is at least partially outside the boundary of the second reference picture, steps 401-411 follow step 305.

[0172] In step 401, a first non-out-of-bounds rectangular block Z L0 is determined. The first non-out-of-bounds rectangular block Z L0 includes at least one sample of the first reference block p L0 within the boundary of the first reference picture.

[0173] In step 402, a second non-out-of-bounds rectangular block Z L1 is determined. The second non-out-of-bounds rectangular block Z L1 includes at least one sample of the second reference block p L1 within the boundary of the second reference picture.

[0174] Figure 10 Shows an example of bidirectional motion compensation-based prediction according to an embodiment. In this example, some samples of the first reference block p L0 are outside the boundary of the first reference picture, and all samples of the second reference block p L1 are within the boundary of the second reference picture. Then, the first non-out-of-bounds rectangular block Z L0 is composed of the samples of the first reference block P L0 within the boundary of the first reference picture, and the second non-out-of-bounds rectangular block Z L1 contains all samples of the second reference block p L1 .

[0175] In step 403, the current sub-block position (i,j) is considered.

[0176] In step 404, method 400 checks whether the sub-block sP L0 at the current sub-block position (i,j) in the first reference block P L0 (i,j) is at least partially outside the first non-out-of-bounds rectangular block Z L0 . Then method 400 assigns a value to the boolean value OOB L0 . If the sub-block sP L0(i,j) is at least partially within the first non-out-of-bounds rectangular block Z L0 outside, then the boolean value OOB L0 is true, otherwise the boolean value OOB L0 is false.

[0177] In step 405, method 400 checks the sub-block sP L1 at the current sub-block position (i,j) within the second reference block P L1 (i,j) to see if it is at least partially outside the second non-out-of-bounds rectangular block Z L1 . Then method 400 assigns a value to the boolean value OOB L1 . If the sub-block sP L1 (i,j) is at least partially outside the second non-out-of-bounds rectangular block Z L1 , then the boolean value OOB L1 is true, otherwise the boolean value OOB L1 is false.

[0178] In step 406, method 400 checks whether the boolean value OOB L0 is true and whether the boolean value OOB L1 is false.

[0179] If the boolean value OOB L0 is true and the boolean value OOB L1 is false, that is, if the first sub-block sP L0 (i,j) is at least partially outside the first non-out-of-bounds rectangular block Z L0 and the second sub-block sP L1 (i,j) is within the second non-out-of-bounds rectangular block Z L1 , then in step 407, method 400 derives the prediction sub-block sP final (i,j) based on bidirectional motion compensation as the second sub-block sP L1 (i,j).

[0180] Otherwise, in step 408, method 400 checks whether the boolean value OOB L0 is false and whether the boolean value OOB L1 is true.

[0181] If the boolean value OOB L0 is false and the boolean value OOB L1 is true, that is, if the first sub-block sP L0 (i,j) is within the first non-out-of-bounds rectangular block Z L0 and the second sub-block sP L1 (i,j) is at least partially outside the second non-out-of-bounds rectangular block Z L1Otherwise, in step 409, method 400 derives the prediction sub-block sP final (i,j) based on bidirectional motion compensation as the first sub-block sP L0 (i,j).

[0182] Otherwise, in step 410, method 400 derives the prediction sub-block sP final (i,j) based on bidirectional motion compensation as the average of the samples of the first sub-block sP L0 (i,j) and the samples of the second sub-block sP L1 (i,j).

[0183] Step 410 of method 400 can also be applied in the case where the first sub-block sP L0 (i,j) is completely within the first non-out-of-bounds rectangular block Z L0 and the second sub-block sP L1 (i,j) is completely within the second non-out-of-bounds rectangular block Z L1 .

[0184] After steps 407, 409, and 410 is step 411, in which method 400 checks whether all sub-block positions have been considered. In this case, method 400 ends. Otherwise, after step 411 is step 403, in which another current sub-block position is considered.

[0185] Figure 11 FIG. schematically shows a block diagram of the steps of a method 500 for predicting blocks based on adaptive bidirectional motion compensation according to an embodiment.

[0186] If the boolean value OOB L0 is true and / or the boolean value OOB L1 is true (step 305), i.e., if the first reference block P L0 is at least partially outside the boundary of the first reference picture and / or if the second reference block P L1 is at least partially outside the boundary of the second reference picture, then after step 305 are steps 401, 402, and 501 - 503.

[0187] In step 401, a first non-out-of-bounds rectangular block Z L0 is determined. The first non-out-of-bounds rectangular block Z L0 includes at least one sample of the first reference block P L0 within the boundary of the first reference picture.

[0188] In step 402, a second non-out-of-bounds rectangular block Z L1 is determined. The second non-out-of-bounds rectangular block Z L1 includes the second reference block P within the boundary of the second reference pictureL1 at least one sampling

[0189] In step 501, method 400: When the at least one sampling is within the first non - out - of - bounds rectangular block Z L0 copy at least one sampling of the first reference block P L0 to the first intermediate block P′ L0 and copy at least one sampling of the second reference block P L1 to at least one sampling positions of the first intermediate block P′ L0 where at least one sampling positions of the first intermediate block correspond to at least one sampling positions of the first reference block P L0 which is outside the first non - out - of - bounds rectangular block Z L0 L0

[0190] In step 502, method 400: When the at least one sampling is within the second non - out - of - bounds rectangular block Z L1 copy at least one sampling of the second reference block P L1 to the second intermediate block P′ L1 and copy at least one sampling of the first reference block P L0 to at least one sampling positions of the second intermediate block P′ L1 where at least one sampling positions of the second intermediate block correspond to at least one sampling positions of the second reference block P L1 which is outside the second non - out - of - bounds rectangular block Z L1 L1

[0191] In step 503, method 500: Derive the prediction block P based on bidirectional motion compensation by averaging the samplings of the first and second intermediate blocks final final

[0192] In one embodiment, determine the first and second non - out - of - bounds rectangular blocks according to the motion vectors associated with the first and second reference blocks and the size of the blocks of the video picture

[0193] In one embodiment, define the non - out - of - bounds rectangular block by the positioning of the upper - left corner of the rectangle and the number of columns and rows

[0194] In one embodiment, the coordinates (x Z , y Z ) of the positioning of the upper - left corner of the rectangle block and the number of columns (width) w Z and the number of rows (height) h Z are given in Table 4 below

[0195] Table 4

[0196]

[0197]

[0198] The intermediate block can be easily calculated without any test of the sampling positions, as follows. For example, the first intermediate block P′ L0 is just a copy of the sampling values of the first reference block P L0 or the second reference block P L1 For each sampling location (i, j) within the first non-out-of-bounds rectangular block Z L0 i.e., and set the value of the sample P′ L0 (i, j) to be equal to the value of the sample P L0 (i, j), and for each sampling location (i, j) outside the first non-out-of-bounds rectangular block Z L0 set the value of the sample P′ L0 (i, j) to be equal to the value of the sample P L1 (i, j).

[0199] To calculate the intermediate block P′ without any test L0 , the block P L1 can first be completely copied to the block P′ L0 . Next, a loop can be executed over the sampling positions and , and for each position (i, j), copy the sample P L0 at the position (i, j) of the block P L0 (i, j) to the block P′ L0 at the position (i, j).

[0200] Thus, the intermediate block P′ L0 is obtained in a very straightforward manner without any checks at the sampling or sub-block level. The same remarks also apply to the second intermediate block P′ L1 .

[0201] In one embodiment of step 503, the prediction block P based on bidirectional motion compensation final is derived as follows:

[0202]

[0203] In one embodiment of step 503, the samples of the first and second intermediate blocks or the samples of the first and second sub-blocks are averaged using a weighting factor.

[0204] In one embodiment of step 503, the prediction block P based on bidirectional motion compensation final is derived as follows:

[0205]

[0206] For example, w 0 = w 0 = 1 / 2

[0207] Figure 12 shows an example of the result of bidirectional motion compensation - based prediction according to method 500.

[0208] This example corresponds to the example of the first and second reference blocks given in Figure 10 , where only the first reference block is partially outside the boundary of the first reference picture. Figure 10 In

[0209] Figure 12 's example, it shows that the first intermediate block includes the samples of the first reference block and the samples of the second reference block in the band to the left of the first reference block, and the second intermediate block only includes the samples of the second reference block. Then, the bidirectional motion compensation - based prediction block P is derived by averaging the first and second intermediate blocks final . In the example, the sampled values in the left - hand band of the bidirectional motion compensation - based prediction block P final are equal to the sampled values of the second reference block, and the sampled values within the first non - out - of - bounds rectangular block are the average of the samples of the first and second reference blocks.

[0210] Figure 13 shows an example of the result of bidirectional motion compensation - based prediction according to method 500.

[0211] In this example, the first reference block is partially outside the boundary of the first reference picture, and the second reference block is partially outside the boundary of the second reference picture.

[0212] Figure 13 's example shows that the first intermediate block includes the samples of the first reference block and the samples of the second reference block in the first band to the left of the first reference block, and the second intermediate block includes the samples of the second reference block and the samples of the first reference block in the second band at the bottom of the second reference block. Then, the bidirectional motion compensation - based prediction block P is derived by averaging the first and second intermediate blocks final . In the given example, the sampled values in the first band of the bidirectional motion compensation - based prediction block P final are equal to the sampled values of the second reference block, the sampled values within the intersection area of the first and second non - out - of - bounds rectangular blocks are the average of the samples of the first and second reference blocks, the sampled values in the second band of the bidirectional motion compensation - based prediction block P final are equal to the sampled values of the first reference block, and the sampled values located in the bidirectional motion compensation - based prediction block P finalThe sampled value at the lower left corner is equal to the average of the samples of the first and second reference blocks.

[0213] Method 500 is advantageous because methods 300, 400, and 500 provide the same result but method 500 does not involve any checking operations at the sampling level or sub-block level, as is the case with the related art methods and method 400.

[0214] Accordingly, method 500 has the advantage of maintaining the compression efficiency of the related art methods while significantly reducing its complexity.

[0215] Figure 14 FIG. shows a schematic block diagram illustrating an example of a system 600 in which various aspects and embodiments are implemented.

[0216] System 600 may be embedded as one or more devices, including the various components described below. In various embodiments, system 600 may be configured to implement one or more aspects described in the present disclosure.

[0217] Examples of equipment that may form all or part of system 600 include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or web servers), still or video cameras, encoding or decoding chips, or any other communication device. The elements of system 600 may be implemented singly or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 600 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 600 may be communicatively coupled to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0218] System 600 may include at least one processor 610 configured to execute instructions loaded therein for implementing various aspects described in the present disclosure, for example. The processor 610 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 600 may include at least one memory 620 (e.g., volatile memory devices and / or non-volatile memory devices). System 600 may include a storage device 640, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, the storage device 640 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0219] System 600 may include an encoder / decoder module 630 configured to, for example, process data to provide encoded / decoded video picture data, and the encoder / decoder module 630 may include its own processor and memory. The encoder / decoder module 630 may represent one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding and a decoding module. Additionally, the encoder / decoder module 630 may be implemented as a separate element of System 600 or may be incorporated within the processor 610 as a combination of hardware and software known to those skilled in the art.

[0220] Program code to be loaded into the processor 610 or the encoder / decoder 630 to execute various aspects described in the present disclosure may be stored in the storage device 640 and subsequently loaded into the memory 620 for execution by the processor 610. According to various embodiments, during the execution of the processes described in the present disclosure, one or more of the processor 610, the memory 620, the storage device 640, and the encoder / decoder module 630 may store one or more of various items. Such stored items may include but are not limited to video picture data, information data for encoding / decoding video picture data, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0221] In several embodiments, the memory internal to the processor 610 and / or the encoder / decoder module 630 may be used to store instructions and provide working memory for processing that may be performed during encoding or decoding.

[0222] However, in other embodiments, a memory external to the processing device (e.g., the processing device can be the processor 610 or the encoder / decoder module 630) is used for one or more of these functions. The external memory can be the memory 620 and / or the storage device 640, e.g., dynamic volatile memory and / or non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM can be used as a working memory for video encoding and decoding operations, e.g., for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 video), AVC, HEVC, EVC, VVC, AV1, etc.

[0223] As indicated in block 690, input to the elements of the system 600 can be provided via various input devices. Such input devices include, but are not limited to, (i) an RF section that can receive an RF signal, e.g., transmitted over the air by a broadcast device, (ii) composite input terminals, (iii) USB input terminals, (iv) HDMI input terminals, (v) a bus when the present invention is implemented in the automotive field, such as a CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus.

[0224] In various embodiments, the input devices of block 690 have associated respective input processing elements, as known in the art. For example, the RF section can be associated with elements necessary for (i) selecting a desired frequency (also known as selecting a signal or band-limiting the signal to a band), (ii) down-converting the selected signal, (iii) band-limiting the band again to a narrower band to select a signal band that can be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) de-multiplexing to select a desired data packet stream. The RF section of various embodiments can include one or more elements that perform these functions, e.g., a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a de-multiplexer. The RF section can include a tuner that performs various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0225] In one embodiment of a set-top box, the RF section and its associated input processing elements may receive RF signals transmitted over a wired (e.g., cable) medium. The RF section may then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0226] Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0227] Adding elements may include inserting elements between existing elements, such as inserting an amplifier and an analog-to-digital converter, for example. In various embodiments, the RF section may include an antenna.

[0228] In addition, the USB and / or HDMI terminals may include respective interface processors for connecting the system 600 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within the processor 610 when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within the processor 610 when necessary. The demodulated, error-corrected, and demultiplexed stream may be provided to various processing elements, including, for example, the processor 610 and the encoder / decoder 630, which operate in conjunction with memory and storage elements to process the data stream as necessary for presentation on an output device.

[0229] The various elements of the system 600 may be provided within an integrated housing. Within the integrated housing, a suitable connection arrangement 690, such as internal buses (including I2C buses), wiring, and printed circuit boards known in the art, may be used to interconnect the various elements and transfer data between them.

[0230] The system 600 may include a communication interface 650 that enables communication with other devices via a communication channel 651. The communication interface 650 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 651. The communication interface 650 may include, but is not limited to, a modem or a network card, and the communication channel 651 may be implemented, for example, within a wired and / or wireless medium.

[0231] In various embodiments, Wi-Fi networks such as IEEE 802.11 can be used to stream data to system 600. The Wi-Fi signals of these embodiments can be received through communication channel 651 and communication interface 650 suitable for Wi-Fi communication. The communication channel 651 of these embodiments can generally be connected to an access point or a router that provides access to an external network including the Internet to allow streaming applications and other over-the-top communications.

[0232] Other embodiments can use a set-top box to provide streamed data to system 600, and the set-top box delivers data through the HDMI connection of input box 690.

[0233] Still other embodiments can use the RF connection of input box 690 to provide streamed data to system 600.

[0234] The streamed data can be used as a way for system 600 to transmit signaling information. The signaling information can include bitstream B and / or information such as the number of video picture pixels and / or any codec / decoding setting parameters.

[0235] It should be recognized that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. can be used to transmit information to the corresponding decoder.

[0236] System 600 can provide output signals to various output devices, including display 661, speaker 671, and other peripheral devices 681. In various examples of the embodiments, other peripheral devices 681 can include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices based on the output providing functions of system 600.

[0237] In various embodiments, control signals can be communicated between system 600 and display 661, speaker 671, or other peripheral devices 681 using signaling of communication protocols such as AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention.

[0238] The output devices can be communicatively connected to system 600 through dedicated connections via corresponding interfaces 660, 670, and 680.

[0239] Alternatively, the output devices can be connected to system 600 via communication channel 651 using communication interface 650. Display 661 and speaker 671 can be integrated into a single unit with other components of system 600 in an electronic device such as a television.

[0240] In various embodiments, the display interface 660 may include a display driver, such as, for example, a timing controller (TCon) chip.

[0241] For example, if the RF portion of the input terminal 690 is part of a separate set-top box, then the display 661 and the speaker 671 may optionally be separate from one or more of the other components. In various embodiments where the display 661 and the speaker 671 may be external components, output signals may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0242] In Figure 1-14 this document, various methods are described, and each method includes one or more steps or actions to implement the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0243] Some examples are described with reference to block diagrams and / or operational flowcharts. Each block represents a circuit element, module, or portion of code that includes one or more executable instructions for implementing the specified logical function(s). It should also be noted that in other implementations, the function(s) noted in the blocks may not occur in the order indicated. For example, depending on the functions involved, two blocks shown in succession may actually be executed substantially concurrently, or sometimes the blocks may be executed in the reverse order.

[0244] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if discussed only in the context of a single form of embodiment (e.g., only as a method), the embodiments of the features discussed may be implemented in other forms (e.g., an apparatus or a computer program).

[0245] A method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes a communication device.

[0246] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values generated by the embodiments) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having computer-readable program code executable by a computer implemented thereon. Considering the inherent ability to store information therein and the inherent ability to retrieve information provided therefrom, a computer-readable storage medium as used herein can be considered a non-transitory storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. It should be recognized that the following, although providing more specific examples of computer-readable storage media to which the present embodiment can be applied, are merely illustrative and not exhaustive as readily recognized by those of ordinary skill in the art: portable computer floppy disks; hard disks; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination of the foregoing.

[0247] The instructions can form an application program tangibly implemented on a processor-readable medium.

[0248] For example, the instructions can be in hardware, firmware, software, or a combination. For example, the instructions can be found in an operating system, a separate application, or a combination of both. Thus, a processor can be characterized as, for example, a device configured to execute a process and a device including a processor-readable medium (such as a storage device) having instructions for executing the process. Additionally, in addition to or instead of the instructions, the processor-readable medium can store data values generated by the embodiments.

[0249] The apparatus can be implemented in, for example, appropriate hardware, software, and firmware. Examples of such apparatus include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing the output from a video decoder, pre-processors for providing input to a video encoder, web servers, set-top boxes, and any other devices for processing video frames, or other communication devices. It should be clear that the equipment can be mobile and even installed in a moving vehicle.

[0250] Computer software can be implemented by processor 610 or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments can also be implemented by one or more integrated circuits. Memory 620 can be of any type suitable for the technical environment and can be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). Processor 610 can be of any type suitable for the technical environment and can encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples).

[0251] Embodiments of the present disclosure also provide an electronic device, which includes: a processor; and a memory for storing instructions executable by the processor. The processor is configured to execute any of the methods described above.

[0252] As will be apparent to those of ordinary skill in the art, embodiments can generate various signals that are formatted to carry information that can be stored or transmitted, for example. The information can include, for example, instructions for performing a method or data generated by one of the described embodiments. For example, the signal can be formatted to carry a bitstream of the described embodiment. Such a signal can be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. As is known, signals can be transmitted over various different wired or wireless links. The signal can be stored on a processor-readable medium.

[0253] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an", and "the" may also be intended to include the plural forms, unless the context clearly indicates otherwise. It will be further understood that when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Also, when an element is referred to as being "responsive to" or "connected to" or "associated with" another element, it can be directly responsive or connected to or associated with the other element, or intervening elements may be present. In contrast, when an element is referred to as being "directly responsive to" or "directly connected to" or "directly associated with" another element, no intervening elements are present.

[0254] It should be recognized that, for example, in the case of "A / B", "A and / or B", and "at least one of A and B", the use of any one of the symbols / terms " / ", "and / or", and "at least one of" can be intended to cover the selection of only the first-listed option (A), or only the second-listed option (B), or the selection of both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to cover the selection of only the first-listed option (A), or only the second-listed option (B), or only the third-listed option (C), or the selection of only the first and second-listed options (A and B), or the selection of only the first and third-listed options (A and C), or the selection of only the second and third-listed options (B and C), or the selection of all three options (A and B and C). As will be clear to those of ordinary skill in the art and related fields, this can be extended to as many items as are listed.

[0255] A variety of numerical values can be used in the present disclosure. Specific values can be used for illustrative purposes and the described aspects are not limited to these specific values.

[0256] It will be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the teachings of the present disclosure, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. No ordering is implied between the first element and the second element.

[0257] References to "an embodiment" or "embodiments" or "an implementation" or "implementations" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in connection with the embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearance of the phrases "in an embodiment" or "in embodiments" or "in an implementation" or "in implementations" and any other variations thereof throughout the present disclosure does not necessarily all refer to the same embodiment.

[0258] Similarly, references herein to "according to an embodiment / example / implementation" or "in an embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an embodiment / example / implementation) may be included in at least one embodiment / example / implementation. Therefore, the expressions "according to an embodiment / example / implementation" or "in an embodiment / example / implementation" appearing in various places in this disclosure do not necessarily all refer to the same embodiment / example / implementation, nor are separate or alternative embodiments / examples / implementations necessarily mutually exclusive of other embodiments / examples / implementations.

[0259] Reference numerals appearing in the claims are for illustration purposes only and have no limiting effect on the scope of the claims.The present embodiments / examples and variations may be employed in any combination or sub-combination although not explicitly described.

[0260] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0261] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0262] Various embodiments relate to decoding. As used in the present disclosure, "decoding" may encompass, for example, all or part of a process performed on a received video picture (which may include a received bitstream encoding one or more video pictures) to produce a final output suitable for display or further processing in a reconstructed video domain. In various embodiments, such a process includes one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process also includes or optionally includes a process performed by a decoder of the various embodiments described in the present disclosure.

[0263] As a further example, in one embodiment "decoding" may refer only to dequantization, in one embodiment "decoding" may refer to entropy decoding, in another embodiment "decoding" may refer only to differential decoding, and in another embodiment "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. Based on the context of the specific description, whether the phrase "decoding process" may be intended to specifically refer to a subset of operations, or generally refer to a broader decoding process will be clear and is believed to be well understood by those skilled in the art.

[0264] Various embodiments relate to encoding. In a manner similar to the above discussion regarding "decoding", "encoding" as used in the present disclosure can cover, for example, all or part of the process performed on an input video picture to generate an output bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder. In various embodiments, such processes also include or optionally include the processes performed by the encoders of the various embodiments described in the present disclosure.

[0265] As a further example, in one embodiment "encoding" can refer only to quantization, in one embodiment "encoding" can refer only to entropy encoding, in another embodiment, "encoding" can refer only to differential encoding, and in another embodiment, "encoding" can refer to a combination of quantization, differential encoding, and entropy encoding. Based on the context of a particular description, it will be clear whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process, and it is believed that this will be well understood by those skilled in the art.

[0266] In addition, the present disclosure may refer to "obtaining" various information. Obtaining information can include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from a memory, processing information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0267] In addition, the present disclosure may refer to "receiving" each piece of information. Receiving information can include, for example, one or more of the following: accessing information or receiving information from a communication network.

[0268] Moreover, as used herein, the word "signaling" particularly refers to indicating something to a corresponding decoder, etc. For example, in certain embodiments, an encoder signals specific information, such as codec parameters or encoded video picture data. In this way, in an embodiment, the same parameters can be used on the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicitly signal) specific parameters to a decoder such that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, then signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameters. By avoiding the transmission of any actual functionality, bit savings are achieved in various embodiments. It should be recognized that signaling can be accomplished in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. Although the foregoing relates to the verb form of the word "signal", the word "signaling" can also be used as a noun herein.

[0269] Multiple embodiments have been described. However, it should be understood that various modifications can be made. For example, elements of different embodiments can be combined, supplemented, modified, or removed to produce other embodiments. In addition, those of ordinary skill in the art will understand that other structures and processes can replace the disclosed structures and processes, and the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) ways to achieve at least substantially the same (one or more) results as the disclosed embodiments. Accordingly, the present disclosure contemplates these and other embodiments.

Claims

1. A method for predicting a block of a video picture based on bidirectional motion compensation, wherein the method comprises: - determining (301) a first reference block as a block of a first reference picture in a first reference picture list, the first reference block being pointed to by a first motion vector starting from the block of the video picture; - determining (302) a second reference block as a block of a second reference picture in a second reference picture list, the second reference block being pointed to by a second motion vector starting from the block of the video picture; - checking (305) whether the sampling of at least one of the first or second reference blocks is at least partially outside the boundary of the first or second reference picture; wherein the method further comprises: - If at least one sample of the first reference block is outside the boundary of the first reference picture, determine (401) a first non-out-of-bounds rectangular block (Z L0 ) that includes at least one sample of the first reference block that is within the boundary of the first reference picture; - If at least one sample of the second reference block is at least partially outside the boundary of the second reference picture, determine (402) a second non-out-of-bounds rectangular block (Z L1 ), the second non-out-of-bounds rectangular block including at least one sample of the second reference block that is within the boundary of the second reference picture; and - when the first reference block is at least partially outside the first non-overlapping rectangular block, deriving (403-411, 501-503) a prediction block based on bidirectional motion compensation by replacing at least one sampling of the first reference block with at least one co-located sampling of the second reference block, and when the second reference block is at least partially outside the second non-overlapping rectangular block, deriving the prediction block based on bidirectional motion compensation by replacing at least one sampling of the second reference block with at least one co-located sampling of the first reference block.

2. The method according to claim 1, wherein the first and second reference blocks are split into sub-blocks, and for each current sub-block position, the method further comprises: - If a first sub-block (sP L0 (i,j)) at the current sub-block position in the first reference block is at least partially outside the first non-overflow rectangular block (Z L0 ) and a second sub-block (sP L1 (i,j)) is within the second non-overflow rectangular block (Z L1 ), a prediction sub-block (sP final (i,j)) based on bidirectional motion compensation is derived (470) as the second sub-block (sP L1 (i,j)); - If the first sub-block (sP L0 (i,j)) located at the current sub-block position in the first reference block is within the first non-out-of-bounds rectangular block (Zj L0 ) and the second sub-block (sP L1 (i,j)) is at least partially outside the second non-out-of-bounds rectangular block (Z L1 ), the prediction sub-block (sP final (i,j)) based on bidirectional motion compensation is derived (409) as the first sub-block (sP L0 (i,j)); - Otherwise, the predictor block (Sp final (i, j)) of the bidirectional motion compensation is derived (410) as the average of the samples of the first sub-block (sP L0 (i, j)) and the samples of the second sub-block (Sp L1 (i, j)).

3. The method according to claim 2, wherein if the first sub-block (Sp L0 (i,j)) is completely within the first non-overflow rectangular block (Z L0 ), and the second sub-block (sP L1 (i,j)) is completely within the second non-overflow rectangular block (Z L1 ), then the method further comprises deriving (410) the bi-directional motion compensated prediction sub-block (sP final (i,j)) as the average of the samples of the first sub-block (sP L0 (i,j)) and the samples of the second sub-block (sP L1 (i,j)).

4. The method according to any one of claims 1 to 3, wherein the method further comprises: - if at least one sampling of the first reference block is outside the first non-overlapping rectangular block, or at least one sampling of the second reference block is outside the second non-overlapping rectangular block, then - When at least one of the samplings is within the first non-out-of-bounds rectangular block (Z L0 ), copy (501) at least one sampling of the first reference block (P L0 ) to the first intermediate block (P' L0 ), and copy at least one sampling of the second reference block (P L1 ) to at least one sampling position of the first intermediate block (P' L0 ), where at least one sampling position of the first intermediate block corresponds to at least one sampling position of the first reference block (P L0 ) that is outside the first non-out-of-bounds rectangular block (Z L0 ); - When at least one sample is within the second non - out - of - bounds rectangular block (Z L1 ), copy (502) at least one sample of the second reference block (P L1 ) to the second intermediate block (P' L1 ), and copy at least one sample of the first reference block (P L0 ) to at least one sample position of the second intermediate block (P' L1 ), where at least one sample position of the second intermediate block corresponds to at least one sample position of the second reference block (P L1 ) that is outside the second non - out - of - bounds rectangular block (Z L1 ); - deriving (503) a prediction block based on bidirectional motion compensation by averaging the samplings of the first and second intermediate blocks.

5. The method according to any one of claims 2 to 4, wherein the samplings of the first and second intermediate blocks or the samplings of the first and second sub-blocks are averaged using a weighting factor.

6. The method according to any one of claims 1 to 5, wherein the first and second non-overlapping rectangular blocks are determined according to the motion vectors associated with the first and second reference blocks and the size of the block of the video picture.

7. The method according to any one of claims 1 to 6, wherein the first and second non-overlapping rectangular blocks are defined by the positioning of the upper left corner of the rectangle and the number of columns and rows.

8. The method according to claim 7, wherein the positioning of the upper left corner of the rectangle is related to the upper left corner of the block of the video picture.

9. A method for encoding a block of a video picture into a bitstream of encoded video picture data, the method comprising performing temporal prediction on the block of the video picture according to the method of any one of claims 1 to 8.

10. A method for decoding a block of a video picture from a bitstream of encoded video picture data, the method comprising performing temporal prediction on the block of the video picture according to the method of any one of claims 1 to 8.

11. An apparatus, comprising means for performing one of the methods according to any one of claims 1 to 10.

12. A computer program product comprising instructions which, when the program is executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 10.

13. A non-transitory storage medium carrying instructions of program code for performing the method according to any one of claims 1 to 10.

14. An electronic device, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to perform one of the methods according to any one of claims 1 to 10.

Images