Method and apparatus for adaptive constraints on bidirectional prediction for out-of-boundary conditions - Patents.com

Abstract

A method and apparatus for inter prediction executed by at least one processor. The method for inter prediction includes receiving a coded video bitstream including a current picture and a first and a second reference picture associated with the current picture, the current picture including at least one block encoded according to an inter bi-prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture, determining whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied, and for the inter bi-prediction mode, adaptively disallowing bi-prediction to have MVs pointing outside the reference picture boundary.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority under 35 U.S.C. §119 from U.S. Provisional Application No. 63 / 323,749, filed in the U.S. Patent and Trademark Office on March 25, 2022, and U.S. Application No. 17 / 945,679, filed on September 15, 2022, the disclosures of which are incorporated by reference in their entireties.

[0002] [Technical field] A method and apparatus according to an example embodiment of the present disclosure relates to an MV constraint for inter bi-prediction modes that adaptively disallows bi-prediction from having MVs that point outside the reference picture boundaries. [Background technology]

[0003] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3) and 2016 (version 4). In 2015, these two standards bodies jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing the next video coding standard beyond HEVC. In October 2017, they issued a Joint Call for Proposal (CfP) on video compression with capabilities beyond HEVC. By February 15, 2018, a total of 22 CfP responses for standard dynamic range (SDR), 12 CfP responses for high dynamic range (HDR), and 12 CfP responses for 360 video category were submitted. In April 2018, all received CfP responses were evaluated at the 122 MPEG / 10th JVET meeting. As a result of this meeting, JVET formally started the standardization process for next-generation video coding beyond HEVC, and the new standard was named VVC (Versatile Video Coding), and JVET was renamed Joint Video Experts Team. In 2020, ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the VVC video coding standard (version 1).

[0004] For each inter-prediction coding unit (CU), motion parameters consisting of motion vectors, reference picture indices and reference picture list usage indices, as well as additional information required by the new coding features of VVC, are used to generate inter-prediction samples.

[0005] A picture may have at least one block coded according to a bi-prediction mode, and the at least one block includes a first motion vector (MV) pointing to a first reference picture and a second MV pointing to a second reference picture. When either the first MV or the second MV points to a position outside a frame boundary, a constraint on the bi-prediction mode may be applied. For example, the constraint may change the coding mode of at least one block from a bi-prediction mode to another coding mode, such as a uni-prediction mode. Current efforts to apply constraints for inter-bi-prediction when a motion vector (MV) crosses a frame boundary may not always be beneficial for coding efficiency. Summary of the Invention

[0006] The following presents a simplified summary of one or more embodiments of the present disclosure in order to provide a basic understanding of such embodiments. This summary is not an extensive overview of all possible embodiments, and is not intended to identify key or critical elements of all embodiments or to delineate the scope of any or all embodiments. Its sole purpose is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0007] A method, apparatus, and non-transitory computer-readable medium for MV constraints that, for inter bi-prediction modes, adaptively disallow bi-prediction from having MVs that point outside the reference picture boundaries.

[0008] According to an exemplary embodiment, a method for inter prediction is executed by at least one processor. The method includes receiving a coded video bitstream including a current picture and a first and a second reference picture associated with the current picture, the current picture including at least one block encoded according to an inter bi-prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture. The method further includes determining whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied. The method further includes determining whether a disabling condition for disabling the out-of-bounds condition is satisfied in response to determining that the out-of-bounds condition is satisfied and the disabling condition is not satisfied. The method further includes (i) changing the at least one block from the inter bi-prediction mode to another coding mode, and (ii) decoding the at least one block according to the other coding mode. The method further includes, in response to determining that the out of bounds condition is not met or the disabling condition is met, decoding at least one block according to an inter prediction bi-prediction mode.

[0009] According to an exemplary embodiment, an apparatus for inter prediction includes at least one memory configured to store computer program code and at least one processor configured to access the computer program code and operate as instructed by the computer program code. The computer program code includes a receiving code configured to cause the at least one processor to receive a coded video bitstream including a current picture and a first and a second reference picture associated with the current picture, the current picture including at least one block encoded according to an inter bidirectional prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture. The computer program code further includes a first decision code configured to cause the at least one processor to determine whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied. The computer program code further includes a second decision code configured to cause the at least one processor to determine whether a disabling condition for disabling the out-of-bounds condition is satisfied in response to determining that the out-of-bounds condition is satisfied. The computer program code further includes modification code and decoding code, where in response to determining that the out-of-bounds condition is met and the disabling condition is not met, (i) the modification code causes the at least one processor to modify the at least one block from the inter bi-prediction mode to another coding mode, and (ii) the decoding code causes the at least one processor to decode the at least one block according to the other coding mode. The computer program code further includes decoding code for causing the at least one processor to decode the at least one block according to the inter-prediction bi-prediction mode in response to determining that the out-of-bounds condition is not met or the disabling condition is met.

[0010] According to an exemplary embodiment, a non-transitory computer-readable medium is provided having instructions stored thereon that, when executed by a processor, cause the processor to perform a method for inter prediction. The method includes receiving a coded video bitstream including a current picture and a first and a second reference picture associated with the current picture, the current picture including at least one block encoded according to an inter bi-prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture. The method further includes determining whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied. The method further includes determining whether a disabling condition for disabling the out-of-bounds condition is satisfied in response to determining that the out-of-bounds condition is satisfied and the disabling condition is not satisfied. The method further includes (i) changing at least one block from the inter bi-prediction mode to another coding mode, and (ii) decoding the at least one block according to the other coding mode. The method further includes, in response to determining that the out of bounds condition is not met or the disabling condition is met, decoding at least one block according to an inter prediction bi-prediction mode.

[0011] Further embodiments will be set forth in the description that follows, and in part will be obvious from the description, and / or may be learned by practice of the presented embodiments of the present disclosure. [Brief description of the drawings]

[0012] The above and other aspects, features and aspects of embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Diagram 2] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Diagram 3] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Diagram 5] 1 is an example of spatial merge candidate locations according to one embodiment. [Figure 6] 1 is an example of candidate pairs considered for redundancy checking of spatial merge candidates according to one embodiment. [Figure 7] 1 is an example of motion vector scaling for temporal merge candidates according to one embodiment. [Figure 8] 1 is an example of candidate positions C0 and C1 for temporal merge candidates according to one embodiment. [Figure 9A] 4 is an example of an affine motion field for a block according to one embodiment. [Figure 9B] 4 is an example of an affine motion field for a block according to one embodiment. [Figure 10] 4 is an example of an affine motion vector field for a block according to one embodiment. [Figure 11] 1 is an example of spatial merge candidate locations according to one embodiment. [Figure 12] 1 is an example of control point motion vector inheritance according to one embodiment. [Figure 13] 11 is an example of locations of candidate positions for a construct affine merge mode according to one embodiment. [Figure 14] 4 is an example of sub-blocks with motion vectors according to one embodiment. [Figure 15A] 1 is an example of spatial neighboring blocks used by ATVMP. [Figure 15B] 1 is an example of an SbTMVP process in a VVC according to one embodiment. [Figure 16] 1 is an example of an extended CU region used in BDOF according to one embodiment. [Figure 17] 1 is an example of decode-side motion vector refinement according to one embodiment. [Figure 18]1 is an example of GPM divisions grouped by identical angles according to one embodiment. [Figure 19] 1 is an example of one-way predictive motion vector selection for geometric partition mode according to one embodiment. [Figure 20] 13 is an example of bending weight generation using a geometric partition mode according to one embodiment. [Figure 21] 1 is a table of specifications for AmvrShift in VVC according to one embodiment. [Figure 22] 1 is an example of sub-block level bi-prediction constraints according to one embodiment. [Figure 23] 1 shows a flowchart of a method performed according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following detailed description of the exemplary embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0014] The above disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations. Furthermore, one or more features or components of some embodiments may be combined or incorporated in some embodiments (or one or more features of some embodiments). Furthermore, in the flowcharts and operational descriptions provided below, it is understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least partially), and the order of one or more operations may be rearranged.

[0015] It is apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to any specific software code, and it is understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0016] Although particular combinations of features are recited in the claims and / or disclosed herein, these combinations do not limit the disclosure of possible implementations. Indeed, many of these features may be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0017] No element, act, or instruction used herein should be construed as critical or essential unless expressly stated otherwise. Additionally, the singular terms used herein are intended to include one or more items and may be used interchangeably with "one or more." Additionally, terms such as "has," "have," "having," "include," "including," and the like, used herein are intended to be open-ended terms. Additionally, the phrase "based on" is intended to mean "based at least in part on," unless expressly stated otherwise. Additionally, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood as including only A, only B, or both A and B.

[0018] Throughout this specification, references to "some embodiments," "one embodiment," or similar language means that a particular feature, structure, or characteristic described in connection with the illustrated embodiment is included in at least some embodiments of the solution. Thus, appearances of the phrases "some embodiments," "one embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0019] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize in light of the description herein that the present disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in a particular embodiment that may not be present in all embodiments of the present disclosure.

[0020] The disclosed methods may be used separately or combined in any order. Additionally, the disclosed methods may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0021] Inter Prediction in VVC For each inter-predicted CU, motion parameters consisting of a motion vector, a reference picture index and a reference picture list usage index, as well as additional information required for the new coding features of VVC, are used for inter-predicted sample generation. The motion parameters may be signaled explicitly or implicitly. When a CU is coded in skip mode, it is associated with one PU and has no significant residual coefficients, coded motion vector deltas or reference picture indexes. A merge mode is specified, whereby motion parameters for the current CU, including spatial and temporal candidates and further schedules introduced in VVC, are obtained from neighboring CUs. The merge mode may be applied to any inter-predicted CU, not just skip mode. An alternative to the merge mode is explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index of each reference picture list, and a reference picture list usage flag, as well as other necessary information, are explicitly signaled for each CU.

[0022] In VVC, the VTM reference software includes several new and improved inter-predictive coding tools, which are listed below.

[0023] Enhanced Merge Prediction Merge motion vector difference (MMVD) AMVP mode with symmetric MVD signalling Affine motion compensation prediction Subblock-based temporal motion vector prediction (SbTMVP) Adaptive motion vector resolution (AMVR) Motion field storage: 1 / 16 luma sample MV storage and 8x8 motion field compression Bi-prediction with CU-level weights (BCW) Bi-directional optical flow (BDOF) Decoder side motion vector refinement (DMVR) Combined inter and intra prediction (CIIP) Geometric partitioning mode (GPM) The following text provides more details on inter prediction and related methods.

[0024] In VTM4, a merge candidate list is constructed by including the following five types of candidates, in order:

[0025] Spatial MVP from spatially adjacent CUs Time MVP from co-located CU History-based MVP from a FIFO table Pairwise average MVP Zero's music video The size of the merge list is signaled in the slice header, and the maximum allowed size of a merge list is 6 in VTM4. For each CU code in merge mode, the index of the best merge candidate is coded using truncated unary binarization (TU). The first bin of the merge index is coded with the context, and bypass coding is used for the other bins.

[0026] The generation process for each category of merge candidates is provided in this session.

[0027] The derivation of spatial merge candidates in VVC is the same as that in HEVC. Up to four merge candidates are selected from the candidates in the positions shown in FIG. 5. The order of derivation is B1, A1, B0, A0, B2. Position B2 is considered only when any CU in positions A0, B0, B1, A1 is unavailable (e.g., because it belongs to another slice or tile) or is intra-coded. After the candidate in position A1 is added, the addition of the remaining candidates is subjected to a redundancy check to ensure that candidates with the same motion information are removed from the list so that the coding efficiency is improved. In order to reduce the computational complexity, not all possible candidate pairs are considered in the above redundancy check. Instead, only the pairs linked by arrows in FIG. 6 are considered, and a candidate is added to the list only if the corresponding candidate used in the redundancy check does not have the same motion information.

[0028] In this operation, only one candidate is added to the list. In particular, in this derivation of the temporal merge candidate, a scaled motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list to be used for the derivation of the co-located CU is explicitly signaled in the slice header. The scaled motion vector for the temporal merge candidate is obtained as shown by the dotted line in FIG. 7, which is scaled from the motion vector of the co-located CU using POC distances tb and td. Here, tb is defined to be the POC difference between the reference picture of the current picture and the current picture, and td is defined to be the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to 0.

[0029] The location of the temporal candidate is selected between candidates C0 and C1 as shown in Figure 8. If the CU at location C0 is not available, is intra-coded, or is outside the current row of the CTU, then location C1 is used. Otherwise, location C0 is used in the derivation of the temporal merge candidate.

[0030] In HEVC, only translational motion model is applied to motion compensation prediction (MCP). In the real world, there are many kinds of motions, such as zoom in / out, rotation, perspective motion and other irregular motions. In the current VTM, block-based affine transformation motion compensation prediction is applied. As shown in Figure 9A and Figure 9B, the affine motion field of a block is described by the motion information of two control points (four parameters) or three control point motion vectors (six parameters).

[0031] For a four-parameter affine motion model, the motion vector at a sample location (x,y) within a block is derived as follows:

number

number

number

number

[0032] To simplify the motion compensation prediction, block-based affine transformation prediction is applied. To derive the motion vector of each 4×4 luma sub-block, the motion vector of the center sample of each sub-block is calculated according to the above formula and rounded to 1 / 16 fractional precision, as shown in FIG. 10. Then, a motion compensation interpolation filter is applied to generate a prediction for each sub-block with the derived motion vector. The sub-block size of the chroma components is also set to 4×4. The MV of a 4×4 chroma sub-block is calculated as the average of the MVs of the four corresponding 4×4 luma sub-blocks.

[0033] As is done for translational motion inter prediction, there are also two affine motion inter prediction modes: affine merge mode and affine AMVP mode.

[0034] The AF_MERGE mode may be applied to CUs whose width and height are both equal to or greater than 8. In this mode, the CPMV of the current CU is generated based on the motion information of spatially neighboring CUs. There may be up to five CPMVP candidates, and an index is signaled to indicate the one to be used for the current CU. The following three types of CPVM candidates are used to form the affine merge candidate list:

[0035] Inherited affine merge candidates extrapolated from the CPMV of adjacent CUs Construct affine merge candidate CPMVP derived using translational MVs of neighboring CUs Zero's music video In VTM3, there are at most two inherited affine candidates derived from the affine motion models of the neighboring blocks, one from the left neighboring CU and one from the top neighboring CU. The candidate blocks are shown in Figure 11. For the left predictor, the scanning order is A0→A1, and for the top predictor, the scanning order is B0→B1→B2. Only the first inherited candidate from each edge is selected. No pruning check is performed between the two inherited candidates. When a neighboring affine CU is identified, its control point motion vector is used to derive a CPMVP candidate in the affine merge list of the current CU. As shown in Figure 12, when the neighboring bottom-left block A is coded in affine mode, the motion vectors v2, v3 and v4 of the top-left corner, top-right corner and bottom-left corner of the CU containing block A are obtained. When block A is coded with a four-parameter affine model, the two CPMVs of the current CU are calculated according to v2 and v3. If block A is coded with a six-parameter affine model, the three CPMVs of the current CU are calculated according to v2, v3, and v4.

[0036] Constructing affine candidates means that the candidates are constructed by combining the translational motion information of each control point's neighbors. The motion information for a control point is derived from the specified spatial and temporal neighbors shown in Figure 13. CPMV k (k=1,2,3,4) represents the kth control point. For CPMV1, blocks B2->B3->A2 are examined and the MV of the first available block is used. For CPMV2, blocks B1->B0 are examined, and for CPMV3, blocks A1->A0 are examined. TMVP is used as CPMV4 if available.

[0037] After the MVs of the four control points are obtained, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used in order to construct it:

[0038] {CPMV1,CPMV2,CPMV3},{CPMV1,CPMV2,CPMV4},{CPMV1,CPMV3,CPMV4},{CPMV2,CPMV3,CPMV4},{CPMV1,CPMV2},{CPMV1,CPMV3} A combination of three CPMVs constructs a six-parameter affine merge candidate, and a combination of two CPMVs constructs a four-parameter affine merge candidate. To avoid the motion scaling process, if the reference indices of the control points are different, the associated combination of control point MVs is discarded.

[0039] After the inheritance and construction affine merge candidates have been examined, if the list is not already full, a MV of zero is inserted at the end of the list.

[0040] Affine AMVP mode may be applied to CUs whose width and height are both 16 or more. An affine flag at the CU level is signaled in the bitstream to indicate whether affine AMVP mode is used or not, and then another flag is signaled to indicate whether it is 4-parameter affine or 6-parameter affine. In this mode, the difference between the CPMV of the current CU and these predictor CPMVPs is signaled in the bitstream. The affine AVMP candidate list size is 2 and is generated by using the following four types of CPVM candidates in order:

[0041] Inherited affine AMVP candidates extrapolated from CPMVs of adjacent CUs Construct affine AMVP candidate CPMVP derived using translational MVs of neighboring CUs Translational MV from adjacent CU Zero's music video The testing order of inherited affine AMVP candidates is the same as that of inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as the current block are considered. When inserting an inherited affine motion predictor into the candidate list, the pruning process is not applied.

[0042] The constructed AMVP candidates are derived from the specified spatial neighborhood shown in Figure 13. The same check order is used as done in the construction of the affine merge candidate. In addition, the reference picture indexes of the neighboring blocks are also checked. The first block in the check order that is inter-coded and has the same reference picture as the current CU is used. When the current CU is coded in a 4-parameter affine mode, if there is only one and both mv0 and mv1 are available, they are added as one candidate in the affine AMVP list. When the current CU is coded in a 6-parameter affine mode and all three CPMVs are available, they are added as one candidate in the affine AMVP list. Otherwise, the constructed AMVP candidate is set as unavailable.

[0043] After the inherited affine AMVP candidates and the constructed AMVP candidates have been examined, if there are still less than two affine AMVP list candidates, then mv0, mv1, and mv2 are added in order as translation MVs to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, zero MVs are used to fill the affine AMVP list.

[0044] Sub-block-based affine motion compensation can save memory access bandwidth and reduce computational complexity compared to pixel-based motion compensation at the expense of a prediction accuracy penalty. To achieve finer granularity of motion compensation, prediction refinement with optical flow (PROF) is used to improve sub-block-based affine motion compensation prediction without increasing memory access bandwidth for motion compensation. In VVC, after sub-block-based affine motion compensation is performed, the luma prediction samples are refined by adding the difference derived by the optical flow equation. PROF is described as the following four operations:

[0045] Action 1) Sub-block based affine motion compensation is performed to generate the sub-block prediction I(i,j).

[0046] Step 2) Spatial gradient of subblock prediction g x (i, j) and g y (i,j) is calculated at each sample position using a 3-tap filter [-1,0,1]. The gradient calculation is exactly the same as the gradient calculation in BDOF.

[0047] g x (i,j)=(I(i+1,j)>>shift1)-(I(i-1,j)>>shift1) (5) g y (i,j)=(I(i,j+1)>>shift1)-(I(i,j-1)>>shift1) (6) shift1 is used to control the precision of the gradient. Sub-block (i.e., 4x4) predictions are extended by one sample on each side for gradient calculation. To avoid additional memory bandwidth and additional interpolation calculations, these extended samples on the extension boundary are copied from the nearest integer pixel location in the reference picture.

[0048] Action 3) Luma prediction refinement is computed according to the optical flow equation:

[0049] ΔI(i,j)=g x (i,j)*Δv x (i,j)+g y (i,j)*Δv y (i,j) (7) where Δv(i,j) is the difference between the sample MV calculated for sample position (i,j) denoted by v(i,j) and the sub-block MV of the sub-block to which sample (i,j) belongs, as shown in Figure 14. Δv(i,j) is quantized in units of 1 / 32 luma sample precision.

[0050] Since the affine model parameters and sample positions relative to the subblock center do not change for each subblock, Δv(i,j) may be calculated for the first subblock and reused for other subblocks in the same CU. dx(i,j) and dy(i,j) are calculated from the sample position (i,j) to the subblock center (x SB ,y SB ), Δv(x,y) may be derived by the following equation:

number

number

[0051] For the four-parameter affine model:

number

number

[0052] Action 4) Finally, the luma prediction refinement ΔI(i,j) is added to the sub-block prediction I(i,j). The final prediction I′ is generated as follows:

[0053] I'(i,j) = I(i,j) + ΔI(i,j) PROF does not apply in two cases for an affine coded CU: 1) all control points MV are the same, which indicates that the CU has only translational motion, and 2) the affine motion parameters are larger than the specified limit because sub-block-based affine MCs are decomposed into CU-based MCs to avoid large memory access bandwidth requirements.

[0054] VVC supports the subblock-based temporal motion vector prediction (SbTMVP) method. Similar to temporal motion vector prediction (TMVP) in HEVC, SbTMVP uses motion fields in co-located pictures to improve motion vector prediction and merge modes for CUs in the current picture. The same co-located pictures used by TMVP are used for SbTVMP. SbTMVP differs from TMVP in two main aspects:

[0055] TMVP predicts motion at the CU level, while SbTMVP predicts motion at the sub-CU level.

[0056] While TMVP fetches the temporal motion vector from the co-located block in the co-located picture (the co-located block is the bottom-right or center block with respect to the current CU), SbTMVP applies a motion shift before fetching the temporal motion information from the co-located picture, where the motion shift is obtained from a motion vector from one of the spatial neighboring blocks of the current CU. The SbTVMP process is shown in Figure 15A and Figure 15B. SbTMVP predicts the motion vectors of sub-CUs in a current CU in two operations. In the first operation, the spatial neighbor A1 in Figure 15A is examined. If A1 has a motion vector that uses a co-located picture as its reference picture, this motion vector is selected to be the motion shift to be applied. If no such motion is identified, the motion shift is set to (0,0).

[0057] In a second operation, the motion shift identified in operation 1 is applied (i.e., added to the coordinates of the current block) to obtain sub-CU level motion information (motion vector and reference index) from the co-located picture, as shown in FIG. 15B. The example in FIG. 15B assumes that the motion shift is set to the motion of block A1. Then, for each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample) in the co-located picture is used to derive the motion information for the sub-CU. After the motion information of the co-located sub-CU is identified, it is converted to the motion vector and reference index of the current sub-CU in a manner similar to the TMVP process of HEVC, where temporal motion scaling is applied to align the reference picture of the temporal motion vector to the reference picture of the current CU.

[0058] In HEVC, a bi-predictive signal is generated by averaging two prediction signals obtained from two different reference pictures and / or by using two different motion vectors. In VVC, the bi-predictive mode is extended beyond a simple average to allow a weighted average of the two prediction signals.

number

[0059] At the encoder, fast search algorithms are applied to find the weight index without significantly increasing the encoder complexity. These algorithms are summarized as follows. For further details, see the VTM software and document JVET-L0646. When combined with AMVR, unequal weights are only conditionally checked for 1-pixel and 4-pixel motion vector accuracy if the current picture is a low-latency picture.

[0060] When combined with affine, affine ME is performed for unequal weights only if the affine mode is selected as the current best mode.

[0061] Unequal weights are only conditionally checked when the two reference pictures in bi-prediction are the same.

[0062] Depending on the POC distance between the current picture and its reference pictures, the coding QP and the temporal level, unequal weights are not explored when certain conditions are met.

[0063] The BCW weight index is coded using one context-coded bin followed by a bypass-coded bin: the first context-coded bin indicates whether equal weights are used or not, and if unequal weights are used, a further bin is signaled using bypass coding to indicate which unequal weights are used.

[0064] Weighted prediction (WP) is a coding tool supported by H.264 / AVC and HEVC standards to efficiently code video content with fading. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weight and offset) to be signaled for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the weight and offset of the corresponding reference picture are applied. WP and BCW are designed for different types of video content. To avoid interactions between WP and BCW that would complicate the design of a VVC decoder, when a CU uses WP, the BCW weight index is not signaled and w is inferred to be 4 (i.e., equal weights are applied). For a merge CU, the weight index is inferred from neighboring blocks based on the merge candidate index. This may be applied to both normal merge mode and inherit affine merge mode. For the construct affine merge mode, the affine motion information is constructed based on the motion information of up to three blocks. The BCW index for a CU using the construction affine merge mode is simply set equal to the BCW index of the first control point MV.

[0065] In VVC, CIIP and BCW cannot be applied together for a CU. When a CU is coded in CIIP mode, the BCW index of the current CU is set to 2, e.g., equal weighting.

[0066] Bi-directional optical flow (BDOF) tools are included in VVC. Formerly called BIO, BDOF is included in JEM. Compared to the JEM version, BDOF in VVC is a simpler version that requires much less computation, especially in terms of the number of multiplications and the size of the multipliers.

[0067] BDOF is used to improve the bidirectional prediction signal of a CU at the 4×4 sub-block level. BDOF is applied to a CU if the CU satisfies all of the following conditions:

[0068] The CU is coded using a "true" bidirectional prediction mode, i.e., one of the two reference pictures is before the current picture in display order and the other is after the current picture in display order.

[0069] The distances from the two reference pictures to the current picture (ie, the POC difference) are the same.

[0070] Both reference pictures are short-term reference pictures.

[0071] The CU is not coded using affine mode or SbTMVP merge mode.

[0072] A CU has more than 64 luma samples.

[0073] Both the CU height and the CU width are equal to or greater than 8 luma samples.

[0074] The BCW weight index indicates equal weighting.

[0075] WP is currently not enabled for CU.

[0076] The CIIP mode is currently not used for the CU.

[0077] BDOF is applied only to the luma component. As the name suggests, the BDOF mode is based on the concept of optical flow, which assumes that the object motion is smooth. For each 4 × 4 sub-block, motion refinement (v x ,v y ) is calculated by minimizing the difference between the L0 predicted samples and the L1 predicted samples. Then, motion refinement is used to adjust the bidirectional predicted sample values ​​in the 4x4 sub-block. In the BDOF process, the following operations are applied:

[0078] First, the horizontal and vertical gradients of the two prediction signals

number

number

[0079] Then the gradient autocorrelations and cross-correlations S1, S2, S3, S5 and S6 are calculated as follows:

number

number

[0080] Then, motion refinement (v x ,v y ) is derived using the cross-correlation and autocorrelation terms using:

number

number

number

number

[0081] Based on the motion refinement and gradients, the following adjustments are calculated for each sample in the 4x4 sub-block:

number

number

[0082] To derive the gradient value, we select some prediction samples I in list k (k=0,1) outside the current CU. (k)(i,j) need to be generated. As shown in FIG. 16, BDOF in VVC uses one extended row / column around the boundary of the CU. To control the computational complexity of generating out-of-bounds predicted samples, the predicted samples in the extended region (white positions) are generated by directly taking the reference samples of the neighboring integer positions (using floor() operation on the coordinates) without interpolation, and a normal 8-tap motion compensation interpolation filter is used to generate the predicted samples in the CU (gray positions). These extended sample values ​​are only used in the gradient calculation. For the remaining operations in the BDOF process, if any sample values ​​and gradient values ​​outside the CU boundary are needed, they are padded (i.e., repeated) from their nearest neighbors.

[0083] When the width and / or height of a CU is larger than 16 luma samples, it is divided into sub-blocks with width and / or height equal to 16 luma samples, and the sub-block boundaries are treated as CU boundaries in the BDOF process. The maximum unit size of the BDOF process is limited to 16x16. For each sub-block, the BDOF process may be skipped. When the SAD between the initial L0 predicted sample and the initial L1 predicted sample is smaller than a threshold, the BDOF process is not applied to the sub-block. The threshold is set equal to (8*W*(H>>1), where W denotes the sub-block width and H denotes the sub-block height. To avoid further complexity of SAD calculation, the SAD between the initial L0 predicted sample and the initial L1 predicted sample calculated in the DVMR process is reused here.

[0084] If BCW is enabled for the current block, i.e., the BCW weight index indicates unequal weights, bidirectional optical flow is disabled. Similarly, if WP is enabled for the current block, i.e., luma_weight_lx_flag is 1 for either of the two reference pictures, BDOF is also disabled. When the CU is coded in symmetric MVD mode or CIIP mode, BDOF is also disabled.

[0085] To increase the accuracy of MV in merge mode, bilateral-matching (BM) based decoder-side motion vector refinement is applied in VVC. In bi-prediction operation, the improved MV is searched around the initial MV in reference picture list L0 and reference picture list L1. The BM method calculates the distortion between two candidate blocks in reference picture list L0 and list L1. As shown in Figure 17, the SAD between red blocks is calculated based on each MV candidate around the initial MV. The MV candidate with the lowest SAD becomes the improved MV and is used to generate the bi-prediction signal.

[0086] In VVC, the application of DMVR is restricted and applies only to CUs coded with the following modes and features:

[0087] CU-level merge mode with bi-predictive MV For a current picture, one reference picture is a past picture and another reference picture is a future picture.

[0088] The distances from the two reference pictures to the current picture (ie, the POC difference) are the same.

[0089] Both reference pictures are short-term reference pictures.

[0090] A CU has more than 64 luma samples.

[0091] Both the CU height and the CU width are equal to or greater than 8 luma samples.

[0092] The BCW weight index indicates equal weighting.

[0093] WP is currently not enabled for blocks.

[0094] CIIP mode is currently not used for blocks.

[0095] The improved MVs derived by the DMVR process are used to generate inter prediction samples and are also used in temporal motion vector prediction for future picture coding, while the original MVs are used in the deblocking process and are also used in spatial motion vector prediction for future CU coding.

[0096] Further features of the DMVR are mentioned in the following subsections.

[0097] In DVMR, the search points surround the initial MV, and the MV offset follows the MV difference mirroring rule. In other words, any point examined by DMVR, indicated by a candidate MV pair (MV0, MV1), follows the following two equations:

[0098] MV0'=MV0+MV_offset (3-2) MV1'=MV1-MV_offset (3-3) where MV_offset represents the refinement offset between the initial MV and the improved MV in one of the reference pictures. The refinement search range is two integer luma samples from the initial MV. The search includes an integer sample offset search stage and a fractional sample refinement stage.

[0099] A full search of 25 points is applied to the integer sample offset search. The SAD of the initial MV pair is calculated first. If the SAD of the initial MV pair is less than a threshold, the integer sample stage of DMVR ends. Otherwise, the SAD of the remaining 24 points are calculated and examined in raster scan order. The point with the smallest SAD is selected as the output of the integer sample offset search stage. To reduce the uncertainty penalty of DMVR refinement, it is disclosed to prioritize the original MV during the DMVR process. The SAD between the reference blocks referenced by the initial MV candidates is reduced by ¼ of the SAD value.

[0100] The integer sample search is followed by fractional sample refinement. To save computational complexity, the fractional sample refinement is derived by using a parametric error surface equation instead of further searching by SAD comparison. The fractional sample refinement is conditionally invoked based on the output of the integer sample search stage. When the integer sample search stage ends on the center with the smallest SAD in either the first iteration or the second iteration search, the fractional sample refinement is further applied.

[0101] In parametric error surface based sub-pixel offset estimation, the cost of the center position and the costs at the four neighboring positions from the center are used to fit a 2D parabolic error surface equation of the following form:

[0102] E(x,y)=A(xx min ) 2 +B(yy min ) 2 +C (3-4) Here, (x min ,y min ) corresponds to the decimal point with the minimum cost, and C corresponds to the minimum cost value. By solving the above equation by using the cost values ​​of the five search points, (x min ,y min ) is calculated as follows:

[0103] x min =(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0))) (3-5) y min =(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0))) (3-6) Since all cost values ​​are positive and the minimum value is E(0,0), x min and y minis automatically constrained to be between -8 and 8. This corresponds to a half pixel offset with 1 / 16 pixel MV precision in VVC. The calculated fraction (x min ,y min ) is added to the integer distance refinement MV to get the delta MV for sub-pixel accurate refinement.

[0104] In VVC, the resolution of the MV is 1 / 16 luma samples. The samples at fractional positions are interpolated using an 8-tap interpolation filter. In DMVR, the search points surround the initial fractional pixel MV with integer sample offsets, and therefore the samples at these fractional positions need to be interpolated for the DMVR search process. To reduce the computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect is that by using a bilinear filter, with a 2-sample search range, the DVMR does not access more reference samples compared to the normal motion compensation process. After the improved MV is obtained in the DMVR search process, a normal 8-tap interpolation filter is applied to generate the final prediction. To avoid having more reference samples accessed by the normal MC process, samples that are not needed for the original MV-based interpolation process but are needed for the improved MV-based interpolation process are padded from these available samples.

[0105] When the width and / or height of a CU is greater than 16 luma samples, the CU is further divided into sub-blocks with width and / or height equal to 16 luma samples. The maximum unit size of the DMVR search process is limited to 16x16.

[0106] In VVC, geometric partitioning mode is supported for inter prediction. The geometric partitioning mode is signaled using a CU level flag as one type of merge mode, other merge modes include normal merge mode, MMVD mode, CIIP mode and sub-block merge mode. A total of 64 partitions are supported for all possible CU sizes w×h=2, excluding 8×64 and 64×8. m ×2 n ,m,n∈{3…6} are supported by the geometric partition mode.

[0107] When this mode is used, the CU is divided into two parts by a geometrically arranged straight line (Figure 18). The position of the division line is mathematically derived from the angle and offset parameters of a particular partition. Each part of the geometric partition within a CU is inter-predicted using its own motion, and only one-way prediction is allowed for each partition, i.e., each part has one motion vector and one reference index. As with conventional bi-prediction, a one-way prediction motion constraint is applied to ensure that only two motion-compensated predictions are needed for each CU. The one-way prediction motion for each partition is derived using the process described in 2.1.6.1.

[0108] If a geometric partition mode is used for the current CU, a geometric partition index indicating the partition mode (angle and offset) of the geometric partition and two merge indices (one per partition) are further signaled. The number of maximum GPM candidate sizes is explicitly signaled in the SPS, which specifies the syntax binarization for the GPM merge index. After predicting each of the parts of the geometric partition, the sample values ​​along the geometric partition edges are adjusted using a blending process with adaptive weights as in 2.1.6.2. This is the prediction signal for the entire CU, and the transformation and quantization process is applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using the geometric partition mode is stored as in 2.1.6.3.

[0109] The unidirectional prediction candidate list is directly derived from the merge candidate list constructed according to the extended merge prediction process. Let n be the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector of the nth extended merge candidate at X equal to the parity of n is used as the nth unidirectional prediction motion vector for the geometric partition mode. These motion vectors are marked with "x" in FIG. 19. If the corresponding LX motion vector of the nth extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the unidirectional prediction motion vector for the geometric partition mode.

[0110] After predicting each part of the geometric partition using its own motion, blending is applied to the two predicted signals to derive samples around the geometric partition edges. The blending weights for each position of the CU are derived based on the distance between the individual position and the partition edges.

[0111] The distance of a position (x,y) to a partition edge is derived as follows:

number

[0112] The weights for each part of the geometric partition are derived as follows:

number

[0113] Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combined Mv of Mv1 and Mv2 are stored in the motion field of a CU coded in geometric partition mode.

[0114] The stored motion vector type for each individual position in the motion field is determined as follows.

[0115] sType=abs(motionIdx)<32?2:(motionIdx≦0?(1-partIdx):partIdx) (3-43) Here, the motion Idx is equal to d(4x+2,4y+2), which is recalculated from equation (3-36). partIdx depends on the angle index i.

[0116] If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise if sType is equal to 2, then the combined Mv from Mv0 and Mv2 is stored. The combined Mv is generated using the following process:

[0117] If Mv1 and Mv2 are from different reference picture lists (one from L0 and the other from L1), Mv1 and Mv2 are simply combined to form a bi-predictive motion vector.

[0118] Otherwise, if Mv1 and Mv2 are from the same list, only the one-way predictive motion Mv2 is stored.

[0119] Video coding standards HEVC and AVC / H.264 use a fixed motion vector resolution of 1 / 4 luma samples. However, it is well known that an optimal tradeoff between displacement vector rate and prediction error rate must be selected to achieve overall rate-distortion optimality. VVC allows for the selection of motion vector resolution at the coding block level, thus allowing a tradeoff between bitrate and fidelity for the signaling of motion parameters. This is made possible by the AMVR mode, which is signaled at the coding block level if at least one component of the MVD is not equal to 0. The motion vector predictor is rounded to a given resolution such that the resulting motion vector is guaranteed to be on a grid of the given resolution. For each given resolution, a corresponding AmvrShift value in Table 1 is also defined to specify the resolution of the motion vector difference due to a left shift operation with the AmvrShift bits. Given motion vector differences, denoted as MvdL0 and MvdL1 in AMVP mode and MvdCpL0 and MvdCpL1 in affine AMVP mode, are modified as follows when AMVR mode is enabled:

[0120] Out-of-bounds constraints for inter-Bi prediction Constraints on inter bi-predictive coding modes may apply to bi-prediction when certain reference samples used in motion compensation lie outside the picture boundaries.

[0121] In one example, when a current pixel with a bidirectional prediction motion vector has a motion vector in one of the two reference lists that points to a position that exceeds a distance threshold from a picture boundary, the motion vector of that reference list is considered out of bounds, and inter prediction is changed to unidirectional prediction. Only the motion vectors of the other reference list that are not out of bounds are used for unidirectional prediction. In some methods, when the MVs of both reference lists are out of bounds, bidirectional prediction is not restricted.

[0122] In another example, the constraint on bidirectional prediction is applied at the subblock level. As shown in Figure 21, for each NxN subblock in a coding block with inter Bi prediction MV, if the motion vector in one of the reference lists points outside the boundary of the reference picture by more than a threshold of M pixels, the subblock may be changed to unidirectional prediction mode using only the MVs on the reference list that do not point beyond the out-of-bounds threshold on the corresponding reference picture.

[0123] In some methods, bi-prediction related tools may be disabled or modified when bi-prediction is changed to uni-prediction due to out-of-bounds conditions. In one example, BDOF may be disabled when bi-prediction restrictions are applied and uni-prediction is used. Embodiments of the present disclosure adaptively apply constraints to bi-prediction modes to improve coding efficiency and / or quality.

[0124] FIG. 1 shows a simplified block diagram of a communication system (100) according to one embodiment of the present disclosure. The communication system (100) includes at least two terminals (110, 120) interconnected via a network (150). For one-way transmission of data, a first terminal (110) may encode video data at a local location for transmission to the other terminal (120) via the network (150). The second terminal (120) may receive the encoded video data of the other terminal from the network (150), decode the encoded data, and display the reconstructed video data. One-way data transmission may be common in media provisioning applications, etc.

[0125] 1 illustrates a second pair of terminals (130, 140) provided to support bidirectional transmission of encoded video that may occur, for example, during a video conference. For bidirectional transmission of data, each terminal (130, 140) may encode video data captured at a local location for transmission to the other terminal over the network (150). Each terminal (130, 140) may also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the reconstructed video data on a local display device.

[0126] In FIG. 1, the terminals (110-140) may be depicted as servers, personal computers, and smartphones, and / or any other type of terminal. For example, the terminals (110-140) may be laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. The network (150) represents any number of networks that convey encoded video data between the terminals (110-140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data in circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network (150) is not important to the operation of the present disclosure, unless otherwise described herein below.

[0127] 2 shows the arrangement of video encoders and decoders in a streaming environment as an example of an application of the disclosed subject matter, which is equally applicable to other video-enabled applications including, for example, videoconferencing, digital TV, storage of compressed video on digital media (including CDs, DVDs, memory sticks, etc.), etc.

[0128] As shown in FIG. 2, the streaming system (200) may include a capture subsystem (213) that may include a video source (201) and an encoder (203). For example, the video source (201) may be a digital camera and may be configured to create an uncompressed video sample stream (202). The uncompressed video sample stream (202) may provide a high amount of data when compared to an encoded video bitstream and may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204) may include a lower amount of data when compared to the sample stream and may be stored on a streaming server (205) for future use. One or more streaming clients (206) may access the streaming server (205) to obtain a video bitstream (209), which may be a copy 208 of the encoded video data (204).

[0129] In an embodiment, the streaming server (205) may also operate as a Media-Aware Network Element (MANE). For example, the streaming server (205) may be configured to prune the encoded video bitstream (204) in order to tailor potentially different bitstreams to one or more of the streaming clients (206). In an embodiment, a MANE may be provided in the streaming system (200) separate from the streaming server (205).

[0130] The streaming client (206) may include a video decoder (210) and a display (212). The video decoder (210) may, for example, decode a video bitstream (209) that is an input copy of the encoded video bitstream (204) and generate an output video sample stream (210) that may be rendered on a display (212) or another rendering device (not shown). In some streaming systems, the video bitstreams (204, 209) may be encoded according to a particular video encoding / compression standard. Examples of these standards include, but are not limited to, ITU-T Recommendation H.265. The developing video encoding standard is informally known as Versatile Video Coding (VVC). Embodiments of the present disclosure may be used in the context of VVC.

[0131] FIG. 3 illustrates an exemplary operational block diagram of a video decoder (210) attached to a display (212) according to one embodiment of the disclosure.

[0132] The video decoder (210) may include a channel (312), a receiver (310), a buffer memory (315), an entropy decoder / parser (320), a scaler / inverse transform unit (351), an intra prediction unit (352), a motion compensated prediction unit (353), an aggregator (355), a loop filter unit (356), a reference picture memory (357), and a current picture memory (). In at least some embodiments, the video decoder (210) may include an integrated circuit, a series of integrated circuits, and / or other electronic circuitry. The video decoder (210) may also be embodied partially or entirely in software running on one or more CPUs with associated memory.

[0133] In this and other embodiments, the receiver (310) may receive one or more coded video sequences to be decoded by the decoder (210), one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (312), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (310) may receive the coded video data along with other data (e.g., coded audio data and / or auxiliary data streams), which may be forwarded to respective usage entities (not shown). The receiver (310) may separate the coded video sequences from the other data. To prevent network jitter, a buffer memory (315) may be coupled between the receiver (310) and the entropy decoder / parser (320) (hereinafter the "parser"). If the receiver 310 is receiving data from a store-and-forward device with sufficient bandwidth and controllability, or from an isosynchronous network, then the buffer 315 may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer 315 may be needed and may be relatively large or adaptively sized.

[0134] The video decoder (210) may include a parser (320) for reconstructing symbols (321) from the entropy-coded video sequence. These categories of symbols may include, for example, information used to manage the operation of the decoder (210) and potentially information for controlling a rendering device, such as a display (212) that may be coupled to the decoder as shown in FIG. 2. The rendering device control information may be in the form of, for example, a Supplementary Enhancement Information (SEI) (SEI message) or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be according to a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. The subgroups may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transformation unit (TU), a prediction unit (PU), etc. The parser (320) may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0135] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer (315) to generate symbols (321).

[0136] The reconstruction of the symbols (321) may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser (320). The flow of such subgroup control information between the parser (320) and the following units is not shown for clarity.

[0137] In addition to the operational blocks described above, the decoder (210) may be conceptually subdivided into a number of operational units, as described below. In a practical implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into the following operational units is appropriate:

[0138] One unit is a scalar / inverse transform unit (462). The scalar / inverse transform unit (351) may receive quantized transform coefficients as symbols (321) from the parser (320), along with control information (including which transform to use, block size, quantization factor, quantization scaling matrix, etc.). The scalar / inverse transform unit (351) may output blocks containing sample values ​​that may be input to an aggregator (355).

[0139] In some cases, the output samples of the scaler / inverse transform (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed part of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates blocks of the same size and shape of the block being reconstructed using surrounding already reconstructed information retrieved from the (partially reconstructed) current picture from a current picture memory (358). In some cases, the aggregator (355) adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0140] In other cases, the output samples of the scalar / inverse transform unit (351) may relate to an inter-coded and potentially motion-compensated block. In such cases, the motion compensation prediction unit (353) may access the reference picture memory (357) to retrieve samples used for prediction. After motion compensating the retrieved samples according to the symbols (321) associated with the block, these samples may be added by the aggregator (355) to the output of the scalar / inverse transform unit (351) (in this case referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (357) from which the motion compensation prediction unit (353) retrieves the prediction samples may be controlled by a motion vector. The motion vector may be made available to the motion compensation prediction unit (353) in the form of a symbol (321), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​retrieved from the reference picture memory (357) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0141] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356). Video compression techniques may include in-loop filter techniques, controlled by parameters contained in the coded video bitstream and made available to the loop filter unit (356) as symbols (321) from the parser (320), that may be responsive to meta-information obtained during decoding of previous portions of the coded pictures or coded video sequences (in decoding order), as well as to previously reconstructed loop filtered sample values.

[0142] The output of the loop filter unit (356) may be a sample stream that may be output to a rendering device, such as a display (212), and may also be stored in a reference picture memory (357) for use in future inter-picture prediction.

[0143] Once a particular coded picture is fully reconstructed, it may be used as a reference picture for future prediction. Once a coded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (320)), the current reference picture may become part of the reference picture memory (357), and a new current picture memory may be reallocated before beginning reconstruction of a subsequent coded picture.

[0144] The video decoder (210) may perform decoding operations according to a given video compression technique, which may be documented in a standard such as ITU-T Rec. H.265. The coded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that the coded video sequence conforms to the syntax of the video compression technique or standard as specified in the video compression technique or standard, and in particular as specified in its profile document. Also, for compliance with some video compression techniques or standards, the complexity of the coded video sequence may be within a range defined by a level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level can be further limited through a Hypothetical Reference Decoder (HRD) specification and metadata about HRD buffer management conveyed in the coded video sequence.

[0145] In one embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0146] FIG. 4 illustrates an exemplary operational block diagram of a video encoder (203) associated with a video source (201) according to one embodiment of the present disclosure.

[0147] The video encoder (203) may include, for example, an encoder, such as a source coder (430), a coding engine (432), a (local) decoder (433), a reference picture memory (434), a predictor (435), a transmitter (440), an entropy coder (445), a controller (450), and a channel (460).

[0148] The encoder (203) may receive video samples from a video source (201) (not part of the encoder), which may capture video images to be encoded by the encoder (203).

[0149] The video source (201) may provide a source video sequence to be encoded by the encoder (203) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 Y CrCB, RGB, etc.), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media presentation system, the video source (201) may be a storage device that stores pre-prepared videos. In a video conferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that convey motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following description will focus on samples.

[0150] According to one embodiment, the encoder (203) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real-time or under any other time constraint required by an application. Achieving an appropriate encoding rate is one operation of the controller (450). The controller (450) may also control and be operatively coupled to other operational units, as described below. Coupling is not shown for clarity. Parameters set by the controller (450) may include rate control related parameters (picture skip, quantization, lambda value for rate distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. One skilled in the art can readily recognize other operations of the controller (450) that may be associated with a video encoder (203) optimized for a particular system design. Some video encoders operate in what those skilled in the art will easily recognize as a "coding loop." As a very simplified description, the coding loop can consist of the coding part of the source coder (430) (responsible for generating symbols based on the input picture to be coded and the reference pictures) and a (local) decoder (433) embedded in the encoder (203). The decoder (433) reconstructs the symbols to generate sample data similar to what a (remote) decoder would generate when the compression between the symbols and the coded video bitstream is lossless for a particular video compression technique. That reconstructed sample stream may be input to a reference picture memory (434). Since the decoding of the symbol stream results in bit-wise accurate results independent of the location of the decoder (local or remote), the contents of the reference picture memory are also bit-wise accurate between the local and remote encoders. In other words, the predictive part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchronization (including the resulting drift when synchronization cannot be maintained, for example, due to channel errors) is known to those skilled in the art.

[0151] The operation of the "local" decoder (433) may be the same as the "remote" decoder (210), which has already been described in detail above in connection with Figure 3. However, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (445) and parser (320) may be lossless, the entropy decoding portion of the decoder (210), including the channel (312), receiver (310), buffer (315) and parser (320), may not be fully implemented in the local decoder (433).

[0152] A consideration that can be made at this point is that any decoder technique, other than analysis / entropy decoding, present in the decoder must be present in the corresponding encoder in substantially the same operational form. For this reason, the subject matter of the disclosure focuses on the decoder operation. A description of the encoder technique may be omitted, as it may be the inverse of the decoder technique described generically. Only in certain areas is a more detailed description necessary, which is provided below.

[0153] As part of its operation, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from the video sequence designated as “reference frames.” In this manner, the coding engine (432) codes differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as predictive references for the input frame.

[0154] The local video decoder (433) may decode the encoded video data of frames that may be designated as reference frames based on the symbols generated by the source coder (430). The operation of the encoding engine (432) may advantageously be a lossy process. If the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence may be a replica of the source video sequence, typically with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder on the reference frames and store the reconstructed reference frames in the reference picture memory (434). In this way, the encoder (203) may locally store copies of reconstructed reference frames that have common content as the reconstructed reference frames (without transmission errors) obtained by the far-end video decoder.

[0155] The predictor (435) may perform a predictive search for the coding engine (432). That is, for a new frame to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata (reference picture motion vectors, block shapes, etc.), which may serve as suitable prediction references for the new picture. The predictor (435) may operate on a sample block-by-pixel block basis to find suitable prediction references. In some cases, the input picture determined by the search results obtained by the predictor (435) may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0156] The controller (450) may manage the encoding operations of the video coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0157] The output of all the above operational units may be subjected to entropy coding in an entropy coder (445), which converts the symbols produced by the various operational units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as, for example, Huffman coding, variable length coding, arithmetic coding, etc.

[0158] The transmitter (440) may buffer the encoded video sequence produced by the entropy coder (445) and prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the encoded video data from the video coder (430) with other data to be transmitted, such as encoded audio data and / or an auxiliary data stream (not shown).

[0159] The controller (450) may manage the operation of the encoder (203). During encoding, the controller (450) may assign each coded picture a particular coding picture type, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as intra pictures (I pictures), predicted pictures (P pictures), or bidirectionally predicted pictures (B pictures).

[0160] The encoder (203) and decoder (210) may adaptively apply inter bi-prediction constraints for out-of-bounds MVs. The disclosed methods may be applied regardless of whether the bi-prediction constraints for out-of-bounds MVs are pixel (e.g., block) based or sub-block based. In some embodiments, the bi-prediction constraints are applied using predefined out-of-bounds conditions.

[0161] In some embodiments, the bidirectional prediction constraint for out-of-bounds MVs may be adaptively applied by the encoder (203) and the decoder (210) according to the time directions of the reference pictures (801) and (802) in Figures 17 and 22. If both reference pictures (801) and (802) for bidirectional prediction are from the same time direction, the bidirectional prediction constraint may not be applied regardless of the reference pixel position, and normal bidirectional prediction may be applied. Otherwise, the bidirectional prediction may be changed to a unidirectional prediction mode according to the out-of-bounds condition.

[0162] In some embodiments, when the MVs of reference list L0 and reference list L1 point to the same reference picture, the bidirectional prediction constraint for the out-of-bounds MVs is not applied. Otherwise, the bidirectional prediction may be changed to a unidirectional prediction mode according to the out-of-bounds condition.

[0163] In some embodiments, when the MVs of reference list L0 and reference list L1 point to the same reference picture and the difference between the MVs of L0 and L1 is within a threshold, the bi-prediction constraint for the out-of-bounds MVs may not be applied. Otherwise, the bi-prediction may be changed to uni-prediction according to the out-of-bounds condition.

[0164] In some embodiments, the threshold may be set as N luma samples, and the bi-prediction constraint may not be applied when the difference between the horizontal component of the MV of L0 (denoted as MV0) and the horizontal component of the MV of L1 (denoted as MV1) is less than or equal to the threshold, and / or when the difference between the vertical component of MV0 and the vertical component of MV1 is less than or equal to the threshold. This condition may be expressed by the following equation:

number

[0165] In an example, the threshold N is 1 / 4. In another example, the threshold N is 1 / 2. In another example, the threshold N is 1.

[0166]

[0023] Embodiments of the present disclosure are further directed to the application of adaptive bi-prediction constraints. In some embodiments, the disclosed adaptive bi-prediction constraints may be applied by default without signaling.

[0167] In some embodiments, the disclosed adaptive bidirectional prediction constraints may be applied according to high-level syntax, which may be signaled at a high level in the bitstream, such as at the sequence level, picture level, slice level, tile level, or tile group level.

[0168] In some embodiments, the selection of inter bi-prediction constraints for out-of-bounds MVs may be performed at the pixel (eg, block) level and at the sub-block level.

[0169] In some embodiments, the selection of pixel-level and sub-block-level bi-prediction constraints may be signaled at a high level in the bitstream, such as at the sequence level, picture level, slice level, tile level, or tile group level. In some embodiments, the selection of pixel-level and sub-block-level bi-prediction constraints may be made adaptively at the block level. In some examples, the adaptive selection may be based on a threshold distance between the reference sample position and a picture boundary.

[0170] In some embodiments, the adaptive selection may be based on a threshold difference between the MVs of reference list L0 and reference list L1. In some embodiments, the adaptive selection may be signaled at the block level. In some embodiments, block level signaling for adaptive selection may be allowed only when a block is at a picture (or slice or tile or sub-picture) boundary.

[0171] In some embodiments, bidirectional prediction constraints for out-of-boundary MVs may be applied to boundaries other than picture boundaries. In some embodiments, bidirectional prediction constraints may be applied to MVs outside sub-picture boundaries. In some embodiments, bidirectional prediction constraints may be applied to MVs outside virtual boundaries. In some embodiments, bidirectional prediction constraints may be applied to MVs outside slice boundaries. In some embodiments, bidirectional prediction constraints may be applied to MVs outside tile or tile group boundaries.

[0172] FIG. 23 shows a flow chart disclosing one embodiment of a process for determining and applying inter bi-prediction constraints. The process may start at operation S901, where a coded video bitstream is received. The bitstream may include a current picture and a first and a second reference picture associated with the current picture. The current picture may include at least one block coded according to an inter bi-prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture.

[0173] The process proceeds to operation S902, where it is determined whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied. The process proceeds to operation S9093, where it is determined whether a disabling condition for disabling the out-of-bounds condition is satisfied. The disabling condition may correspond to disabling a constraint on the bi-prediction mode when one of the first and second motion vectors points to a position on a corresponding reference picture that is outside the picture boundary.

[0174] The process proceeds to operation S904, where if the out-of-bounds condition and the invalidation condition are met, the at least one block may be changed from the inter bi-prediction mode to another coding mode (e.g., a uni-prediction mode) and the at least one block may be decoded according to the other coding mode. The process proceeds to operation S905, where if the out-of-bounds condition is not met or the invalidation condition is met, the at least one block is decoded according to the inter-prediction bi-prediction mode.

[0175] The above disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

[0176] It is understood that the particular order or hierarchy of blocks in the processes / flowcharts disclosed herein is an illustration of an example approach. Based on design preferences, it is understood that the particular order or hierarchy of blocks in the processes / flowcharts may be rearranged. Further, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in an example order, and are not intended to be limited to the particular order or hierarchy presented.

[0177] Some embodiments may relate to systems, methods and / or computer-readable media in any possible level of technical detail integration. Furthermore, one or more of the above components described above may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or media) having computer-readable program instructions for causing a processor to perform operations.

[0178] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves in which instructions are recorded, and any suitable combination of the above. Computer-readable storage media, as used herein, should not be construed as being a transitory signal per se, such as electric waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.

[0179] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium into each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0180] The computer readable program code / instructions for carrying out the operations may be either source code or object code written in any combination of one or more programming languages ​​including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or object oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or wide area network (WAN), or a connection to the external computer may be made (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to perform an aspect or operation.

[0181] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to generate a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer readable storage medium on which the instructions are stored includes an article of manufacture including instructions that implement aspects of the operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0182] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus or other device to cause a sequence of operations to be executed on the computer, other programmable apparatus or other device to generate a computer-implemented process, whereby the instructions executing on the computer, other programmable apparatus or other device implement the operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0183] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of an instruction, including one or more executable instructions for performing specified logical operations. The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks than those shown in the drawings. In some alternative implementations, the operations described in the blocks may occur in a different order than that described in the drawings. For example, two blocks shown in succession may in fact be executed simultaneously or substantially simultaneously, or the blocks may be executed in reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that executes the specified operations or executes a combination of dedicated hardware and computer instructions.

[0184] It is apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to any specific software code, and it is understood that software and hardware may be designed to implement the systems and / or methods based on the description herein.

[0185] The above disclosure also encompasses the embodiments listed below.

[0186] (1) A method for inter prediction performed by at least one processor, comprising: receiving a coded video bitstream including a current picture and first and second reference pictures associated with the current picture, the current picture including at least one block coded according to an inter bidirectional prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture; determining whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied; responsive to determining that the out-of-bounds condition is met, determining whether an override condition is met for overriding the out-of-bounds condition; in response to determining that the out-of-bounds condition is satisfied and that the invalidation condition is not satisfied, (i) changing the at least one block from the inter bi-prediction mode to another coding mode; (ii) decoding the at least one block according to the other coding mode; and in response to determining that the out-of-bounds condition is not met or that the invalidation condition is met, decoding the at least one block according to the inter bi-prediction mode; The method includes:

[0187] (2) The method of feature (1), wherein the out-of-bounds condition is satisfied in response to a determination that (i) the first motion vector points to a position in the first reference picture that is outside a first picture boundary of the first reference picture and has a distance from the first picture boundary that is greater than a distance threshold, or (ii) the second motion vector points to a second position in the second reference picture that is outside a second picture boundary of the second reference picture and has a distance from the second picture boundary that is greater than the distance threshold.

[0188] (3) The method of feature (1) or (2), wherein the out-of-bounds condition is not satisfied in response to a determination that (i) the first motion vector points to a position in the first reference picture that is outside a first picture boundary of the first reference picture and has a distance from the first picture boundary that is greater than a distance threshold, and (ii) the second motion vector points to a second position in the second reference picture that is outside a second picture boundary of the second reference picture and has a distance from the second picture boundary that is greater than the distance threshold.

[0189] (4) The method according to any one of features (1) to (3), wherein the other coding mode is a unidirectional prediction mode.

[0190] (5) The method according to any one of features (1) to (4), wherein the invalidation condition is satisfied in response to a determination that the first reference picture and the second reference picture are from the same time direction.

[0191] (6) The method according to any one of features (1) to (5), wherein the invalidation condition is satisfied in response to a determination that the first reference picture and the second reference picture are the same reference picture.

[0192] (7) The method according to any one of features (1) to (6), wherein the invalidation condition is satisfied in response to a determination that the first reference picture and the second reference picture are the same reference picture and a difference between the first motion vector and the second motion vector is within a predetermined threshold.

[0193] (8) The method according to feature (7), wherein the predetermined threshold is N luma samples, and the invalidation condition is satisfied in response to a determination that (i) a difference between a horizontal component of the first motion vector and a horizontal component of the second motion vector is less than or equal to the predetermined threshold, or (ii) a difference between a vertical component of the first motion vector and a vertical component of the second motion vector is less than or equal to the predetermined threshold.

[0194] (9) The method according to any one of features (1) to (8), wherein the invalidation condition is met in response to a determination that the first reference picture and the second reference picture are the same reference picture and a difference between a first reference position pointed to by the first motion vector and a second reference position pointed to by the second motion vector is within a predetermined threshold, the predetermined threshold being N luma samples.

[0195] (10) The method of any one of features (1) to (9), wherein the coded video bitstream includes an indicator indicating whether the disabling condition is allowed for the at least one block.

[0196] (11) An apparatus for inter prediction, comprising: at least one memory configured to store computer program code; at least one processor configured to access the computer program code and to operate as instructed by the computer program code; Including, The computer program code comprises: receiving code configured to cause the at least one processor to receive a coded video bitstream including a current picture and first and second reference pictures associated with the current picture, the current picture including at least one block encoded according to an inter bidirectional prediction mode, the at least one block including a first motion vector that points to the first reference picture and a second motion vector that points to the second reference picture; a first decision code configured to cause the at least one processor to determine whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied; and second decision code configured to cause the at least one processor to determine, in response to determining that the out-of-bounds condition is met, whether an override condition is met for overriding the out-of-bounds condition; and Modification code and decryption code Including, in response to determining that the out-of-bounds condition is satisfied and that the invalidation condition is not satisfied, (i) the change code causes the at least one processor to change the at least one block from the inter bi-prediction mode to another coding mode; (ii) the decoding code causes the at least one processor to decode the at least one block according to the other coding mode; In response to determining that the out-of-bounds condition is not met or the invalidation condition is met, the decoding code causes at least one processor to decode the at least one block in accordance with the inter bi-prediction mode.

[0197] (12) The apparatus of feature (11), wherein the out-of-bounds condition is satisfied in response to a determination that (i) the first motion vector points to a position in the first reference picture that is outside a first picture boundary of the first reference picture and has a distance from the first picture boundary that is greater than a distance threshold, or (ii) the second motion vector points to a second position in the second reference picture that is outside a second picture boundary of the second reference picture and has a distance from the second picture boundary that is greater than the distance threshold.

[0198] (13) The apparatus of feature (11) or (12), wherein the out-of-bounds condition is not satisfied in response to a determination that (i) the first motion vector points to a position in the first reference picture that is outside a first picture boundary of the first reference picture and has a distance from the first picture boundary that is greater than a distance threshold, and (ii) the second motion vector points to a second position in the second reference picture that is outside a second picture boundary of the second reference picture and has a distance from the second picture boundary that is greater than the distance threshold.

[0199] (14) The device according to any one of features (11) to (13), wherein the other coding mode is a unidirectional prediction mode.

[0200] (15) The device according to any one of features (11) to (14), wherein the invalidation condition is satisfied in response to a determination that the first reference picture and the second reference picture are from the same time direction.

[0201] (16) The apparatus according to any one of features (11) to (16), wherein the invalidation condition is satisfied in response to a determination that the first reference picture and the second reference picture are the same reference picture.

[0202] (17) The device described in any one of features (11) to (16), wherein the invalidation condition is satisfied in response to a determination that the first reference picture and the second reference picture are the same reference picture and a difference between the first motion vector and the second motion vector is within a predetermined threshold.

[0203] (18) The apparatus of feature (17), wherein the predetermined threshold is N luma samples, and the invalidation condition is satisfied in response to a determination that (i) a difference between a horizontal component of the first motion vector and a horizontal component of the second motion vector is less than or equal to the predetermined threshold, or (ii) a difference between a vertical component of the first motion vector and a vertical component of the second motion vector is less than or equal to the predetermined threshold.

[0204] (19) The device described in any one of features (11) to (18), wherein the invalidation condition is met in response to a determination that the first reference picture and the second reference picture are the same reference picture and a difference between a first reference position pointed to by the first motion vector and a second reference position pointed to by the second motion vector is within a predetermined threshold, the predetermined threshold being N luma samples.

[0205] (21) A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for inter prediction, comprising: The method comprises: receiving a coded video bitstream including a current picture and first and second reference pictures associated with the current picture, the current picture including at least one block coded according to an inter bidirectional prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture; determining whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied; responsive to determining that the out-of-bounds condition is met, determining whether an override condition is met for overriding the out-of-bounds condition; in response to determining that the out-of-bounds condition is satisfied and that the invalidation condition is not satisfied, (i) changing the at least one block from the inter bi-prediction mode to another coding mode; (ii) decoding the at least one block according to the other coding mode; and in response to determining that the out-of-bounds condition is not met or that the invalidation condition is met, decoding the at least one block according to the inter bi-prediction mode; A non-transitory computer readable medium comprising:

Claims

1. 1. A method for inter prediction performed by at least one processor, comprising: receiving a coded video bitstream including a current picture and first and second reference pictures associated with the current picture, the current picture including at least one block coded according to an inter bidirectional prediction mode, the at least one block including a first motion vector pointing to the first reference picture and a second motion vector pointing to the second reference picture; determining whether an out-of-bounds condition associated with the first motion vector and the second motion vector is satisfied; responsive to determining that the out-of-bounds condition is met, determining whether an override condition is met for overriding the out-of-bounds condition; in response to determining that the out-of-bounds condition is satisfied and that the invalidation condition is not satisfied, (i) changing the at least one block from the inter bi-prediction mode to another coding mode; (ii) decoding the at least one block according to the other coding mode; and in response to determining that the out-of-bounds condition is not met or that the invalidation condition is met, decoding the at least one block according to the inter bi-prediction mode; The method includes:

2. 2. The method of claim 1, wherein the out-of-bounds condition is satisfied in response to a determination that (i) the first motion vector points to a position in the first reference picture that is outside a first picture boundary of the first reference picture and has a distance from the first picture boundary that is greater than a distance threshold, or (ii) the second motion vector points to a second position in the second reference picture that is outside a second picture boundary of the second reference picture and has a distance from the second picture boundary that is greater than the distance threshold.

3. 2. The method of claim 1, wherein the out-of-bounds condition is not satisfied in response to a determination that (i) the first motion vector points to a position in the first reference picture that is outside a first picture boundary of the first reference picture and has a distance from the first picture boundary that is greater than a distance threshold, and (ii) the second motion vector points to a second position in the second reference picture that is outside a second picture boundary of the second reference picture and has a distance from the second picture boundary that is greater than the distance threshold.

4. The method of claim 1 , wherein the other coding mode is a unidirectional prediction mode.

5. The method of claim 1 , wherein the invalidation condition is met in response to determining that the first reference picture and the second reference picture are from the same temporal direction.

6. The method of claim 1 , wherein the invalidation condition is satisfied in response to determining that the first reference picture and the second reference picture are the same reference picture.

7. 2. The method of claim 1, wherein the invalidation condition is met in response to a determination that the first reference picture and the second reference picture are the same reference picture and a difference between the first motion vector and the second motion vector is within a predetermined threshold.

8. 8. The method of claim 7, wherein the predetermined threshold is N luma samples, and the invalidation condition is satisfied in response to a determination that (i) a difference between a horizontal component of the first motion vector and a horizontal component of the second motion vector is less than or equal to the predetermined threshold, or (ii) a difference between a vertical component of the first motion vector and a vertical component of the second motion vector is less than or equal to the predetermined threshold.

9. 2. The method of claim 1, wherein the invalidation condition is met in response to a determination that the first reference picture and the second reference picture are the same reference picture and a difference between a first reference position pointed to by the first motion vector and a second reference position pointed to by the second motion vector is within a predetermined threshold, the predetermined threshold being N luma samples.

10. The method of claim 1 , wherein the coded video bitstream includes an indicator that indicates whether the disabling condition is allowed for the at least one block.

11. An apparatus for inter prediction, comprising: at least one memory configured to store computer program code; At least one processor configured to access said computer program code and to carry out the method according to any one of claims 1 to 10; An apparatus comprising:

12. A computer program which, when executed by a processor, causes the processor to carry out a method according to any one of claims 1 to 10.

13. 1. A method for inter prediction performed by at least one processor, comprising: determining, for at least one block in a current picture to be coded according to an inter bidirectional prediction mode, whether an out-of-bounds condition associated with a first motion vector pointing to a first reference picture and a second motion vector pointing to a second reference picture is satisfied; responsive to determining that the out-of-bounds condition is met, determining whether an override condition is met for overriding the out-of-bounds condition; in response to determining that the out-of-bounds condition is satisfied and that the invalidation condition is not satisfied, (i) changing the at least one block from the inter bi-prediction mode to another coding mode; (ii) encoding the at least one block according to the other coding mode; and in response to determining that the out-of-bounds condition is not met or that the disable condition is met, encoding the at least one block according to the inter-prediction bi-prediction mode; The method includes:

Images