Derivation of converted unidirectional prediction candidates

Abstract

Techniques for implementing video processing techniques are described. In one example implementation, a method of video processing includes: for a conversion between a current block of a video and a bitstream representation of the video, determining a unidirectional motion vector from a bidirectional motion vector when a condition of a block size is satisfied. The unidirectional motion vector is then used as a Merge candidate for the conversion. The method also includes performing the conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a Chinese national phase application of International Patent Application No. PCT / CN2020 / 080824 filed on March 24, 2020, and this application timely claims priority and benefits of International Patent Application No. PCT / CN2019 / 079397 filed on March 24, 2019. The entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to image and video encoding and decoding. Background Art

[0004] Digital video accounts for the largest amount of bandwidth usage on the Internet and other digital communications networks. The bandwidth required for digital video usage is expected to continue to grow as the number of connected user devices capable of receiving and displaying video increases. Summary of the invention

[0005] This document discloses various video processing techniques that may be used by video encoders and decoders during encoding and decoding operations.

[0006] In one example aspect, a video processing method is disclosed. The method includes determining, for a conversion between a current block of a video and a bitstream representation of the video using an affine coding tool, that a first motion vector of a subblock of the current block and a second motion vector that is a representative motion vector of the current block comply with a size constraint. The method also includes performing the conversion based on the determination.

[0007] In another example aspect, a video processing method is disclosed. The method includes determining an affine model including six parameters for a conversion between a current block of a video and a bitstream representation of the video. The affine model inherits affine encoding information of neighboring blocks of the current block. The method also includes performing the conversion based on the affine model.

[0008] In another example aspect, a video processing method is disclosed. The method includes determining whether a bidirectional predictive coding technique is applicable to a block for conversion between a block of video and a bitstream representation of the video based on a block size having a width W and a height H, W and H being positive integers. The method also includes performing the conversion based on the determination.

[0009] In another example aspect, a video processing method is disclosed. The method includes determining whether a coding tree partitioning process is applicable to a block for conversion between a block of a video and a bitstream representation of the video based on a size of a sub-block, the sub-block being a child coding unit of the block according to the coding tree partitioning process. The sub-block has a width W and a height H, W and H being positive integers. The method also includes performing the conversion based on the determination.

[0010] In another example aspect, a video processing method is disclosed. The method includes determining, for a conversion between a current block of a video and a bitstream representation of the video, whether to derive an index for a bidirectional prediction with a coding unit level weight (BCW) coding mode based on a rule regarding the position of the current block. In the BCW coding mode, a weight set including a plurality of weights is used to generate a bidirectional prediction value for the current block. The method also includes performing the conversion based on the determination.

[0011] In another example aspect, a video processing method is disclosed. The method includes determining, for a conversion between a current block of a video encoded using a combined inter-frame and intra-frame prediction (CIIP) coding technique and a bitstream representation of the video, an intra-frame prediction mode of the current block that is independent of intra-frame prediction modes of adjacent blocks. The CIIP coding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

[0012] In another example aspect, a video processing method is disclosed. The method includes determining an intra-frame prediction mode of the current block according to a first intra-frame prediction mode of a first adjacent block and a second intra-frame prediction mode of a second adjacent block for a conversion between a current block of a video encoded using a combined inter-frame and intra-frame prediction (CIIP) coding technique and a bitstream representation of the video. The first adjacent block is encoded using an intra-frame prediction coding technique, and the second adjacent block is encoded using a CIIP coding technique. The first intra-frame prediction mode is assigned a priority different from that of the second intra-frame prediction mode. The CIIP coding technique uses an intermediate inter-frame prediction value and an intermediate intra-frame prediction value to derive a final prediction value of the current block. The method also includes performing the conversion based on the determination.

[0013] In another example aspect, a video processing method is disclosed. The method includes determining, for a conversion between a current block of a video and a bitstream representation of the video, whether a combined inter-frame and intra-frame prediction (CIIP) process is applicable to a color component of the current block based on the size of the current block. The CIIP encoding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

[0014] In another example aspect, a video processing method is disclosed. The method includes determining, for a conversion between a current block of a video and a bitstream representation of the video, whether to apply a combined inter-frame and intra-frame prediction (CIIP) coding technique to the current block based on characteristics of the current block. The CIIP coding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The method also includes performing the conversion based on the determination.

[0015] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current block of a video and a bitstream representation of the video, determining whether to disable coding tools for the current block based on whether the current block is encoded with a combined inter-frame and intra-frame prediction (CIIP) coding technique. The coding tools include at least one of bidirectional optical flow (BDOF), overlapped block motion compensation (OBMC), or a decoder-side motion vector refinement process (DMVR). The method also includes performing the conversion based on the determination.

[0016] In another example aspect, a video processing method is disclosed. The method includes determining, for conversion between a block of a video and a bitstream representation of the video, a first precision P1 of a motion vector for spatial motion prediction and a second precision P2 of a motion vector for temporal motion prediction. P1 and / or P2 are fractional, and neither P1 nor P2 is signaled in the bitstream representation. The method also includes performing the conversion based on the determination.

[0017] In another example aspect, a video processing method is disclosed. The method includes determining a motion vector (MVx, MVy) with precision (Px, Py) for conversion between a block of a video and a bitstream representation of the video. Px is associated with MVx, and Py is associated with MVy. MVx and MVy are stored as integers each having N bits, and MinX≤MVx≤MaxX and MinY≤MVy≤MaxY, MinX, MaxX, MinY, and MaxY are real numbers. The method also includes performing the conversion based on the determination.

[0018] In another example aspect, a video processing method is disclosed. The method includes determining whether a shared Merge list is applicable to a current block of a video and a bitstream representation of the video for conversion between the current block and the bitstream representation of the video according to the coding mode of the current block. The method also includes performing the conversion based on the determination.

[0019] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current block of a video of size W×H and a bitstream representation of the video, determining a second block of size (W+N-1)x(H+N-1) for motion compensation during the conversion. The second block is determined based on a reference block of size (W+N-1-PW)x(H+N-1-PH). N represents a filter size, and W, H, N, PW, and PH are non-negative integers. PW and PH are not both equal to 0. The method also includes performing the conversion based on the determination.

[0020] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current block of a video of size W x H and a bitstream representation of the video, determining a second block of size (W+N-1)x(H+N-1) for motion compensation during the conversion. W, H are non-negative integers, and N is a non-negative integer and is based on a filter size. During the conversion, a refined motion vector is determined based on a multi-point search according to a motion vector refinement operation on an original motion vector, and pixels along a reference block boundary are determined by repeating one or more non-boundary pixels. The method also includes performing the conversion based on the determination.

[0021] In another example aspect, a video processing method is disclosed. The method includes, for a block of a video encoded using a combined inter-frame and intra-frame prediction (CIIP) coding technique and a conversion between a bitstream representation of the video, determining a prediction value at a position in the block based on a weighted sum of an inter-frame prediction value and an intra-frame prediction value at the position. The weighted sum is based on adding an offset to an initial sum obtained based on the inter-frame prediction value and the intra-frame prediction value, and before performing a right shift operation to determine the weighted sum, adding the offset. The method also includes performing the conversion based on the determination.

[0022] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current block of a video and a bitstream representation of the video, determining a manner of representing encoding information of the current block in the bitstream representation based in part on whether a condition associated with a size of the current block is satisfied. The method also includes performing the conversion based on the determination.

[0023] In another example aspect, a video processing method is disclosed. The method includes determining a modified motion vector set for a conversion between a current block of a video and a bitstream representation of the video, and performing the conversion based on the modified motion vector set. Since the current block satisfies a condition, the modified motion vector set is a modified version of the motion vector set associated with the current block.

[0024] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current block of a video and a bitstream representation of the video, determining a unidirectional motion vector from a bidirectional motion vector if a condition on block size is satisfied, wherein a decoder-side motion vector refinement (DMVR) process is subsequently disabled based on the determination of using the unidirectional motion vector during the conversion; and performing the conversion based on the determination.

[0025] In another example aspect, a video processing method is disclosed. The method includes determining a unidirectional motion vector from a bidirectional motion vector for a conversion between a current block of a video and a bitstream representation of the video, provided that a block size is satisfied. The unidirectional motion vector is then used as a Merge candidate for the conversion. The method also includes performing the conversion based on the determination.

[0026] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current block of a video and a bitstream representation of the video, determining, based on a size of the current block, that motion candidates of the current block are restricted to unidirectional prediction candidates. The method also includes performing the conversion based on the determination.

[0027] In another example aspect, a video processing method is disclosed. The method includes determining a size constraint between a representative motion vector of an affine-encoded current video block and a motion vector of a sub-block of the current video block, and performing a conversion between a bitstream representation and a pixel value of the current video block or the sub-block using the size constraint.

[0028] In another example aspect, a video processing method is disclosed. The method includes determining, for an affine-encoded current video block, one or more sub-blocks of the current video block, wherein each sub-block has a size of MxN pixels, where M and N are multiples of 2 or 4, such that motion vectors of the sub-blocks conform to a size constraint, and conditionally performing a conversion between a bitstream representation and pixel values ​​of the current video block based on a trigger using the size constraint.

[0029] In yet another example aspect, a video processing method is disclosed. The method includes determining that a current video block satisfies a size condition, and based on the determination, performing a conversion between a bitstream representation and pixel values ​​of a previous video block by excluding a bidirectional predictive coding mode of the current video block.

[0030] In yet another example aspect, a video processing method is disclosed. The method includes determining that a current video block satisfies a size condition, and based on the determination, performing a conversion between a bitstream representation and pixel values ​​of the current video block, wherein an inter-frame prediction mode is signaled in the bitstream representation according to the size condition.

[0031] In yet another example aspect, a video processing method is disclosed, the method comprising determining that a current video block satisfies a size condition, and based on the determination, performing a conversion between a bitstream representation and pixel values ​​of the current video block, wherein generation of a Merge candidate list during the conversion depends on the size condition.

[0032] In yet another example aspect, a video processing method is disclosed. The method includes determining that a child coding unit of a current video block satisfies a size condition, and based on the determination, performing a conversion between a bitstream representation and pixel values ​​of the current video block, wherein a coding tree partitioning process for generating the child coding unit depends on the size condition.

[0033] In yet another example aspect, a video processing method is disclosed, the method comprising determining a weight index of a generalized bidirectional prediction (GBi) process of the current video block based on a position of the current video block, and performing conversion between the current video block and its bitstream representation using the weight index to implement the GBi process.

[0034] In yet another example aspect, a video processing method is disclosed. The method includes determining that a current video block is encoded as an intra-frame inter-frame prediction (IIP) coded block, and performing a conversion between the current video block and its bitstream representation using a simplified rule for determining an intra-frame prediction mode or a most probable mode (MPM) for the current video block.

[0035] In yet another example aspect, a video processing method is disclosed. The method includes determining that a current video block meets a simplification criterion, and performing a conversion between the current video block and a bitstream representation by disabling an inter-frame intra-frame prediction mode for the conversion or by disabling an additional coding tool for the conversion.

[0036] In yet another example aspect, a video processing method is disclosed. The method includes using a motion vector-based encoding process to perform a conversion between a current video block and a bitstream representation of the current video block, wherein: (a) during the conversion process, precision P1 is used to store spatial motion prediction results and precision P2 is used to store temporal motion prediction results, where P1 and P2 are fractional, or (b) precision Px is used to store x motion vectors and precision Py is used to store y motion vectors, where Px and Py are fractional.

[0037] In yet another example aspect, a video processing method is disclosed. The method includes interpolating a small sub-block of size W1xH1 within a large sub-block of size W2xH2 of a current video block by extracting a (W2+N–1–PW)*(H2+N–1–PH) block, pixel padding the extracted block, performing boundary pixel repetition on the pixel-padded block, and obtaining pixel values ​​of the small sub-block, wherein W1, W2, H1, H2, PW, and PH are integers, and performing conversion between the current video block and a bitstream representation of the current video block using the interpolated pixel values ​​of the small sub-block.

[0038] In yet another example aspect, a video processing method is disclosed. The method includes performing a motion compensation operation during a conversion between a current video block of WxH size and a bitstream representation of the current video block by extracting (W+N–1–PW)*(W+N–1–PH) reference pixels during the motion compensation operation and filling reference pixels outside the extracted reference pixels, and performing a conversion between the current video block and the bitstream representation of the current video block using a result of the motion compensation operation, wherein W, H, N, PW, and PH are integers.

[0039] In yet another example aspect, a video processing method is disclosed. The method includes determining that bidirectional prediction or unidirectional prediction of the current video block is not allowed based on the size of the current video block, and based on the determination, performing conversion between a bitstream representation and pixel values ​​of the current video block by disabling the bidirectional prediction or unidirectional prediction mode.

[0040] In yet another example aspect, a video processing method is disclosed. The method includes determining that bidirectional prediction or unidirectional prediction of the current video block is not allowed based on the size of the current video block, and based on the determination, performing conversion between a bitstream representation and pixel values ​​of the current video block by disabling the bidirectional prediction or unidirectional prediction mode.

[0041] In yet another example aspect, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the above method.

[0042] In yet another example aspect, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the above method.

[0043] In yet another exemplary aspect, a computer readable medium having code stored thereon is disclosed. The code is in the form of processor executable code embodying one of the methods described herein.

[0044] These and other features are described throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 An example of sub-block based prediction is shown.

[0046] Figure 2A A 4-parameter affine model is shown.

[0047] Figure 2B A 6-parameter affine model is shown.

[0048] Figure 3 An example of the affine motion vector field for each sub-block is shown.

[0049] Figure 4A An example of a candidate for AF_MERGE is shown.

[0050] Figure 4B Another example of a candidate for AF_MERGE is shown.

[0051] Figure 5 Candidate positions for the affine merge mode are shown.

[0052] Figure 6 An example of constrained sub-block motion vectors for a coding unit (CU) in affine mode is shown.

[0053] Fig. 7A An example of a 135 degree partition of a CU into two triangular prediction units is shown.

[0054] Figure 7B An example of a 45-degree partitioning pattern that divides a CU into two triangular prediction units is shown.

[0055] Figure 8 An example of the positions of neighboring blocks is shown.

[0056] Fig. 9 An example of repeated boundary pixels of a reference block before interpolation is shown.

[0057] Fig.10 An example of a coding tree unit (CTU) and a CTU (region) row is shown. A shaded CTU (region) is located in one CTU (region) row and an unshaded CTU (region) is located in another CTU (region) row.

[0058] Fig.11 is a block diagram of an example of a hardware platform for implementing the video decoder or video encoder apparatus described herein.

[0059] Fig.12 is a flow chart of an example method of video processing.

[0060] Fig.13 An example of a motion vector difference MVD(0, 1) mirrored between list 0 and list 1 in DMVR is shown.

[0061] Fig.14Exemplary MVs that may be checked in one iteration are shown.

[0062] Fig.15 Reference points and boundaries for the calculated fill are shown.

[0063] Fig.16 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

[0064] Fig.17 is a flow chart representing a method for video processing according to the present disclosure.

[0065] Fig.18 is a flow chart representing another method for video processing according to the present disclosure.

[0066] Fig.19 is a flow chart representing another method for video processing according to the present disclosure.

[0067] Fig. 20 is a flow chart representing another method for video processing according to the present disclosure.

[0068] Fig.21 is a flow chart representing another method for video processing according to the present disclosure.

[0069] Fig. 22 is a flow chart representing another method for video processing according to the present disclosure.

[0070] Fig.23 is a flow chart representing another method for video processing according to the present disclosure.

[0071] Fig.24 is a flow chart representing another method for video processing according to the present disclosure.

[0072] Fig.25 is a flow chart representing another method for video processing according to the present disclosure.

[0073] Fig.26 is a flow chart representing another method for video processing according to the present disclosure.

[0074] Fig. 27 is a flow chart representing another method for video processing according to the present disclosure.

[0075] Fig.28 is a flow chart representing another method for video processing according to the present disclosure.

[0076] Fig.29 is a flow chart representing another method for video processing according to the present disclosure.

[0077] Fig.30 is a flow chart representing another method for video processing according to the present disclosure.

[0078] Fig.31 is a flow chart representing another method for video processing according to the present disclosure.

[0079] Fig.32 is a flow chart representing another method for video processing according to the present disclosure.

[0080] Fig.33 is a flow chart representing another method for video processing according to the present disclosure.

[0081] Fig.34 is a flow chart representing another method for video processing according to the present disclosure.

[0082] Fig.35 is a flow chart representing another method for video processing according to the present disclosure.

[0083] Fig.36 is a flow chart representing another method for video processing according to the present disclosure.

[0084] Fig.37 is a flow chart representing yet another method for video processing according to the present disclosure. DETAILED DESCRIPTION

[0085] The section headings used in this document are for ease of understanding and do not limit the applicability of the techniques and embodiments disclosed in each section to only that section.

[0086] 1. Overview

[0087] This patent document relates to video / image coding technology. Specifically, it is related to reducing the bandwidth and line buffer of several coding tools in video / image coding. It can be applied to existing video coding standards (such as HEVC), or to a pending standard (Versatile Video Coding). It may also be applicable to future video / image coding standards or video / image codecs.

[0088] 2. Background

[0089] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations worked together to produce the H.262 / MPEG-2 Video and H.264 / MPEG-4 Enhanced Video Coding (AVC) and H.265 / HEVC standards. Starting with H.262, video coding standards are based on a hybrid video coding structure that utilizes temporal prediction and transform coding. In order to explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software called the "Joint Exploration Model" (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was established to work on the Versatile Video Coding (VVC) standard with the goal of reducing bit rate by 50% compared to HEVC.

[0090] 2.1 Inter-frame prediction in HEVC / VVC

[0091] Interpolation Filter

[0092] In HEVC, luma sub-samples are generated by an 8-tap interpolation filter, and chroma sub-samples are generated by a 4-tap interpolation filter.

[0093] The filters are separable in two dimensions. First the samples are filtered horizontally, then vertically.

[0094] 2.2 Sub-block-based prediction techniques

[0095] First, sub-block based prediction was introduced into the video coding standard through HEVC Annex I (3D-HEVC). With sub-block based prediction, a block such as a coding unit (CU) or a prediction unit (PU) is divided into several non-overlapping sub-blocks. Different motion information, such as reference indexes or motion vectors (MVs), can be assigned to different sub-blocks, and motion compensation (MC) is performed separately for each sub-block. Figure 1 A diagram is shown demonstrating the concept of sub-block based prediction.

[0096] In order to explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software called the Joint Exploration Model (JEM).

[0097] In JEM, sub-block based prediction is adopted in several coding tools, such as affine prediction, optional temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), bidirectional optical flow (BIO) and frame rate up conversion (FRUC). Affine prediction has also been used in VVC.

[0098] 2.3 Affine Prediction

[0099] In HEVC, only the translation motion model is applied to motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, a simplified affine transformation motion compensation prediction is applied. Figure 2A-2B As shown, the affine motion field of a block is described by two (in a 4-parameter affine model) or three (in a 6-parameter affine model) control point motion vectors.

[0100] The motion vector field (MVF) of the block is described by the following 4-parameter affine model in equation (1) (where the 4 parameters are defined as variables a, b, e and f)) and 6-parameter affine model in equation (2) (where the 6 parameters are defined as variables a, b, c, d, e and f), respectively:

[0101]

[0102]

[0103] Here, (mv h 0,mv h 0) is the motion vector of the upper left control point (CP), (mv h 1,mv h 1) is the motion vector of the upper right control point, (mv h 2,mv h 2) is the motion vector of the lower left control point, (x, y) represents the coordinates of the representative point relative to the upper left sample point in the current block, and all three motion vectors are called control point motion vectors (CPMV). The CP motion vector can be signaled (such as in affine AMVP mode) or derived on the fly (such as in affine Merge mode). w and h are the width and height of the current block. In practice, division is achieved by rounding and right shifting. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample point in the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2).

[0104] In a division-free design, equations (1) and (2) are implemented as:

[0105]

[0106] For the 4-parameter affine model shown in (1):

[0107]

[0108] For the 6-parameter affine model shown in (2):

[0109]

[0110] final,

[0111]

[0112]

[0113] Off=1<<(S-1)

[0114] Where S represents the calculation precision. For example, in VVC, S=7. In VVC, the MV for the sub-block with the upper left corner sample point (xs, ys) in MC is calculated using equation (6) when x=xs+2 and y=ys+2.

[0115] like Figure 3 As shown, to derive the motion vector of each 4×4 sub-block, the motion vector of the center sample of each sub-block is calculated according to equation (1) or equation (2) and rounded to 1 / 16 fractional accuracy. Then, the motion compensation interpolation filter is applied to generate a prediction of each sub-block with a derived motion vector.

[0116] Affine models can be inherited from spatially adjacent affine encoding blocks, such as Figure 4A The neighboring blocks to the left, top, top right, bottom left, and top left are shown. For example, if Figure 4A The adjacent lower left block A in is encoded in affine mode, such as Figure 4B As shown in A0 in the figure, the control point (CP) motion vector mv0 of the upper left corner, upper right corner and lower left corner of the adjacent CU / PU containing block A is obtained. N 、mv1 N and mv2 N . And based on mv0 N 、mv1 N and mv2 N Calculate the motion vector mv0 of the upper left corner / upper right / lower left corner of the current CU / PU C 、mv1 C and mv2 C(Only for 6-parameter affine model). It should be noted that in VTM-2.0, if the current block is affine coded, the sub-block (e.g., 4×4 block in VTM) LT stores mv0 and RT stores mv1. If the current block is coded using a 6-parameter affine model, LB stores mv2; otherwise (coded using a 4-parameter affine model), LB stores mv2'. Other sub-blocks store MVs for MC.

[0117] It should be noted that when a CU is coded in affine merge mode, for example in AF_MERGE mode, it obtains the first block coded in affine mode from the valid adjacent reconstructed blocks. Figure 4A As shown, the order of selecting candidate blocks is from left, top, top right, bottom left to top left.

[0118] In affine merge mode, the exported CP MV mv0 of the current block C 、mv1 C and mv2 C Can be used as CP MV. Alternatively, they can be used as MVP for affine inter mode in VVC. It should be noted that for Merge mode, if the current block is encoded in affine mode, after deriving the CP MV of the current block, the current block can be further divided into multiple sub-blocks, and each block will derive its motion information based on the derived CP MV of the current block.

[0119] 2.4 Example Embodiment in JVET

[0120] This is different from VTM, where only one affine spatial neighbor can be used to derive the affine motion of a block.In some embodiments, a separate list of affine candidates for AF_MERGE mode is constructed.

[0121] 1) Insert the inherited affine candidate into the candidate list

[0122] Inherited affine candidates are candidates that are derived from valid neighbor reconstructed blocks encoded in affine mode.

[0123] like Figure 5 As shown, the scanning order of candidate blocks is A1, B1, B0, A0 and B2. When a block (for example A1) is selected, a two-step process is performed:

[0124] 1.a First, use the three angular motion vectors of the CU covering the block to derive two / three control points of the current block; and

[0125] 1.b The sub-block motion of each sub-block within the current block is derived based on the control points of the current block.

[0126] 2) Insert the constructed affine candidate

[0127] If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand, the constructed affine candidate is inserted into the candidate list.

[0128] The constructed affine candidates are those constructed by combining the neighbor motion information of each control point.

[0129] First, from Figure 5 The specified spatial and temporal neighbors shown derive the motion information of the control point. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial positions used to predict CPk (k = 1, 2, 3); T is the temporal position used to predict CP4.

[0130] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.

[0131] The motion information for each control point is obtained according to the following priority order:

[0132] 2.a For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is not available, use B3. If both B2 and B3 are not available, use A2. If none of these three candidates are available, the motion information of CP1 cannot be obtained.

[0133] 2.b For CP2, check the priority from B1->B0;

[0134] 2.c For CP3, check the priority from A1->A0;

[0135] 2.d For CP4, use T.

[0136] Second, a combination of control points is used to construct a motion model.

[0137] The motion vectors of three control points are required to calculate the transformation parameters in the 6-parameter affine model. The three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). For example, the 6-parameter affine motion model is constructed using the CP1, CP2, and CP3 control points, expressed as Affine (CP1, CP2, CP3).

[0138] The motion vectors of two control points are required to calculate the transformation parameters in the 4-parameter affine model. Two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). For example, the 4-parameter affine motion model is constructed using the CP1 and CP2 control points, denoted as Affine (CP1, CP2).

[0139] Insert the combinations of constructed affine candidates into the candidate list in the following order: {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

[0140] 3) Insert zero motion vector

[0141] If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand, zero motion vectors are inserted into the candidate list until the list is full.

[0142] 2.5 Affine Merge Candidate List

[0143] 2.5.1 Affine Merge Mode

[0144] In the Affine Merge mode of VTM-2.0.1, only the first available affine neighbor can be used to derive motion information for the Affine Merge mode. In some embodiments, a candidate list for the Affine Merge mode is constructed by searching for valid affine neighbors and combining neighbor motion information for each control point.

[0145] The affine merge candidate list is constructed as follows:

[0146] 1) Insert inherited affine candidates

[0147] An inherited affine candidate is a candidate derived from the affine motion model of its valid neighbor affine coded blocks. Figure 5 As shown, on a common basis, the scanning order of candidate positions is: A1, B1, B0, A0 and B2.

[0148] After deriving the candidates, a full pruning process is performed to check if the same candidate has been inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0149] 2) Insert the constructed affine candidate

[0150] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (set to 5 in this contribution), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate refers to the candidate constructed by combining the neighbor motion information of each control point.

[0151] First, the motion information of the control point is derived from the specified spatial and temporal neighbors. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial positions used to predict CPk (k = 1, 2, 3); T is the temporal position used to predict CP4.

[0152] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.

[0153] The motion information for each control point is obtained according to the following priority order:

[0154] For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is not available, use B3. If neither B2 nor B3 is available, use A2. If none of the three candidates are available, the motion information of CP1 cannot be obtained.

[0155] For CP2, the check priority is B1->B0;

[0156] For CP3, the check priority is A1->A0;

[0157] For CP4, use T.

[0158] Second, a combination of control points is used to construct a motion model.

[0159] The motion information of three control points is required to construct a 6-parameter affine candidate. Three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted to a 6-parameter motion model represented by the top left, top right, and bottom left control points.

[0160] The motion information of two control points is required to construct a 4-parameter affine candidate. Two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). The combination {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4} will be converted to a 4-parameter motion model point represented by the top left and top right control points.

[0161] Insert the combinations of constructed affine candidates into the candidate list in the following order:

[0162] {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

[0163] For a combined reference list X (X is 0 or 1), the reference index with the highest usage in the control point is selected as the reference index of list X, and the motion vector pointing to the difference reference picture is scaled.

[0164] After deriving the candidates, a full pruning process is performed to check if the same candidate has been inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0165] 3) Fill with zero motion vectors

[0166] If the number of candidates in the affine merge candidate list is less than 5, zero motion vectors with zero reference index are inserted into the candidate list until the list is full.

[0167] 2.5.2 Example of Affine Merge Mode

[0168] In some embodiments, the affine merge mode can be simplified as follows:

[0169] 1) The pruning process of inherited affine candidates is simplified by comparing coding units covering adjacent positions instead of comparing affine candidates derived in VTM-2.0.1. At most 2 inherited affine candidates are inserted into the affine merge list. The pruning process of constructed affine candidates is completely removed.

[0170] 2) Remove the MV scaling operation in the constructed affine candidate. If the reference index of the control point is different, the constructed motion model is discarded.

[0171] 3) The number of constructed affine candidates is reduced from 10 to 6.

[0172] 4) In some embodiments, other Merge candidates with sub-block predictions (such as ATMVP) are also put into the affine Merge candidate list. In that case, the affine Merge candidate list can be renamed with some other names such as the sub-block Merge candidate list.

[0173] 2.6 Example of control point MV shift for affine merge mode

[0174] Generate new affine Merge candidates based on the CPMV offset of the first affine Merge candidate. If the first affine Merge candidate enables the 4-parameter affine model, then by offsetting the 2 CPMVs of the first affine Merge candidate, 2 CPMVs can be derived for each new affine Merge candidate. Otherwise (enable the 6-parameter affine model), then by offsetting the 3 CPMVs of the first affine Merge candidate, 3 CPMVs are derived for each new affine Merge candidate. In unidirectional prediction, the CPMV offset is applied to the CPMV of the first candidate. In bidirectional prediction with list 0 and list 1 in the same direction, the CPMV offset is applied to the first candidate as follows:

[0175] MV new(L0),i =MV old(L0) +MV offset(i) Equation (8)

[0176] MV new(L1),i =MV old(L1) +MV offset(i) Equation (9)

[0177] In bidirectional prediction with list 0 and list 1 in opposite directions, the CPMV offset is applied to the first candidate as follows:

[0178] MV new(L0),i =MV old(L0) +MV offset(i) Equation (10)

[0179] MV new(L1),i =MV old(L1) -MV offset(i) Equation (11)

[0180] Various offset directions with various offsets can be used to generate new affine merge candidates. Two implementations have been tested:

[0181] (1) Generate 16 new affine merge candidates with 8 different offset directions and 2 different offsets, as shown in the following offset set:

[0182] Offset set = {(4, 0), (0, 4), (-4, 0), (0, -4), (-4, -4), (4, -4), (4, 4), (-4, 4), (8, 0), (0, 8), (-8, 0), (0, -8), (-8, -8), (8, -8), (8, 8), (-8, 8)}.

[0183] For this design, the affine merge list increases to 20. The total number of potential affine merge candidates is 31.

[0184] (2) Generate 4 new affine merge candidates with 4 different offset directions and 1 offset, as shown in the following offset set:

[0185] Offset set = {(4, 0), (0, 4), (-4, 0), (0, -4)}.

[0186] As in VTM-2.0.1, the affine merge list remains at 5. Remove the affine merge candidates constructed in the four time domains to keep the number of potential affine merge candidates unchanged, for example, a total of 15. Assume that the coordinates of CPMV1, CPMV2, CPMV3, and CPMV4 are (0, 0), (W, 0), (H, 0), and (W, H). Note that Figure 6 As shown, CPMV4 is derived from the time domain MV. The removed candidates are the following four time domain related constructed affine merge candidates: {CP2, CP3, CP4}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}.

[0187] 2.7 Bandwidth Issues in Affine Motion Compensation

[0188] Since the current block is divided into 4×4 sub-blocks for the luma component and 2×2 sub-blocks for the two chroma components for motion compensation, the total bandwidth requirement is much higher than that of non-sub-block inter prediction. To solve the bandwidth problem, several methods have been proposed.

[0189] 2.7.1 Example 1

[0190] 4x4 blocks are used as the sub-block size of a unidirectional affine coded CU, while 8x4 / 4x8 blocks are used as the sub-block size of a bidirectional affine coded CU.

[0191] 2.7.2 Example 2

[0192] For affine mode, the sub-block motion vectors of the affine CU are constrained to be within a predefined motion vector field. Assume that the motion vector of the first (upper left) sub-block is (v 0x ,v 0y ), the motion vector of the second sub-block is (v ix ,viy ), then v ix and v iy The values ​​of exhibit the following constraints:

[0193] v ix ∈[v 0x -H,v 0x +H] Equation (12)

[0194] v iy ∈[v 0y -V,v 0y +V] Equation (13)

[0195] If the motion vector of any sub-block exceeds the predefined motion vector field, the motion vector is clipped. Figure 6 An illustration of the idea of ​​constraining sub-block motion vectors is given in .

[0196] Assuming that memory is retrieved per CU instead of per subblock, the values ​​of H and V are chosen so that the worst-case memory bandwidth of an affine CU will not exceed the bandwidth of a normal inter MC for an 8×8 bi-predicted block. Note that the values ​​of H and V apply to the CU size and to uni- or bi-prediction.

[0197] 2.7.3 Example 3

[0198] To reduce the memory bandwidth requirement in affine prediction, each 8x8 block in the block is treated as a basic unit. The MVs of all four 4x4 sub-blocks in the 8x8 block are constrained so that the maximum difference between the integer parts of the MVs of the four 4x4 sub-blocks does not exceed 1 pixel. The bandwidth is (8+7+1)*(8+7+1) / (8*8)=4 samples / pixel.

[0199] In some cases, after calculating the MVs of all sub-blocks within the current block using the affine model, the MVs of the sub-blocks containing the control points are first replaced with the corresponding control point MVs. This means that the MVs of the upper left, upper right, and lower left sub-blocks are replaced by the upper left, upper right, and lower left control point MVs, respectively. Then, for each 8x8 block within the current block, the MVs of all four 4x4 sub-blocks are clipped to ensure that the maximum difference between the integer parts of the four MVs does not exceed 1 pixel. It should be noted here that the sub-blocks containing control points (upper left, upper right, and lower left sub-blocks) participate in the MV clipping process using the corresponding control point MVs. During the clipping process, the MV of the upper right control point remains unchanged.

[0200] The cropping process applied to each 8x8 block is described as follows:

[0201] 1. First, determine the minimum and maximum values ​​MVminx, MVminy, MVmaxx, MVmaxy of the MV components for each 8x8 block as follows:

[0202] a) Get the minimum MV component among the four 4×4 sub-block MVs

[0203] MVminx=min(MVx0,MVx1,MVx2,MVx3)

[0204] MVminy=min(MVy0,MVy1,MVy2,MVy3)

[0205] b) Use the integer part of MVminx and MVminy as the minimum MV component

[0206] MVminx=MVminx>>MV_precision< <MV_precision

[0207] MVminy=MVminy>>MV_precision< <MV_precision

[0208] c) The maximum MV component is calculated as follows:

[0209] MVmaxx=MVminx+(2< <MV_precision)–1

[0210] MVmaxy=MVminy+(2< <MV_precision)–1

[0211] d) If the upper right control point is in the current 8x8 block

[0212] If (MV1x>MVmaxx)

[0213] MVminx=(MV1x>>MV_precision< <MV_precision)–(1<<MV_precision)

[0214] MVmaxx=MVminx+(2< <MV_precision)–1

[0215] If (MV1y>MVmaxy)

[0216] MVminy=(MV1y>>MV_precision< <MV_precision)–(1<<MV_precision)

[0217] MVmaxy=MVminy+(2< <MV_precision)–1

[0218] 2. The MV components of each 4x4 block within the 8x8 block are cropped as follows:

[0219] MVxi=max(MVminx,min(MVmaxx,MVxi))

[0220] MVyi=max(MVminy,min(MVmaxy,MVyi))

[0221] Among them, (MVxi, MVyi) is the MV of the i-th sub-block in an 8×8 block, where i is 0, 1, 2, 3; (MV1x, MV1y) is the MV of the upper right control point; MV_precision is equal to 4, corresponding to 1 / 16 motion vector fractional accuracy. Since the difference between the integer parts of MVminx and MVmaxx (MVminy and MVmaxy) is 1 pixel, the maximum difference between the integer parts of the four 4×4 sub-block MVs does not exceed 1 pixel.

[0222] In some embodiments, a similar approach may also be used to process planar patterns.

[0223] 2.7.4 Example 4

[0224] In some embodiments, the constraints of the affine mode are used for worst-case bandwidth reduction. To ensure that the worst-case bandwidth of the affine block is no worse than that of the INTER_4x8 / INTER_8x4 block or even the INTER_9x9 block, the motion vector differences between the affine control points are used to determine whether the sub-block size of the affine block is 4x4 or 8x8.

[0225] General affine constraint for worst-case bandwidth reduction

[0226] The memory bandwidth reduction of the affine mode is controlled by constraining the motion vector differences between the affine control points (also called control point differences). In general, if the control point differences satisfy the following constraints, the affine motion will use 4x4 sub-blocks (i.e., 4x4 affine mode). Otherwise, it will use 8x8 sub-blocks (8x8 affine mode). The constraints for the 6-parameter and 4-parameter models are as follows.

[0227] To derive constraints for different block sizes (wxh), the motion vector differences of the control points are normalized to:

[0228]

[0229]

[0230]

[0231] Norm(v 2x -v 0x )=(v 2x -v 0x )*128 / h Equation (14)

[0232] In the 4-parameter affine model, (v 2x -v 0x ) and (v 2y -v 0y ) is set as follows:

[0233] (v 2x -v 0x )=-(v 1y -v 0y )

[0234] (v 2y -v 0y )=-(v 1x -v 0x ) Equation (15)

[0235] Therefore, (v 2x -v 0x ) and (v 2y -v 0y ) is:

[0236] Norm(v 2x -v 0x )=-Norm(v 1y -v 0y )

[0237] Norm(v 2y -v 0y )=Norm(v 1x -v 0x ) Equation (16)

[0238] The constraint to ensure the worst case bandwidth is to achieve INTER_4x8 or INTER_8x4:

[0239] |Norm(v 1x -v 0x )+Norm(v 2x -v 0x )+128|+|Norm(v 1y -v 0y )+Norm(v 2y -v 0y )+128|+|Norm(v 1x -v 0x )-Norm(v 2x -v 0x )|+|Norm(v 1y -v 0y )-Norm(v 2y -v 0y )|

[0240] <128*3.25 Equation (17)

[0241] The left side of equation (18) represents the reduction or stride level of the sub-affine block, while (3.25) indicates a 3.25 pixel offset.

[0242] The worst case bandwidth constraint is to achieve INTER_9x9

[0243] (4*Norm(v 1x -v 0x )>-4*pel&&+4*Norm(v 1x -v 0x ) <pel)&&

[0244] (4*Norm(v 1y -v 0y )>-pel&&4*Norm(v 1y -v 0y ) <pel)&&

[0245] (4*Norm(v 2x -v 0x )>-pel&&4*Norm(v 2x -v 0x ) <pel)&&

[0246] (4*Norm(v 2y -v 0y )>-4*pel&&4*Norm(v 2y -v 0y ) <pel)&&

[0247] ((4*Norm(v 1x -v 0x )+4*Norm(v 2x -v 0x )>-4*pel)&&

[0248] (4*Norm(v 1x -v 0x )+4*Norm(v 2x -v 0x ) <pel))&&

[0249] ((4*Norm(v 1y -v 0y )+4*Norm(v 2y -v 0y )>-4*pel)&&

[0250] (4*Norm(v 1y -v 0y) + 4 * Norm(v 2y - v 0y ) < pel)) Equation (18)

[0251] where pel = 128 * 16 (128 and 16 indicate the normalization factor and the motion vector precision respectively).

[0252] 2.8. Generalized Bidirectional Prediction Improvement

[0253] Some embodiments improve the gain-complexity trade-off of GBi and are adopted in BMS2.1 GBi, also known as bidirectional prediction with CU-level weights (BCW). BMS2.1 GBi applies unequal weights to the predictors from L0 and L1 in the bidirectional prediction mode. In the inter-frame prediction mode, multiple weight pairs including the equal weight pair (1 / 2, 1 / 2) are evaluated based on rate-distortion optimization (RDO), and the GBi index of the selected weight pair is signaled to the decoder. In the Merge mode, the GBi index is inherited from neighboring CUs. In BMS2.1 GBi, Equation (19) shows the predictor generation in the bidirectional prediction mode.

[0254] PGB i = (w0 * PL0 + w1 * PL1 + RoundingOffsetGBi) >> shiftNumGBi, Equation (19)

[0255] where PGB i is the final predictor of GBi. w0 and w1 are the selected GBi weight pairs and are applied to the predictors of list 0 (L0) and list 1 (L1) respectively. RoundingOffsetGBi and shiftNumGBi are used to normalize the final predictor in GBi. The supported set of w1 weights is {-1 / 4, 3 / 8, 1 / 2, 5 / 8, 5 / 4}, where the five weights correspond to one equal weight pair and four unequal weight pairs respectively. The mixing gain, such as the sum of w1 and w0, is fixed at 1.0. Therefore, the corresponding set of w0 weights is {5 / 4, 5 / 8, 1 / 2, 3 / 8, -1 / 4}. The weight pair selection is at the CU level.

[0256] For non-low-delay pictures, the weight setting size is reduced from five to three, where the w1 weights are set to {3 / 8, 1 / 2, 5 / 8}, and the w0 weights are set to {5 / 8, 1 / 2, 3 / 8}. In this contribution, the reduction of the weight setting size for non-low-delay pictures is applicable to BMS2.1 GBi and all GBi tests.

[0257] In some embodiments, the following modifications are made based on the existing GBi design in BMS2.1 to further improve the GBi performance.

[0258] 2.8.1 GBi encoder bug fixes

[0259] To reduce GBi encoding time, in the current encoder design, the encoder will store the unidirectional prediction motion vector estimated based on GBi weight equal to 4 / 8 and reuse it for unidirectional prediction search for other GBi weights. This fast encoding method is applied to both translational motion models and affine motion models. In VTM 2.0, a 6-parameter affine model is adopted together with a 4-parameter affine model. When the GBi weight is equal to 4 / 8 and the BMS2.1 encoder stores a unidirectional prediction affine MV, the BMS2.1 encoder does not distinguish between the 4-parameter affine model and the 6-parameter affine model. Therefore, after encoding with GBi weight 4 / 8, the 4-parameter affine MV can be overwritten with the 6-parameter affine MV. For other GBi weights, the stored 6-parameter affine MV can be used for the 4-parameter affine ME, or the stored 4-parameter affine MV can be used for the 6-parameter affine ME. The proposed GBi encoder error fix is ​​to store the 4-parameter affine and 6-parameter affine MV separately. When GBi weight is equal to 4 / 8, the encoder will store these affine MVs based on the affine model type and reuse the corresponding affine MVs for other GBi weights based on the affine model type.

[0260] 2.8.2 GBi CU Size Constraints

[0261] In this method, GBi is disabled for small CUs. In inter prediction mode, if bi-prediction is used and the CU area is smaller than 128 luma samples, GBi is disabled without signaling.

[0262] 2.8.3 Using GBi's Merge Mode

[0263] In Merge mode, the GBi index is not signaled. Instead, it is inherited from the neighboring block it is merged into. When a TMVP candidate is selected, GBi is turned off in this block.

[0264] 2.8.4 Affine prediction using GBi

[0265] GBi can be used when the current block is encoded using affine prediction. For affine inter mode, the GBi index will be signaled. For affine merge mode, the GBi index is inherited from the neighboring block being merged into. If the constructed affine model is selected, GBi will be turned off in this block.

[0266] 2.9 Example of Inter-intra prediction mode (IIP)

[0267] In the inter-intra prediction mode, the multi-hypothesis prediction combines an intra prediction and a merge index prediction. Such a block is regarded as a special inter coding block.

[0268] In Merge CU, when the flag is true, a flag is signaled for Merge mode to select the intra mode from the intra candidate list. For luma component, the intra candidate list is derived from 4 intra prediction modes including DC, planar, horizontal and vertical modes, and the size of the intra candidate list can be 3 or 4 based on the block shape. When the CU width is greater than twice the CU height, the horizontal mode does not include the intra mode list, and when the CU height is greater than twice the CU width, the vertical mode is removed from the intra mode list. One intra prediction mode selected by the intra mode index and one Merge index prediction selected by the Merge index are combined using weighted average. For chroma components, DM is always applied without additional signaling.

[0269] The weights of combined prediction are described as follows. When DC or planar mode is selected, or CB width or height is less than 4, equal weights are applied. For CBs with CB width and height greater than or equal to 4, when horizontal / vertical mode is selected, one CB is first divided vertically / horizontally into four equal-area regions. Each weight set, denoted as (w_intra i ,w_inter i ), where i ranges from 1 to 4 and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5) and (w_intra4, w_inter4) = (2, 6) will be applied to the corresponding areas. (w_intra1, w_inter1) is used for the area closest to the reference sample, and (w_intra4, w_inter4) is used for the area farthest from the reference sample. The combined prediction can then be calculated by adding the two weighted predictions and shifting right by 3 bits. In addition, the intra prediction mode for the intra hypothesis of the predictor can be saved for reference by subsequent neighboring CUs.

[0270] Assume that the intra-frame and inter-frame prediction values ​​are PIntra and Pinter, and the weighting factors are w_intra and w_inter respectively. The prediction value of the position (x, y) is calculated as (PIntra(x, y)*w_intra(x, y)+PInter(x, y)*w_inter(x, y))>>N, where w_intra(x, y)+w_iner(x, y)=2^N.

[0271] Signaling intra prediction mode in IIP coded blocks

[0272] When inter-intra mode is used, one of the four allowed intra prediction modes, DC, planar, horizontal, and vertical, is selected and signaled. Three most likely modes (MPMs) are constructed from the left and top of the neighboring blocks. The intra prediction mode of an intra-coded neighboring block or an IIP-coded neighboring block is considered as one MPM. If the intra prediction mode is not one of the four allowed intra prediction modes, it will be rounded to a vertical mode or a horizontal mode depending on the angle difference. The neighboring block must be in the same CTU row as the current block.

[0273] Assume that the width and height of the current block are W and H. If W>2*H or H>2*W, only one of the three MPMs can be used in inter-intra mode. Otherwise, all four valid intra prediction modes can be used in inter-intra mode.

[0274] It should be noted that the intra prediction mode in the inter-intra mode cannot be used to predict the intra prediction mode in a normal intra-coded block.

[0275] Inter-intra prediction can be used only when W*H>=64.

[0276] 2.10 Example of triangular prediction mode

[0277] The concept of triangular prediction mode (TPM) is to introduce a new triangular partitioning for motion compensated prediction. Fig. 7A and 7B As shown, it divides the CU into two triangular prediction units along the diagonal direction or the reverse diagonal direction. Each triangular prediction unit in the CU uses the reference frame index derived from the unidirectional prediction candidate list and its own unidirectional prediction motion vector for inter-frame prediction. After predicting the triangular prediction unit, adaptive weighting is performed on the diagonal edges. Then, the transformation and quantization process is applied to the entire CU. Note that this mode is only applied to the skip and merge modes.

[0278] 2.10.1 One-way prediction candidate list for TPM

[0279] The unidirectional prediction candidate list consists of five unidirectional prediction motion vector candidates. Figure 8 As shown, it is derived from seven neighboring blocks including five spatial neighboring blocks (1 to 5) and two temporally collocated blocks (6 to 7). According to the order of unidirectional prediction motion vector, L0 motion vector of bi-prediction motion vector, L1 motion vector of bi-prediction motion vector, and average motion vector of L0 and L1 motion vector of bi-prediction motion vector, the motion vectors of the seven neighboring blocks are collected and put into the unidirectional prediction candidate list. If the number of candidates is less than five, the zero motion vector is added to the list. The motion candidate added to this list is called TPM motion candidate.

[0280] More specifically, the following steps are involved:

[0281] 1) Without any cropping operation, from A1, B1, B0, A0, B2, Col and Col2 (corresponding to Figure 8 Motion candidates are obtained in blocks 1-7).

[0282] 2) Set the variable numCurrMergeCand = 0

[0283] 3) For each motion candidate derived from A1, B1, B0, A0, B2, Col and Col2, and numCurrMergeCand is less than 5, if the motion candidate is unidirectionally predicted (from list 0 or list 1), it is added to the Merge list and numCurrMergeCand is increased by 1. Such added motion candidates are called "original unidirectionally predicted candidates".

[0284] Apply full cropping.

[0285] 4) For each motion candidate derived from A1, B1, B0, A0, B2, Col and Col2, and numCurrMergeCand is less than 5, if the motion candidate is bidirectionally predicted, the motion information in list 0 is added to the Merge list (i.e., modified to unidirectional prediction from list 0), and numCurrMergeCand is increased by 1. Such an added motion candidate is called a "truncated list 0 predicted candidate".

[0286] Apply full cropping.

[0287] 5) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is bidirectionally predicted, the motion information in list 1 is added to the Merge list (i.e., modified to be unidirectionally predicted from list 1), and numCurrMergeCand is increased by 1. Such an added motion candidate is called a "truncated list 1 predicted candidate".

[0288] Apply full cropping.

[0289] 6) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is bidirectionally predicted,

[0290] – If the slice quantization parameter QP of the list 0 reference picture is less than the slice QP of the list 1 reference picture, the motion information of list 1 is first scaled to the list 0 reference picture, and the average of two MVs (one from the original list 0 and the other is the scaled MV from list 1) is added to the Merge list, which is the average unidirectional prediction from the list 0 motion candidate, and numCurrMergeCand is increased by 1.

[0291] – Otherwise, first scale the motion information of list 0 to the list 1 reference picture, then add the average of the two MVs (one from the original list 1 and the other is the scaled MV from list 0) to the Merge list, which is the average unidirectional prediction from the list 1 motion candidates, and numCurrMergeCand is increased by 1.

[0292] Apply full cropping.

[0293] 7) If numCurrMergeCand is less than 5, add a zero motion vector candidate.

[0294] 2.11 Decoder-side Motion Vector Refinement (DMVR) in VVC

[0295] For DMVR in VVC, it is assumed that the MVD mirroring between list 0 and list 1 is as follows Fig.13 As shown, bilateral matching is performed to refine the MV, for example, to find the best MVD among several MVD candidates. The MVs of the two reference picture lists are represented by MVL0 (L0X, L0Y) and MVL1 (L1X, L1Y). The MVD of list 0 represented by (MvdX, MvdY) is defined as the best MVD, which can minimize the cost function (e.g., SAD). For the SAD function, it is defined as the SAD between the reference block of list 0 (derived using the motion vector (L0X+MvdX, L0Y+MvdY) in list 0) and the reference block of list 1 (derived using the motion vector (L1X-MvdX, L1Y-MvdY) in list 1).

[0296] The motion vector refinement process can be iterated twice. In each iteration, up to 6 MVDs (with integer pixel precision) can be checked in two steps, such as Fig.14 As shown. In the first step, MVD (0, 0), (-1, 0), (1, 0), (0, -1), (0, 1) is checked. In the second step, one of MVD (-1, -1), (-1, 1), (1, -1) or (1, 1) can be selected and further checked. Assume that the function Sad(x, y) returns the SAD value of MVD (x, y). The MVD represented by (MvdX, MvdY) checked in the second step is determined as follows:

[0297] MvdX = -1;

[0298] MvdY=-1;

[0299] If (Sad(1,0) <Sad(-1,0))

[0300] MvdX = 1;

[0301] If (Sad(0,1) <Sad(0,-1))

[0302] MvdY=1;

[0303] In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the best MVD selected in the first iteration. DMVR is applicable only when one reference picture is the previous picture and the other reference picture is the next picture and both reference pictures have the same picture order count distance to the current picture.

[0304] To further simplify the DMVR process, the following key features may be implemented in some embodiments:

[0305] 1. When the SAD at position (0, 0) between list 0 and list 1 is less than the threshold, terminate early.

[0306] 2. For some positions, when the SAD between list 0 and list 1 is zero, terminate early.

[0307] 3. Block size of DMVR: W*H>=64&&H>=8, where W and H are the width and height of the block.

[0308] 4. For DMVR with CU size > 16*16, the CU is divided into multiple 16x16 sub-blocks. If only the width or height of the CU is greater than 16, it is divided only in the vertical or horizontal direction.

[0309] 5. Reference block size (W+7)*(H+7) (for luma).

[0310] 6.25-point SAD-based integer pixel search (e.g., (+-)2 refinement search range, single level)

[0311] 7. DMVR based on bilinear interpolation.

[0312] 8. Sub-pixel refinement based on the “parametric error surface equation”. This process is performed only when the minimum SAD cost is not equal to zero and the best MVD is (0, 0) in the previous MV refinement iteration.

[0313] 9. Luma / Chroma MC with reference block filling (if needed).

[0314] 10. Refinement MV for MC and TMVP only.

[0315] 2.11.1 Use of DMVR

[0316] DMVR can be enabled when all of the following conditions are met:

[0317] - The DMVR enabled flag in SPS (e.g., sps_dmvr_enabled_flag) is equal to 1.

[0318] -TPM flag, inter-frame affine flag, sub-block Merge flag (ATMVP or affine Merge), and MMVD flag are all equal to 0.

[0319] The -Merge flag is equal to 1.

[0320] - The current block is bi-directionally predicted, and the picture order count (POC) distance between the reference pictures in list 1 and the current picture is equal to the POC distance between the reference pictures in list 0 and the current picture.

[0321] -The height of the current CU is greater than or equal to 8.

[0322] - The number of luma samples (CU width * height) is greater than or equal to 64.

[0323] 2.11.2 Sub-pixel refinement based on “parametric error surface equation”

[0324] The method is summarized as follows:

[0325] 1. The parameter error surface fit is calculated only if the center location is the best cost location in a given iteration.

[0326] 2. The center position cost and the center distance (-1,0), (0,-1), (1,0) and (0,1) positions are used to fit a two-dimensional parabolic error surface equation of the following form

[0327] E(x,y)=A(x-x0) 2 +B(y-y0) 2 +C

[0328] Where (x0, y0) corresponds to the location with the lowest cost and C corresponds to the minimum cost value. By solving the 5 equations in the 5 unknowns, (x0, y0) can be calculated as:

[0329] x0=(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0)))

[0330] y0=(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0))

[0331] (x0, y0) can be computed to any desired sub-pixel precision by adjusting the precision with which the division is performed (e.g., how many bits of the quotient are computed). For 1 / 16 pixel precision, only 4 bits of the absolute value of the quotient need to be computed, which lends itself to implementing the 2 divisions required per CU based on fast-shift subtraction.

[0332] 3. Add the calculated (x0, y0) to the integer distance refinement MV to obtain a sub-pixel accurate refinement increment MV.

[0333] 2.11.3 Reference points required in DMVR

[0334] For a block of size W*H, assuming the maximum allowed MVD value is + / -offset (e.g., 2 in VVC), and the filter size is filteSize (e.g., 8 for luma and 4 for chroma in VVC), then (W+2*offSet+filterSize–1)*(H+2*offSet+filterSize–1) reference samples are required. To reduce memory bandwidth, only the center (W+filterSize–1)*(H+filterSize–1) reference samples are extracted, and other pixels are generated by repeating the boundaries of the extracted samples. Fig.15 An example of an 8*8 block is shown in FIG. 1 , 15*15 reference samples are extracted (marked as 1502 ), and the boundaries of the extracted samples are repeated (marked by 1504 ) to generate a 17*17 region (marked by 1506 ).

[0335] During motion vector refinement, these reference samples are used to perform bilinear motion compensation. At the same time, these reference samples are also used to perform final motion compensation.

[0336] 2.12 Bandwidth calculation for different block sizes

[0337] Based on the current 8-tap luma interpolation filter and 4-tap chroma interpolation filter, the memory bandwidth of each block unit (4:2:0 color format, one MxN luma block with two M / 2x N / 2 chroma blocks) is listed in Table 1 below.

[0338] Table 1 Memory bandwidth examples

[0339]

[0340] Similarly, based on the current 8-tap luma interpolation filter and 4-tap chroma interpolation filter, the storage bandwidth of each MxN luma block unit is listed in Table 2 below:

[0341] Table 2 Memory bandwidth examples

[0342]

[0343]

[0344] Therefore, regardless of the color format, the bandwidth requirements for each block size, in descending order, are:

[0345] 4*4 bidirectional>4*8 bidirectional>4*16 bidirectional>4*4 unidirectional>8*8 bidirectional>4*32 bidirectional>4*64 bidirectional>4*128 bidirectional>8*16 bidirectional>4*8 unidirectional>8*32 bidirectional>….

[0346] 2.12 Motion vector accuracy issues in VTM-3.0

[0347] In VTM-3.0, the MV accuracy is 1 / 16 brightness pixel in storage. When signaling MV, the highest accuracy is 1 / 4 brightness pixel.

[0348] 3. Examples of Problems Solved by the Disclosed Embodiments

[0349] 1. The bandwidth control method for affine prediction is not clear enough and should be more flexible.

[0350] 2. Note that in the HEVC design, even if the coding unit (CU) can be partitioned with asymmetric prediction modes (such as splitting a 16x16 into two PUs of size 4x16 and 12x16), the worst case of memory bandwidth requirements is still 8x8 bidirectional prediction. In VVC, due to the new QTBT partitioning structure, a CU can be set to 4x16 and bidirectional prediction can be enabled. Compared with the bidirectionally predicted 8x8CU, the bidirectionally predicted 4x16CU requires higher memory bandwidth. How to handle block sizes that require higher bandwidth (such as 4×16 or 16×4) is not yet known.

[0351] 3. New encoding tools (such as GBi) introduce more line buffer issues

[0352] 4. Inter-frame Intra-frame modes require more memory and logic to signal the intra-frame prediction mode used in inter-coded blocks.

[0353] 5.1 / 16 brightness pixel MV accuracy requires higher memory storage.

[0354] 6. To interpolate four 4x4 blocks in one 8x8 block, (8+7+1)*(8+7+1) reference pixels need to be extracted, and about 14% more pixels are required compared to non-affine / non-planar mode 8x8 blocks.

[0355] 7. The averaging operation in hybrid intra- and inter-prediction should be consistent with other coding tools, such as weighted prediction, local illumination compensation, OBMC, and triangular prediction, where an offset is added before the shift.

[0356] 4. Examples of Embodiments

[0357] The techniques disclosed herein can reduce the bandwidth and line buffers required in affine prediction and other new encoding tools.

[0358] The following description should be considered as an example to explain the general concept and should not be interpreted in a narrow sense. In addition, the embodiments can be combined in any manner.

[0359] In the following discussion, the width and height of the current CU affine-coded are w and h, respectively. Assume that the interpolation filter taps (in motion compensation) are N (eg, 8, 6, 4, or 2), and the current block size is WxH.

[0360] Bandwidth control for affine prediction

[0361] Example 1: Assume that the motion vector of the subblock SB in the affine coded block is MV SB (expressed as (MVx, MVy)), MV SB It can be within a certain range relative to the representative motion vector MV' (MV'x, MV'y).

[0362] In some embodiments, MVx>=MV'x-DH0 and MVx<=MV'x+DH1 and MVy>=MV'y-DV0 and MVy<=MV'y+DV1, where MV'=(MV'x,MV'y). In some embodiments, DH0 may be equal to or not equal to DH1; DV0 may be equal to or not equal to DV1. In some embodiments, DH0 may be equal to or not equal to DV0; DH1 may be equal to or not equal to DV1. In some embodiments, DH0 may be equal to or not equal to DV0; DH1 may be equal to or not equal to DV1. In some embodiments, DH0 may be unequal to DH1; DV0 may be unequal to DV1. In some embodiments, DH0, DH1, DV0, and DV1 may be signaled from the encoder to the decoder, such as in a VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU / PU. In some embodiments, DH0, DH1, DV0, and DV1 may be specified differently for different standard profiles / levels / tiers. In some embodiments, DH0, DH1, DV0, and DV1 may depend on the width and height of the current block. In some embodiments, DH0, DH1, DV0 and DV1 may depend on whether the current block is unidirectionally predicted or bidirectionally predicted. In some embodiments, DH0, DH1, DV0 and DV1 may depend on the position of the sub-block SB. In some embodiments, DH0, DH1, DV0 and DV1 may depend on how to obtain MV'.

[0363] In some embodiments, MV' may be a CPMV, such as MV0, MV1, or MV2.

[0364] In some embodiments, MV′ may be the MV for one of the corner sub-blocks in MC, such as Figure 3 MV0', MV1' or MV2' in.

[0365] In some embodiments, MV' may be the MV derived for any position inside or outside the current block using the affine model of the current block. For example, it may be derived for the center position of the current block (eg, x=w / 2 and y=h / 2).

[0366] In some embodiments, MV′ may be the MV for any sub-block of the current block in MC, such as one of the center sub-blocks ( Figure 3 C0, C1, C2 or C3 shown in ).

[0367] In some embodiments, if MV SB If the constraint is not satisfied, MV SB In some embodiments, the MV is clipped to the valid range. SB Stored in the MV buffer, the cropped MV SB Will be used to predict the MV of the subsequent coding block. In some embodiments, in the MV SBIt is stored in the MV buffer before being cropped.

[0368] In some embodiments, if MV SB If the constraints are not met, the bitstream is considered non-compliant with the standard (invalid). In one example, the MV SB A constraint must or should be satisfied. Any conforming encoder shall follow the constraint, otherwise the encoder will be considered non-conforming.

[0369] In some embodiments, the MV may be represented with signaling MV precision (such as quarter-pixel precision) SB and MV'. In some embodiments, MV SB and MV' can be expressed with the stored MV precision (such as 1 / 16 precision). In some embodiments, MV SB and MV' are rounded to a precision different from the signaling or storage precision (such as integer precision).

[0370] Example 2: For affine coded blocks, each MxN (such as 8x4, 4x8 or 8x8) block within the block is considered as a basic unit. The MVs of all 4x4 sub-blocks within the MxN are constrained so that the maximum difference between the integer parts of the four 4x4 sub-block MVs does not exceed K pixels.

[0371] In some embodiments, whether and how to apply the constraint depends on whether the current block applies bidirectional prediction or unidirectional prediction. For example, the constraint only applies to bidirectional prediction and not to unidirectional prediction. As another example, M, N, and K are different for bidirectional prediction and unidirectional prediction.

[0372] In some embodiments, M, N, and K may depend on the width and height of the current block.

[0373] In some embodiments, whether the constraints are applied can be signaled from the encoder to the decoder in, for example, VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU / PU. For example, an on / off flag is signaled to indicate whether the constraints are applied. As another example, M, N, and K are signaled.

[0374] In some embodiments, M, N, and K may be specified differently for different standard profiles / levels / tiers.

[0375] Example 3 : For different affine-coded blocks, the width and height of the sub-block can be calculated in different ways.

[0376] In some embodiments, the calculation method is different for blocks using affine coding of unidirectional prediction and bidirectional prediction. In one example, for blocks using unidirectional prediction, the subblock size is fixed (such as 4×4, 4×8 or 8×4). In another example, the subblock size is calculated for blocks using bidirectional prediction. In this case, the subblock size can be different for two different bidirectional prediction affine blocks.

[0377] In some embodiments, for a bidirectionally predicted affine block, the width and / or height of the sub-block from reference list 0 may be different from the width and / or height of the sub-block from reference list 1. In one example, assume that the width and height of the sub-block from reference list 0 are derived as Wsb0 and Hsb0, respectively; the width and height of the sub-block from reference list 1 are derived as Wsb1 and Hsb1, respectively. Then, for reference list 0 and reference list 1, the final width and height of the sub-block are calculated as Max(Wsb0, Wsb1) and Max(Hsb0, HSb1), respectively.

[0378] In some embodiments, the calculated width and height of the sub-block only applies to the luma component. For the chroma component, it is always fixed, such as a 4×4 chroma sub-block corresponding to an 8×8 luma block using a 4:2:0 color format.

[0379] In some embodiments, MVx-MV'x and MVy-MV'y are calculated to determine the width and height of the sub-block. (MVx, MVy) and (MV'x, MV'y) are defined in Example 1.

[0380] In some embodiments, the MVs involved in the calculations may be represented with signaling MV precision (such as quarter pixel precision). In one example, these MVs may be represented with storage MV precision (such as 1 / 16 precision). As another example, these MVs may be rounded to a different precision than the signaling or storage precision (such as integer precision).

[0381] In some embodiments, the thresholds used in the calculation to determine the width and height of the sub-blocks may be signaled from the encoder to the decoder in, for example, VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU / PU.

[0382] In some embodiments, the thresholds used in the calculation to determine the width and height of a sub-block may be different for different standard profiles / levels / tiers.

[0383] Example 4:To interpolate a W1xH1 sub-block within a W2xH2 sub-block / block, first extract the (W2+N–1–PW)*(H2+N–1–PH) block, then apply the pixel filling method described in Example 6 (e.g., the boundary pixel repetition method) to generate a larger block, which is then used to interpolate the W1xH1 sub-block. For example, W2=H2=8, W1=H1=4, and PW=PH=0.

[0384] In some embodiments, the integer part of the MV of any W1xH1 sub-block can be used to extract the entire W2xH2 sub-block / block, and therefore different boundary pixel repetition methods may be required. For example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 1 pixel, the integer part of the MV of the upper left W1xH1 sub-block is used to extract the entire W2xH2 sub-block / block. The right and lower boundaries of the reference block are repeated once. As another example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 1 pixel, the integer part of the MV of the lower right W1xH1 sub-block is used to extract the entire W2xH2 sub-block / block. The left and upper boundaries of the reference block are repeated once.

[0385] In some embodiments, the MV of any W1xH1 sub-block may be modified first and then used to extract the entire W2xH2 sub-block / block, and therefore different boundary pixel repetition methods may be required. For example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 2 pixels, the integer part of the MV of the upper left W1xH1 sub-block may be added to (1, 1) (where 1 represents 1 integer pixel distance) and then used to extract the entire W2xH2 sub-block / block. In this case, the left, right, upper, and lower boundaries of the reference block are repeated once. As another example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 2 pixels, the integer part of the MV of the lower right W1xH1 sub-block may be added to (-1, -1) (where 1 represents 1 integer pixel distance) and then used to extract the entire W2xH2 sub-block / block. In this case, the left, right, upper, and lower boundaries of the reference block are repeated once.

[0386] Bandwidth control for specific block sizes

[0387] Example 5: If w and h of the current block satisfy one or more of the following conditions, bidirectional prediction is not allowed.

[0388] Aw is equal to T1 and h is equal to T2, or h is equal to T1 and w is equal to T2. In one example, T1=4 and T2=16.

[0389] Bw is equal to T1 and h is not greater than T2, or h is equal to T1 and w is not greater than T2. ​​In one example, T1=4 and T2=16.

[0390] Cw is not greater than T1 and h is not greater than T2, or h is not greater than T1 and w is not greater than T2. ​​In one example, T1=8 and T2=8. In another example, T1==8, T2==4. In yet another example, T1==4 and T2==4.

[0391] In some embodiments, bidirectional prediction may be disabled for 4x8 blocks. In some embodiments, bidirectional prediction may be disabled for 8x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16 blocks. In some embodiments, bidirectional prediction may be disabled for 16x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x8, 8x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16, 16x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x8, 16x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16, 8x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16, 8x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4xN blocks, for example, N≤16. In some embodiments, bidirectional prediction may be disabled for Nx4 blocks, for example, N≤16. In some embodiments, bidirectional prediction may be disabled for 8xN blocks, for example, N<=16. In some embodiments, bidirectional prediction may be disabled for Nx8 blocks, for example, N<=16. In some embodiments, bidirectional prediction may be disabled for 4x8, 8x4, 4x16 blocks. In some embodiments, bidirectional prediction may be disabled for 4x8, 8x4, 16x4 blocks. In some embodiments, bi-prediction may be disabled for 8x4, 4x16, 16x4 blocks. In some embodiments, bi-prediction may be disabled for 4x8, 8x4, 4x16, 16x4 blocks.

[0392] In some embodiments, the block sizes disclosed herein may refer to one color component, such as a luma component, and the decision on whether to disable bidirectional prediction may apply to all color components. For example, if bidirectional prediction is disabled based on the block size of the luma component of a block, bidirectional prediction will also be disabled for corresponding blocks of other color components. In some embodiments, the block sizes disclosed herein may refer to one color component, such as a luma component, and the decision on whether to disable bidirectional prediction applies only to that color component.

[0393] In some embodiments, if bi-prediction is disabled for a block, and the selected Merge candidate is bi-predictive, only one MV from reference list 0 or reference list 1 of the Merge candidate is assigned to the block.

[0394] In some embodiments, if bi-prediction is disabled for a block, triangular prediction mode (TPM) is not allowed for the block.

[0395] In some embodiments, how the prediction direction (unidirectional prediction from list 0 / 1, bidirectional prediction) is signaled may depend on the block size. In one example, when 1) block width * block height < 64 or 2) block width * block height = 64 but width is not equal to height, an indication of unidirectional prediction from list 0 / 1 may be signaled. As another example, when 1) block width * block height > 64 or 2) block width * block height = 64 and width is equal to height, an indication of bidirectional prediction or unidirectional prediction from list 0 / 1 may be signaled.

[0396] In some embodiments, for 4×4 blocks, both unidirectional prediction and bidirectional prediction can be disabled. In some embodiments, it can be disabled for affine-coded blocks. Alternatively, it can be disabled for non-affine-coded blocks. In some embodiments, the indication of quadtree segmentation for 8x8 blocks, binary tree segmentation for 8x4 or 4x8 blocks, and ternary tree segmentation for 4x16 or 16x4 blocks can be skipped. In some embodiments, 4×4 blocks must be encoded as intra blocks. In some embodiments, the MV of a 4×4 block must be integer precision. For example, the IMV flag of a 4x4 block must be 1. As another example, the MV of a 4x4 block must be rounded to integer precision.

[0397] In some embodiments, bidirectional prediction is allowed. However, assuming that the tap of the interpolation filter is N, only (W+N–1–PW)*(W+N–1–PH) reference pixels are extracted instead of (W+N–1)*(H+N–1) reference pixels. At the same time, the pixels at the reference block boundaries (top, left, bottom, and right boundaries) are repeated to generate the following Fig. 9 The (W+N–1)*(H+N–1) block for the final interpolation is shown. In some embodiments, PH is zero and only the left or / and right boundary is repeated. In some embodiments, PW is zero and only the upper or / and lower boundary is repeated. In some embodiments, both PW and PH are greater than zero and the left or / and right boundary is repeated first, followed by the upper or / and lower boundary. In some embodiments, both PW and PH are greater than zero and the upper or / and lower boundary is repeated first, followed by the left and right boundaries. In some embodiments, the left boundary is repeated M1 times and the right boundary is repeated PW–M1 times. In some embodiments, the upper boundary is repeated M2 times and the lower boundary is repeated PH–M2 times. In some embodiments, this boundary pixel repetition method may be applicable to some or all reference blocks. In some embodiments, PW and PH may be different for different color components such as Y, Cb, and Cr.

[0398] Fig. 9 An example of repeating boundary pixels of a reference block before interpolation is shown.

[0399] Example 6: In some embodiments, (W+N–1–PW)*(W+N–1–PH) reference pixels (instead of (W+N–1)*(H+N–1) reference pixels) may be extracted for motion compensation of the WxH block. Samples outside the range of (W+N–1–PW)*(W+N–1–PH) but within the range of (W+N–1)*(H+N–1) are padded for the interpolation process. In one padding method, pixels at the reference block boundaries (top, left, bottom, and right boundaries) are repeated to generate the following: Fig.11 The (W+N–1)*(H+N–1) blocks used for the final interpolation are shown.

[0400] In some embodiments, the PH is zero and only the left or / and right borders are repeated.

[0401] In some embodiments, the PW is zero, and only the upper or / and lower boundaries are repeated.

[0402] In some embodiments, both PW and PH are greater than zero, and the left or / and right boundary is repeated first, and then the upper or / and lower boundary is repeated.

[0403] In some embodiments, both PW and PH are greater than zero, and the upper and / or lower boundaries are repeated first, followed by the left and right boundaries.

[0404] In some embodiments, the left border is repeated M1 times, and the right border is repeated PW-M1 times.

[0405] In some embodiments, the upper boundary is repeated M2 times, and the lower boundary is repeated PH-M2 times.

[0406] In some embodiments, this boundary pixel repetition method may be applied to some or all reference blocks.

[0407] In some embodiments, PW and PH may be different for different color components such as Y, Cb, and Cr.

[0408] In some embodiments, PW and PH may be different for different block sizes or shapes.

[0409] In some embodiments, PW and PH may be different for unidirectional prediction and bidirectional prediction.

[0410] In some embodiments, no padding may be performed in affine mode.

[0411] In some embodiments, samples outside the range (W+N–1–PW)*(W+N–1–PH) but within the range (W+N–1)*(H+N–1) are set to a single value. In some embodiments, the single value is 1<<(BD-1), where BD is the bit depth of the sample, such as 8 or 10. In some embodiments, the single value is signaled from the encoder to the decoder in the VPS / SPS / PPS / slice header / slice group header / slice / CTU row / CTU / CU / PU. In some embodiments, the single value is derived from samples in the range (W+N–1–PW)*(W+N–1–PH).

[0412] Example 7: In DMVR, instead of extracting (W+filterSize–1)*(H+filterSize–1) reference samples, (W+filterSize–1–PW)*(H+filterSize–1–PH) reference samples may be extracted, and all other required samples may be generated by repeating the boundaries of the extracted reference samples, where PW>=0 and PH>=0.

[0413] In some embodiments, the method proposed in Example 6 can be used to fill in the unextracted sample points.

[0414] In some embodiments, in the final motion compensation of DMVR, padding may no longer be performed.

[0415] In some embodiments, whether to apply the above method may depend on the block size.

[0416] Example 8: The signaling method of inter_pred_idc may depend on whether w and h satisfy the conditions in Example 5. An example is shown in Table 3 below:

[0417] Table 3

[0418]

[0419] Another example is shown in Table 4 below:

[0420] Table 4

[0421]

[0422] Another example is shown in Table 5 below:

[0423] Table 5

[0424]

[0425] Example 9:The Merge candidate list construction process may depend on whether w and h satisfy the conditions in Example 4. The following embodiment describes the case when w and h satisfy the conditions.

[0426] In some embodiments, if a Merge candidate uses bidirectional prediction, only the prediction from reference list 0 is retained, and the Merge candidate is considered as a unidirectional prediction referring to reference list 0.

[0427] In some embodiments, if a Merge candidate uses bidirectional prediction, only the prediction from reference list 1 is retained, and the Merge candidate is considered as a unidirectional prediction referring to reference list 1.

[0428] In some embodiments, if a Merge candidate uses bidirectional prediction, the candidate is considered unavailable, that is, such a Merge candidate is removed from the Merge list.

[0429] In some embodiments, the Merge candidate list construction process for triangular prediction mode is used instead.

[0430] Example 10: The coding tree splitting process may depend on whether the width and height of the sub-CU after the splitting meet the conditions in Example 5.

[0431] In some embodiments, if the width and height of the sub-CU after the split meet the conditions in Example 5, the split is not allowed. In some embodiments, the signaling of the coding tree split may depend on whether a split is allowed. In one example, if a split is not allowed, the codeword representing the split is omitted.

[0432] Example 11: Signaling of the skip flag and / or the intra-block copy (IBC) flag may depend on whether the width and / or height of the block satisfies certain conditions (eg, the conditions described in Example 5).

[0433] In some embodiments, the condition is that the luma block contains no more than X samples. For example, X=16;

[0434] In some embodiments, the condition is that the luma block contains X samples, for example X=16.

[0435] In some embodiments, the condition is that the width and height of the luminance block are both equal to X. For example, X=4;

[0436] In some embodiments, when one or some of the above conditions are true, inter mode and / or IBC mode are not allowed for such blocks.

[0437] In some embodiments, if inter mode is not allowed for a block, no skip flag may be signaled for it. Alternatively, furthermore, the skip flag may be inferred to be false.

[0438] In some embodiments, if inter mode and IBC mode are not allowed for a block, the skip flag may not be signaled for it and the skip flag may be implicitly derived as false (eg, the block is derived as encoded in non-skipped mode).

[0439] In some embodiments, if inter mode is not allowed for a block but IBC mode is allowed for the block, the skip flag may still be signaled. In some embodiments, if the block is encoded in skip mode, the IBC flag may not be signaled and the IBC flag may be implicitly derived as true (e.g., the block is derived as encoded in IBC mode).

[0440] Example 12: The signaling of the prediction mode may depend on whether the width and / or height of the block satisfies certain conditions (eg, the conditions described in Example 5).

[0441] In some embodiments, the condition is that the luma block contains no more than X samples, eg X=16.

[0442] In some embodiments, the condition is that the luma block contains X samples, eg, X=16.

[0443] In some embodiments, the condition is that the width and height of the luminance block are both equal to X, for example X=4.

[0444] In some embodiments, when one or some of the above conditions are true, inter mode and / or IBC mode are not allowed for such blocks.

[0445] In some embodiments, signaling of indications of certain modes may be skipped.

[0446] In some embodiments, if inter mode and IBC mode are not allowed for a block, signaling of indication of inter and IBC modes may be skipped, remaining allowed modes may still be signaled, such as intra mode or palette mode.

[0447] In some embodiments, if inter mode and IBC mode are not allowed for a block, the prediction mode may not be signaled. Alternatively, furthermore, the prediction mode may be implicitly derived as intra mode.

[0448] In some embodiments, if inter-mode is not allowed for a block, the signaling of the indication of inter-mode is skipped, and the remaining allowed modes can still be signaled, such as intra-mode or IBC mode. Alternatively, the remaining allowed modes can still be signaled, such as intra-mode or IBC mode or palette mode.

[0449] In some embodiments, if inter mode is not allowed for a block but IBC mode and intra mode are allowed, an IBC flag may be signaled to indicate whether the block is encoded in IBC mode.Alternatively, furthermore, the prediction mode may not be signaled.

[0450] Example 13: Signaling of the triangular pattern may depend on whether the width and / or height of the block satisfies certain conditions (eg, the conditions described in Example 5).

[0451] In some embodiments, the condition is that the luma block size is one of certain specific sizes. For example, the specific size may include 4x16 or / and 16x4.

[0452] In some embodiments, when the above condition is true, the triangular mode may be disabled and the flag indicating whether the current block is encoded in the triangular mode may not be signaled and may be derived as false.

[0453] Example 14: The signaling of the inter prediction direction may depend on whether the width and / or height of the block satisfies certain conditions (eg, the conditions described in Example 5).

[0454] In some embodiments, the condition is that the luma block size is one of certain specific sizes. For example, the specific sizes may include 8x4 or / and 4x8 or / and 4x16 or / and 16x4.

[0455] In some embodiments, when the above condition is true, the block may be only unidirectionally predicted, and a flag indicating whether the current block is bidirectionally predicted may not be signaled and may be derived as false.

[0456] Example 15: The signaling of the SMVD (Symmetrical MVD) flag may depend on whether the width and / or height of the block meets certain conditions (eg, the conditions described in Example 5).

[0457] In some embodiments, the condition is that the luma block size is one of certain specific sizes. In some embodiments, the condition is defined as whether the block size has no more than 32 samples. In some embodiments, the condition is defined as whether the block size is 4×8 or 8×4. In some embodiments, the condition is defined as whether the block size is 4×4, 4×8 or 8×4. In some embodiments, the specific sizes may include 8x4 or / and 4x8 or / and 4x16 or / and 16x4.

[0458] In some embodiments, when certain conditions are true, an indication of the use of SMVD (such as an SMVD flag) may not be signaled and may be derived as false.For example, a block may be set to be unidirectionally predicted.

[0459] In some embodiments, when certain conditions are true, an indication of the use of SMVD may still be signaled (such as an SMVD flag), however, only list 0 or list 1 motion information may be used in the motion compensation process.

[0460] Example 16: The motion vector used for IBC (such as the motion vector derived in regular Merge mode, ATMVP mode, MMVD Merge mode, MMVD Skip mode, etc.) or the block vector may be modified depending on whether the width and / or height of the block meets certain conditions.

[0461] In some embodiments, the condition is that the luma block size is one of some specific sizes. For example, the specific sizes may include 8x4 or / and 4x8 or / and 4x16 or / and 16x4.

[0462] In some embodiments, when the above conditions are true, if the derived motion information is bidirectionally predicted (e.g., inherited from a neighboring block with some offset), the motion vector of the block or the block vector may be changed to a unidirectional motion vector. Such a process is called a conversion process, and the resulting unidirectional motion vector is called a "converted unidirectional" motion vector. In some embodiments, the motion information of reference picture list X (e.g., X is 0 or 1) may be retained, and the motion information of list Y (Y is 1-X) may be discarded. In some embodiments, the motion information of reference picture list X (e.g., X is 0 or 1) and the motion information of list Y (Y is 1-X) may be jointly utilized to derive a new motion candidate point for list X. In one example, the motion vector of the new motion candidate may be the average motion vector of the two reference picture lists. As another example, the motion information of list Y may first be scaled to list X. Then, the motion vector of the new motion candidate may be the average motion vector of the two reference picture lists. In some embodiments, the motion vector in the prediction direction X may not be used (e.g., the motion vector in the prediction direction X is changed to (0, 0), and the reference index in the prediction direction X is changed to -1), and the prediction direction may be changed to 1-X, where X=0 or 1. In some embodiments, the converted unidirectional motion vector may be used to update the HMVP lookup table. In some embodiments, the derived bidirectional motion information, such as the bidirectional MV before conversion to the unidirectional MV, may be used to update the HMVP lookup table. In some embodiments, the converted unidirectional motion vector may be stored and may be used for motion prediction, TMVP, deblocking, etc. of a subsequently coded block. In some embodiments, the derived bidirectional motion information, such as the bidirectional MV before conversion to the unidirectional MV, may be stored and may be used for motion prediction, TMVP, deblocking, etc. of a subsequently coded block. In some embodiments, the converted unidirectional motion vector may be used for motion refinement. In some embodiments, the derived bidirectional motion information may be used for motion refinement and / or sample refinement, such as using an optical flow method. In some embodiments, the prediction blocks generated according to the derived bi-directional motion information may be refined first, and then only one prediction block may be utilized to derive a final prediction and / or reconstructed block for one block.

[0463] In some embodiments, when certain conditions are true, a (bi-directionally predicted) motion vector may be converted to a uni-directional motion vector before being used as a base Merge candidate in MMVD.

[0464] In some embodiments, when certain conditions are true (eg, the size of the block satisfies the conditions as specified in Example 5 above), the (bi-directionally predicted) motion vectors may be converted to uni-directional motion vectors before being inserted into the Merge list.

[0465] In some embodiments, the converted unidirectional motion vector may only come from reference list 0. In some embodiments, when the current slice / slice group / picture is bi-directionally predicted, the converted unidirectional motion vector may come from reference list 0 or list 1. In some embodiments, when the current slice / slice group / picture is bi-directionally predicted, the converted unidirectional motion vectors from reference list 0 and list 1 may be interleaved in the Merge list or / and the MMVD basic Merge candidate list.

[0466] In some embodiments, how the motion information is converted to a unidirectional motion vector may depend on the reference pictures. In some embodiments, if all reference pictures of one video data unit (such as a slice / slice group) are past pictures in display order, list 1 motion information may be utilized. In some embodiments, if at least one of the reference pictures of one video data unit (such as a slice / slice group) is a past picture and at least one is a future picture in display order, list 0 motion information may be utilized. In some embodiments, how the motion information is converted to a unidirectional motion vector may depend on a low latency check flag.

[0467] In some embodiments, the conversion process may be called immediately before the motion compensation process. In some embodiments, the conversion process may be called immediately after the motion candidate list (e.g., Merge list) construction process. In some embodiments, the conversion process may be called before the add MVD process in the MMVD process is called. That is, the add MVD process follows the design of unidirectional prediction rather than bidirectional prediction. In some embodiments, the conversion process may be called before the sample refinement process in the PROF process is called. That is, the add MVD process follows the design of unidirectional prediction rather than bidirectional prediction. In some embodiments, the conversion process may be called before the BIO (also called BDOF) process is called. That is, in some cases, BIO may be disabled because it has been converted to unidirectional prediction. In some embodiments, the conversion process may be called before the DMVR process is called. That is, in some cases, DMVR may be disabled because it has been converted to unidirectional prediction.

[0468] Example 17: In some embodiments, how the motion candidate list is generated may depend on the block size, for example, as described in Example 5 above.

[0469] In some embodiments, for certain block sizes, all motion candidates derived from spatial blocks and / or temporal blocks and / or HMVP and / or other kinds of motion candidates may be restricted to be unidirectionally predicted.

[0470] In some embodiments, for certain block sizes, if a motion candidate derived from a spatial block and / or a temporal block and / or an HMVP and / or other types of motion candidates is bidirectionally predicted, it may first be converted to unidirectionally predicted before being added to the candidate list.

[0471] Example 18: Whether sharing of Merge lists is allowed may depend on the encoding mode.

[0472] In some embodiments, for blocks encoded in the normal Merge mode, sharing of Merge lists may not be allowed, while for blocks encoded in the IBC mode, sharing of Merge lists may be allowed.

[0473] In some embodiments, when a block split from a parent shared node is encoded in a normal Merge mode, the update of the HMVP table may be disabled after encoding / decoding the block.

[0474] Example 19: In the examples disclosed above, the block size / width / height of the luma block may also be changed to the block size / width / height of the chroma block, such as Cb, Cr or G / B / R.

[0475] GBi mode line buffer reduction

[0476] Example 20 : Whether the GBi weighted index (including CABAC context selection) can be inherited or predicted from neighboring blocks depends on the position of the current block.

[0477] In some embodiments, the GBi weighted index cannot be inherited or predicted from neighboring blocks that are not in the same coding tree unit (CTU, also called largest coding unit LCU) as the current block.

[0478] In some embodiments, the GBi weight index cannot be inherited or predicted from neighboring blocks that are not in the same CTU line or CTU row as the current block.

[0479] In some embodiments, the GBi weighted index cannot be inherited or predicted from neighboring blocks in the MxN region that is no longer the current block. For example, M = N = 64. In this case, the slice / strip / picture is divided into multiple non-overlapping MxN regions.

[0480] In some embodiments, the GBi weighted index cannot be inherited or predicted from a neighboring block that is not in the same M×N region line or M×N region row as the current block. For example, M=N=64. Fig.10 CTU lines / rows and region lines / rows are depicted in FIG.

[0481] In some embodiments, assuming that the upper left corner (or any other position) of the current block is (x, y) and the upper left corner (or any other position) of the neighboring block is (x', y'), it cannot be inherited or predicted from the neighboring block if one of the following conditions is met.

[0482] (1) x / M!=x' / M. For example, M=128 or 64.

[0483] (2) y / N!=y' / N. For example, N=128 or 64.

[0484] (3) ((x / M!=x' / M)&&(y / N!=y' / N)). For example, M=N=128 or M=N=64.

[0485] (4) ((x / M!=x' / M)||(y / N!=y' / N)) For example, M=N=128 or M=N=64.

[0486] (5) x>>M!=x'>>M. For example, M=7 or 6.

[0487] (6) y>>N!=y'>>N. For example, N=7 or 6.

[0488] (7) ((x>>M!=x'>>M)&&(y>>N!=y'>>N)). For example, M=N=7 or M=N=6.

[0489] (8) ((x>>M!=x'>>M)||(y>>N!=y'>>N)). For example, M=N=7 or M=N=6.

[0490] In some embodiments, a flag is signaled in the PPS or slice header or slice group header or slice to indicate whether GBi can be applied in a picture / slice / slice group / slice. In some embodiments, whether to use GBi and how to use GBi (such as how many candidate weights and the value of the weights) can be derived for a picture / slice / slice group / slice. In some embodiments, the derivation can depend on information such as QP, temporal layer, POC distance, etc.

[0491] Fig.10 An example of a CTU (region) row is shown. A shaded CTU (region) is in one CTU (region) row and an unshaded CTU (region) is in another CTU (region) row.

[0492] Simplification of Inter-frame and Intra-frame Prediction (IIP)

[0493] Example 21: The intra prediction mode in an IIP-coded block is coded independently of the intra prediction mode of the neighboring IIP-coded blocks.

[0494] In some embodiments, such as during the MPM list construction process, only intra prediction modes of intra coded blocks may be used in the encoding of intra prediction modes of IIP coded blocks.

[0495] In some embodiments, the intra prediction modes in the IIP coding block are encoded without mode prediction from any neighboring blocks.

[0496] Example 22: When the intra prediction mode of the IIP coding block and the intra prediction mode of the intra coding block are both used to encode the intra prediction mode of the new IIP coding block, the priority of the intra prediction mode of the IIP coding block may be lower than the intra prediction mode of the intra coding block.

[0497] In some embodiments, when deriving the MPM of the IIP coded block, the intra prediction modes of the IIP coded block and the intra-coded neighboring blocks are utilized. However, the intra prediction modes from the intra-coded neighboring blocks can be inserted into the MPM list before the intra prediction modes from the IIP coded neighboring blocks.

[0498] In some embodiments, the intra prediction modes from the intra-coded neighboring blocks may be inserted into the MPM list after the intra prediction modes from the IIP-coded neighboring blocks.

[0499] Example 23: The intra prediction mode in the IIP coding block can also be used to predict the intra prediction mode of the intra coding block.

[0500] In some embodiments, the intra prediction mode in the IIP coding block can be used to derive the MPM of the normal intra coding block. In some embodiments, when the intra prediction mode of the IIP coding block and the intra prediction mode of the intra coding block are both used to derive the MPM of the normal intra coding block, the priority of the intra prediction mode of the IIP coding block can be lower than the intra prediction mode of the intra coding block.

[0501] In some embodiments, the intra prediction mode in the IIP coding block may also be used to predict the intra prediction mode of the normal intra coding block or the IIP coding block only when one or more of the following conditions are met:

[0502] 1. Both blocks are in the same CTU row.

[0503] 2. Both blocks are in the same CTU.

[0504] 3. The two blocks are in the same M×N area, such as M=N=64.

[0505] 4. Both blocks are in the same M×N region row, such as M=N=64.

[0506] Example 24: In some embodiments, the MPM construction process for IIP coded blocks should be the same as the MPM construction process for normal intra coded blocks.

[0507] In some embodiments, six MPMs are used for inter-coded blocks using inter-intra prediction.

[0508] In some embodiments, only some of the MPMs are used for the IIP coding blocks. In some embodiments, the first MPM is always used. Alternatively, furthermore, the MPM flag and the MPM index do not need to be signaled. In some embodiments, the first four MPMs may be utilized. Alternatively, furthermore, the MPM flag does not need to be signaled, but the MPM index needs to be signaled.

[0509] In some embodiments, each block may select one from the MPM list according to the intra prediction modes included in the MPM list, such as selecting a mode with a smallest index compared to a given mode (eg, planar).

[0510] In some embodiments, each block may select a subset of modes from the MPM list and signal the mode index in the subset.

[0511] In some embodiments, the context used to encode the intra MPM mode is reused for encoding the intra mode in the IIP coding block. In some embodiments, a different context used to encode the intra MPM mode is used for encoding the intra mode in the IIP coding block.

[0512] Example 25: In some embodiments, for angular intra prediction modes other than horizontal and vertical directions, equal weights are used for intra prediction blocks and inter prediction blocks generated for the IIP coding block.

[0513] Example 26: In some embodiments, for certain positions, zero weights may be applied in the IIP encoding process.

[0514] In some embodiments, zero weights may be applied to intra-prediction blocks used in the IIP encoding process.

[0515] In some embodiments, zero weights may be applied to inter-prediction blocks used in the IIP encoding process.

[0516] Example 27: In some embodiments, no matter what the size of the current block is, the intra prediction mode of the IIP coding block can only be selected as one of the MPMs.

[0517] In some embodiments, the MPM flag is not signaled and is inferred to be 1, regardless of the size of the current block.

[0518] Example 28: For IIP coding blocks, the chroma components are intra-predicted using the luma-predicted chroma mode (LM) mode instead of the derived mode (DM) mode.

[0519] In some embodiments, both DM and LM may be allowed.

[0520] In some embodiments, multiple intra-prediction modes may be allowed for chroma components.

[0521] In some embodiments, whether multiple modes are allowed for chroma components may depend on the color format. In one example, for a 4:4:4 color format, the allowed chroma intra prediction modes may be the same as the chroma intra prediction modes for luma components.

[0522] Example 29 : In one or more of the following specific cases, inter-frame and intra-frame prediction may not be allowed:

[0523] Aw==T1||h==T1, for example, T1=4

[0524] Bw>T1||h>T1, for example, T1=64

[0525] C.(w==T1&&h==T2)||(w==T2&&h==T1), for example, T1=4, T2=16.

[0526] Example 30: For blocks using bi-directional prediction, inter-frame and intra-frame prediction may not be allowed.

[0527] In some embodiments, if the selected Merge candidate for the IIP coding block uses bidirectional prediction, it will be converted to a unidirectional prediction Merge candidate. In some embodiments, only the prediction from reference list 0 is retained, and the Merge candidate is considered to be a unidirectional prediction referencing reference list 0. In some embodiments, only the prediction from reference list 1 is retained, and the Merge candidate is considered to be a unidirectional prediction referencing reference list 1.

[0528] In some embodiments, a restriction is added that the selected Merge candidate should be a uni-prediction Merge candidate. Alternatively, the signaled Merge index of the IIP coding block indicates the index of the uni-prediction Merge candidate (ie, bi-prediction Merge candidates are not counted).

[0529] In some embodiments, the Merge candidate list construction process used in triangular prediction mode may be used to derive the motion candidate list for the IIP coding block.

[0530] Example 31: When inter-frame and intra-frame prediction are applied, some coding tools may not be allowed.

[0531] In some embodiments, bidirectional optical flow (BIO) is not applied for bidirectional prediction.

[0532] In some embodiments, overlapped block motion compensation (OBMC) is not applied.

[0533] In some embodiments, the decoder side motion vector derivation / refinement process is not allowed.

[0534] Example 32: The intra prediction process for inter-frame intra prediction may be different from the intra prediction process for normal intra-coded blocks.

[0535] In some embodiments, the neighboring samples may be filtered in different ways. In some embodiments, the neighboring samples are not filtered before performing intra prediction for inter-frame intra-frame prediction.

[0536] In some embodiments, position-dependent intra prediction sample filtering is not performed for intra prediction used for inter-frame intra prediction. In some embodiments, multi-line intra prediction is not allowed in inter-frame intra prediction. In some embodiments, wide-angle intra prediction is not allowed in inter-frame intra prediction.

[0537] Example 33: Assume that the intra and inter prediction values ​​in the hybrid intra and inter prediction are PIntra and Pinter, and the weighting factors are w_intra and w_inter, respectively. The prediction value of the position (x, y) is calculated as (PIntra(x, y)*w_intra(x, y)+PInter(x, y)*w_inter(x, y)+offset(x, y))>>N, where w_intra(x, y)+w_iner(x, y)=2^N, and offset(x, y)=2^(N–1). In one example, N=3.

[0538] Example 34: In some embodiments, the MPM flag signaled in a normal intra coded block and in an IIP coded block should share the same arithmetic coding context.

[0539] Example 35: In some embodiments, no MPM is needed to encode intra prediction modes in an IIP coding block (assuming the width and height of the block are w and h).

[0540] In some embodiments, the four modes {planar, DC, vertical, horizontal} are binarized as 00, 01, 10, and 11 (any mapping rule such as 00-planar, 01-DC, 10-vertical, 11-horizontal may be used).

[0541] In some embodiments, the four modes {planar, DC, vertical, horizontal} are binarized into 0, 10, 110 and 111 (any mapping rule such as 0-planar, 10-DC, 110-vertical, 111-horizontal can be used).

[0542] In some embodiments, the four modes {planar, DC, vertical, horizontal} are binarized as 1, 01, 001, and 000 (any mapping rule such as 1-planar, 01-DC, 001-vertical, 000-horizontal may be used).

[0543] In some embodiments, when W>N*H can be used (N is an integer such as 2), only three modes {planar, DC, vertical} can be used. The three modes are binarized into 1, 01, 11 (any mapping rules such as 1-planar, 01-DC, 11-vertical can be used).

[0544] In some embodiments, when W>N*H can be used (N is an integer such as 2), only three modes {planar, DC, vertical} can be used. The three modes are binarized into 0, 10, 00 (any mapping rule such as 0-planar, 10-DC, 00-vertical can be used).

[0545] In some embodiments, when H>N*W can be used (N is an integer such as 2), only three modes {plane, DC, horizontal} can be used. The three modes are binarized into 1, 01, 11 (any mapping rules such as 1-plane, 01-DC, 11-horizontal can be used).

[0546] In some embodiments, when H>N*W (N is an integer such as 2) can be used, only three modes {plane, DC, horizontal} can be used. The three modes are binarized into 0, 10, 00 (any mapping rule such as 0-plane, 10-DC, 00-horizontal can be used).

[0547] Example 36: In some embodiments, only DC and planar modes are used in the IIP coding block. In some embodiments, a flag is signaled to indicate whether DC or planar is used.

[0548] Example 37: In some embodiments, IIP is performed differently for different color components.

[0549] In some embodiments, chroma components (such as Cb and Cr) are not inter-frame or intra-frame predicted.

[0550] In some embodiments, in the IIP coding block, the intra prediction mode for the chrominance component is different from the intra prediction mode for the luma component. In some embodiments, the DC mode is always used for chrominance. In some embodiments, the planar mode is always used for chrominance. In some embodiments, the LM mode is always used for chrominance.

[0551] In some embodiments, how different color components are IIPed may depend on the color format (such as 4:2:0 or 4:4:4).

[0552] In some embodiments, how to perform IIP on different color components may depend on the block size. For example, when the width or height of the current block is equal to or less than 4, no inter-frame or intra-frame prediction is performed on chrominance components (such as Cb and Cr).

[0553] MV accuracy issue

[0554] In the following discussion, the precision of the MV stored for spatial motion prediction is denoted as P1, and the precision of the MV stored for temporal motion prediction is denoted as P2.

[0555] Example 38: P1 and P2 may be the same, or they may be different.

[0556] In some embodiments, P1 is a 1 / 16 brightness pixel and P2 is a 1 / 4 brightness pixel. In some embodiments, P1 is a 1 / 16 brightness pixel and P2 is a 1 / 8 brightness pixel. In some embodiments, P1 is a 1 / 8 brightness pixel and P2 is a 1 / 4 brightness pixel. In some embodiments, P1 is a 1 / 8 brightness pixel and P2 is a 1 / 8 brightness pixel. In some embodiments, P2 is a 1 / 16 brightness pixel and P1 is a 1 / 4 brightness pixel. In some embodiments, P2 is a 1 / 16 brightness pixel and P1 is a 1 / 8 brightness pixel. In some embodiments, P2 is a 1 / 8 brightness pixel and P1 is a 1 / 4 brightness pixel.

[0557] Example 39: P1 and P2 may not be fixed. In some embodiments, P1 / P2 may be different for different standard profiles / levels / tiers. In some embodiments, P1 / P2 may be different for pictures in different temporal layers. In some embodiments, P1 / P2 may be different for pictures with different widths / heights. In some embodiments, P1 / P2 may be signaled from the encoder to the decoder in the VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU.

[0558] Example 40: For MV(MVx,MVy), the precision of MVx and MVy can be different, denoted as Px and Py.

[0559] In some embodiments, Px / Py may be different for different standard profiles / levels / tiers. In some embodiments, Px / Py may be different for pictures in different temporal layers. In some embodiments, Px may be different for pictures with different widths. In some embodiments, Py may be different for pictures with different heights. In some embodiments, Px / Py may be signaled from the encoder to the decoder in the VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU.

[0560] Example 41: Before putting the MVs (MVx, MVy) into storage for temporal motion prediction, they should be converted to the correct precision.

[0561] In some embodiments, if P1>=P2, then MVx=Shift(MVx,P1-P2), MVy=Shift(MVy,P1-P2). In some embodiments, if P1>=P2, then MVx=SignShift(MVx,P1-P2), MVy=SignShift(MVy,P1-P2). In some embodiments, if P1<P2, then MVx=MVx<<(P2-P1), MVy=MVy<<(P2-P1).

[0562] Example 42: Assume that the precision of MV(MVx, MVy) is Px and Py, and MVx or MVy is stored by an integer with N bits. The range of MV(MVx, MVy) is MinX<=MVx<=MaxX, and MinY<=MVy<=MaxY.

[0563] In some embodiments, MinX may be equal to MinY, or may not be equal to MinY. In some embodiments, MaxX may be equal to MaxY, or may not be equal to MaxY. In some embodiments, {MinX, MaxX} may depend on Px. In some embodiments, {MinY, MaxY} may depend on Py. In some embodiments, {MinX, MaxX, MinY, MaxY} may depend on N. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for MVs stored for spatial motion prediction and MVs stored for temporal motion prediction. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for different standard profiles / levels / tiers. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for pictures in different temporal layers. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for pictures with different widths / heights. In some embodiments, {MinX, MaxX, MinY, MaxY} may be signaled from the encoder to the decoder in the VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU. In some embodiments, {MinX, MaxX} may be different for pictures with different widths. In some embodiments, {MinY, MaxY} may be different for pictures with different heights. In some embodiments, MVx is cropped to [MinX, MaxX] before being placed in storage for spatial motion prediction. In some embodiments, MVx is cropped to [MinX, MaxX] before being placed in storage for temporal motion prediction. In some embodiments, MVy is cropped to [MinY, MaxY] before being placed in storage for spatial motion prediction. In some embodiments, MVy is cropped to [MinY, MaxY] before being placed in storage for temporal motion prediction.

[0564] Affine Merge Mode Row Buffer Reduction

[0565] Example 43: The affine model (derived CPMV or affine parameters) inherited by an affine merge candidate from neighboring blocks is always a 6-parameter affine model.

[0566] In some embodiments, if the neighboring block is encoded using a 4-parameter affine model, the affine model is still inherited as a 6-parameter affine model.

[0567] In some embodiments, whether the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model or a 4-parameter affine model may depend on the position of the current block. In some embodiments, if the neighboring block is not in the same coding tree unit (CTU, also called the largest coding unit LCU) as the current block, the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model. In some embodiments, if the neighboring block is not in the same CTU line or CTU row as the current block, the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model. In some embodiments, if the neighboring block is not in the M×N region of the current block, the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model. For example, M=N=64. In this case, the slice / strip / picture is divided into multiple non-overlapping MxN regions. In some embodiments, if the neighboring block is not in the same M×N region line or M×N region row as the current block, the 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model. For example, M=N=64. In Fig.10 CTU lines / rows and region lines / rows are depicted in FIG.

[0568] In some embodiments, assuming that the upper left corner (or any other position) of the current block is (x, y), and the upper left corner (or any other position) of the neighboring block is (x', y'), if the neighboring block satisfies one or more of the following conditions, the 4-parameter affine model of the neighboring block is inherited as a 6-parameter affine model:

[0569] (a) x / M! = x' / M. For example, M = 128 or 64.

[0570] (b) y / N!=y' / N. For example, N=128 or 64.

[0571] (c) ((x / M!=x' / M)&& (y / N!=y' / N)). For example, M=N=128 or M=N=64.

[0572] (d) ((x / M!=x' / M)||(y / N!=y' / N)) For example, M=N=128 or M=N=64.

[0573] (e) x>>M!=x'>>M. For example, M=7 or 6.

[0574] (f) y>>N!=y'>>N. For example, N=7 or 6.

[0575] (g) ((x>>M!=x'>>M)&&(y>>N!=y'>>N)). For example, M=N=7 or M=N=6.

[0576] (h)((x>>M!=x'>>M)||(y>>N!=y'>>N)). For example, M=N=7 or M=N=6.

[0577] 5. Examples

[0578] The following description shows an example of how the disclosed technology can be implemented within the syntax structure of the current VVC standard. New additions are shown in bold, and deleted content is shown in italics.

[0579] 5.1 Embodiment #1 (4x4 inter prediction disabled, and bidirectional prediction disabled for 4x8, 8x4, 4x16, and 16x4 blocks)

[0580] 7.3.6.6 Code unit syntax

[0581]

[0582]

[0583]

[0584] 7.4.7.6 Code unit semantics

[0585] pred_mode_flag equal to 0 specifies that the current coding unit is encoded in inter prediction mode. pred_mode_flag equal to 1 indicates that the current coding unit is encoded in intra prediction mode. For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, the variable CuPredMode[x][y] is derived as follows:

[0586] – If pred_mode_flag is equal to 0, set CuPredMode[x][y] equal to MODE_INTER.

[0587] – Otherwise (pred_mode_flag is equal to 1), set CuPredMode[x][y] equal to MODE_INTRA.

[0588] When pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4; and it is inferred to be equal to 0 when decoding a P or B slice group.

[0589] pred_mode_ibc_flag equal to 1 specifies that the current coding unit is encoded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not encoded in IBC prediction mode.

[0590] When pred_mode_ibc_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 and encoded in skip mode; it is inferred to be equal to 0 when decoding a P or B slice group.

[0591] When pred_mode_ibc_flag is equal to 1, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the variable CuPredMode[x][y] is set equal to MODE_IBC.

[0592] inter_pred_idc[x0][y0] specifies whether list0, list1 or bi-directional prediction is used for the current coding unit according to Tables 7-9. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered coding block relative to the top left luma sample of the picture.

[0593] Table 7-9 – Association with inter prediction mode names

[0594]

[0595] When inter_pred_idc[x0][y0] is not present, it is inferred to be equal to PRED_L0.

[0596] 8.5.2.1 Overview

[0597] The inputs to this process are:

[0598] – The luminance position (xCb, yCb) of the upper left sample of the current luminance coding block relative to the upper left luminance sample of the current picture,

[0599] – The variable cbWidth specifies the width of the current coding block in luminance samples,

[0600] – The variable cbHeight specifies the height of the current coding block in luma samples.

[0601] The output of this process is:

[0602] – Luma motion vectors mvL0[0][0] and mvL1[0][0] with a precision of 1 / 16 fractional samples,

[0603] – reference indices refIdxL0 and refIdxL1,

[0604] – The prediction list uses flags predFlagL0[0][0] and predFlagL1[0][0],

[0605] – Bidirectional prediction weight index gbiIdx.

[0606] Let variable LX be the RefPicList[X] of the current picture, where X is 0 or 1.

[0607] For the derivation of the variables mvL0[0][0] and mvL1[0][0], refIdxL0 and refIdxL1, and predFlagL0[0][0] and predFlagL1[0][0], the following applies:

[0608] – If merge_flag[xCb][yCb] is equal to 1, the luma position (xCb, yCb), variables cbWidth and cbHeight are used as input to call the luma motion vector derivation process for Merge mode specified in Section 8.5.2.2, with the output being luma motion vectors mvL0[0][0], mvL1[0][0], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0] and bidirectional prediction weight index gbiIdx.

[0609] – Otherwise, the following applies:

[0610] – For the case where predFlagLX[0][0], mvLX[0][0] and refIdxLX in PRED_LX, and where X is replaced by 0 or 1 in the variable PRED_LX, and in the syntax elements ref_idx_lX and MvdLX, the following sequential steps apply:

[0611] 1. The variables refIdxLX and predFlagLX[0][0] are derived as follows:

[0612] - if inter_pred_idc[xCb][yCb] is equal to PRED_LX or PRED_BI,

[0613] refIdxLX=ref_idx_lX[xCb][yCb] (8-266)

[0614] predFlagLX[0][0]=1 (8-267)

[0615] - Otherwise, assign variables refIdxLX and predFlagLX[0][0] to:

[0616] refIdxLX=-1 (8-268)

[0617] predFlagLX[0][0]=0 (8-269)

[0618] 2. The variable mvdLX is derived as follows:

[0619] mvdLX[0]=MvdLX[xCb][yCb][0] (8-270)

[0620] mvdLX[1]=MvdLX[xCb][yCb][1] (8-271)

[0621] 3. When predFlagLX[0][0] is equal to 1, the luma coding block position (xCb, yCb), coding block width cbWidth, coding block height cbHeight and variable refIdxLX are used as input to call the derivation process of luma motion vector prediction in Section 8.5.2.8, and the output is mvpLX.

[0622] 4. When predFlagLX[0][0] is equal to 1, the luminance motion vector mvLX[0][0] is derived as follows:

[0623] uLX[0]=(mvpLX[0]+mvdLX[0]+2 18 )%2 18 (8-272)

[0624] mvLX[0][0][0]=(uLX[0]>=2 17 )? (uLX[0]-2 18 ):

[0625] uLX[0] (8-273)

[0626] uLX[1]=(mvpLX[1]+mvdLX[1]+2 18 )%2 18 (8-274)

[0627] mvLX[0][0][1]=(uLX[1]>=2 17 )? (uLX[1]-2 18 ):

[0628] uLX[1] (8-275)

[0629] Note 1 – As mentioned above, the resulting values ​​of mvLX[0][0][0] and mvLX[0][0][1] will always be between -2 17 to 2 17 –1 (inclusive).

[0630] – Set the bidirectional prediction weight index gbiIdx equal to gbi_idx[xCb][yCb].

[0631] refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0 when all of the following conditions are true:

[0632] –predFlagL0[0][0] is equal to 1.

[0633] –predFlagL1[0][0] is equal to 1.

[0634] ––(cbWidth+cbHeight==8)||(cbWidth+cbHeight==12)||(cbWidth+cbHeight==20)

[0635] –cbWidth is equal to 4; cbHeight is equal to 4.

[0636] The update process of the history-based motion vector predictor list specified in Section 8.5.2.16 is called with the luma motion vectors mvL0[0][0] and mvL1[0][0], the reference indices refIdxL0 and refIdxL1, the prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0], and the bidirectional prediction weight index gbiIdx.

[0637] 9.5.3.8 Binarization process of inter_pred_idc

[0638] The input to this process is a binarized request for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

[0639] The output of this process is the binarization of the syntax elements.

[0640] The binarization of the syntax element inter_pred_idc is specified in Table 9-9.

[0641] Table 9-9 – Binarization of inter_pred_idc

[0642]

[0643] 9.5.4.2.1 Overview

[0644] Table 9-10 – Assignment of ctxInc to syntax elements with context coded bins

[0645]

[0646] 5.2 Example #2 (disable 4x4 inter prediction)

[0647] 7.3.6.6 Code unit syntax

[0648]

[0649]

[0650] 7.4.7.6 Code unit semantics

[0651] pred_mode_flag equal to 0 specifies that the current coding unit is encoded in inter prediction mode. pred_mode_flag equal to 1 indicates that the current coding unit is encoded in intra prediction mode. For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, the variable CuPredMode[x][y] is derived as follows:

[0652] – If pred_mode_flag is equal to 0, set CuPredMode[x][y] equal to MODE_INTER.

[0653] – Otherwise (pred_mode_flag is equal to 1), set CuPredMode[x][y] equal to MODE_INTRA.

[0654] When pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4; and it is inferred to be equal to 0 when decoding a P or B slice group.

[0655] pred_mode_ibc_flag equal to 1 specifies that the current coding unit is encoded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not encoded in IBC prediction mode.

[0656] When pred_mode_ibc_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 and encoded in skip mode; it is inferred to be equal to 0 when decoding a P or B slice group.

[0657] When pred_mode_ibc_flag is equal to 1, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the variable CuPredMode[x][y] is set equal to MODE_IBC.

[0658] 5.3 Example #3 (disabling bidirectional prediction for 4x8, 8x4, 4x16 and 16x4 blocks)

[0659] 7.4.7.6 Code unit semantics

[0660] inter_pred_idc[x0][y0] specifies whether list0, list1 or bi-directional prediction is used for the current coding unit according to Tables 7-9. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered coding block relative to the top left luma sample of the picture.

[0661] Table 7-9 – Association with inter prediction mode names

[0662]

[0663] When inter_pred_idc[x0][y0] is not present, it is inferred to be equal to PRED_L0.

[0664] 8.5.2.1 Overview

[0665] The inputs to this process are:

[0666] – The luminance position (xCb, yCb) of the upper left sample of the current luminance coding block relative to the upper left luminance sample of the current picture,

[0667] – The variable cbWidth specifies the width of the current coding block in luminance samples,

[0668] – The variable cbHeight specifies the height of the current coding block in luma samples.

[0669] The output of this process is:

[0670] – Luma motion vectors mvL0[0][0] and mvL1[0][0] with a precision of 1 / 16 fractional samples,

[0671] – reference indices refIdxL0 and refIdxL1,

[0672] – The prediction list uses flags predFlagL0[0][0] and predFlagL1[0][0],

[0673] – Bidirectional prediction weight index gbiIdx.

[0674] Let variable LX be the RefPicList[X] of the current picture, where X is 0 or 1.

[0675] For the derivation of the variables mvL0[0][0] and mvL1[0][0], refIdxL0 and refIdxL1, and predFlagL0[0][0] and predFlagL1[0][0], the following applies:

[0676] – If merge_flag[xCb][yCb] is equal to 1, the luma position (xCb, yCb), variables cbWidth and cbHeight are used as input to call the luma motion vector derivation process for Merge mode specified in Section 8.5.2.2, with the output being luma motion vectors mvL0[0][0], mvL1[0][0], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0] and bidirectional prediction weight index gbiIdx.

[0677] – Otherwise, the following applies:

[0678] – For the case where predFlagLX[0][0], mvLX[0][0] and refIdxLX in PRED_LX, and where X is replaced by 0 or 1 in the variable PRED_LX, and in the syntax elements ref_idx_lX and MvdLX, the following sequential steps apply:

[0679] 5. The variables refIdxLX and predFlagLX[0][0] are derived as follows:

[0680] - if inter_pred_idc[xCb][yCb] is equal to PRED_LX or PRED_BI,

[0681] refIdxLX=ref_idx_lX[xCb][yCb] (8-266)

[0682] predFlagLX[0][0]=1 (8-267)

[0683] - Otherwise, assign variables refIdxLX and predFlagLX[0][0] to:

[0684] refIdxLX=-1 (8-268)

[0685] predFlagLX[0][0]=0 (8-269)

[0686] 6. The variable mvdLX is derived as follows:

[0687] mvdLX[0]=MvdLX[xCb][yCb][0] (8-270)

[0688] mvdLX[1]=MvdLX[xCb][yCb][1] (8-271)

[0689] 7. When predFlagLX[0][0] is equal to 1, the luma coding block position (xCb, yCb), coding block width cbWidth, coding block height cbHeight and variable refIdxLX are used as input to call the derivation process of luma motion vector prediction in Section 8.5.2.8, and the output is mvpLX.

[0690] 8. When predFlagLX[0][0] is equal to 1, the luminance motion vector mvLX[0][0] is derived as follows:

[0691] uLX[0]=(mvpLX[0]+mvdLX[0]+2 18 )%2 18 (8-272)

[0692] mvLX[0][0][0]=(uLX[0]>=2 17 )? (uLX[0]-2 18 ):

[0693] uLX[0] (8-273)

[0694] uLX[1]=(mvpLX[1]+mvdLX[1]+2 18 )%2 18 (8-274)

[0695] mvLX[0][0][1]=(uLX[1]>=2 17 )? (uLX[1]-2 18 ):

[0696] uLX[1] (8-275)

[0697] Note 1 – As mentioned above, the resulting values ​​of mvLX[0][0][0] and mvLX[0][0][1] will always be between -2 17 to 2 17 -1 (inclusive).

[0698] – Set the bidirectional prediction weight index gbiIdx equal to gbi_idx[xCb][yCb].

[0699] refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0 when all of the following conditions are true:

[0700] –predFlagL0[0][0] is equal to 1.

[0701] –predFlagL1[0][0] is equal to 1.

[0702] ––(cbWidth+cbHeight==8)||(cbWidth+cbHeight==12)||(cbWidth+cbHeight==20)

[0703] –cbWidth is equal to 4; cbHeight is equal to 4.

[0704] The update process of the history-based motion vector predictor list specified in Section 8.5.2.16 is called with the luma motion vectors mvL0[0][0] and mvL1[0][0], the reference indices refIdxL0 and refIdxL1, the prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0], and the bidirectional prediction weight index gbiIdx.

[0705] 9.5.3.8 Binarization process of inter_pred_idc

[0706] The input to this process is a binarized request for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

[0707] The output of this process is the binarization of the syntax elements.

[0708] The binarization of the syntax element inter_pred_idc is specified in Table 9-9.

[0709] Table 9-9 – Binarization of inter_pred_idc

[0710]

[0711] 9.5.4.2.1 Overview

[0712] Table 9-10 – Assignment of ctxInc to syntax elements with context coded bins

[0713]

[0714] Example #4 (disable 4x4 inter prediction and disable bidirectional prediction for 4x8, 8x4 blocks)

[0715] 7.3.6.6 Code unit syntax

[0716]

[0717]

[0718] 7.4.7.6 Code unit semantics

[0719] pred_mode_flag equal to 0 specifies that the current coding unit is encoded in inter prediction mode. pred_mode_flag equal to 1 indicates that the current coding unit is encoded in intra prediction mode. For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, the variable CuPredMode[x][y] is derived as follows:

[0720] – If pred_mode_flag is equal to 0, set CuPredMode[x][y] equal to MODE_INTER.

[0721] – Otherwise (pred_mode_flag is equal to 1), set CuPredMode[x][y] equal to MODE_INTRA.

[0722] When pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4; and it is inferred to be equal to 0 when decoding a P or B slice group.

[0723] pred_mode_ibc_flag equal to 1 specifies that the current coding unit is encoded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not encoded in IBC prediction mode.

[0724] When pred_mode_ibc_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 and encoded in skip mode; it is inferred to be equal to 0 when decoding a P or B slice group.

[0725] When pred_mode_ibc_flag is equal to 1, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the variable CuPredMode[x][y] is set equal to MODE_IBC.

[0726] inter_pred_idc[x0][y0] specifies whether list0, list1 or bi-directional prediction is used for the current coding unit according to Tables 7-9. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered coding block relative to the top left luma sample of the picture.

[0727] Table 7-9 – Association with inter prediction mode names

[0728]

[0729] When inter_pred_idc[x0][y0] is not present, it is inferred to be equal to PRED_L0.

[0730] 8.5.2.1 Overview

[0731] The inputs to this process are:

[0732] – The luminance position (xCb, yCb) of the upper left sample of the current luminance coding block relative to the upper left luminance sample of the current picture,

[0733] – The variable cbWidth specifies the width of the current coding block in luminance samples,

[0734] – The variable cbHeight specifies the height of the current coding block in luma samples.

[0735] The output of this process is:

[0736] – Luma motion vectors mvL0[0][0] and mvL1[0][0] with a precision of 1 / 16 fractional samples,

[0737] – reference indices refIdxL0 and refIdxL1,

[0738] – The prediction list uses flags predFlagL0[0][0] and predFlagL1[0][0],

[0739] – Bidirectional prediction weight index gbiIdx.

[0740] Let variable LX be the RefPicList[X] of the current picture, where X is 0 or 1.

[0741] For the derivation of the variables mvL0[0][0] and mvL1[0][0], refIdxL0 and refIdxL1, and predFlagL0[0][0] and predFlagL1[0][0], the following applies:

[0742] – If merge_flag[xCb][yCb] is equal to 1, the luma position (xCb, yCb), variables cbWidth and cbHeight are used as input to call the luma motion vector derivation process for Merge mode specified in Section 8.5.2.2, with the output being luma motion vectors mvL0[0][0], mvL1[0][0], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0] and bidirectional prediction weight index gbiIdx.

[0743] – Otherwise, the following applies:

[0744] – For the case where predFlagLX[0][0], mvLX[0][0] and refIdxLX in PRED_LX, and where X is replaced by 0 or 1 in the variable PRED_LX, and in the syntax elements ref_idx_lX and MvdLX, the following sequential steps apply:

[0745] 1. The variables refIdxLX and predFlagLX[0][0] are derived as follows:

[0746] - if inter_pred_idc[xCb][yCb] is equal to PRED_LX or PRED_BI,

[0747] refIdxLX=ref_idx_lX[xCb][yCb] (8-266)

[0748] predFlagLX[0][0]=1 (8-267)

[0749] - Otherwise, assign variables refIdxLX and predFlagLX[0][0] to:

[0750] refIdxLX=-1 (8-268)

[0751] predFlagLX[0][0]=0 (8-269)

[0752] 2. The variable mvdLX is derived as follows:

[0753] mvdLX[0]=MvdLX[xCb][yCb][0] (8-270)

[0754] mvdLX[1]=MvdLX[xCb][yCb][1] (8-271)

[0755] 3. When predFlagLX[0][0] is equal to 1, the luma coding block position (xCb, yCb), coding block width cbWidth, coding block height cbHeight and variable refIdxLX are used as input to call the derivation process of luma motion vector prediction in Section 8.5.2.8, and the output is mvpLX.

[0756] 4. When predFlagLX[0][0] is equal to 1, the luminance motion vector mvLX[0][0] is derived as follows:

[0757] uLX[0]=(mvpLX[0]+mvdLX[0]+2 18 )%2 18 (8-272)

[0758] mvLX[0][0][0]=(uLX[0]>=2 17 )? (uLX[0]-2 18 ):

[0759] uLX[0] (8-273)

[0760] uLX[1]=(mvpLX[1]+mvdLX[1]+2 18 )%2 18 (8-274)

[0761] mvLX[0][0][1]=(uLX[1]>=2 17 )? (uLX[1]-2 18 ):

[0762] uLX[1] (8-275)

[0763] Note 1 – As mentioned above, the resulting values ​​of mvLX[0][0][0] and mvLX[0][0][1] will always be between -2 17 to 2 17 -1 (inclusive).

[0764] – Set the bidirectional prediction weight index gbiIdx equal to gbi_idx[xCb][yCb].

[0765] refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0 when all of the following conditions are true:

[0766] –predFlagL0[0][0] is equal to 1.

[0767] –predFlagL1[0][0] is equal to 1.

[0768] –(cbWidth+cbHeight==8)||(cbWidth+cbHeight==12) – cbWidth is equal to 4;

[0769] –cbHeight is equal to 4.

[0770] The update process of the history-based motion vector predictor list specified in Section 8.5.2.16 is called with the luma motion vectors mvL0[0][0] and mvL1[0][0], the reference indices refIdxL0 and refIdxL1, the prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0], and the bidirectional prediction weight index gbiIdx.

[0771] 9.5.3.8 Binarization process of inter_pred_idc

[0772] The input to this process is a binarized request for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

[0773] The output of this process is the binarization of the syntax elements.

[0774] The binarization of the syntax element inter_pred_idc is specified in Table 9-9.

[0775] Table 9-9 – Binarization of inter_pred_idc

[0776]

[0777] 9.5.4.2.1 Overview

[0778] Table 9-10 – Assignment of ctxInc to syntax elements with context coded bins

[0779]

[0780] 5.5 Embodiment #5 (disable 4x4 inter-frame prediction, disable bidirectional prediction for 4x8 and 8x4 blocks, and disable the shared Merge list of the normal Merge mode)

[0781] 7.3.6.6 Code unit syntax

[0782]

[0783]

[0784]

[0785] 7.4.7.6 Code unit semantics

[0786] pred_mode_flag equal to 0 specifies that the current coding unit is encoded in inter prediction mode. pred_mode_flag equal to 1 indicates that the current coding unit is encoded in intra prediction mode. For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, the variable CuPredMode[x][y] is derived as follows:

[0787] – If pred_mode_flag is equal to 0, set CuPredMode[x][y] equal to MODE_INTER.

[0788] – Otherwise (pred_mode_flag is equal to 1), set CuPredMode[x][y] equal to MODE_INTRA.

[0789] When pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4; and it is inferred to be equal to 0 when decoding a P or B slice group.

[0790] pred_mode_ibc_flag equal to 1 specifies that the current coding unit is encoded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not encoded in IBC prediction mode.

[0791] When pred_mode_ibc_flag is not present, it is inferred to be equal to 1 when decoding an I slice group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 and encoded in skip mode; it is inferred to be equal to 0 when decoding a P or B slice group.

[0792] When pred_mode_ibc_flag is equal to 1, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the variable CuPredMode[x][y] is set equal to MODE_IBC.

[0793] inter_pred_idc[x0][y0] specifies whether list0, list1 or bi-directional prediction is used for the current coding unit according to Tables 7-9. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered coding block relative to the top left luma sample of the picture.

[0794] Table 7-9 – Association with inter prediction mode names

[0795]

[0796] When inter_pred_idc[x0][y0] is not present, it is inferred to be equal to PRED_L0.

[0797] 8.5.2.1 Overview

[0798] The inputs to this process are:

[0799] – The luminance position (xCb, yCb) of the upper left sample of the current luminance coding block relative to the upper left luminance sample of the current picture,

[0800] – The variable cbWidth specifies the width of the current coding block in luminance samples,

[0801] – The variable cbHeight specifies the height of the current coding block in luma samples.

[0802] The output of this process is:

[0803] – Luma motion vectors mvL0[0][0] and mvL1[0][0] with a precision of 1 / 16 fractional samples,

[0804] – reference indices refIdxL0 and refIdxL1,

[0805] – The prediction list uses flags predFlagL0[0][0] and predFlagL1[0][0],

[0806] – Bidirectional prediction weight index gbiIdx.

[0807] Let variable LX be the RefPicList[X] of the current picture, where X is 0 or 1.

[0808] For the derivation of the variables mvL0[0][0] and mvL1[0][0], refIdxL0 and refIdxL1, and predFlagL0[0][0] and predFlagL1[0][0], the following applies:

[0809] – If merge_flag[xCb][yCb] is equal to 1, the luma position (xCb, yCb), variables cbWidth and cbHeight are used as input to call the luma motion vector derivation process for Merge mode specified in Section 8.5.2.2, with the output being luma motion vectors mvL0[0][0], mvL1[0][0], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0] and bidirectional prediction weight index gbiIdx.

[0810] – Otherwise, the following applies:

[0811] – For the case where predFlagLX[0][0], mvLX[0][0] and refIdxLX in PRED_LX, and where X is replaced by 0 or 1 in the variable PRED_LX, and in the syntax elements ref_idx_lX and MvdLX, the following sequential steps apply:

[0812] 5. The variables refIdxLX and predFlagLX[0][0] are derived as follows:

[0813] - if inter_pred_idc[xCb][yCb] is equal to PRED_LX or PRED_BI,

[0814] refIdxLX=ref_idx_lX[xCb][yCb] (8-266)

[0815] predFlagLX[0][0]=1 (8-267)

[0816] - Otherwise, assign variables refIdxLX and predFlagLX[0][0] to:

[0817] refIdxLX=-1 (8-268)

[0818] predFlagLX[0][0]=0 (8-269)

[0819] 6. The variable mvdLX is derived as follows:

[0820] mvdLX[0]=MvdLX[xCb][yCb][0] (8-270)

[0821] mvdLX[1]=MvdLX[xCb][yCb][1] (8-271)

[0822] 7. When predFlagLX[0][0] is equal to 1, the luma coding block position (xCb, yCb), coding block width cbWidth, coding block height cbHeight and variable refIdxLX are used as input to call the derivation process of luma motion vector prediction in Section 8.5.2.8, and the output is mvpLX.

[0823] 8. When predFlagLX[0][0] is equal to 1, the luminance motion vector mvLX[0][0] is derived as follows:

[0824] uLX[0]=(mvpLX[0]+mvdLX[0]+2 18 )%2 18 (8-272)

[0825] mvLX[0][0][0]=(uLX[0]>=2 17 )? (uLX[0]-2 18 ):

[0826] uLX[0](8-273)

[0827] uLX[1]=(mvpLX[1]+mvdLX[1]+2 18 )%2 18 (8-274)

[0828] mvLX[0][0][1]=(uLX[1]>=2 17 )? (uLX[1]-2 18 ):

[0829] uLX[1](8-275)

[0830] NOTE 1 – As mentioned above, the resulting values ​​of mvLX[0][0][0] and mvLX[0][0][1] will always be between -2 17 to 2 17 –1 (inclusive).

[0831] – Set the bidirectional prediction weight index gbiIdx equal to gbi_idx[xCb][yCb].

[0832] refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0 when all of the following conditions are true:

[0833] –predFlagL0[0][0] is equal to 1.

[0834] –predFlagL1[0][0] is equal to 1.

[0835] ––(cbWidth+cbHeight==8)||(cbWidth+cbHeight==12)–cbWidth is equal to 4;

[0836] –cbHeight is equal to 4.

[0837] The update process of the history-based motion vector predictor list specified in Section 8.5.2.16 is called with the luma motion vectors mvL0[0][0] and mvL1[0][0], the reference indices refIdxL0 and refIdxL1, the prediction list utilization flags predFlagL0[0][0] and predFlagL1[0][0], and the bidirectional prediction weight index gbiIdx.

[0838] 8.5.2.2 Derivation of Luma Motion Vectors in Merge Mode

[0839] This process is called only when merge_flag[xCb][yPb] is equal to 1, where (xCb, yCb) specifies the top left corner sample of the current luma coding block relative to the top left corner luma sample of the current picture.

[0840] The inputs to this process are:

[0841] – The luminance position (xCb, yCb) of the upper left sample of the current luminance coding block relative to the upper left luminance sample of the current picture,

[0842] – The variable cbWidth specifies the width of the current coding block in luminance samples,

[0843] – The variable cbHeight specifies the height of the current coding block in luma samples.

[0844] The output of this process is:

[0845] – Luma motion vectors mvL0[0][0] and mvL1[0][0] with a precision of 1 / 16 fractional samples,

[0846] – reference indices refIdxL0 and refIdxL1,

[0847] – The prediction list uses flags predFlagL0[0][0] and predFlagL1[0][0],

[0848] – Bidirectional prediction weight index gbiIdx.

[0849] Set the bidirectional prediction weight index gbiIdx equal to 0.

[0850] The variables xSmr, ySmr, smrWidth, smrHeight, and smrNumHmvpCand are derived as follows:

[0851] xSmr=IsInSmr[xCb][yCb]? SmrX[xCb][yCb]:xCb (8-276)

[0852] ySmr=IsInSmr[xCb][yCb]? SmrY[xCb][yCb]:yCb (8-277)

[0853] smrWidth=IsInSmr[xCb][yCb]? SmrW[xCb][yCb]:cbWidth (8-278)

[0854] smrHeight=IsInSmr[xCb][yCb]? SmrH[xCb][yCb]:cbHeight (8-279)

[0855] smrNumHmvpCand=IsInSmr[xCb][yCb]? NumHmvpSmrCand:NumHmvpCand(8-280)

[0856] 8.5.2.6 The derivation process of history-based merge candidates

[0857] The inputs to this process are:

[0858] –Merge candidate list mergeCandList,

[0859] – The variable isInSmr specifies whether the current coding unit is within the shared Merge candidate region.

[0860] – The number of available merge candidates in the list numCurrMergeCand.

[0861] The output of this process is:

[0862] – Modified Merge candidate list mergeCandList,

[0863] – The number of modified merge candidates in the list numCurrMergeCand.

[0864] Set both the variables isPrunedA1 and isPrunedB1 equal to FALSE.

[0865] The array smrHmvpCandList and the variable smrNumHmvpCand are derived as follows:

[0866] smrHmvpCandList=isInSmr? HmvpSmrCandList:HmvpCandList (8-353)

[0867] smrNumHmvpCand=isInSmr? NumHmvpSmrCand:NumHmvpCand

[0868] (8-354)

[0869] For each candidate in smrHmvpCandList[hMvpIdx] with index hMvpIdx=1..smrNumHmvpCand, repeat the following ordered steps until numCurrMergeCand equals (MaxNumMergeCand-1):

[0870] 1. The variable sameMotion is derived as follows:

[0871] – If for any Merge candidate N where N is A1 or B1, all of the following conditions are true, then set both sameMotion and isPrunedN equal to TRUE:

[0872] –hMvpIdx is less than or equal to 2.

[0873] –Candidate smrHmvpCandList[smrNumHmvpCand-hMvpIdx] is equal to Merge candidate N.

[0874] –isPrunedN is equal to FALSE.

[0875] – Otherwise, set sameMotion equal to FALSE.

[0876] 2. When sameMotion is equal to FALSE, add the candidate smrHmvpCandList[smrNumHmvpCand-hMvpIdx] to the Merge candidate list as follows:

[0877] mergeCandList[numCurrMergeCand++]=smrHmvpCandList[smrNumHmvpCand-hMvpIdx](8-355)

[0878] 9.5.3.8 Binarization process of inter_pred_idc

[0879] The input to this process is a binarized request for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

[0880] The output of this process is the binarization of the syntax elements.

[0881] The binarization of the syntax element inter_pred_idc is specified in Table 9-9.

[0882] Table 9-9 – Binarization of inter_pred_idc

[0883]

[0884] 9.5.4.2.1 Overview

[0885] Table 9-10 – Assignment of ctxInc to syntax elements with context coded bins

[0886]

[0887] Fig.11 1 is a block diagram of a video processing device 1100. Device 1100 may be used to implement one or more methods described herein. Device 1100 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. Device 1100 may include one or more processors 1102, one or more memories 1104, and video processing hardware 1106. Processor 4302 may be configured to implement one or more methods described herein. Although some embodiments may operate without memory, (one or more) memories 1104 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1106 may be used to implement some of the techniques described in this document in hardware circuits.

[0888] Fig.12 12 is a flow chart of an example method 1200 for video processing. The method 1200 includes determining (1202) a size constraint between a representative motion vector of an affine-encoded current video block and a motion vector of a sub-block of the current video block, and performing (1204) a conversion between a bitstream representation and pixel values ​​of the current video block or sub-block using the size constraint.

[0889] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of the current video block may correspond, for example, to bits that are concatenated or interspersed in different locations within the bitstream, as defined by the syntax. For example, a macroblock may be encoded based on a transformed and encoded error residual value and also using bits in a header and other fields in the bitstream.

[0890] It should be appreciated that the disclosed techniques may be used to implement embodiments where the implementation complexity of video processing is reduced through reduced memory requirements or line buffer size requirements.Some of the presently disclosed techniques may be described using the following clause-based description.

[0891] 1. A video processing method, comprising:

[0892] determining a size limit between a representative motion vector of an affine-encoded current video block and a motion vector of a sub-block of the current video block; and

[0893] By using the size constraints, conversion between the bitstream representation and the pixel values ​​of the current video block or sub-block is performed.

[0894] 2. A method according to clause 1, wherein performing the conversion comprises generating a bitstream representation from the pixel values.

[0895] 3. A method according to clause 1, wherein performing the conversion includes generating pixel values ​​from a bitstream representation.

[0896] 4. A method according to any of clauses 1 to 3, wherein the size restriction comprises constraining the value of the motion vector (MVx, MVy) of the sub-block according to: MVx>=MV'x-DH0 and MVx<=MV'x+DH1 and MVy>=MV'y-DV0 and MVy<=MV'y+DV1, where MV'=(MV'x, MV'y);

[0897] wherein MV′ denotes a representative motion vector; and wherein DH0, DH1, DV0, and DV1 denote positive numbers.

[0898] 5. The method of clause 4, wherein the size restriction comprises at least one of:

[0899] i. DH0 is equal to DH1 or DV0 is equal to DV1.

[0900] ii. DH0 is equal to DV0 or DH1 is equal to DV1.

[0901] iii. DH0 and DH1 are different or DV0 and DV1 are different.

[0902] iv. DH0, DH1, DV0 and DV1 are signaled in the bitstream representation at the video parameter set level, or the sequence parameter set level, or the picture parameter set level, or the slice header level, or the slice group header level, or the slice level, or the coding tree unit level, or the coding unit level, or the prediction unit level.

[0903] v.DH0, DH1, DV0, and DV1 are functions of the video processing mode.

[0904] vi. DH0, DH1, DV0 and DV1 depend on the width and height of the current video block.

[0905] vii. DH0, DH1, DV0 and DV1 depend on whether the current video block is encoded using unidirectional prediction or bidirectional prediction.

[0906] viii. DH0, DH1, DV0 and DV1 depend on the location of the sub-block.

[0907] 6. A method according to any of clauses 1 to 5, wherein the representative motion vector corresponds to a control point motion vector of the current video block.

[0908] 7. A method according to any of clauses 1 to 5, wherein the representative motion vector corresponds to a motion vector of a corner sub-block of the current video block.

[0909] 8. A method according to any of clauses 1 to 7, wherein the precision of the motion vectors and the representative motion vectors for the sub-blocks corresponds to the motion vector signalling precision in the bitstream representation.

[0910] 9. A method according to any of clauses 1 to 7, wherein the precision of the motion vectors and the representative motion vectors for the sub-blocks corresponds to a storage precision used for storing the motion vectors.

[0911] 10. A video processing method, comprising:

[0912] Determining, for the affine-encoded current video block, one or more sub-blocks of the current video block, wherein each sub-block has a size of MxN pixels, where M and N are multiples of 2 or 4;

[0913] Adapting the motion vectors of the sub-blocks to a size constraint; and

[0914] By using size constraints, conversion between the bitstream representation and pixel values ​​of the current video block is performed conditionally based on a trigger.

[0915] 11. A method according to clause 10, wherein performing the conversion comprises generating a bitstream representation from the pixel values.

[0916] 12. The method of clause 10, wherein performing the conversion comprises generating pixel values ​​from a bitstream representation.

[0917] 13. A method according to any of clauses 10 to 12, wherein the size constraint constrains the maximum difference between the integer parts of the sub-block motion vectors of the current video block to be less than or equal to K pixels, where K is an integer.

[0918] 14. A method according to any of clauses 10-13, wherein the method is applied only if the current video block is encoded using bi-directional prediction.

[0919] 15. A method according to any of clauses 10-13, wherein the method is applied only if the current video block is encoded using uni-directional prediction.

[0920] 16. A method according to any of clauses 10-13, wherein the value of M, N or K is a function of a uni-prediction or bi-prediction mode of the current video block.

[0921] 17. A method according to any of clauses 10-13, wherein the value of M, N or K is a function of the height or width of the current video block.

[0922] 18. A method according to any one of clauses 10 to 17, wherein the trigger is included in the bitstream representation at a video parameter set level, or a sequence parameter set level, or a picture parameter set level, or a slice header level, or a slice group header level, or a slice level, or a coding tree unit level, or a coding unit level, or a prediction unit level.

[0923] 19. The method of clause 18, wherein the trigger signals a value of M, N or K.

[0924] 20. A method according to any of clauses 10 to 19, wherein the one or more sub-blocks of the current video block are calculated based on a type of affine encoding used for the current video block.

[0925] 21. A method according to clause 20, wherein two different methods are used to compute sub-blocks of uni-predictive and bi-predictive affine prediction modes.

[0926] 22. The method of clause 21, wherein, in case the current video block is a bi-predicted affine block, the width or height of the sub-blocks from different reference lists are different.

[0927] 23. A method according to any of clauses 20-22, wherein one or more sub-blocks correspond to a luma component.

[0928] 24. A method according to any of clauses 10 to 23, wherein the width and height of one of the one or more sub-blocks are determined using a motion vector difference between a motion vector value of the current video block and a motion vector value of one of the one or more sub-blocks.

[0929] 25. A method according to any of clauses 20 to 23, wherein the calculation is based on a pixel precision signalled in the bitstream representation.

[0930] 26. A video processing method, comprising:

[0931] Determining that the current video block meets a size condition; and

[0932] Based on the determination, a conversion between a bitstream representation and pixel values ​​of a previous video block is performed by excluding a bidirectional predictive coding mode of the current video block.

[0933] 27. A video processing method, comprising:

[0934] Determining that the current video block meets a size condition; and

[0935] Based on the determination, a conversion between a bitstream representation and pixel values ​​of the current video block is performed, wherein the inter prediction mode is signaled in the bitstream representation according to the size condition.

[0936] 28. A video processing method, comprising:

[0937] Determining that the current video block meets a size condition; and

[0938] Based on the determination, a conversion between the bitstream representation and the pixel values ​​of the current video block is performed, wherein generation of a Merge candidate list during the conversion depends on a size condition.

[0939] 29. A video processing method, comprising:

[0940] Determining that a sub-coding unit of the current video block satisfies a size condition; and

[0941] Based on the determination, a conversion between a bitstream representation and pixel values ​​of the current video block is performed, wherein a coding tree partitioning process for generating sub-coding units depends on the size condition.

[0942] 30. A method according to any of clauses 26 to 29, wherein the size condition is one of:

[0943] (a) w is equal to T1 and h is equal to T2, or h is equal to T1 and w is equal to T2;

[0944] (b) w is equal to T1 and h is not greater than T2, or h is equal to T1 and w is not greater than T2;

[0945] (c) w is not greater than T1 and h is not greater than T2, or h is not greater than T1 and w is not greater than T2.

[0946] 31. A method according to clause 30, wherein T1=8 and T2=8 or T1=8, T2=4 or T1=4 and T2=4, or T1=4 and T2=16.

[0947] 32. A method according to any of clauses 26 to 29, wherein the converting comprises generating a bitstream representation from pixel values ​​of the current video block or generating pixel values ​​of the current video block from the bitstream representation.

[0948] 33. A video processing method, comprising:

[0949] Determining a weight index of a generalized bidirectional prediction (GBi) process of the current video block based on the position of the current video block; and

[0950] The conversion between the current video block and its bitstream representation is performed using the weight index to implement the GBi process.

[0951] 34. A method according to clause 33, wherein converting comprises generating a bitstream representation from pixel values ​​of the current video block or generating pixel values ​​of the current video block from the bitstream representation.

[0952] 35. A method according to any one of clauses 33 or 34, wherein determining comprises: for a current video block at a first position, inheriting or predicting another weight index of an adjacent block, and for a current video block at a second position, calculating the GBI without inheriting from an adjacent block.

[0953] 36. The method of clause 35, wherein the second position comprises the current video block being in a different coding tree unit than the neighboring block.

[0954] 37. A method according to clause 35, wherein the second position corresponds to the current video block being in a different coding tree unit line or a different coding tree unit row than the neighbouring block.

[0955] 38. A video processing method, comprising:

[0956] Determining that the current video block is encoded as an intra-frame inter-frame prediction (IIP) encoding block; and

[0957] The conversion between the current video block and its bitstream representation is performed using simplified rules for determining the intra prediction mode or Most Probable Mode (MPM) for the current video block.

[0958] 39. A method according to clause 38, wherein converting comprises generating a bitstream representation from pixel values ​​of the current video block or generating pixel values ​​of the current video block from the bitstream representation.

[0959] 40. The method of any of clauses 38-39, wherein the simplified rule specifies determining an intra prediction coding mode of a current video block that is intra inter prediction (IIP) coded independently of another intra prediction coding mode of a neighboring video block.

[0960] 41. A method according to any of clauses 38 to 39, wherein the intra prediction coding mode is represented in the bitstream representation using coding independent of neighbouring blocks.

[0961] 42. A method according to any of clauses 38 to 40, wherein the simplification rule specifies a preference for coding modes of intra-coded blocks over coding modes of intra-prediction coded blocks.

[0962] 43. The method of clause 38, wherein the simplified rule specifies that the MPM is determined by inserting the intra prediction modes from the intra-coded neighboring blocks before inserting the intra prediction modes from the IIP-coded neighboring blocks.

[0963] 44. A method according to clause 38, wherein the simplified rule specifies that the MPM is determined using the same construction process as used for another normal intra-coded block.

[0964] 45. A video processing method, comprising:

[0965] Determining that the current video block satisfies a simplification criterion; and

[0966] The conversion between the current video block and the bitstream representation is performed by disabling the inter-intra prediction modes used for the conversion or by disabling additional coding tools used for the conversion.

[0967] 46. ​​The method of clause 45, wherein converting comprises generating a bitstream representation from pixel values ​​of the current video block or generating pixel values ​​of the current video block from the bitstream representation.

[0968] 47. A method according to any of clauses 45-46, wherein the simplification criterion comprises that the width or height of the current video block is equal to T1, wherein T1 is an integer.

[0969] 48. A method according to any of clauses 45-46, wherein the simplification criterion comprises that a width or a height of the current video block is greater than T1, wherein T1 is an integer.

[0970] 49. The method of any of clauses 45-46, wherein the simplification criteria comprises that the width of the current video block is equal to T1, and the height of the current video block is equal to T2.

[0971] 48. A method according to any of clauses 45-46, wherein the simplified criterion specifies that the current video block uses a bi-prediction mode.

[0972] 49. A method according to any of clauses 45-46, wherein the additional encoding tool comprises bidirectional optical flow (BIO) encoding.

[0973] 50. A method according to any of clauses 45-46, wherein the additional coding tools comprise an overlapped block motion compensation mode.

[0974] 51. A video processing method, comprising:

[0975] Using a motion vector based encoding process, a conversion between a current video block and a bitstream representation of the current video block is performed, where:

[0976] (a) During the conversion process, precision P1 is used to store the spatial motion prediction results, and precision P2 is used to store the temporal motion prediction results, where P1 and P2 are fractional values, or

[0977] (b) Precision Px is used to store the x motion vector, and precision Py is used to store the y motion vector, where Px and Py are fractional values.

[0978] 52. The method according to clause 51, wherein P1, P2, Px and Py are different numbers.

[0979] 53. A method according to clause 52, wherein:

[0980] P1 is a 1 / 16 brightness pixel, and P2 is a 1 / 4 brightness pixel, or

[0981] P1 is a 1 / 16 brightness pixel, and P2 is a 1 / 8 brightness pixel, or

[0982] P1 is a 1 / 8 brightness pixel, and P2 is a 1 / 4 brightness pixel, or

[0983] P1 is a 1 / 8 brightness pixel, and P2 is a 1 / 8 brightness pixel, or

[0984] P2 is a 1 / 16 brightness pixel, and P1 is a 1 / 4 brightness pixel, or

[0985] P2 is a 1 / 16 brightness pixel, and P1 is a 1 / 8 brightness pixel, or

[0986] P2 is a 1 / 8 brightness pixel, and P1 is a 1 / 4 brightness pixel.

[0987] 54. A method according to clauses 51-52, wherein P1 and P2 are different for different pictures in different temporal layers included in the bitstream representation.

[0988] 55. A method according to clauses 51-52, wherein the calculated motion vectors are processed through a precision correction process before being stored as temporal motion predictions.

[0989] 56. A method according to clauses 51-52, wherein storing comprises storing the x motion vector and the y motion vector as N-bit integers, and wherein the range of values ​​of the x motion vector is [MinX, MaxX] and the range of values ​​of the y motion vector is [MinY, MaxY], wherein the ranges satisfy one or more of the following:

[0990] a.MinX is equal to MinY,

[0991] b.MaxX is equal to MaxY,

[0992] c.{MinX,MaxX} depends on Px;

[0993] d.{MinY,MaxY} depends on Py;

[0994] e.{MinX, MaxX, MinY, MaxY} depends on N.

[0995] f. For an MV stored for spatial motion prediction and another MV stored for temporal motion prediction, {MinX, MaxX, MinY, MaxY} are different.

[0996] g. For pictures in different temporal layers, {MinX, MaxX, MinY, MaxY} are different.

[0997] h. For images with different width or height, {MinX, MaxX, MinY, MaxY} are different.

[0998] i. For images with different widths, {MinX, MaxX} are different.

[0999] j. For images with different heights, {MinY, MaxY} are different.

[1000] k. Clip MVx to [MinX, MaxX] before storing for spatial motion prediction.

[1001] l. Clip MVx to [MinX, MaxX] before storing for temporal motion prediction.

[1002] m. Clip MVy to [MinY, MaxY] before storing for spatial motion prediction.

[1003] n. Clip MVy to [MinY, MaxY] before storing for temporal motion prediction.

[1004] 59. A video processing method, comprising: interpolating a small sub-block of size W1xH1 within a large sub-block of size W2xH2 of a current video block by extracting a (W2+N–1–PW)*(H2+N–1–PH) block, pixel-filling the extracted block, performing boundary pixel repetition on the pixel-filled block, and obtaining pixel values ​​of the small sub-block, wherein W1, W2, H1, H2, PW, and PH are integers; and performing conversion between a current video block and a bitstream representation of the current video block using the interpolated pixel values ​​of the small sub-block.

[1005] 60. The method of clause 59, wherein converting comprises generating the current video block from a bitstream representation or generating a bitstream representation from a current sub-block.

[1006] 61. A method according to any of clauses 59-60, wherein W2=H2=8, W1=H1=4 and PW=PH=0.

[1007] 62. A video processing method, comprising:

[1008] During conversion of the current video block of WxH size and the bitstream representation of the current video block, performing a motion compensation operation by extracting (W+N–1–PW)*(W+N–1–PH) reference pixels during the motion compensation operation and filling reference pixels outside the extracted reference pixels; and

[1009] Conversion between the current video block and a bitstream representation of the current video block is performed using the result of the motion compensation operation, where W, H, N, PW, and PH are integers.

[1010] 63. The method of clause 62, wherein converting comprises generating the current video block from a bitstream representation or generating a bitstream representation from a current sub-block.

[1011] 64. A method according to any of clauses 62-63, wherein padding comprises repeating a left or right border of the extracted pixels.

[1012] 65. A method according to any of clauses 62-63, wherein padding comprises repeating an upper or lower boundary of the acquired pixels.

[1013] 66. A method according to any of clauses 62-63, wherein padding comprises setting pixel values ​​to a constant.

[1014] 67. A method according to clause 38, wherein the rule specifies to use the same arithmetic coding context during conversion as used for other intra-coded blocks.

[1015] 68. The method of clause 38, wherein the transforming of the current video block does not include using an MPM for the current video block.

[1016] 69. The method of clause 38, wherein the simplification rule specifies that only DC and planar modes are to be used for the bitstream representation of the current video block as an IIP coded block.

[1017] 70. The method of clause 38, wherein the simplified rule specifies different intra-prediction modes for luma and chroma components.

[1018] 71. The method of clause 44, wherein a subset of the MPM is used for the current video block being IIP encoded.

[1019] 72. A method according to clause 38, wherein the simplified rule indicates that the MPM is selected based on the intra-prediction modes included in the MPM list.

[1020] 73. A method according to clause 38, wherein the simplification rules indicate that a subset of MPMs is to be selected from the MPM list and a mode index associated with the subset is signalled.

[1021] 74. The method of clause 38, wherein the context used for encoding the intra MPM mode is used for encoding the intra mode of the current video block being IIP encoded.

[1022] 75. The method of clause 44, wherein equal weights are used for intra-prediction blocks and inter-prediction blocks generated for a current video block, the current video block being an IIP coded block.

[1023] 76. The method of clause 44, wherein a zero weight is used for a position in the IIP encoding process of the current video block.

[1024] 77. A method according to clause 77, wherein zero weights are applied to intra-predicted blocks used in the IIP encoding process.

[1025] 78. A method according to clause 77, wherein zero weight is applied to inter-prediction blocks used in the IIP encoding process.

[1026] 79. A video processing method, comprising:

[1027] Based on the size of the current video block, determining not to allow bidirectional prediction or unidirectional prediction of the current video block; and

[1028] Based on the determination, a conversion between a bitstream representation and pixel values ​​of the current video block is performed by disabling a bi-directional prediction or uni-directional prediction mode. For example, the disallowed mode is not used to encode or decode the current video block. The conversion operation may represent video encoding or compression, or video decoding or decompression.

[1029] 80. The method of clause 79, wherein the current video block is 4x8, and the determining comprises determining that bi-prediction is not allowed. Further examples are given in Example 5.

[1030] 81. The method of clause 79, wherein the current video block is 4x8 or 8x4, and determining comprises determining that bi-prediction is not allowed.

[1031] 82. The method of clause 79, wherein the current video block is 4xN, where N is an integer <= 16, and determining comprises determining that bi-prediction is not allowed.

[1032] 83. A method according to any of clauses 26-29 or 79-82, wherein the size of the current block corresponds to the size of a color component or a luma component of the current block.

[1033] 84. The method of clause 83, wherein disabling bi-directional prediction or uni-directional prediction is applicable to all three components of the current video block.

[1034] 85. A method according to clause 83, wherein disabling bi-directional prediction or uni-directional prediction applies only to color components whose size is used as the size of the current block.

[1035] 86. A method according to any one of clauses 79 to 85, wherein the conversion is performed by disabling bi-directional prediction and further using Merge candidates of bi-directional prediction and then assigning only one motion vector from one reference list to the current video block.

[1036] 87. The method of clause 79, wherein the current video block is 4x4, and determining comprises determining to disable bi-directional prediction and uni-directional prediction.

[1037] 88. The method of clause 87, wherein the current video block is encoded as an intra block.

[1038] 89. The method of clause 87, wherein the current video block is restricted to use integer-pixel motion vectors.

[1039] Additional examples and embodiments of clauses 78-89 are described in Example 5.

[1040] 90. A method for processing video, comprising:

[1041] Determining a video encoding condition of the current video block based on a size of the current video block; and

[1042] Based on the video coding condition, a conversion between the current video block and a bitstream representation of the current video block is performed.

[1043] 91. A method according to clause 90, wherein the video coding condition specifies selectively signaling a skip flag or an intra-block coding flag in the bitstream representation.

[1044] 92. A method according to clause 90 or 91, wherein the video coding condition specifies selective signaling of a prediction mode for a current video block.

[1045] 93. A method according to any of clauses 90 to 92, wherein the video encoding condition specifies selectively signalling triangular mode encoding of the current video block.

[1046] 94. A method according to any of clauses 90 to 93, wherein the video coding conditions specify selective signalling of an inter prediction direction for a current video block.

[1047] 95. A method according to any of clauses 90 to 94, wherein the video encoding condition specifies selective modification of a motion vector or a block vector for intra block copying of a current video block.

[1048] 96. A method according to any of clauses 90-95, wherein the video condition depends on the height in pixels of the current video block.

[1049] 97. A method according to any of clauses 90-96, wherein the video condition depends on the width in pixels of the current video block.

[1050] 98. A method according to any of clauses 90-95, wherein the video condition depends on whether the current video block is a square.

[1051] Additional examples of clauses 90 to 98 are provided in items 11 to 16 listed in Section 4 of this document.

[1052] 99. A video encoder apparatus comprising a processor configured to perform the methods of one or more of clauses 1 to 98.

[1053] 100. A video decoder device comprising a processor configured to perform the method of one or more of clauses 1 to 98.

[1054] 101. A computer readable medium having stored thereon code which, when executed by a processor, causes the processor to implement the method of any one or more of clauses 1 to 98.

[1055] Fig.16 1 is a block diagram illustrating an example video processing system 1600 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1600. System 1600 may include an input 1602 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or may be received in a compressed or encoded format. Input 1602 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[1056] System 1600 may include a coding component 1604 that can implement various encodings or encoding methods described herein. Coding component 1604 can reduce the average bit rate of the video from the input 1602 of coding component 1604 to the output to generate a coded representation of the video. Therefore, coding technology is sometimes referred to as video compression or video transcoding technology. As represented by component 1606, the output of coding component 1604 can be stored or sent via the connected communication. The storage or transmitted bitstream (or encoding) representation of the video received at input 1602 can be used by component 1608 to generate pixel values ​​or displayable video sent to display interface 1610. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "coding" operations or tools, it should be understood that coding tools or operations are used at the encoder, and the corresponding decoding tools or operations of the reverse coding results will be performed by the encoder.

[1057] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described herein may be embodied in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[1058] Fig.17 1 is a flowchart representation of a video processing method 1700 according to the present disclosure. The method 1700 includes, at operation 1702, for a conversion between a current block of a video and a bitstream representation of the video using an affine coding tool, determining that a first motion vector of a sub-block of the current block and a second motion vector that is a representative motion vector of the current block comply with a size constraint. The method 1700 also includes, at operation 1704, performing the conversion based on the determination.

[1059] In some embodiments, the first motion vector of the sub-block is denoted as (MVx, MVy), and the second motion vector is denoted as (MV'x, MV'y). The size constraint indicates that MVx>=MV'x-DH0, MVx<=MV'x+DH1, MVy>=MV'y-DV0, and MVy<=MV'y+DV1, and DH0, DH1, DV0, and DV1 are positive numbers. In some embodiments, DH 0=DH 1. In some embodiments, DH0≠DH1. In some embodiments, DV0=DV1. In some embodiments, DV0≠DV0. In some embodiments, DH0=DV0. In some embodiments, DH0≠DV0. In some embodiments, DH1=DV1. In some embodiments, DH1≠DV1.

[1060] In some embodiments, at least one of DH0, DH1, DV0 or DV1 is signaled in the bitstream representation at a video parameter set level, a sequence parameter set level, a picture parameter set level, a slice header, a slice group header, a slice level, a coding tree unit level, a coding unit level or a prediction unit level. In some embodiments, DH0, DH1, DV0 and DV1 are different for different profiles, levels or levels of conversion. In some embodiments, DH0, DH1, DV0 and DV1 are based on the weight or height of the current block. In some embodiments, DH0, DH1, DV0 and DV1 are based on the prediction mode of the current block, which is a unidirectional prediction mode or a bidirectional prediction mode. In some embodiments, DH0, DH1, DV0 and DV1 are based on the position of the sub-block in the current block.

[1061] In some embodiments, the second motion vector comprises a control point motion vector of the current block. In some embodiments, the second motion vector comprises a motion vector of a second sub-block of the current block. In some embodiments, the second sub-block comprises a center sub-block of the current block. In some embodiments, the second sub-block comprises a corner sub-block of the current block. In some embodiments, the second motion vector comprises a motion vector derived from an internal or external position of the current block, the position being encoded using the same affine model as the current block. In some embodiments, the position comprises a center position of the current block.

[1062] In some embodiments, the first motion vector is adjusted to satisfy a size constraint. In some embodiments, the bitstream is invalid if the first motion vector does not satisfy a size constraint on the second motion vector. In some embodiments, the first motion vector and the second motion vector are represented according to a motion vector signaling precision in the bitstream representation. In some embodiments, the first motion vector and the second motion vector are represented according to a storage precision used to store the motion vectors. In some embodiments, the first motion vector and the second motion vector are represented according to a precision that is different from a motion vector signaling precision or a storage precision used to store the motion vectors.

[1063] Fig.18 1 is a flow chart representation of a video processing method 1800 according to the present disclosure. The method 1800 includes, at operation 1802, determining an affine model including six parameters for a conversion between a current block of a video and a bitstream representation of the video. The affine model is inherited from affine encoding information of neighboring blocks of the current block. The method 1800 includes, at operation 1804, performing the conversion based on the affine model.

[1064] In some embodiments, the adjacent block is encoded using a second affine model having six parameters, which is the same as the second affine model. In some embodiments, the adjacent block is encoded using a third affine model having four parameters. In some embodiments, the affine model is determined based on the position of the current block. In some embodiments, when the adjacent block is not in the same coding tree unit (CTU) as the current block, the affine model is determined according to the third affine model. In some embodiments, when the adjacent block is not in the same CTU line or the same CTU row as the current block, the affine model is determined according to the third affine model.

[1065] In some embodiments, the slice, strip or map is divided into a plurality of non-overlapping regions. In some embodiments, the affine model is determined according to a third affine model when the adjacent block is not in the same region as the current block. In some embodiments, the affine model is determined according to a third affine model when the adjacent block is not in the same region line or region row as the current block. In some embodiments, each region has a size of 64×64. In some embodiments, the upper left corner of the current block is represented as (x, y), and the upper left corner of the adjacent block is represented as (x', y'), and wherein, the affine model is determined according to the third affine model when conditions regarding x, y, x' and y' are satisfied. In some embodiments, the condition indicates x / M≠x' / M, and M is a positive integer. In some embodiments, M is 128 or 64. In some embodiments, the condition indicates y / N≠y' / N, and N is a positive integer. In some embodiments, N is 128 or 64. In some embodiments, the condition indicates x / M≠x' / M and y / N≠y' / N, M and N are positive integers. In some embodiments, M=N=128 or M=N=64. In some embodiments, the condition indicates x>>M≠x'>>M, M is a positive integer. In some embodiments, M is 6 or 7. In some embodiments, the condition indicates y>>N≠y'>>N, N is a positive integer. In some embodiments, N is 6 or 7. In some embodiments, the condition indicates x>>M≠x'>>M and y>>N≠y'>>N, M and N are positive integers. In some embodiments, M=N=6 or M=N=7.

[1066] Fig.19 1 is a flowchart representation of a video processing method 1900 according to the present disclosure. The method 1900 includes, at operation 1902, determining whether a bidirectional predictive coding technique is applicable to a block for conversion between a block of video and a bitstream representation of the video based on a block size having a width W and a height H, W and H being positive integers. The method 1900 includes, at operation 1904, performing conversion based on the determination.

[1067] In some embodiments, when W=T1 and H=T2, T1 and T2 are positive integers, the bidirectional predictive coding technique is not applicable. In some embodiments, when W=T2 and H=T1, T1 and T2 are positive integers, the bidirectional predictive coding technique is not applicable. In some embodiments, when W=T1 and H≤T2, T1 and T2 are positive integers, the bidirectional predictive coding technique is not applicable. In some embodiments, when W≤T2 and H=T1, T1 and T2 are positive integers, the bidirectional predictive coding technique is not applicable. In some embodiments, T1=4 and T2=16. In some embodiments, when W≤T1 and H≤T2, T1 and T2 are positive integers, the bidirectional predictive coding technique is not applicable. In some embodiments, T1=T2=8. In some embodiments, T1=8 and T2=4. In some embodiments, T1=T2=4. In some embodiments, T1=4 and T2=8.

[1068] In some embodiments, when the bidirectional predictive coding technique is applicable, an indicator indicating information about the bidirectional predictive coding technique is signaled in the bitstream. In some embodiments, when the bidirectional predictive coding technique is not applicable to a block, an indicator indicating information about the bidirectional predictive coding technique for the block is excluded from the bitstream. In some embodiments, when the block size is one of 4×8 or 8×4, the bidirectional predictive coding technique is not applicable. In some embodiments, when the block size is 4xN or Nx4, N is a positive integer and N≤16, the bidirectional predictive coding technique is not applicable. In some embodiments, the size of the block corresponds to the first color component of the block, and for the first color component and the remaining color components of the block, it is determined whether the bidirectional predictive coding technique is applicable. In some embodiments, the size of the block corresponds to the first color component of the block, and it is determined whether the bidirectional predictive coding technique is applicable only for the first color component. In some embodiments, the first color component includes a luminance component.

[1069] In some embodiments, the method further comprises, when bidirectional prediction coding technology is not applicable to the current block, assigning a single motion vector from the first reference list or the second reference list when determining that the selected Merge candidate is encoded using the bidirectional prediction coding technology. In some embodiments, the method further comprises, when bidirectional prediction coding technology is not applicable to the current block, determining that the triangular prediction mode is not applicable to the block. In some embodiments, whether the bidirectional prediction coding technology is applicable is associated with a prediction direction, which is further associated with the unidirectional prediction coding technology, and wherein the prediction direction is signaled in the bitstream based on the size of the block. In some embodiments, information about the unidirectional prediction coding technology is signaled in the bitstream in the following cases: (1) WxH<64 or (2) WxH=64, W is not equal to H. In some embodiments, information about the unidirectional prediction coding technology is signaled in the bitstream in the following cases: (1) W x H>64 or (2) W x H=64, W is equal to H.

[1070] In some embodiments, the restriction indicates that neither the bidirectional coding technique nor the unidirectional technique is applicable to the block when the size of the block is 4x4. In some embodiments, the restriction applies when the block is affine coded. In some embodiments, the restriction applies when the block is not affine coded. In some embodiments, the restriction applies when the block is intra coded. In some embodiments, the restriction does not apply when the motion vector of the block has integer precision.

[1071] In some embodiments, signaling of blocks generated based on a partition of a parent block is skipped in the bitstream, the size of the parent block being (1) 8x8 for a quadtree partition, (2) 8x4 or 4x8 for a binary tree partition, (3) or 4x16 or 16x4 for a ternary tree partition. In some embodiments, an indicator indicating that the motion vector has integer precision is set to 1 in the bitstream. In some embodiments, the motion vector of the block is rounded to integer precision.

[1072] In some embodiments, the bidirectional predictive coding technique is applied to blocks. The size of the reference block is (W+N-1-PW)x(H+N-1-PH), and the boundary pixels of the reference block are repeated to generate a second block of size (W+N-1)x(H+N-1) for interpolation operation, N represents the interpolation filter tap, and N, PW and PH are integers. In some embodiments, PH=0, and the pixels of at least the left or right boundary are repeated to generate the second block. In some embodiments, PW=0, and the pixels of at least the upper or lower boundary are repeated to generate the second block. In some embodiments, PW>0 and PH>0, and the second block is generated by repeating at least the pixels of the left or right boundary after repeating at least the pixels of the upper or lower boundary. In some embodiments, PW>0 and PH>0, and the second block is generated by repeating at least the pixels of the left or right boundary after repeating at least the pixels of the upper or lower boundary. In some embodiments, the pixels of the left boundary are repeated M1 times, and the pixels of the right boundary are repeated (PW-M1) times. In some embodiments, the pixels of the upper boundary are repeated M2 times, and wherein the pixels of the lower boundary are repeated (PH-M2) times. In some embodiments, for conversion, how the boundary pixels of the reference block are repeated applies to some or all reference blocks. In some embodiments, PW and PH are different for different components of a block.

[1073] In some embodiments, a Merge candidate list construction process is performed based on the size of the block. In some embodiments, a Merge candidate is considered to be a unidirectional prediction candidate that references a first reference list in a unidirectional prediction coding technique in the following cases: (1) the Merge candidate is encoded using a bidirectional prediction coding technique, and (2) bidirectional prediction is not applicable to the block based on the size of the block. In some embodiments, the first reference list includes reference list 0 or reference list 1 of the unidirectional prediction coding technique. In some embodiments, a Merge candidate is considered to be unavailable in the following cases: (1) the Merge candidate is encoded using a bidirectional prediction coding technique, and (2) bidirectional prediction is not applicable to the block based on the size of the block. In some embodiments, in the Merge candidate list construction process, unavailable Merge candidates are deleted from the Merge candidate list. In some embodiments, in the case of a block for which bidirectional prediction is not applicable based on the size of the block, a Merge candidate list construction process of a triangular prediction mode is called.

[1074] Fig. 202 is a flowchart representation of a video processing method 2000 according to the present disclosure. The method 2000 includes, at operation 2002, determining whether a coding tree partitioning process is applicable to a block for conversion between a block of a video and a bitstream representation of the video based on a size of a sub-block, the sub-block being a sub-coding unit of the block according to the coding tree partitioning process. The sub-block has a width W and a height H, W and H being positive integers. The method 2000 also includes, at operation 2004, performing conversion based on the determination.

[1075] In some embodiments, when W=T1 and H=T2, T1 and T2 are positive integers, the coding tree partitioning process is not applicable. In some embodiments, when W=T2 and H=T1, T1 and T2 are positive integers, the coding tree partitioning process is not applicable. In some embodiments, when W=T1 and H≤T2, T1 and T2 are positive integers, the coding tree partitioning process is not applicable. In some embodiments, when W≤T2 and H=T1, T1 and T2 are positive integers, the coding tree partitioning process is not applicable. In some embodiments, T1=4 and T2=16. In some embodiments, when W≤T1 and H≤T2, T1 and T2 are positive integers, the coding tree partitioning process is not applicable. In some embodiments, T1=T2=8. In some embodiments, T1=8 and T2=4. In some embodiments, T1=T2=4. In some embodiments, T1=4. In some embodiments, T2=4. In an embodiment, in the case where the coding tree partitioning process is not applicable to the current block, the signaling of the coding tree partitioning process is omitted from the bitstream.

[1076] Fig.21 21 is a flowchart representation of a video processing method 2100 according to the present disclosure. The method 2100 includes, at operation 2102, for a conversion between a current block of a video and a bitstream representation of the video, determining whether to derive an index for a bidirectional prediction with a coding unit level weight (BCW) coding mode based on a rule regarding the position of the current block. In the BCW coding mode, a weight set including a plurality of weights is used to generate a bidirectional prediction value for the current block. The method 2100 also includes performing the conversion based on the determination at operation 2104.

[1077] In some embodiments, the bidirectional prediction value of the current block is generated as a non-average weighted sum of two motion vectors, with at least one weight in the weight set applied. In some embodiments, the rule specifies that when the current block and the adjacent block are located in different coding tree units or maximum coding units, the index is not derived from the adjacent block. In some embodiments, the rule specifies that when the current block and the adjacent block are located in different lines or rows in the coding tree unit, the index is not derived from the adjacent block. In some embodiments, the rule specifies that when the current block and the adjacent block are located in different non-overlapping areas of a slice, strip, or picture of the video, the index is not derived from the adjacent block. In some embodiments, the rule specifies that when the current block and the adjacent block are located in different rows of non-overlapping areas of a slice, strip, or picture of the video, the index is not derived from the adjacent block. In some embodiments, each area has a size of 64×64.

[1078] In some embodiments, the top angle of the current block is represented as (x, y), and the top angle of the adjacent block is represented as (x', y'). The rule specifies that the index is not derived from the adjacent block when (x, y) and (x', y') meet the condition. In some embodiments, the condition indicates x / M≠x' / M, and M is a positive integer. In some embodiments, M is 128 or 64. In some embodiments, the condition indicates y / N≠y' / N, and N is a positive integer. In some embodiments, N is 128 or 64. In some embodiments, the condition indicates (x / M≠x' / M) and (y / N≠y' / N), and M and N are positive integers. In some embodiments, M=N=128 or M=N=64. In some embodiments, the condition indicates x>>M≠x'>>M, and M is a positive integer. In some embodiments, M is 6 or 7. In some embodiments, the condition indicates y>>N≠y'>>N, and N is a positive integer. In some embodiments, N is 6 or 7. In some embodiments, the conditions indicate (x>>M≠x'>>M) and (y>>N≠y'>>N), M and N are positive integers. In some embodiments, M=N=6 or M=N=7.

[1079] In some embodiments, whether the BCW coding mode is applicable to a picture, a slice, a slice group or a slice is signaled in a bitstream in a picture parameter set, a slice header, a slice group header or a slice, respectively. In some embodiments, whether the BCW coding mode is applicable to a picture, a slice, a slice group or a slice is derived based on information associated with the picture, the slice, the slice group or the slice. In some embodiments, the information includes at least a quantization parameter (QP), a temporal layer or a picture order count distance.

[1080] Fig. 2222 is a flowchart representation of a video processing method 2200 according to the present disclosure. The method 2200 includes, at operation 2202, for a conversion between a current block of a video encoded using a combined inter-frame and intra-frame prediction (CIIP) coding technique and a bitstream representation of the video, determining an intra-frame prediction mode of the current block that is independent of intra-frame prediction modes of neighboring blocks. The CIIP coding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The method 2200 also includes performing the conversion based on the determination at operation 2204.

[1081] In some embodiments, the intra prediction mode of the current block is determined without reference to the intra prediction mode of any neighboring blocks. In some embodiments, the neighboring blocks are encoded using CIIP encoding techniques. In some embodiments, the intra prediction of the current block is determined based on the intra prediction mode of a second neighboring block encoded using intra prediction encoding techniques. In some embodiments, whether to determine the intra prediction mode of the current block according to the second intra prediction mode is based on whether a condition is satisfied, which specifies a relationship between the current block as the first block and the second neighboring block as the second block. In some embodiments, the determination is part of a most probable mode (MPM) construction process for the current block for deriving a list of MPM modes.

[1082] Fig.23 23 is a flowchart representation of a video processing method 2300 according to the present disclosure. The method 2300 includes, at operation 2302, for a conversion between a current block of a video encoded using a combined inter-frame and intra-frame prediction (CIIP) coding technique and a bitstream representation of the video, determining an intra-frame prediction mode of the current block according to a first intra-frame prediction mode of a first adjacent block and a second intra-frame prediction mode of a second adjacent block. The first adjacent block is encoded using an intra-frame prediction coding technique, and the second adjacent block is encoded using a CIIP coding technique. The first intra-frame prediction mode is assigned a different priority than the second intra-frame prediction mode. The CIIP coding technique uses an intermediate inter-frame prediction value and an intermediate intra-frame prediction value to derive a final prediction value of the current block. The method 2300 also includes performing a conversion based on the determination at operation 2304.

[1083] In some embodiments, the determination is part of a most probable mode (MPM) construction process for the current block used to derive a list of MPM modes. In some embodiments, the first intra-frame prediction mode is located before the second intra-frame prediction mode in the MPM candidate list. In some embodiments, the first intra-frame prediction mode is located after the second intra-frame prediction mode in the MPM candidate list. In some embodiments, the encoding of the intra-frame prediction mode bypasses the most probable mode (MPM) construction process of the current block. In some embodiments, the method also includes determining the intra-frame prediction mode of the subsequent block based on the intra-frame prediction mode of the current block, wherein the subsequent block is encoded using an intra-frame prediction encoding technique and the current block is encoded using a CIIP encoding technique. In some embodiments, the determination is part of a most probable mode (MPM) construction process for the subsequent block. In some embodiments, in the MPM construction process for the subsequent block, the intra-frame prediction mode of the current block is given a lower priority than the intra-frame prediction mode of another neighboring block encoded using the intra-frame prediction encoding technique. In some embodiments, whether to determine the intra-frame prediction mode of the subsequent block based on the intra-frame prediction mode of the current block is based on whether a condition is satisfied, which specifies a relationship between the subsequent block as the first block and the current block as the second block. In some embodiments, the condition includes at least one of: (1) the first block and the second block are located in the same row of a coding tree unit (CTU), (2) the first block and the second block are located in the same CTU, (3) the first block and the second block are in the same region, or (4) the first block and the second block are in the same row of a region. In some embodiments, the width of the region is the same as the height of the region. In some embodiments, the region has a size of 64×64.

[1084] In some embodiments, only a subset of the list of most probable modes (MPMs) of normal intra-coding techniques is used for the current block. In some embodiments, the subset includes a single MPM mode in the list of MPM modes of normal intra-coding techniques. In some embodiments, the single MPM mode is the first MPM mode in the list. In some embodiments, the index indicating the single MPM mode is omitted in the bitstream. In some embodiments, the subset includes the first four MPM modes in the MPM mode list. In some embodiments, the index indicating the MPM mode in the subset is signaled in the bitstream. In some embodiments, the coding context used to encode the intra-coded block is reused for encoding the current block. In some embodiments, the first MPM flag of the intra-coded block and the second MPM flag of the current block share the same coding context in the bitstream. In some embodiments, regardless of the size of the current block, the intra-prediction mode of the current block is selected from the MPM mode list. In some embodiments, the MPM construction process is enabled by default, and the flag indicating the MPM construction process is omitted in the bitstream. In some embodiments, the current block does not require an MPM list construction process.

[1085] In some embodiments, a luminance prediction chrominance mode is used to process the chrominance component of the current block. In some embodiments, a derived mode is used to process the chrominance component of the current block. In some embodiments, multiple intra-frame prediction modes are used to process the chrominance component of the current block. In some embodiments, multiple intra-frame prediction modes are used based on the color format of the chrominance component. In some embodiments, when the color format is 4:4:4, the multiple intra-frame prediction modes are the same as the intra-frame prediction mode of the luminance component of the current block. In some embodiments, one or more bits are used to encode each of the four intra-frame prediction modes, which include a plane mode, a DC mode, a vertical mode, and a horizontal mode. In some embodiments, 00, 01, 10, and 11 are used to encode the four intra-frame prediction modes. In some embodiments, 0, 10, 110, and 111 are used to encode the four intra-frame prediction modes. In some embodiments, 1, 01, 001, and 000 are used to encode the four intra-frame prediction modes. In some embodiments, when the width W and height H of the current block meet the conditions, only a subset of the four intra-frame prediction modes can be used. In some embodiments, when W>N×H and N is an integer, the subset includes a plane mode, a DC mode, and a vertical mode. In some embodiments, 1, 01, and 11 are used to encode the plane mode, the DC mode, and the vertical mode. In some embodiments, 0, 10, and 00 are used to encode the plane mode, the DC mode, and the vertical mode. In some embodiments, when H>N×W and N is an integer, the subset includes a plane mode, a DC mode, and a horizontal mode. In some embodiments, 1, 01, and 11 are used to encode the plane mode, the DC mode, and the horizontal mode. In some embodiments, 0, 10, and 00 are used to encode the plane mode, the DC mode, and the horizontal mode. In some embodiments, N=2. In some embodiments, only the DC mode and the plane mode are used for the current block. In some embodiments, an indicator indicating the DC mode or the plane mode is signaled in the bitstream.

[1086] Fig.24 24 is a flowchart representation of a video processing method 2400 according to the present disclosure. The method 2400 includes, at operation 2402, for conversion between a current block of video and a bitstream representation of the video, based on the size of the current block, determining whether a combined inter-frame and intra-frame prediction (CIIP) process is applicable to color components of the current block. The CIIP encoding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The method 2400 also includes performing the conversion based on the determination at operation 2404.

[1087] In some embodiments, the color component includes a chroma component, and wherein the CIIP process is not performed on the chroma component when the width of the current block is less than 4. In some embodiments, the color component includes a chroma component, and wherein the CIIP process is not performed on the chroma component when the height of the current block is less than 4. In some embodiments, the intra-frame prediction mode of the chroma component of the current block is different from the intra-frame prediction mode of the luma component of the current block. In some embodiments, the chroma component uses one of the DC mode, the planar mode, or the luma prediction chroma mode. In some embodiments, the intra-frame prediction mode of the chroma component is determined based on the color format of the chroma component. In some embodiments, the color format includes 4:2:0 or 4:4:4.

[1088] Fig.25 25 is a flowchart representation of a video processing method 2500 according to the present disclosure. The method 2500 includes, at operation 2502, for conversion between a current block of a video and a bitstream representation of the video, based on characteristics of the current block, determining whether to apply a combined inter-frame and intra-frame prediction (CIIP) coding technique to the current block. The CIIP coding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The method 2500 also includes, at operation 2504, performing the conversion based on the determination.

[1089] In some embodiments, the characteristic includes the size of the current block having a width W and a height H, W and H being integers, and if the size of the block satisfies the condition, the inter-frame intra-frame prediction coding technique is disabled for the current block. In some embodiments, the condition indicates that W is equal to T1, and T1 is an integer. In some embodiments, the condition indicates that H is equal to T1, and T1 is an integer. In some embodiments, T1=4. In some embodiments, T1=2. In some embodiments, the condition indicates that W is greater than T1 or H is greater than T1, and T1 is an integer. In some embodiments, T1=64 or 128. In some embodiments, the condition indicates that W is equal to T1 and H is equal to T2, and T1 and T2 are integers. In some embodiments, the condition indicates that W is equal to T2 and H is equal to T1, and T1 and T2 are integers. In some embodiments, T1=4 and T2=16.

[1090] In some embodiments, the characteristic includes a coding technique applied to the current block, and when the coding technique satisfies the condition, the CIIP coding technique is disabled for the current block. In some embodiments, the condition indicates that the coding technique is a bidirectional predictive coding technique. In some embodiments, a bidirectional predictive coding Merge candidate is converted to a unidirectional predictive coding Merge candidate to allow the inter-frame intra-frame predictive coding technique to be applied to the current block. In some embodiments, the converted Merge candidate is associated with a reference list 0 of the unidirectional predictive coding technique. In some embodiments, the converted Merge candidate is associated with a reference list 1 of the unidirectional predictive coding technique. In some embodiments, only the unidirectional predictive coding Merge candidate of the conversion selection block is selected. In some embodiments, the bidirectional predictive coding Merge candidate is discarded to determine the Merge index indicating the Merge candidate in the bitstream representation. In some embodiments, the inter-frame intra-frame predictive coding technique is applied to the current block according to the determination. In some embodiments, the Merge candidate list construction process for the triangular prediction mode is used to derive the motion candidate list of the current block.

[1091] Fig.26 is a flowchart representation of a video processing method 2600 according to the present disclosure. The method 2600 includes, at operation 2602, for a conversion between a current block of a video and a bitstream representation of the video, determining whether to disable coding tools for the current block based on whether the current block is encoded with a combined inter-frame and intra-frame prediction (CIIP) coding technique. The CIIP coding technique uses intermediate inter-frame prediction values ​​and intermediate intra-frame prediction values ​​to derive a final prediction value for the current block. The coding tools include at least one of bidirectional optical flow (BDOF), overlapped block motion compensation (OBMC), or a decoder-side motion vector refinement process (DMVR). The method 2500 also includes performing the conversion based on the determination at operation 2604.

[1092] In some embodiments, the intra prediction process of the current block is different from the intra prediction process of a second block encoded using the intra prediction coding technique. In some embodiments, in the intra prediction process of the current block, filtering of neighboring samples is skipped. In some embodiments, in the intra prediction process of the current block, position-dependent filtering of intra prediction samples is disabled. In some embodiments, in the intra prediction process of the current block, multi-line intra prediction process is disabled. In some embodiments, in the intra prediction process of the current block, wide-angle intra prediction process is disabled.

[1093] Fig. 2727 is a flowchart representation of a method 2700 of video processing according to the present disclosure. The method 2700 includes, at operation 2702, for conversion between a block of a video and a bitstream representation of the video, determining a first precision P1 of a motion vector for spatial motion prediction and a second precision P2 of a motion vector for temporal motion prediction. P1 and / or P2 are fractional. P1 and P2 are different from each other. The method 2600 also includes, at operation 2704, performing conversion based on the determination.

[1094] In some embodiments, the first precision is 1 / 16 brightness pixel, and the second precision is 1 / 4 brightness pixel. In some embodiments, the first precision is 1 / 16 brightness pixel, and the second precision is 1 / 8 brightness pixel. In some embodiments, the first precision is 1 / 8 brightness pixel, and the second precision is 1 / 4 brightness pixel. In some embodiments, the first precision is 1 / 16 brightness pixel, and the second precision is 1 / 4 brightness pixel. In some embodiments, the first precision is 1 / 16 brightness pixel, and the second precision is 1 / 8 brightness pixel. In some embodiments, the first precision is 1 / 8 brightness pixel, and the second precision is 1 / 4 brightness pixel. In some embodiments, at least one of the first or second precision is lower than 1 / 16 brightness pixel.

[1095] In some embodiments, at least one of the first or second precision is variable. In some embodiments, the first precision or the second precision varies according to a profile, level, or hierarchy of the video. In some embodiments, the first precision or the second precision varies according to a temporal layer of a picture in the video. In some embodiments, the first precision or the second precision varies according to a size of a picture in the video.

[1096] In some embodiments, at least one of the first or second precision is signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a slice group header, a slice, a coding tree unit, or a coding unit in a bitstream representation. In some embodiments, a motion vector is represented as (MVx, MVy) and the precision of the motion vector is represented as (Px, Py), and wherein Px is associated with MVx and Py is associated with MVy. In some embodiments, Px and Py are variables according to a profile, level, or hierarchy of the video. In some embodiments, Px and Py vary according to the temporal layer of a picture in the video. In some embodiments, Px and Py are variables according to the width of a picture in the video. In some embodiments, Px and Py are signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a slice group header, a slice, a coding tree unit, or a coding unit in a bitstream representation. In some embodiments, the decoded motion vector is recorded as (MVx, MVy), and wherein the motion vector is adjusted according to the second precision before the motion vector is stored as a temporal motion prediction motion vector. In some embodiments, the temporal motion prediction motion vector is adjusted to (Shift(MVx, P1-P2), Shift(MVy, P1-P2)), P1 and P2 are integers, and P1≥P2, Shift represents a right shift operation on an unsigned number. In some embodiments, the temporal motion prediction motion vector is adjusted to (SignShift(MVx, P1-P2), SignShift(MVy, P1-P2)), P1 and P2 are integers and P1≥P2, SignShift represents a right shift operation on a signed number. In some embodiments, the temporal motion prediction motion vector is adjusted to (MVx<<(P1-P2)), MVy<<(P1-P2)), P1 and P2 are integers and P1≥P2, << represents a left shift operation on a signed or unsigned number.

[1097] Fig.28 28 is a flowchart representation of a method 2800 of video processing according to the present disclosure. The method 2800 includes, at operation 2802, for a conversion between a block of a video and a bitstream representation of the video, determining a motion vector (MVx, MVy) with precision (Px, Py). Px is associated with MVx, and Py is associated with MVy. MVx and MVy are represented by N bits, and MinX≤MVx≤MaxX and MinY≤MVy≤MaxY, MinX, MaxX, MinY and MaxY are real numbers. The method 2700 also includes, at operation 2804, performing a conversion based on the determination.

[1098] In some embodiments, MinX = MinY. In some embodiments, MinX≠MinY. In some embodiments, MaxX = MaxY. In some embodiments, MaxX≠MaxY.

[1099] In some embodiments, at least one of MinX or MaxX is based on Px. In some embodiments, the motion vector has a precision recorded as (Px, Py), and wherein at least one of MinY or MaxY is based on Py. In some embodiments, at least one of MinX, MaxX, MinY or MaxY is based on N. In some embodiments, at least one of MinX, MaxX, MinY or MaxY of a spatial motion prediction motion vector is different from MinX, MaxX, MinY or MaxY of a corresponding temporal motion prediction motion vector. In some embodiments, at least one of MinX, MaxX, MinY or MaxY varies according to a profile, level or hierarchy of a video. In some embodiments, at least one of MinX, MaxX, MinY or MaxY varies according to a temporal layer of a picture in a video. In some embodiments, at least one of MinX, MaxX, MinY or MaxY varies according to the size of a picture in a video. In some embodiments, at least one of MinX, MaxX, MinY or MaxY is signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a slice group header, a slice, a coding tree unit or a coding unit in a bitstream representation. In some embodiments, MVx is clipped to [MinX, MaxX] before being used for spatial motion prediction or temporal motion prediction. In some embodiments, MVy is clipped to [MinY, MaxY] before being used for spatial motion prediction or temporal motion prediction.

[1100] Fig.29 2 is a flowchart representation of a video processing method 2900 according to the present disclosure. The method 2900 includes, at operation 2902, for a conversion between a current block of a video and a bitstream representation of the video, determining whether a shared Merge list is applicable to the current block according to a coding mode of the current block. The method 2800 includes, at operation 2904, performing the conversion based on the determination.

[1101] In some embodiments, when the current block is encoded using the conventional Merge mode, the shared Merge list is not applicable. In some embodiments, when the current block is encoded using the intra-block copy (IBC) mode, the shared Merge list is applicable. In some embodiments, the method further includes: before performing the conversion, maintaining the motion candidate table based on the past conversion represented by the video and the bitstream; and after performing the conversion, when the current block is a child block of a parent block to which the shared Merge list is applicable and the current block is encoded using the conventional Merge mode, disabling the update of the motion candidate table.

[1102] Fig.30 is a flowchart representation of a video processing method 3000 according to the present disclosure. The method 3000 includes, at operation 3002, for a conversion between a current block of a video of size W×H and a bitstream representation of the video, determining a second block of size (W+N-1)x(H+N-1) for motion compensation during the conversion. The second block is determined based on a reference block of size (W+N-1-PW)x(H+N-1-PH). N represents a filter size, and W, H, N, PW, and PH are non-negative integers. PW and PH are not both equal to 0. The method 3000 also includes, at operation 3004, performing the conversion based on the determination.

[1103] In some embodiments, pixels in the second block that are outside the reference block are determined by repeating one or more boundaries of the reference block. In some embodiments, PH=0, and at least the left boundary or the right boundary of the reference block is repeated to generate the second block. In some embodiments, PW=0, and at least the upper boundary or the lower boundary of the reference block is repeated to generate the second block. In some embodiments, PW>0 and PH>0, and the second block is generated by repeating at least the left boundary or the right boundary of the reference block and then repeating at least the upper boundary or the lower boundary of the reference block. In some embodiments, PW>0 and PH>0, and the second block is generated by repeating at least the upper boundary or the lower boundary of the reference block and then repeating at least the left boundary or the right boundary of the reference block.

[1104] In some embodiments, the left boundary of the reference block is repeated M1 times, and the right boundary of the reference block is repeated (PW-M1) times, where M1 is a positive integer. In some embodiments, the upper boundary of the reference block is repeated M2 times, and the lower boundary of the reference block is repeated (PH-M2) times, where M2 is a positive integer. In some embodiments, at least one of PW or PH is different for different color components of the current block, and the color component includes at least a luminance component or one or more chrominance components. In some embodiments, at least one of PW or PH varies according to the size or shape of the current block. In some embodiments, at least one of PW or PH varies according to the coding characteristics of the current block, and the coding characteristics include unidirectional predictive coding or bidirectional predictive coding.

[1105] In some embodiments, pixels in the second block that are outside the reference block are set to a single value. In some embodiments, the single value is 1<<(BD-1), where BD is the bit depth of the pixel samples in the reference block. In some embodiments, BD is 8 or 10. In some embodiments, the single value is derived based on the pixel samples of the reference block. In some embodiments, the single value is signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a slice group header, a slice, a coding tree unit row, a coding tree unit line, a coding unit, or a prediction unit. In some embodiments, padding of pixels in the second block that are outside the reference block is disabled when the current block is affine coded.

[1106] Fig.31 is a flowchart representation of a video processing method 3000 according to the present disclosure. The method 3000 includes, at operation 3102, for a conversion between a current block of a video of size W x H and a bitstream representation of the video, determining a second block of size (W+N-1)x(H+N-1) for motion compensation during the conversion. W, H are non-negative integers, and wherein N is a non-negative integer and is based on a filter size. During the conversion, a refined motion vector is determined based on a multi-point search based on a motion vector refinement operation on the original motion vector, and wherein pixels along a reference block boundary are determined by repeating one or more non-boundary pixels. The method 3100 also includes, at operation 3104, performing the conversion based on the determination.

[1107] In some embodiments, processing the current block includes filtering the current block in a motion vector refinement operation. In some embodiments, it is determined whether the reference block is suitable for processing the current block based on the size of the current block. In some embodiments, interpolating the current block includes: interpolating multiple sub-blocks of the current block based on the second block. The size of each sub-block is W1×H1, and W1 and H1 are non-negative integers. In some embodiments, W1=H1=4, W=H=8, and PW=PH=0. In some embodiments, the second block is determined entirely based on the integer part of the motion vector of at least one of the multiple sub-blocks. In some embodiments, when the maximum difference between the integer parts of the motion vectors of all the multiple sub-blocks is equal to or less than 1 pixel, the reference block is determined based on the integer part of the motion vector of the upper left sub-block of the current block, and each of the right boundary and the lower boundary of the reference block is repeated once to obtain the second block. In some embodiments, when the maximum difference between the integer parts of the motion vectors of all the multiple sub-blocks is equal to or less than 1 pixel, the reference block is determined based on the integer part of the motion vector of the lower right sub-block of the current block, and each of the left boundary and the upper boundary of the reference block is repeated once to obtain the second block. In some embodiments, the second block is determined based entirely on the modified motion vector of one of the plurality of sub-blocks.

[1108] In some embodiments, when the maximum difference between the integer parts of the motion vectors of all the plurality of sub-blocks is equal to or less than two pixels, the motion vector of the upper left sub-block of the current block is modified by adding an integer pixel distance to each component to obtain a modified motion vector, a reference block is determined based on the modified motion vector, and each of the left boundary, the right boundary, the upper boundary, and the lower boundary of the reference block is repeated once to obtain a second block.

[1109] In some embodiments, when the maximum difference between the integer parts of the motion vectors of all the plurality of sub-blocks is equal to or less than two pixels, the motion vector of the lower right sub-block of the current block is modified by subtracting an integer pixel distance from each component to obtain a modified motion vector, a reference block is determined based on the modified motion vector, and each of the left boundary, the right boundary, the upper boundary, and the lower boundary of the reference block is repeated once to obtain a second block.

[1110] Fig.323200 is a flowchart representation of a video processing method 3200 according to the present disclosure. The method 3200 includes, at operation 3202, for a conversion between a block of a video encoded using a combined inter-frame and intra-frame prediction (CIIP) coding technique and a bitstream representation of the video, determining a prediction value at a location in the block based on a weighted sum of an inter-frame prediction value and an intra-frame prediction value at the location. The weighted sum is based on adding an offset to an initial sum obtained based on the inter-frame prediction value and the intra-frame prediction value, and adding the offset before performing a right shift operation to determine the weighted sum. The method 3200 also includes, at operation 3204, performing a conversion based on the determination.

[1111] In some embodiments, the position in the block is represented as (x, y), the inter-frame prediction value at the position (x, y) is represented as Pinter(x, y), the intra-frame prediction value at the position (x, y) is represented as Pintra(x, y), the inter-frame prediction weight at the position (x, y) is represented as w_inter(x, y), and the intra-frame prediction weight at the position (x, y) is represented as w_intra(x, y). The prediction value at the position (x, y) is determined as (Pintra(x, y)×w_intra(x, y)+Pinter(x, y)×w_inter(x, y)+offset(x, y))>>N, where w_intra(x, y)+w_inter(x, y)=2^N and offset(x, y)=2^(N-1), and N is a positive integer. In some embodiments, N=2.

[1112] In some embodiments, equal weights are used for the inter-prediction values ​​and intra-prediction values ​​at the position to determine the weighted sum. In some embodiments, zero weights are used according to the position in the block to determine the weighted sum. In some embodiments, zero weights are applied to the inter-prediction values. In some embodiments, zero weights are applied to the intra-prediction values.

[1113] Fig.33 3300 is a flowchart representation of a video processing method 3300 according to the present disclosure. The method 3300 includes, at operation 3302, for a conversion between a current block of video and a bitstream representation of the video, determining a manner of representing encoding information of the current block in the bitstream representation based in part on whether a condition associated with a size of the current block is satisfied. The method 3300 also includes, at operation 3304, performing the conversion based on the determination.

[1114] In some embodiments, the coding information is signaled in the bitstream representation if the condition is met. In some embodiments, the coding information is omitted in the bitstream representation if the condition is met. In some embodiments, at least one of the intra-block copy (IBC) coding tool or the inter-frame prediction coding tool is disabled for the current block if the condition is met. In some embodiments, the coding information is omitted in the bitstream representation if the IBC coding tool and the inter-frame prediction coding tool are disabled. In some embodiments, the coding information is signaled in the bitstream representation if the IBC coding tool is enabled and the inter-frame prediction tool is disabled.

[1115] In some embodiments, the coding information includes an operating mode associated with an IBC tool and / or an inter-frame prediction coding tool, the operating mode including at least a normal mode or a skip mode. In some embodiments, the coding information includes an indicator indicating the use of a prediction mode associated with the coding tool. In some embodiments, when the coding information is omitted in the bitstream representation, the coding tool is not used. In some embodiments, when the coding information is omitted in the bitstream representation, the coding tool is used. In some embodiments, the coding tool includes an intra-block copy (IBC) coding tool or an inter-frame prediction coding tool.

[1116] In some embodiments, in the case where the indicator is omitted in the bitstream, one or more indicators indicating other one or more coding tools are signaled in the bitstream representation. In some embodiments, the other one or more coding tools include at least an intra-frame coding tool or a palette coding tool. In some embodiments, in the case where the inter-frame prediction coding tool is disabled and the IBC coding tool is enabled, the indicator distinguishes between inter-frame mode and intra-frame mode, and the one or more indicators include a first indicator indicating the IBC mode and a second indicator indicating the palette mode signaled in the bitstream.

[1117] In some embodiments, the encoding information includes a third indicator for a skip mode, the skip mode being an inter-frame skip mode or an IBC skip mode. In some embodiments, in a case where the inter-frame prediction coding tool is disabled and the IBC coding tool is enabled, the third indicator for the skip mode is signaled. In some embodiments, in a case where the inter-frame prediction coding tool is disabled and the IBC coding tool is enabled, the prediction mode is determined to be the IBC mode, even though the indicator indicating the use of the IBC mode is omitted in the bitstream representation, and wherein the skip mode is applicable to the current block.

[1118] In some embodiments, the encoding information includes a triangular pattern. In some embodiments, the encoding information includes an inter-frame prediction direction. In some embodiments, in the case where a unidirectional prediction encoding tool is used to encode the current block, the encoding information is omitted in the bitstream representation. In some embodiments, the encoding information includes an indicator indicating the use of a symmetric motion vector difference (SMVD) method. In some embodiments, if a condition is met, the current block is set to unidirectional prediction despite the presence of an indicator indicating the use of the SMVD method in the bitstream representation. In some embodiments, in the case where an indicator indicating the use of the SMVD method is excluded from the bitstream representation, only List 0 or List 1 associated with the unidirectional prediction encoding tool is used in the motion compensation process.

[1119] In some embodiments, the condition indicates that the width of the current block is T1 and the height of the current block is T2, and T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is T2 and the height of the current block is T1, and T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is T1 and the height of the current block is less than or equal to T2, and T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is T2 and the height of the current block is less than or equal to T1, and T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is less than or equal to T1 and the height of the current block is less than or equal to T2, and T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is greater than or equal to T1 and the height of the current block is greater than or equal to T2, and T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is greater than or equal to T1 or the height of the current block is greater than or equal to T2, and T1 and T2 are positive integers. In some embodiments, T1=4 and T2=16. In some embodiments, T1=T2=8. In some embodiments, T1=8 and T2=4. In some embodiments, T1=T2=4. In some embodiments, T1=T2=32. In some embodiments, T1=T2=64. In some embodiments, T1=T2=128.

[1120] In some embodiments, the condition indicates that the size of the current block is 4x8, 8x4, 4x16, or 16x4. In some embodiments, the condition indicates that the size of the current block is 4×8, 8×4. In some embodiments, the condition indicates that the size of the current block is 4xN or Nx4, N is a positive integer and N≤16. In some embodiments, the condition indicates that the size of the current block is 8xN or Nx8, N is a positive integer and N≤16. In some embodiments, the condition indicates that the color component of the current block includes less than or equal to N samples. In some embodiments, N=16. In some embodiments, N=32.

[1121] In some embodiments, the condition indicates that the color components of the current block have a width equal to the height. In some embodiments, the condition indicates that the color components of the current block are sized 4x8, 8x4, 4x4, 4x16, or 16x4. In some embodiments, the color components include a luma component or one or more chroma components. In some embodiments, where the condition indicates that the width of the current block is 4 and the height is 4, the inter-frame prediction tool is disabled. In some embodiments, where the condition indicates that the width of the current block is 4 and the height is 4, the IBC prediction tool is disabled.

[1122] In some embodiments, list 0 associated with unidirectional predictive coding tools is used in the motion compensation process when: (1) the width of the current block is 4 and the height is 8, or (2) the width of the current block is 8 and the height is 4. Bidirectional predictive coding tools are disabled in the motion compensation process.

[1123] Fig.34 3400 is a flowchart representation of a video processing method 3400 according to the present disclosure. The method 3400 includes, at operation 3402, determining a modified motion vector set for a conversion between a current block of a video and a bitstream representation of the video. The method 3400 includes, at operation 3404, performing the conversion based on the modified motion vector set. Since the current block satisfies the condition, the modified motion vector set is a modified version of the motion vector set associated with the current block.

[1124] In some embodiments, the motion vector set is derived using one of a conventional Merge coding technique, a sub-block temporal motion vector prediction coding technique (sbTMVP), or a motion vector difference Merge (MMVD) coding technique. In some embodiments, the motion vector set includes block vectors for an intra-block copy (IBC) coding technique. In some embodiments, the condition specifies that the size of the luminance component of the current block is the same as a predefined size. In some embodiments, the predefined size includes at least one of 8x4 or 4x8.

[1125] In some embodiments, modifying the motion vector set includes converting the motion vector set to a unidirectional motion vector when it is determined that the motion information of the current block is bidirectional and includes a first motion vector that references a first reference picture in a first reference picture list for a first prediction direction and a second motion vector that references a second reference picture in a second reference picture list for a second prediction direction. In some embodiments, the motion information of the current block is derived based on the motion information of an adjacent block. In some embodiments, the method includes discarding information in one of the first prediction direction or the second prediction direction. In some embodiments, the discarding information includes modifying the motion vector in the discarded prediction direction to (0,0). In some embodiments, the discarding information includes modifying the reference index in the discarded prediction direction to -1. In some embodiments, the discarding information includes modifying the discarded prediction direction to another prediction direction.

[1126] In some embodiments, the method further comprises deriving a new motion candidate based on the information of the first prediction direction and the information of the second prediction direction. In some embodiments, deriving the new motion candidate comprises determining a motion vector of the new motion candidate based on an average of the first motion vector in the first prediction direction and the second motion vector in the second prediction direction. In some embodiments, deriving the new motion candidate comprises determining a scaled motion vector by scaling the first motion vector in the first prediction direction according to the information of the second prediction direction; and determining the motion vector of the new motion candidate based on an average of the scaled motion vector and the second motion vector in the second prediction direction.

[1127] In some embodiments, the first reference picture list is list 0 and the second reference picture list is list 1. In some embodiments, the first reference picture list is list 1 and the second reference picture list is list 0. The method includes updating a motion candidate table determined based on past conversions using the converted motion vector. In some embodiments, the table includes a historical motion vector prediction (HMVP) table.

[1128] In some embodiments, the method includes updating a motion candidate table determined based on past conversions using information of the first prediction direction and the second prediction direction before modifying the motion vector set. In some embodiments, the converted motion vector is used for motion compensation operations of subsequent blocks. In some embodiments, the converted motion vector is used to predict motion vectors of subsequent blocks.

[1129] In some embodiments, information of the first prediction direction and the second prediction direction before the modified motion vector set is used for motion compensation operation of a subsequent block. In some embodiments, information before the modified motion vector set is used to predict motion vectors of subsequent blocks. In some embodiments, the converted motion vector is used for motion refinement operation of the current block. In some embodiments, information of the first prediction direction and the second prediction direction before the modified motion vector set is used for refinement operation of the current block. In some embodiments, the refinement operation includes an optical flow encoding operation, which includes at least a bidirectional optical flow (BDOF) operation or a prediction (PROF) operation with optical flow refinement.

[1130] In some embodiments, the method includes generating an intermediate prediction block based on information of the first prediction direction and information of the second prediction direction, refining the intermediate prediction block, and determining a final prediction block based on one of the intermediate prediction blocks.

[1131] Fig.35 35 is a flow chart of a method 3500 for video processing according to the present disclosure. The method 3500 includes, at operation 3502, for a conversion between a current block of a video and a bitstream representation of the video, determining a unidirectional motion vector from a bidirectional motion vector when a block size is satisfied. The unidirectional motion vector is then used as a Merge candidate for the conversion. The method 3500 also includes, at operation 3504, performing the conversion according to the determination.

[1132] In some embodiments, the converted unidirectional motion vector is used as a basic Merge candidate in a motion vector difference Merge (MMVD) encoding process. In some embodiments, wherein the converted unidirectional motion vector is subsequently inserted into a Merge list. In some embodiments, the unidirectional motion vector is converted based on one prediction direction of the bidirectional motion vector. In some embodiments, the unidirectional motion vector is converted based on only one prediction direction, and wherein the only one prediction direction is associated with reference picture list 0. In some embodiments, in the case where the video data unit in which the current block is located is bidirectionally predicted, the only one prediction direction of the first candidate in the candidate list is associated with reference picture list 0, and the only one prediction direction of the second candidate in the candidate list is associated with reference picture list 1. The candidate list includes a Merge candidate list or an MMVD basic candidate list.

[1133] In some embodiments, the video data unit comprises a current slice, slice group or picture of the video. In some embodiments, the unidirectional motion vector is determined based on the reference picture list 1 when all reference pictures of the video data unit are past pictures according to display order. In some embodiments, the unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture according to display order and at least a second reference picture of the reference pictures of the video data unit is a future picture according to display order. In some embodiments, the unidirectional motion vectors determined based on different reference picture lists are placed in a Merge list in an interleaved manner.

[1134] In some embodiments, the unidirectional motion vector is determined based on a low latency check indicator. In some embodiments, the unidirectional motion vector is determined before a motion compensation process during the conversion. In some embodiments, the unidirectional motion vector is determined after a motion candidate list construction process during the conversion. In some embodiments, the unidirectional motion vector is determined before adding a motion vector difference during a motion vector difference Merge (MMVD) encoding process.

[1135] In some embodiments, the condition of the block size indicates that the width of the current block is T1, and the height of the current block is T2, and T1 and T2 are positive integers. In some embodiments, the condition of the block size indicates that the width of the current block is T2, and the height of the current block is T1, and T1 and T2 are positive integers. In some embodiments, the condition of the block size indicates that the width of the current block is T1, and the height of the current block is less than or equal to T2, and T1 and T2 are positive integers. In some embodiments, the condition of the block size indicates that the width of the current block is T2, and the height of the current block is less than or equal to T1, and T1 and T2 are positive integers.

[1136] Fig.36 3600 is a flow chart representing a method 3600 for video processing according to the present disclosure. The method 3600 includes, at operation 3602, for a conversion between a current block of a video and a bitstream representation of the video, determining that motion candidates of the current block are restricted to unidirectional prediction candidates based on the size of the current block. The method 3600 also includes, at operation 3604, performing the conversion based on the determination.

[1137] In some embodiments, when the size of the current block satisfies a condition, all motion candidates of the current block are restricted to unidirectional prediction candidates. In some embodiments, when the size of the current block satisfies a condition, bidirectional motion candidates are converted to the unidirectional prediction candidates. In some embodiments, the size of the current block indicates that the width of the current block is T1, and the height of the current block is T2, and T1 and T2 are positive integers. In some embodiments, the condition of the block size indicates that the width of the current block is T2, and the height of the current block is T1, and T1 and T2 are positive integers. In some embodiments, the size of the current block indicates that the width of the current block is T1, and the height of the current block is less than or equal to T2, and T1 and T2 are positive integers. In some embodiments, the size of the current block indicates that the width of the current block is T2, and the height of the current block is less than or equal to T1, and T1 and T2 are positive integers. In some embodiments, T1 is equal to 4, and T2 is equal to 8. In some embodiments, T1 is equal to 4, and T2 is equal to 4. In some embodiments, T1 is equal to 8, and T2 is equal to 8.

[1138] Fig.37 3700 for video processing according to the present disclosure. The method 3700 includes, at operation 3702, for a conversion between a current block of a video and a bitstream representation of the video, determining a unidirectional motion vector from a bidirectional motion vector when a block size is satisfied. A decoder-side motion vector refinement (DMVR) process is then disabled based on a determination to use the unidirectional motion vector during the conversion. The method 3700 includes, at operation 3704, performing the conversion based on the determination.

[1139] In some embodiments, the unidirectional motion vector is determined prior to a sample refinement process. In some embodiments, the DMVR process derives the refined motion vector based on a prediction sample cost calculation. In some embodiments, the prediction sample cost calculation calculates the sum of absolute differences (SAD) between prediction samples.

[1140] In some embodiments, the unidirectional motion vector is determined prior to a bidirectional optical flow (BDOF) process. In some embodiments, the BDOF process is subsequently disabled during the conversion based on the unidirectional motion vector. In some embodiments, the BDOF process derives refined prediction samples based on prediction sample gradient calculations. In some embodiments, the unidirectional motion vector is determined prior to a prediction refinement (PROF) process using optical flow.

[1141] In some embodiments, performing the conversion comprises generating the bitstream representation based on the current block of the video. In some embodiments, performing the conversion comprises generating the current block of the video from the bitstream representation.

[1142] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more thereof. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, for example, one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium, for data processing devices to execute or control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing unit" or "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[1143] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[1144] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[1145] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory may be supplemented by, or incorporated into, dedicated logic circuits.

[1146] Although this patent document contains many details, they should not be interpreted as limitations on any subject matter or the scope of the claims, but rather as descriptions of features of specific embodiments of specific technologies. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially required to be so, in some cases, one or more features in the claim combination can be removed from the combination, and the combination of claims can be directed to subcombinations or variations of subcombinations.

[1147] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1148] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: for a conversion between a current block of a video and a bitstream of the video, determining a unidirectional motion vector from the bidirectional motion vectors if a condition on block size is satisfied, wherein a decoder-side motion vector refinement (DMVR) process is subsequently disabled based on the determination using the unidirectional motion vector during the conversion; and performing said converting based on said determining; The condition of satisfying the block size is that the luminance block size is one of specific sizes, and the specific size includes 4x16 and / or 16x4. converting the unidirectional motion vector based on a prediction direction of the bidirectional motion vector, In a case where the video data unit where the current block is located is bidirectionally predicted, the one prediction direction of the first candidate in the candidate list is associated with reference picture list 0, and the one prediction direction of the second candidate in the candidate list is associated with reference picture list 1, wherein the candidate list includes a Merge candidate list or an MMVD basic candidate list, The unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture according to a display order.

2. The method according to claim 1, wherein: The unidirectional motion vector is determined before the sample refinement process.

3. The method according to claim 1, wherein: The DMVR process derives refined motion vectors based on prediction sample cost calculation.

4. The method according to claim 3, wherein: The prediction sample cost calculation calculates the sum of absolute differences (SAD) between prediction samples.

5. The method according to claim 1, wherein: The unidirectional motion vector is determined before the bidirectional optical flow (BDOF) process.

6. The method according to claim 5, wherein: The BDOF process is then disabled during the transition based on the unidirectional motion vector.

7. The method according to claim 6, wherein: The BDOF process derives refined prediction samples based on prediction sample gradient calculations.

8. The method according to any one of claims 1 to 7, wherein: The unidirectional motion vector is determined prior to a prediction refinement (PROF) process using optical flow.

9. A video processing method, comprising: For a conversion between a current block of a video and a bitstream of the video, determining a unidirectional motion vector from the bidirectional motion vectors when a block size condition is satisfied, wherein the unidirectional motion vector is subsequently used as a Merge candidate for the conversion; and performing said converting based on said determining; The condition of satisfying the block size is that the luminance block size is one of specific sizes, and the specific size includes 4x16 and / or 16x4. converting the unidirectional motion vector based on a prediction direction of the bidirectional motion vector, In a case where the video data unit in which the current block is located is bidirectionally predicted, the only one prediction direction of a first candidate in the candidate list is associated with a reference picture list 0, and the only one prediction direction of a second candidate in the candidate list is associated with a reference picture list 1, wherein the candidate list includes a Merge candidate list or an MMVD basic candidate list, The unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture according to a display order.

10. The method according to claim 9, wherein: In the motion vector difference Merge (MMVD) encoding process, the converted unidirectional motion vector is used as a basic Merge candidate.

11. The method according to claim 9, wherein: The converted unidirectional motion vector is then inserted into the Merge list.

12. The method according to claim 9, wherein: The unidirectional motion vector is converted based on only one prediction direction, and wherein the only one prediction direction is associated with reference picture list 0.

13. The method according to claim 9, wherein: The video data unit includes a current slice, a slice group, or a picture of the video.

14. The method according to claim 9, wherein: In the case where all reference pictures of the video data unit are past pictures according to display order, the unidirectional motion vector is determined based on the reference picture list 1.

15. The method according to claim 9, wherein: The unidirectional motion vectors determined based on different reference picture lists are placed in the Merge list in an interleaved manner.

16. The method according to claim 9, wherein: The unidirectional motion vector is determined based on a low latency check indicator.

17. The method according to claim 9, wherein: The unidirectional motion vector is determined prior to a motion compensation process during the conversion.

18. The method according to claim 9, wherein: The unidirectional motion vector is determined after a motion candidate list construction process during the conversion.

19. The method according to claim 9, wherein: The unidirectional motion vector is determined before adding the motion vector difference in the Motion Vector Difference Merge (MMVD) encoding process.

20. The method of any one of claims 1 to 19, wherein performing the conversion comprises generating the bitstream based on a current block of the video.

21. The method of any one of claims 1 to 19, wherein performing the conversion comprises generating a current block of the video from the bitstream.

22. A video processing device comprising a processor and a non-transitory memory having instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to: for a conversion between a current block of a video and a bitstream of the video, determining a unidirectional motion vector from the bidirectional motion vectors if a condition on block size is satisfied, wherein a decoder-side motion vector refinement (DMVR) process is subsequently disabled based on the determination using the unidirectional motion vector during the conversion; and performing said converting based on said determining; The condition of satisfying the block size is that the luminance block size is one of specific sizes, and the specific size includes 4x16 and / or 16x4. converting the unidirectional motion vector based on a prediction direction of the bidirectional motion vector, In a case where the video data unit where the current block is located is bidirectionally predicted, the one prediction direction of the first candidate in the candidate list is associated with reference picture list 0, and the one prediction direction of the second candidate in the candidate list is associated with reference picture list 1, wherein the candidate list includes a Merge candidate list or an MMVD basic candidate list, The unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture according to a display order.

23. A computer readable medium having stored thereon instructions which, when executed by a processor, cause the processor to: For a conversion between a current block of a video and a bitstream of the video, a unidirectional motion vector is determined from the bidirectional motion vectors when a condition of the block size is satisfied, wherein: then disabling a decoder-side motion vector refinement (DMVR) process based on the determination of using the unidirectional motion vector during the conversion; as well as performing said converting based on said determining; The condition of satisfying the block size is that the luminance block size is one of specific sizes, and the specific size includes 4x16 and / or 16x4. converting the unidirectional motion vector based on a prediction direction of the bidirectional motion vector, In a case where the video data unit where the current block is located is bidirectionally predicted, the one prediction direction of the first candidate in the candidate list is associated with reference picture list 0, and the one prediction direction of the second candidate in the candidate list is associated with reference picture list 1, wherein the candidate list includes a Merge candidate list or an MMVD basic candidate list, The unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture according to a display order.

24. A non-transitory computer-readable medium storing a bitstream of a video, wherein the bitstream is generated by a video processing device executing a method, the method comprising: for a conversion between a current block of a video and a bitstream of the video, determining a unidirectional motion vector from the bidirectional motion vectors if a condition on block size is satisfied, wherein a decoder-side motion vector refinement (DMVR) process is subsequently disabled based on the determination using the unidirectional motion vector during the conversion; and generating the bitstream based on the determination; The condition of satisfying the block size is that the luminance block size is one of specific sizes, and the specific size includes 4x16 and / or 16x4. converting the unidirectional motion vector based on a prediction direction of the bidirectional motion vector, In a case where the video data unit where the current block is located is bidirectionally predicted, the one prediction direction of the first candidate in the candidate list is associated with reference picture list 0, and the one prediction direction of the second candidate in the candidate list is associated with reference picture list 1, wherein the candidate list includes a Merge candidate list or an MMVD basic candidate list, The unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture according to a display order.

25. A method for storing a bitstream, comprising: for a conversion between a current block of a video and a bitstream of the video, determining a unidirectional motion vector from the bidirectional motion vectors if a condition on block size is satisfied, wherein a decoder-side motion vector refinement (DMVR) process is subsequently disabled based on the determination using the unidirectional motion vector during the conversion; generating the bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable storage medium; The condition of satisfying the block size is that the luminance block size is one of specific sizes, and the specific size includes 4x16 and / or 16x4. converting the unidirectional motion vector based on a prediction direction of the bidirectional motion vector, In a case where the video data unit where the current block is located is bidirectionally predicted, the one prediction direction of the first candidate in the candidate list is associated with reference picture list 0, and the one prediction direction of the second candidate in the candidate list is associated with reference picture list 1, wherein the candidate list includes a Merge candidate list or an MMVD basic candidate list, The unidirectional motion vector is determined based on the reference picture list 0 when at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture according to a display order.

Images