Interaction between mv precision and mv difference coding

Abstract

Interactions between MV precision and MV difference coding are described. In one example aspect, a method for video processing includes determining, based on at least one of motion information, motion vector difference (MVD) precision, and motion vector (MV) precision of a first block of a video, whether a symmetric motion vector difference (SMVD) mode of the first block is enabled or disabled and / or how to apply the SMVD mode of the first block for a conversion between the first block and a bitstream representation of the first block, and performing the conversion based on the determination.

Description

[0001] Cross Reference to Related Applications

[0002] This application is made to timely claim priority to and the benefit of International Patent Application No. PCT / CN2019 / 074681 filed on February 3, 2019, under the applicable patent law and / or pursuant to the provisions of the Paris Convention. International Patent Application No. PCT / CN2019 / 074681 is hereby incorporated by reference as part of the disclosure of this application. TECHNICAL FIELD

[0003] This patent document relates to video coding techniques, devices, and systems. BACKGROUND

[0004] Despite advances in video compression, digital video still accounts for the largest bandwidth use on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video usage is expected to continue to grow. SUMMARY

[0005] Devices, systems, and methods related to digital video coding are described, in particular, to the interaction between motion vector precision and motion vector coding. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video coding. The method includes determining, based on at least one of motion information of a current video block, motion vector difference (MVD) precision, or motion vector (MV) precision, between enabling and disabling a symmetric motion vector difference (SMVD) mode for the current video block; and reconstructing the current video block based on the determination.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes determining, based on at least one of motion information of a first block of a video, motion vector difference (MVD) precision, and motion vector (MV) precision, for a conversion between the first block and a bitstream representation of the first block whether or how a symmetric motion vector difference (SMVD) mode of the first block is enabled or disabled and / or applied for the conversion; and performing the conversion based on the determination.

[0008] In another representative aspect, the above method is embodied in the form of processor- executable code and stored in a computer-readable program medium.

[0009] In yet another representative aspect, an apparatus configured or operable to perform the above-described methods is disclosed. The apparatus can include a processor programmed to implement the methods.

[0010] In yet another representative aspect, a video decoder apparatus can implement the methods as described herein.

[0011] The above and other aspects and features of the disclosed technology are more fully described in the following detailed description, in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 An example of constructing a Merge candidate list is shown.

[0013] Figure 2 An example of the location of spatial candidates is shown.

[0014] Figure 3 An example of a pair of candidates subject to a redundancy check for spatial Merge candidates is shown.

[0015] Figure 4A And Figure 4B An example of the location of a second prediction unit (PU) based on the size and shape of the current block is shown.

[0016] Figure 5 An example of motion vector scaling for temporal Merge candidates is shown.

[0017] Figure 6 An example of the candidate location for temporal Merge candidates is shown.

[0018] Figure 7 An example of generating a combined bi-predictive Merge candidate is shown.

[0019] Figure 8 An example of constructing a motion vector prediction candidate is shown.

[0020] Figure 9 An example of motion vector scaling for spatial motion vector candidates is shown.

[0021] Figure 10 An example of a simplified affine motion model is shown.

[0022] Figure 11 An example of an affine motion vector field (MVF) per sub-block is shown.

[0023] Figure 12 An example of motion vector prediction (MVP) for AF INTER affine mode is shown.

[0024] Figure 13A and 13B Examples of 4-parameter and 6-parameter affine models are shown, respectively.

[0025] Figure 14A and 14B An example candidate for AF_MERGE affine motion mode is shown.

[0026] Figure 15 An example of optical flow tracks used by a bi-directional optical flow (BIO) algorithm is shown.

[0027] Figure 16A and 16B An example snapshot using a bi-directional optical flow (BIO) algorithm without block expansion is shown.

[0028] Figure 17 An example of a decoder-side motion vector refinement (DMVR) algorithm based on bilateral template matching is shown.

[0029] Figure 18 An example of a symmetric mode is shown.

[0030] Figure 19 A flowchart of an example method for video processing is shown.

[0031] Figure 20 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0032] Figure 21 A flowchart of an example method for video processing is shown. DETAILED DESCRIPTION

[0033] Video coding methods and techniques are ubiquitous in modern technology due to the ever-increasing demand for higher resolution videos. Video codecs, which typically include electronic circuits or software that compress or decompress digital video, are constantly being improved to provide higher coding efficiency. Video codecs convert uncompressed video into compressed format and vice versa. There is a complex relationship between video quality, amount of data used to represent the video (determined by the bit rate), complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). The compressed format typically conforms to a standard video compression specification, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the Versatile Video Coding standard under finalization, or other current and / or future video coding standards.

[0034] Embodiments of the disclosed technology can be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve readability of the description and do not restrict the discussion or embodiments (and / or implementations) to the respective section in any way.

[0035] 1. Example of HEVC / H.265 inter prediction

[0036] Video coding standards have improved significantly over the years and now partly provide high coding efficiency and support for higher resolutions. The latest standards such as HEVC and H.265 are based on a hybrid video coding structure where temporal prediction plus transform coding is utilized.

[0037] 1.1 Example of prediction modes

[0038] Each inter predicted PU (prediction unit) has motion parameters of one or two reference picture lists. In some embodiments, the motion parameters include a motion vector and a reference picture index. In other embodiments, inter_pred_idc can also be used to signal the use of one of the two reference picture lists. In other embodiments, the motion vector can be explicitly coded as a delta relative to a predictor.

[0039] When a CU is coded with skip mode, the PU associated with the CU and there are no significant residual coefficients, no motion vector delta or reference picture index to be coded. Merge mode is specified, through which the motion parameters of the current PU can be obtained from neighboring PUs, including spatial and temporal candidate points. Merge mode is not only applied to skip mode, but also to any inter predicted PU. An alternative to Merge mode is explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index for each reference picture list and the reference picture list usage are explicitly signaled per PU.

[0040] When signaling indicates that one of the two reference picture lists is to be used, the PU is produced from one sample block. This is referred to as "uni-prediction". Uni-prediction is available for both P slices and B slices.

[0041] When signaling indicates that both reference picture lists are to be used, the PU is produced from two sample blocks. This is referred to as "bi-prediction". Bi-prediction is only available for B slices.

[0042] 1.1.1 Embodiments of constructing Merge mode candidates

[0043] When predicting the PU using the Merge pattern, the indices pointing to entries in the Merge candidate list are parsed from the bitstream and used to retrieve motion information. The construction of this list can be summarized as follows:

[0044] Step 1: Initial Candidate Derivation

[0045] Step 1.1: Spatial Candidate Derivation

[0046] Step 1.2: Redundancy check for airspace candidates

[0047] Step 1.3: Time-domain candidate derivation

[0048] Step 2: Add candidate insertions

[0049] Step 2.1: Creation of bidirectional prediction candidates

[0050] Step 2.2: Insertion of zero motion candidates

[0051] Figure 1 An example of constructing a Merge candidate list based on the sequence of steps summarized above is shown. For spatial Merge candidate derivation, up to four Merge candidates are selected from candidates located at five distinct positions. For temporal Merge candidate derivation, up to one Merge candidate is selected from two candidates. Since a constant number of candidates is assumed for each PU at the decoder, additional candidates are generated when the number of candidates does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the strip header. Because the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (TU). If the CU size is equal to 8, all PUs of the current CU share a single Merge candidate list, which is identical to the Merge candidate list of a 2N×2N prediction unit.

[0052] 1.1.2 Constructing Spatial Merge Candidates

[0053] In the derivation of the spatial Merge candidate, from the position located Figure 2 Up to four merge candidates are selected from the candidates at the indicated positions. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another stripe or slice) or is intra-frame encoded / decoded. After the candidate at position A1 is added, the addition of the remaining candidates undergoes a redundancy check, which ensures that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding efficiency.

[0054] To reduce the computational complexity, not all possible pairs of candidates are considered in the mentioned redundancy check. Instead, only Figure 3 pairs linked with arrows in Fig. 1 1 are considered and a candidate is only added to the list if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of duplicated motion information is the "second PU" associated with a partition different from 2Nx2N. As an example, Figure 4A and Figure 4B depict the second PU for the N x 2N and 2N x N case, respectively. When the current PU is partitioned as N x 2N, the candidate at position Al is not considered for list construction. In some embodiments, adding this candidate can lead to two prediction units having the same motion information, which is redundant in a coding unit having only one PU. Similarly, when the current PU is partitioned as 2N x N, position Bl is not considered.

[0055] 1.1.3 Construction of temporal Merge candidates

[0056] In this step, only one candidate is added to the list. In particular, in the derivation of this temporal Merge candidate, a scaled motion vector is derived based on a co-located PU belonging to the picture within the given reference picture list having the smallest POC difference to the current picture. The reference picture list to be used for the derivation of the co-located PU is explicitly signaled in the slice header.

[0057] Figure 5 An example of the derivation of the scaled motion vector for a temporal Merge candidate (as indicated by the dashed line) is shown in Fig. 12, which is scaled from the motion vector of the co-located PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal Merge candidate is set to zero. For B slices, two motion vectors are obtained, one for reference picture list 0 and the other for reference picture list 1, and they are combined to form a bi-predictive Merge candidate.

[0058] In the co-located PU (Y) belonging to the reference frame, the position for the temporal candidate is chosen between the candidates CO and Cl, as shown in Figure 6 If the PU at position CO is not available, intra coded or outside the current CTU, the Cl position is used. Otherwise, the CO position is used in the derivation of the temporal Merge candidate.

[0059] 1.1.4 Construction of additional types of Merge candidates

[0060] In addition to the temporal merge candidate, there are two additional types of merge candidates: combined bi-predictive merge candidates and zero merge candidates. Combined bi-predictive merge candidates are generated by exploiting the temporal merge candidate. Combined bi-predictive merge candidates are only used for B slices. A combined bi-predictive candidate is generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they will form a new bi-predictive candidate.

[0061] Figure 7 An example of this process is shown, where two candidates in the original list (710, on the left) with mvL0 and refldxL0 or mvL1 and refldxL1 are used to create a combined bi-predictive merge candidate that is added to the final list (720, on the right).

[0062] Zero motion candidates are inserted to fill the remaining entries in the merge candidate list and thus reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts from zero and is increased every time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for uni- and bi-prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

[0063] 1.1.5 Example of motion estimation region for parallel processing

[0064] To speed up the encoding process, motion estimation can be performed in parallel, thus deriving the motion vectors of all the prediction units inside a given region at the same time. Deriving merge candidates from spatial neighbors can interfere with parallel processing, as a prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is completed. To mitigate the trade-off between coding efficiency and processing latency, a motion estimation region (MER) can be defined. The size of the MER can be signaled in the picture parameter set (PPS) using the “log2_parallel_merge_level_minus2” syntax element. When an MER is defined, merge candidates falling into the same region are marked as unavailable and thus not considered in the list construction.

[0065] 1.2 Embodiments of advanced motion vector prediction (AMVP)

[0066] AMVP exploits the spatio-temporal correlation of motion vectors with the neighboring PUs, which is used for the explicit transmission of motion parameters. It builds the motion vector candidate list by first checking the availability of temporally neighboring PU positions on the left, removing redundant candidates and adding zero vectors to make the candidate list length constant. Then, the encoder can select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate is coded using truncated unary. In this case, the maximum value to be coded is 2 (see Figure 8 ). In the following section, details about the derivation process of the motion vector prediction candidates are provided.

[0067] 1.2.1 Example of building motion vector prediction candidates

[0068] Figure 8 The derivation process of the motion vector prediction candidates is summarized and can be implemented for each reference picture list with refidx as input.

[0069] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For spatial motion vector candidate derivation, two motion vector candidates are derived based on the motion vectors of each PU located in five different positions as shown previously in Figure 2

[0070] For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates, which are derived based on two different co-located positions. After forming the first spatial motion vector candidate list, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than 2, the motion vector candidates whose reference picture index is greater than 1 within the associated reference picture list are removed from the list. If the number of spatial-temporal motion vector candidates is less than 2, additional zero motion vector candidates are added to the list.

[0071] 1.2.2 Building spatial motion vector candidates

[0072] In the derivation of spatial motion vector candidates, at most two candidates are considered among five potential candidates, which are derived from the PUs located in five different positions as shown previously in Figure 2 ​The positions derived in the PUs of the shown positions are the same as the positions of the motion Merge. The derivation order for the left side of the current PU is defined as A0, A1 and scaled A0, scaled A1. The derivation order for the top side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Thus, for each side, there are four cases that can be used as motion vector candidates, where two cases do not need to use spatial scaling, and two cases use spatial scaling. The four different cases are summarized as follows:

[0073] -- no spatial scaling

[0074] (1) Same reference picture list and same reference picture index (same POC)

[0075] (2) Different reference picture list but same reference picture (same POC)

[0076] -- spatial scaling

[0077] (3) Same reference picture list but different reference picture (different POC)

[0078] (4) Different reference picture list and different reference picture (different POC)

[0079] The no spatial scaling case is checked first, and then the spatial scaling allowed case is checked. Spatial scaling is considered when the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, regardless of the reference picture list. If all PUs of the left candidate are not available or are intra coded, scaling is allowed for the upper motion vector to help parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

[0080] As shown in the example in Figure 9 , for the spatial scaling case, the motion vector of the neighboring PU is scaled in a similar way as the temporal scaling. One difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the temporal scaling process.

[0081] 1.2.3 Construction of temporal motion vector candidates

[0082] Except for the reference picture index derivation, all the processes for deriving the temporal Merge candidates are the same as those for deriving the spatial motion vector candidates (as shown in the example in Figure 6 ). In some embodiments, the reference picture index is signaled to the decoder.

[0083] 2. Example of inter prediction method in Joint Exploration Model (JEM)

[0084] In some embodiments, reference software known as Joint Exploration Model (JEM) is used to explore future video coding technologies. In JEM, sub-block based prediction is adopted in several coding tools, such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), bi-directional optical flow (BIO), frame-rate up conversion (FRUC), locally adaptive motion vector resolution (LAMVR), overlapped block motion compensation (OBMC), local illumination compensation (LIC), and decoder-side motion vector refinement (DMVR).

[0085] 2.1 Example of Adaptive Motion Vector Difference Resolution

[0086] In some embodiments, when use_integer_mv_flag is equal to 0 in slice header, the motion vector difference (MVD) (between the motion vector of a PU and the predicted motion vector) is signaled in quarter luma sample units. In JEM, locally adaptive motion vector resolution (LAMVR) is introduced. In JEM, the MVD can be coded in quarter luma sample, integer luma sample, or four luma sample units. The MVD resolution is controlled at the coding unit (CU) level, and for each CU with at least one non-zero MVD component, a MVD resolution flag is conditionally signaled.

[0087] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or four luma sample MV precision is used.

[0088] When the first MVD resolution flag of a CU is zero, or no MVD is coded for the CU (meaning all MVDs in the CU are zero), quarter luma sample MV resolution is used for the CU. When a CU uses integer luma sample MV precision or quarter luma sample MV precision, the MVPs in the AMVP candidate list for the CU are rounded to the corresponding precision.

[0089] In the encoder, CU-level RD checks are used to determine which MVD resolution will be used for a CU. That is, for each MVD resolution, the CU-level RD check is performed three times. To speed up the encoder, the following encoding scheme is applied in JEM:

[0090] - During the RD check of a CU with normal quarter luma sample MVD resolution, the motion information (integer luma sample precision) of the current CU is stored. During the RD check of the same CU with integer luma sample and 4 luma sample MVD resolutions, the stored motion information (after rounding) is used as a starting point for further small range motion vector refinement, so that the time-consuming motion estimation process is not repeated three times.

[0091] - The RD check of a CU with 4 luma sample MVD resolution is conditionally invoked. For a CU, when the RD cost of integer luma sample MVD resolution is much larger than that of quarter luma sample MVD resolution, the RD check of the CU with 4 luma sample MVD resolution will be skipped.

[0092] 2.2 Example of higher motion vector storage precision

[0093] In HEVC, the motion vector precision is one quarter of a pixel (pel) (one quarter of a luma sample and one eighth of a chroma sample for 4:2:0 video). In JEM, the precision of the internal motion vector storage and Merge candidates is increased to 1 / 16 pel. The higher motion vector precision (1 / 16 pel) is used in the motion compensated inter prediction for CUs coded in skip / Merge mode. For CUs coded in normal AMVP mode, integer pel or quarter pel motion is used.

[0094] The SHVC up-sampling interpolation filter with the same filter length and normalization factor as the HEVC motion compensation interpolation filter is used as the motion compensation interpolation filter for additional fractional pel positions. In JEM, the chroma component motion vector precision is 1 / 32 sample, the additional interpolation filter for 1 / 32 pel fractional positions is derived by using the average of the filters of two neighboring 1 / 16 pel fractional positions.

[0095] 2.3 Example of affine motion compensation prediction

[0096] In HEVC, only translational motion model is applied for motion compensation prediction (MCP). However, cameras and objects can have multiple kinds of motion, e.g., zooming, rotation, perspective motion, and / or other irregular motion. On the other hand, JEM applies a simplified affine transform motion compensation prediction. Figure 10 An example of an affine motion field of a block 1000 described by two control point motion vectors V0and V1is shown. The motion vector field (MVF) of the block 1000 can be described by the following equation:

[0097]

[0098] As shown in Figure 10 (v 0x , v 0y ) is the motion vector of the top-left control point, and (v 1x , v 1y ) is the motion vector of the top-right control point. To simplify motion compensation prediction, a subblock-based affine transform prediction can be applied. The subblock size M x N is derived as follows:

[0099]

[0100] Here, MvPreis the motion vector fractional precision (e.g., 1 / 16 in JEM). (v 2x , v 2y ) is the motion vector of the bottom-left control point calculated according to equation (1). If necessary, M and N can be adjusted down to be divisors of w and h, respectively.

[0101] Figure 11 An example of the affine motion vector of each subblock of a block 1100 is shown. To derive the motion vector of each M x N subblock, the motion vector of the center sample of each subblock can be calculated according to equation (1) and rounded to the motion vector fractional precision (e.g., 1 / 16 in JEM). Then, a motion compensation interpolation filter can be applied to generate the prediction of each subblock with the derived motion vector. After MCP, the high-precision motion vector of each subblock is rounded and saved to the same precision as the normal motion vector.

[0102] 2.3.1 Embodiment of AF INTER mode

[0103] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. AF_INTER mode can be applied to CUs whose width and height are both greater than 8. Signaling in the bitstream informs the affine flag in the CU level to indicate whether AF_INTER mode is used. In AF_INTER mode, neighboring blocks are used to construct motion vector pairs {(v0,v1)|v0={v...}. A ,v B ,v c},v1={v D ,v E The candidate list of}}.

[0104] Figure 12 An example of motion vector prediction (MVP) for block 1200 in AF_INTER mode is shown. Figure 12 As shown, v0 is selected from the motion vectors of sub-blocks A, B, or C. Motion vectors from neighboring blocks can be scaled according to a reference list. The motion vectors can also be scaled based on the relationship between the Picture Order Count (POC) of the references of neighboring blocks, the POC of the current CU's reference, and the POC of the current CU. The scheme for selecting v1 from neighboring sub-blocks D and E is similar. If the number of candidates in the candidate list is less than two, the list is filled with motion vector pairs formed by copying each AMVP candidate. When the candidate list is greater than two, the candidates can be sorted first based on neighboring motion vectors (e.g., based on the similarity of two motion vectors in a pair). In some implementations, the first two candidates are retained. In some embodiments, a Rate Distortion (RD) cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) for the current CU. Signaling notifications in the bitstream indicate the index of the CPMVP in the candidate list. After determining the CPMVP of the current affine CU, affine motion estimation is applied, and the control point motion vector (CPMV) is found. Then, the difference between CPMV and CPMVP is communicated via signaling in the bitstream.

[0105] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required, and therefore 2 / 3 MVD encoding / decoding is needed for these control points, such as... Figure 13A and 13B As shown. In existing implementations, MV can be derived as follows, for example, it predicts mvd1 and mvd2 from mvd0.

[0106]

[0107]

[0108]

[0109] Here, mvd i and mv1 are the predicted motion vector, motion vector difference and motion vector of the top-left pixel (i = 0), top-right pixel (i = 1) or bottom-left pixel, respectively, as shown in Figure 13B In some embodiments, the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) equals the sum of the two components, respectively. For example, newMV = mvA + mvB means that the two components of newMV are set to (xA + xB) and (yA + yB), respectively.

[0110] 2.3.3 Embodiments of AF_MERGE mode

[0111] When a CU applies in AF_MERGE mode, it obtains the first block coded with affine mode from the valid neighboring reconstructed blocks. Figure 14A An example of the selection order of the candidate blocks of the current CU 1400 is shown. As shown in Figure 14A The selection order can be from the left (1401), top (1402), top-right (1403), bottom-left (1404) to the top-left (1405) of the current CU 1400. Figure 14B Another example of the candidate blocks of the current CU 1400 in AF_Merge mode is shown. If the neighboring bottom-left block 1401 is coded with affine mode, as shown in Figure 14B The motion vectors v2, v3 and v4 containing the top-left, top-right and bottom-left corners of the CU of the sub-block 1401 are derived. The motion vector v0 of the top-left corner on the current CU 1400 is calculated based on v2, v3 and v4. The motion vector v1 of the top-right of the current CU can be calculated accordingly.

[0112] After the CPMVs v0 and v1 of the current CU are calculated according to the affine motion model in equation (1), the MVFs of the current CU can be generated. To identify whether the current CU is coded with AF_MERGE mode, an affine flag can be signaled in the bitstream when at least one neighboring block is coded with affine mode.

[0113] 2.4 Example of bi-directional optical flow (BIO)

[0114] The bidirectional optical flow (BIO) method is a simple-wise motion refinement performed on top of bidirectional predicted block-wise motion compensation. In some implementations, the simple-wise motion refinement does not use signaling notification.

[0115] Let I (k) The brightness value from reference k (k = 0, 1) after block motion compensation, and... and Represented as I (k) The horizontal and vertical components of the gradient. Assuming optical flow is effective, the motion vector field (v) x v y The following formula is given:

[0116]

[0117] To determine the motion trajectory of each sample point, this optical flow equation is combined with Hermite interpolation to obtain a unique third-order polynomial, which is related to the function value I. (k) and the derivative at the endpoint. and Both match. The value of the polynomial at t=0 is the BIO prediction:

[0118]

[0119] Figure 15 An example of optical flow trajectories in the Two-Way Optical Flow (BIO) method is shown. Here, τ0 and τ1 represent the distances to the reference frames. The distances τ0 and τ1 are calculated based on the POCs of Ref0 and Ref1: τ0 = POC(current) - POC(Ref0), τ1 = POC(Ref1) - POC(current). If both predictions originate from the same time direction (either both from the past or both from the future), then the signs are different (e.g., τ0·τ1 < 0). In this case, if the predictions do not originate from the same time (e.g., τ0 ≠ τ1), then BIO is applied. Both reference regions have non-zero motion (e.g., MVx0, MVy0, MVx1, MVy1 ≠ 0), and the block motion vector is proportional to the time distance (e.g., MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0120] Motion vector field (v) x ,v y It is determined by minimizing the difference Δ between the values ​​at points A and B. Figure 9 An example is shown where the motion trajectory intersects with the reference frame plane. The model uses only the first linear term of the local Taylor expansion for Δ:

[0121]

[0122] All values ​​in the above equations depend on the sample location, represented as (i′,j′). Assuming the motion is consistent in the local surrounding region, Δ can be minimized within a (2M+1)×(2M+1) square window Ω centered at the currently predicted point (i,j), where M equals 2:

[0123]

[0124] Then minimize it in the horizontal direction. This yields the following result:

[0125]

[0126]

[0127] in,

[0128]

[0129] To avoid division by zero or very small values, regularization parameters r and m can be introduced in equations (9) and (10), where:

[0130] r = 500·4 d-8 Equation (12)

[0131] m = 700·4 d-8 Equation (13)

[0132] Here, d is the bit depth of the video sample.

[0133] To maintain consistency between memory access for BIO and motion compensation for regular bidirectional prediction, all predictions and gradient values ​​are calculated for the position within the current block. Figure 16A An example of an access location outside block 1600 is shown. Figure 16A As shown, in equation (9), a (2M+1)×(2M+1) square window Ω centered on the currently predicted point on the boundary of the predicted block needs to access locations outside the block. In JEM, the location outside the block... The value is set to be equal to the nearest available value within that block. For example, this could be implemented to fill region 1601, such as... Figure 16B As shown.

[0134] Using BIO, the motion field can be refined for each sample point. To reduce computational complexity, a block-based design of BIO is used in JEM. Motion refinement can be calculated based on 4×4 blocks. In block-based BIO, the s of all samples in the 4×4 block are in equation (9). nThe values can be aggregated and then aggregated s n The values are used to derive the BIO motion vector offsets for 4x4 blocks. More specifically, the following equation can be used for block-based BIO derivation:

[0135]

[0136] Here, b k denotes the set of samples belonging to the k-th 4x4 block of the predicted block. The s n n,bk in equations (9) and (10) are replaced by ((s n,bk >>4) to derive the associated motion vector offsets.

[0137] In some scenarios, the MV regiment of BIO can be unreliable due to noise or irregular motion. Therefore, in BIO, the size of the MV regiment is clipped to a threshold. The threshold is determined based on whether the reference pictures of the current picture all come from one direction. For example, if all the reference pictures of the current picture come from one direction, the threshold is set to 12x2 14-d ; otherwise, it is set to 12x2 13-d .

[0138] The gradients of BIO can be computed simultaneously using motion compensation interpolation with operations consistent with the HEVC motion compensation process (e.g., 2D separable Finite Impulse Response (FIR)). In some embodiments, the input to the 2D separable FIR is the same reference frame samples as for the reference frame samples of the motion compensation process and is according to the fractional position (fracX, fracY) of the fractional part of the block motion vector. For the horizontal gradient The signal is first vertically interpolated using the BIO filter S that corresponds to the fractional position fracY with a de-scaling shift d-8. Then the gradient filter BIO filter G is applied in the horizontal direction that corresponds to the fractional position fracX with a de-scaling shift of 18-d. For the vertical gradient The BIOfilterG corresponding to the fractional position fracY with a de- scaling shift d-8 is used. Then, a signal displacement is performed using the BIOfilterS in the horizontal direction corresponding to the fractional position fracX with a de- scaling shift of 18-d. The length of the interpolation filter BIOfilterG for gradient computation and the interpolation filter BIOfilterF for signal displacement can be shorter (e.g., 6-tap) in order to keep reasonable complexity. Table 1 shows example filters that can be used for gradient computation for different fractional positions of the block motion vector in BIO. Table 2 shows example interpolation filters that can be used for prediction signal generation in BIO.

[0139] Table 1: Example filters for gradient computation in BIO

[0140] Fractional pixel position Interpolation filter for gradients (BIOfilterG) 0 {8,-39,-3,46,-17,5} 1 / 16 {8,-32,-13,50,-18,5} 1 / 8 {7,-27,-20,54,-19,5} 3 / 16 {6,-21,-29,57,-18,5} 1 / 4 {4,-17,-36,60,-15,4} 5 / 16 {3,-9,-44,61,-15,4} 3 / 8 {1,-4,-48,61,-13,3} 7 / 16 {0,1,-54,60,-9,2} 1 / 2 {-1,4,-57,57,-4,1}

[0141] Table 2: Example interpolation filters for prediction signal generation in BIO

[0142] Fractional pixel position Interpolation filter for prediction signal (BIOfilterS) 0 {0,0,64,0,0,0} 1 / 16 {1,-3,64,4,-2,0} 1 / 8 {1,-6,62,9,-3,1} 3 / 16 {2,-8,60,14,-5,1} 1 / 4 {2,-9,57,19,-7,2} 5 / 16 {3,-10,53,24,-8,2} 3 / 8 {3,-11,50,29,-9,2} 7 / 16 {3,-11,44,35,-10,3} 1 / 2 {3,-10,35,44,-11,3}

[0143] In JEM, BIO can be applied to all bi-predicted blocks when the two predictions come from different reference pictures. BIO can be disabled when local illumination compensation (LIC) is enabled for a CU.

[0144] In some embodiments, OBMC is applied for a block after the normal MC process. To reduce the computational complexity, BIO can not be applied during the OBMC process. This means that BIO is applied in the MC process for a block when using its own MV, while not applied in the MC process when using the MV of the neighboring block during the OBMC process.

[0145] 2.5 Example of decoder-side motion vector refinement (DMVR)

[0146] In bi-prediction operation, two prediction blocks formed using the motion vector (MV) of listO and the MV of listl, respectively, are combined to form a single prediction signal for the prediction of one block region. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of bi-prediction are further refined by a bilateral template matching process. The bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference picture in order to obtain the refined MV without transmitting additional motion information.

[0147] In DMVR, a bilateral template is generated as Figure 17The weighted combination (i.e., averaging) of the two prediction blocks shown, the initial MV0 from listO and the initial MV1 from listl. The template matching operation includes computing a cost measure between the generated template and a region of samples in the reference picture (a region around the block of the initial prediction). For each of the two reference pictures, the MV that produces the minimum template cost is considered as the updated MV of that list to replace the original MV. In JEM, nine MV candidates are searched for each list. The nine candidate MVs include the original MV and 8 surrounding MVs with one luma sample offset in the horizontal or vertical direction or both directions relative to the original MV. Finally, two new MVs, MV0' and MV1', shown are used to generate the final bi-predicted result. The sum of absolute difference (SAD) is used as the cost measure. Figure 17

[0148] DMVR is applied to the Merge mode of bi-prediction with one MV from a reference picture from the past and another MV from a reference picture from the future without transmission of additional syntax elements. In JEM, DMVR is not applied when LIC, affine motion, FRUC or sub-CU Merge candidates are enabled for a CU.

[0149] 2.6 Example of symmetric motion vector difference (SMVD) mode

[0150] Symmetric motion vector difference (SMVD) is proposed to encode MVD more efficiently. First, at slice level, the variables BiDirPredFlag, RefIdxSymL0 and RefIdxSymL1 are derived as follows:

[0151] o Search the closest forward reference picture in reference picture list 0 to the current picture. If found, RefIdxSymL0 is set equal to the reference index of the forward picture.

[0152] o Search the closest backward reference picture in reference picture list 1 to the current picture. If found, RefIdxSymL1 is set equal to the reference index of the backward picture.

[0153] o If both forward and backward pictures are found, BiDirPredFlag is set to 1.

[0154] o Otherwise, the following applies:

[0155] ​• The closest backward reference picture in reference picture list 0 to the current reference picture is searched. If found, RefldxSymLo is set equal to the reference index of the backward picture.

[0156] • The closest forward reference picture in reference picture list 1 to the current reference picture is searched. If found, RefldxSymLl is set equal to the reference index of the forward picture.

[0157] • If both backward and forward pictures are found, BiDirPredFlag is set to 1. Otherwise, BiDirPredFlag is set equal to 0.

[0158] Second, at the CU level, if the prediction direction of the CU is bi-prediction and BiDirPredFlag is equal to 1, a symmetric mode flag indicating whether to use the symmetric mode is explicitly signaled.

[0159] When the flag is true, only mvp_l0_flag, mvp_l1_flag and MVD0 are explicitly signaled. For list 0 and list 1, the reference indices are set equal to RefldxSymLo, RefldxSymLl, respectively. MVDl is only set equal to -MVD0. The final motion vector is shown in the following equation:

[0160]

[0161] The modification of the coding unit syntax is shown in Table 3.

[0162] Table 3: Modification of the coding unit syntax

[0163]

[0164]

[0165] 7.3.4.9 Motion vector difference syntax

[0166]

[0167] 2.6.1 Symmetric MVD for affine bi-prediction coding

[0168] A SMVD for affine mode is proposed. In this scheme, the concept of SMVD is extended to the coding of CPMVs of affine coded blocks. In the following description, this method is referred to as affine SMVD.

[0169] 3. Drawbacks of existing implementations

[0170] In some existing implementations, SMVD and affine SMVD modes have been introduced to reduce the overhead of coding MVD. In addition, the goal of AMVR and affine AMVR modes is the same. However, the interaction between these two different types of modes is not well defined.

[0171] 4. Example methods of interaction between MV precision and MV coding

[0172] Embodiments of the presently disclosed technology overcome the shortcomings of existing implementations to provide video coding with higher coding efficiency. Based on the disclosed technology, the interaction between motion vector precision and motion vector coding can enhance existing and future video coding standards, which will be illustrated in the examples described below for various implementations. The examples of the disclosed technology provided below explain general concepts and are not meant to be interpreted as limiting. In the examples, the various features described in these examples can be combined unless explicitly indicated to the contrary.

[0173] In this document (and in the examples below), the terms “affine SMVD” and “affine AMVR” apply to blocks coded in affine mode, while other terms like “SMVD” and “AMVR” apply to non-affine coded blocks. In addition, lower MVD precision means coarser MVD precision; for example, 1-pixel MVD precision is lower than ¼-pixel MVD precision. Higher MVD precision means finer MVD precision; for example, 1 / 16-pixel MVD precision is higher than ¼-pixel MVD precision.

[0174] 1. Whether SMVD is enabled and / or how SMVD is applied for a block can depend on the motion information and / or MVD precision and / or MV precision of the block.

[0175] a. In one example, SMVD can be disabled for some CUs depending on MVD precision, etc.

[0176] i. In one example, SMVD can be disabled when MVD precision is lower than or equal to one precision (e.g., integer-pixel precision).

[0177] ii. In one example, SMVD can be disabled when MVD precision is higher than or equal to one precision (e.g., integer-pixel precision).

[0178] iii. Alternatively, some MVD precisions can be disabled when SMVD is applied. For example, 4-pixel precision MVD can be disabled in SMVD mode.

[0179] iv. In addition, alternatively, the signaling of MVD precision (AMVR indication) can depend on the use of SMVD.

[0180] 1. In one example, when SMVD is disabled for one MVD precision, the signaling of MDV precision (e.g., amvr_mode in the norm for non-affine inter modes) can be modified accordingly.

[0181] 2. In one example, if one MVD precision is not allowed when SMVD is enabled, only one bin can be coded for AMVR mode.

[0182] b. In one example, SMVD can be disabled when all MVD components of prediction direction 0 or / and 1 are zero.

[0183] c. In one example, SMVD mode can be signaled after the motion information of prediction direction 0 or 1.

[0184] i. In one example, SMVD mode is not signaled if all MVD components of prediction direction 0 or 1 are zero.

[0185] ii. In one example, SMVD mode is not signaled if the absolute value of horizontal or / and vertical MVD component of prediction direction 0 or 1 is less than a threshold.

[0186] iii. In one example, SMVD mode is not signaled if the absolute sum of horizontal and vertical MVD components of prediction direction 0 or 1 is less than a threshold.

[0187] iv. In one example, SMVD mode is not signaled if the absolute value of horizontal or / and vertical MVD component of prediction direction 0 or 1 is greater than a threshold.

[0188] v. In one example, SMVD mode is not signaled if the absolute sum of horizontal and vertical MVD components of prediction direction 0 or 1 is greater than a threshold.

[0189] vi. When SMVD mode is not signaled, it can be implicitly derived as false, e.g., SMVD mode is not applied.

[0190] 2. For an affine coded block, whether affine SMVD is enabled and / or how SMVD is applied can depend on the affine motion information and / or MVD precision and / or MV precision of the block.

[0191] a. In one example, affine SMVD can be disabled for some CUs according to affine MVD precision, etc.

[0192] i. In one example, affine SMVD can be disabled when affine MVD precision is lower than or equal to one precision (e.g., integer pixel precision).

[0193] ii. In one example, affine MVD precision higher than or equal to one precision (e.g., integer pixel precision) can disable affine SMVD.

[0194] iii. Alternatively, some affine MVD precisions can be disabled when SMVD is applied. For example, in SMVD mode, integer pixel precision affine MVD can be disabled.

[0195] iv. Furthermore, alternatively, the signaling of affine MVD precision (affine AMVR indication) can depend on the usage of SMVD.

[0196] 1. In one example, furthermore, when SMVD is disabled for one MVD precision, the signaling of MDV precision (e.g., amvr_mode in the specification for affine inter mode) can be modified accordingly.

[0197] 2. In one example, if one MVD precision is not allowed when SMVD is enabled, only one bin can be coded for AMVR mode.

[0198] b. In one example, affine SMVD can be disabled when all affine MVD components of prediction 0 or / and 1 are zero.

[0199] c. In one example, affine SMVD mode can be signaled after the motion information of prediction direction 0 or 1.

[0200] i. In one example, affine SMVD mode is not signaled if all affine MVD components of prediction direction 0 or 1 are zero.

[0201] ii. In one example, affine SMVD mode is not signaled if the absolute values of horizontal or / and vertical affine MVD components of N (N >= 1) control points on prediction direction 0 or 1 are smaller than a threshold.

[0202] iii. In one example, SMVD mode is not signaled if the absolute sum of horizontal and vertical affine MVD components of N (N >= 1) control points on prediction direction 0 or 1 are smaller than a threshold.

[0203] iv. In one example, affine SMVD mode is not signaled if the absolute values of horizontal or / and vertical affine MVD components of N (N >= 1) control points on prediction direction 0 or 1 are larger than a threshold.

[0204] v. In one example, SMVD mode is not signaled if the absolute sum of horizontal and vertical affine MVD components of N (N >= 1) control points on prediction direction 0 or 1 are larger than a threshold.

[0205] vi. When affine SMVD mode is not signaled, it can be implicitly derived as false, i.e., affine SMVD mode is not applied.

[0206] 3. The signaling of motion vector difference can depend on the usage of SMVD / affine SMVD.

[0207] a. In one example, when SMVD is enabled for a block and the horizontal component of motion vector difference is equal to 0, the vertical component of motion vector difference shall not be equal to 0.

[0208] b. In one example, the vertical component of motion vector difference (e.g., abs_mvd_greater0_flag[1]) can be conditionally signaled.

[0209] i. In one example, the signaling of the vertical component can depend on whether SMVD or affine SMVD is enabled for the current block.

[0210] ii. In one example, the signaling of abs_mvd_greater0_flag[1] can depend on the following condition:

[0211] 1. SMVD flag (or affine SMVD) is true and the value of the horizontal component of motion vector difference is not equal to 0 (e.g., abs_mvd_greater0_flag[0] is true); or SMVD flag (or affine SMVD) is false.

[0212] 2. When the above condition is false, the signaling of abs_mvd_greater0_flag[1] is skipped.

[0213] a. In this case, abs_mvd_greater0_flag[1] can be implicitly derived as true.

[0214] 4. The allowed set of MVD precisions can depend on the usage of SMVD.

[0215] a. The allowed set of affine MVD precisions can depend on the usage of affine SMVD.

[0216] b. In one example, when SMVD (or affine SMVD) is enabled, lower MVD precisions can be allowed, such as (1 / 4-pixel, 2-pixel, 8-pixel).

[0217] c. In one example, when SMVD (or affine SMVD) is enabled, higher MVD precisions can be allowed, such as (1 / 4-pixel, 1 / 2-pixel, 1 / 8-pixel).

[0218] 5. It can be signaled in SPS / PPS / VPS / sequence header / picture header / tile group header / tile / set of CTUs, etc. whether to enable or disable the above methods.

[0219] a. Alternatively, it can be signaled in SPS / PPS / VPS / sequence header / picture header / tile group header / tile / set of CTUs, etc. which method to use.

[0220] b. Alternatively, whether to enable or disable the above methods and / or which method to apply can depend on block size of the current block (such as reference picture, uni-prediction or bi-prediction), Virtual Pipelining Data Unit (VPDU), picture type, low delay check flag, coded information of the current block or previously coded blocks.

[0221] The examples described above can be incorporated in the context of the methods described below, such as method 1900, which can be implemented at a video decoder or a video encoder.

[0222] Figure 19 A flowchart of an example method for video processing is shown. The method 1900 includes, at step 1910, determining between enabling and disabling a symmetric motion vector difference (SMVD) mode for a current video block based on at least one of motion information, motion vector difference (MVD) precision, or motion vector (MV) precision of the current video block.

[0223] The method 1900 includes, at step 1920, reconstructing the current video block based on the determination.

[0224] In some embodiments, the SMVD mode is disabled due to the MVD precision being lower than or equal to a threshold precision. In other embodiments, the SMVD mode is disabled due to the MVD precision being higher than or equal to a threshold precision. In an example, the threshold precision is integer pixel precision.

[0225] In some embodiments, the SMVD mode is disabled due to all MVD components of a first prediction direction or a second prediction direction of the current video block being zero.

[0226] In some embodiments, the SMVD mode is signaled after the motion information of a first prediction direction or a second prediction direction of the current video block. In an example, an absolute value of a horizontal or vertical MVD component of the first prediction direction or the second prediction direction is lower than a MVD threshold. In another example, the absolute value of the horizontal or vertical MVD component of the first prediction direction or the second prediction direction is higher than the MVD threshold.

[0227] In some embodiments, the at least one MVD is signaled based on the determination.

[0228] In some embodiments, the SMVD mode is enabled and the MVD precision is ¼ pixel, 2 pixels, or 8 pixels. In other embodiments, the SMVD mode is enabled and the MVD precision is ¼ pixel, ½ pixel, or 1 / 8 pixel.

[0229] In some embodiments, the determining is further based on signaling in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a slice header, a slice header, a sequence header, a picture header, a tile, or a group of coding tree units (CTUs).

[0230] In some embodiments, the determining is further based on a height or a width of the current video block, a virtual pipeline data unit (VPDU) picture type, a low delay check flag, or coding information associated with the current video block or a previously coded block.

[0231] In some embodiments, and in the context of Example 2 and with reference to method 1900, the motion information is affine motion information, the MVD precision is affine MVD precision, the MV precision is affine MV precision, and the SMVD mode is affine SMVD mode.

[0232] 5. Example implementations of the disclosed technology

[0233] Figure 20 is a block diagram of a video processing apparatus 2000. The apparatus 2000 can be used to implement one or more methods described herein. The apparatus 2000 can be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, and the like. The apparatus 2000 can include one or more processors 2002, one or more memories 2004, and video processing hardware 2006. The processor(s) 2002 can be configured to implement one or more methods described in the present document, including but not limited to method 1900. The memory(ies) 2004 can be used for storing data and code used for implementing the methods and techniques described here. The video processing hardware 2006 can be used to implement, in hardware circuitry, some of the techniques described in the present document.

[0234] In some embodiments, the video coding method can be implemented using an apparatus implemented on a hardware platform as described with respect to Figure 20 In some embodiments, the video coding method can be implemented using an apparatus implemented on a hardware platform as described with respect to

[0235] Figure 21is a flowchart of a method 2100 of processing a video. The method 2100 includes determining (2102), for a first block of the video and a conversion between the first block and a bitstream representation of the first block, whether a symmetric motion vector difference (SMVD) mode for the first block is enabled or disabled and / or how to apply the SMVD mode for the first block based on at least one of motion information for the first block, a motion vector difference (MVD) precision, and a motion vector (MV) precision; and performing (2104) the conversion based on the determination.

[0236] In some examples, the SMVD mode is disabled for a partial coding unit (CU) depending on the MVD precision.

[0237] In some examples, the SMVD mode is disabled when the MVD precision is lower than or equal to a threshold precision.

[0238] In some examples, the SMVD mode is disabled when the MVD precision is higher than or equal to a threshold precision.

[0239] In some examples, the threshold precision is an integer pixel precision.

[0240] In some examples, a partial MVD precision is disabled when the SMVD mode is applied.

[0241] In some examples, signaling of the MVD precision depends on the use of the SMVD.

[0242] In some examples, the signaling of the MVD precision is modified accordingly when the SMVD mode is disabled for one MVD precision.

[0243] In some examples, only one bit is coded for the AMVR mode if one MVD precision is not allowed when the SMVD mode is enabled.

[0244] In some examples, the SMVD mode is disabled when all MVD components of reference picture list 0 and / or reference picture list 1 for the first block are zero.

[0245] In some examples, the SMVD mode is signaled after the motion information of reference picture list 0 and / or reference picture list 1 for the first block.

[0246] In some examples, the SMVD mode is not signaled if all MVD components of reference picture list 0 or reference picture list 1 for the first block are zero.

[0247] In some examples, the SMVD mode is not signaled if the absolute value of the horizontal or / and vertical MVD components of reference picture list 0 or reference picture list 1 for the first block is less than a threshold.

[0248] In some examples, the SMVD mode is not signaled if the absolute sum of the horizontal and vertical MVD components of the reference picture list 0 or the reference picture list 1 of the first block is less than a threshold.

[0249] In some examples, the SMVD mode is not signaled if the absolute value of the horizontal or / and vertical MVD component of the reference picture list 0 or the reference picture list 1 of the first block is greater than a threshold.

[0250] In some examples, the SMVD mode is not signaled if the absolute sum of the horizontal and vertical MVD components of the reference picture list 0 or the reference picture list 1 of the first block is greater than a threshold.

[0251] In some examples, the SMVD mode is not applied when the SMVD mode is not signaled.

[0252] In some examples, the signaling of the MVD depends on a determination of whether the SMVD mode is enabled or disabled for the first block.

[0253] In some examples, the vertical component of the MVD is not equal to 0 when the SMVD mode is enabled for the first block and the horizontal component of the MVD is equal to 0.

[0254] In some examples, the vertical component of the MVD is conditionally signaled.

[0255] In some examples, the signaling of the vertical component of the MVD depends on a determination of whether the SMVD mode is enabled or disabled for the first block.

[0256] In some examples, the signaling of an indication of whether the vertical component of the MVD is non-zero depends on one of the following conditions: the SMVD flag is true and the value of the horizontal component of the MVD is not equal to 0; or the SMVD flag is false.

[0257] In some examples, the signaling of the indication of whether the vertical component of the MVD is non-zero is skipped when the condition is false.

[0258] In some examples, the vertical component of the MVD is implicitly derived to be non-zero.

[0259] In some examples, the allowed set of MVD precisions depends on a determination of whether the SMVD mode is enabled or disabled for the first block.

[0260] In some examples, a lower MVD precision is allowed when the SMVD mode is enabled, wherein the lower MVD precision includes at least one of ¼ pixel, 2 pixels, and 8 pixels.

[0261] In some examples, when the SMVD mode is enabled, higher MVD precision is allowed, where the higher MVD precision includes at least one of ¼-pixel, ½-pixel, or 1 / 8-pixel.

[0262] In some examples, the determination is further based on an indication signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a tile group header, a slice header, a sequence header, a picture header, a tile, or a group of coding tree units (CTUs).

[0263] In some examples, the determination is further dependent on at least one of a block size, a virtual pipeline data unit (VPDU), a picture type, a low delay check flag, or coding information associated with the first block or a previously coded block.

[0264] In some examples, when the first block is coded in the affine mode, the SMVD mode is an affine SMVD mode, the motion information is affine motion information, the MVD precision is affine MVD precision, the MV precision is affine MV precision, and the SMVD MVD component is an affine SMVD MVD component.

[0265] In some examples, the affine SMVD mode is not signaled if absolute values of horizontal or / and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are less than a threshold, where N is an integer greater than and / or equal to 1.

[0266] In some examples, the affine SMVD mode is not signaled if an absolute sum of horizontal and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are less than a threshold, where N is an integer greater than and / or equal to 1.

[0267] In some examples, the affine SMVD mode is not signaled if absolute values of horizontal or / and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are greater than a threshold, where N is an integer greater than and / or equal to 1.

[0268] In some examples, the affine SMVD mode is not signaled if an absolute sum of horizontal and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are greater than a threshold, where N is an integer greater than and / or equal to 1.

[0269] In some examples, the conversion generates the first block of the video from the bitstream representation.

[0270] In some examples, the conversion generates the bitstream representation from the first block of the video.

[0271] From the foregoing, it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications can be made without deviating from the scope of the application. Accordingly, the presently disclosed technology is not limited except as by the appended claims.

[0272] Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term“data processing apparatus” or“data processing device” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0273] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0274] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, and apparatuses can also be implemented as such special purpose logic circuitry.

[0275] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0276] The specification and drawings are, of course, exemplary only and serve only as an example. As used in this specification, the use of the singular is intended to include the plural unless the context clearly indicates otherwise.

[0277] Although the patent document includes many details, these should not be construed as limiting the scope of any invention or claimed invention, but as a description of particular features of a particular embodiment. Some of the features described in the patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any appropriate

[0278] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring this particular order or sequence of operations, or that all illustrated operations be performed, to achieve desirable results. In addition, the separation of various system components in the embodiments described in the patent document should not be understood as requiring such separation in all embodiments.

[0279] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method of video processing, comprising: determining, based on at least one of motion information of a first block of a video, motion vector difference (MVD) precision, and motion vector (MV) precision, whether a symmetric motion vector difference (SMVD) mode for the first block is enabled or disabled and / or how to apply the SMVD mode for the first block for a conversion between the first block and a bitstream of the first block; and performing the conversion based on the determining; wherein signaling of the MVD precision depends on usage of the SMVD mode; when the SMVD mode is disabled for one MVD precision, signaling of the MVD precision is modified accordingly; when one MVD precision is not allowed and the SMVD mode is enabled, only one bit is coded for an adaptive motion vector resolution (AMVR) mode. the SMVD mode is disabled for a partial coding unit (CU) according to the MVD precision.

2. The method of claim 1, wherein, the SMVD mode is disabled when the MVD precision is lower than or equal to a threshold precision.

3. The method of claim 1, wherein, the SMVD mode is disabled when the MVD precision is higher than or equal to a threshold precision.

4. The method of claim 1, wherein, the threshold precision is an integer pixel precision.

5. The method of claim 3, wherein, the threshold precision is an integer pixel precision.

6. The method of claim 4, wherein, a partial MVD precision is disabled when the SMVD mode is applied.

7. The method of claim 1, wherein, the SMVD mode is disabled when all MVD components of reference picture list 0 and / or reference picture list 1 of the first block are zero.

8. The method of claim 1, wherein, the SMVD mode is signaled after motion information of reference picture list 0 and / or reference picture list 1 of the first block.

9. The method of claim 1, wherein, the SMVD mode is not signaled if all MVD components of reference picture list 0 or reference picture list 1 of the first block are zero.

10. The method of claim 9, wherein, the SMVD mode is not signaled if absolute values of horizontal or / and vertical MVD components of reference picture list 0 or reference picture list 1 of the first block are less than a threshold.

11. The method of claim 9, wherein, the SMVD mode is not signaled if an absolute sum of horizontal and vertical MVD components of reference picture list 0 or reference picture list 1 of the first block is less than a threshold.

12. The method of claim 9, wherein, the SMVD mode is not signaled if absolute values of horizontal or / and vertical MVD components of reference picture list 0 or reference picture list 1 of the first block are greater than a threshold.

13. The method of claim 9, wherein, the SMVD mode is not signaled if an absolute sum of horizontal and vertical MVD components of reference picture list 0 or reference picture list 1 of the first block is greater than a threshold.

14. The method of claim 9, wherein, the SMVD mode is not applied when the SMVD mode is not signaled.

15. The method of claim 9, wherein, signaling of MVD depends on the determination of whether the SMVD mode for the first block is enabled or disabled.

16. The method of claim 1, wherein, a vertical component of MVD is not equal to 0 when the SMVD mode is enabled for the first block and a horizontal component of MVD is equal to 0.

17. The method of claim 16, wherein, a vertical component of MVD is conditionally signaled.

18. The method of claim 16, wherein, signaling of the vertical component of MVD depends on the determination of whether the SMVD mode for the first block is enabled or disabled.

19. The method of claim 18, wherein, ​ 20. The method of claim 18, wherein, The signaling of the indication of whether the vertical component of the MVD is non-zero depends on one of the following conditions: the SMVD flag is true and the value of the horizontal component of the MVD is not equal to 0; or the SMVD flag is false.

21. The method of claim 20, wherein, The signaling of the indication of whether the vertical component of the MVD is non-zero is skipped when the condition is false.

22. The method of claim 21, wherein, The vertical component of the MVD is implicitly derived to be non-zero.

23. The method of claim 1, wherein, The allowed set of MVD precisions depends on a determination of whether the SMVD mode is enabled or disabled for the first block.

24. The method of claim 23, wherein, When the SMVD mode is enabled, lower MVD precisions are allowed, wherein the lower MVD precisions include at least one of ¼-pixel, 2-pixel, and 8-pixel.

25. The method of claim 23, wherein, When the SMVD mode is enabled, higher MVD precisions are allowed, wherein the higher MVD precisions include at least one of ¼-pixel, ½-pixel, or 1 / 8-pixel.

26. The method of claim 1, wherein, The determination is further based on an indication signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a tile group header, a slice header, a sequence header, a picture header, a tile, or a group of coding tree units (CTUs).

27. The method of claim 1, wherein, The determination further depends on at least one of a block size, a virtual pipeline data unit (VPDU), a picture type, a low delay check flag, or coding information associated with the first block or a previously coded block.

28. The method of claim 1, wherein, When the first block is coded in an affine mode, the SMVD mode is an affine SMVD mode, the motion information is affine motion information, the MVD precision is affine MVD precision, the MV precision is affine MV precision, and the SMVD MVD component is an affine SMVD MVD component.

29. The method of claim 28, wherein, The affine SMVD mode is not signaled if absolute values of horizontal or / and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are less than a threshold, wherein N is an integer greater than and / or equal to 1.

30. The method of claim 28, wherein, The affine SMVD mode is not signaled if an absolute sum of horizontal and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are less than a threshold, wherein N is an integer greater than and / or equal to 1.

31. The method of claim 28, wherein, The affine SMVD mode is not signaled if absolute values of horizontal or / and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are greater than a threshold, wherein N is an integer greater than and / or equal to 1.

32. The method of claim 28, wherein, The affine SMVD mode is not signaled if an absolute sum of horizontal and vertical affine MVD components of N control points in reference picture list 0 or reference picture list 1 of the first block are greater than a threshold, wherein N is an integer greater than and / or equal to 1.

33. The method of any one of claims 1 to 32, wherein, The conversion generates the first block of video from the bitstream.

34. The method of any one of claims 1 to 32, wherein, The conversion generates the bitstream from the first block of video.

35. An apparatus in a video system, the apparatus comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement a method recited in any of claims 1-34.

36. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code which, when executed by a processor, implements the method of any one of claims 1 to 34.

Images