Fast algorithm for symmetric motion vector difference coding mode

Abstract

A method of video processing is provided. The method includes determining, for a conversion between a current video block of a video and a coded representation of the current video block, whether to use a symmetric motion vector difference (SMVD) mode based on a currently selected best mode for the conversion, and performing the conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 074216, filed on January 31, 2019, and International Patent Application No. PCT / CN2019 / 074433, filed on February 1, 2019, under applicable patent law and / or rules pursuant to the Paris Convention. The disclosures of the aforementioned applications are hereby incorporated by reference into this disclosure for all purposes. Technical Field

[0003] This patent document relates to video processing technology, devices and systems. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth requirements for digital video usage are expected to continue to grow. Summary of the Invention

[0005] Devices, systems, and methods related to digital video coding are described, and in particular, motion vector prediction (MVP) derivation and signaling for an affine mode with adaptive motion vector resolution (AMVR) are described. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Codec (HEVC)) and future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a video processing method. The method includes: determining that a conversion between a current video block of a video and a codec representation of the current video block is based on a non-affine inter-AMVR mode; and performing the conversion based on the determination, wherein the codec representation of the current video block is based on context-based coding, and wherein context for coding the current video block is modeled without using affine AMVR mode information of neighboring blocks during the conversion.

[0007] In another representative aspect, the disclosed technology can be used to provide a video processing method. The method includes: determining that a conversion between a current video block of a video and a codec representation of the current video block is based on an affine adaptive motion vector resolution (affine AMVR) mode; and performing the conversion based on the determination, wherein the codec representation of the current video block is based on a context-based codec, and wherein variables control two probability update rates of the context.

[0008] In another representative aspect, the disclosed technology can be used to provide a video processing method. The method includes: determining that a conversion between a current video block of a video and a codec representation of the current video block is based on an affine AMVR mode; and performing the conversion based on the determination, wherein the codec representation of the current video block is based on context-based coding, and wherein codec information of neighboring blocks is used to model a context for coding the current video block, the codec information of the neighboring blocks using both an affine inter mode and a normal inter mode AMVR mode during the conversion.

[0009] In another representative aspect, the disclosed technology can be used to provide a video processing method, the method comprising: determining, for a conversion between a current video block of a video and a codec representation of the current video block, using a plurality of contexts for conversion; and performing the conversion based on the determination, wherein the plurality of contexts is utilized to encode a syntax element indicating coarse motion accuracy.

[0010] In another representative aspect, the disclosed technology can be used to provide a video processing method. The method includes: for converting between a current video block of a video and a codec representation of the current video block, determining whether to use a symmetric motion vector difference (SMVD) mode based on a currently selected best mode for the conversion; and performing the conversion based on the determination.

[0011] In another representative aspect, the disclosed technology can be used to provide a video processing method. The method includes: for converting between a current video block of a video and a codec representation of the current video block, determining whether to use an affine SMVD mode based on a currently selected best mode for the conversion; and performing the conversion based on the determination.

[0012] In another representative aspect, the above method is implemented in the form of processor-executable code and stored in a computer-readable program medium.

[0013] In yet another representative aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

[0014] In yet another representative aspect, a video decoder device may implement a method as described herein.

[0015] The above-described aspects and features and other aspects and features of the disclosed technology are described in more detail in the drawings, the description, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 An example of constructing a Merge candidate list is shown.

[0017] Figure 2Examples of locations of spatial candidates are shown.

[0018] Figure 3 An example of candidate pairs subjected to redundancy check for spatial domain Merge candidates is shown.

[0019] Figure 4A and Figure 4B An example of the position of the second prediction unit PU based on the size and shape of the current block is shown.

[0020] Figure 5 An example of motion vector scaling for temporal merge candidates is shown.

[0021] Figure 6 An example of candidate positions for time-domain Merge candidates is shown.

[0022] Figure 7 An example of generating a combined bi-predictive Merge candidate is shown.

[0023] Figure 8 An example of constructing motion vector prediction candidates is shown.

[0024] Figure 9 An example of motion vector scaling for spatial motion vector candidates is shown.

[0025] Figure 10 An example of motion prediction using an alternative temporal motion vector prediction (ATMVP) algorithm for a coding unit (CU) is shown.

[0026] Figure 11 An example of a codec unit (CU) with sub-blocks and neighboring blocks used by a spatial-temporal motion vector prediction (STMVP) algorithm is shown.

[0027] Figure 12A and Figure 12B Example snapshots of sub-blocks are shown when using an overlapped block motion compensation (OBMC) algorithm.

[0028] Figure 13 An example of neighboring samples used to derive parameters for a local illumination compensation (LIC) algorithm is shown.

[0029] Figure 14 An example of a simplified affine motion model is shown.

[0030] Figure 15 An example of the affine motion vector field (MVF) for each sub-block is shown.

[0031] Figure 16 An example of motion vector prediction (MVP) for AF_INTER affine motion mode is shown.

[0032] Figure 17A and 17B Examples of a 4-parameter affine model and a 6-parameter affine model are shown respectively.

[0033] Figure 18A and 18B Example candidates for the AF_MERGE affine motion mode are shown.

[0034] Figure 19 An example of bilateral matching in the Pattern Matching Motion Vector Derivation (PMMVD) mode, which is based on a special Merge mode of the Frame Rate Up-Conversion (FRUC) algorithm, is shown.

[0035] Figure 20 An example of template matching in the FRUC algorithm is shown.

[0036] Figure 21 An example of unidirectional motion estimation in the FRUC algorithm is shown.

[0037] Figure 22 An example of an optical flow trajectory used by the Bidirectional Optical Flow (BIO) algorithm is shown.

[0038] Figure 23A and 23B Example snapshots using the Bidirectional Optical Flow (BIO) algorithm without block expansion are shown.

[0039] Figure 24 An example of a decoder-side motion vector refinement (DMVR) algorithm based on bilateral template matching is shown.

[0040] Figures 25A-25F A flow chart illustrating an example method for video processing based on an implementation of the disclosed technology is shown.

[0041] Figure 26 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0042] Figure 27 An example of a symmetric pattern is shown.

[0043] Figure 28 Another block diagram illustrating an example of a hardware platform for implementing the video processing system described in this document. DETAILED DESCRIPTION

[0044] Due to the increasing demand for higher resolution video, video coding and decoding methods and techniques are ubiquitous in modern technology. Video codecs typically consist of electronic circuits or software that compress or decompress digital video and are constantly being improved to provide higher coding and decoding efficiency. Video codecs convert uncompressed video into a compressed format and vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). The compression format typically conforms to a standard video compression specification, such as the High Efficiency Video Codec (HEVC) standard (also known as H.265 or MPEG-H Part 2) [1], the yet-to-be-finalized general video codec standard, or other current and / or future video codec standards.

[0045] Embodiments of the disclosed technology can be applied to existing video codec standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve the readability of the description and do not in any way limit the discussion or embodiments (and / or implementations) to only the corresponding section.

[0046] 1. Example of inter-frame prediction in HEVC / H.265

[0047] Over the years, video codec standards have improved significantly and now offer, in part, high codec efficiency and support for higher resolutions. The latest standards, such as HEVC and H.265, are based on a hybrid video codec structure that uses temporal prediction plus transform coding.

[0048] 1.1 Example of prediction model

[0049] Each inter-predicted prediction unit (PU) has motion parameters for one or two reference picture lists. In some embodiments, the motion parameters include a motion vector and a reference picture index. In other embodiments, inter_pred_idc may be used to signal the use of one of the two reference picture lists. In still other embodiments, the motion vector may be explicitly encoded as a delta relative to the predictor.

[0050] When a codec unit (CU) is encoded or decoded using skip mode, a PU is associated with the CU and there are no significant residual coefficients, no coded motion vector increments or reference picture indices. Merge mode is specified, whereby the motion parameters for the current PU are obtained from neighboring PUs, including spatial and temporal candidates. Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to merge mode is explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index for each reference picture list, and the reference picture list usage are explicitly signaled for each PU.

[0051] When signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of samples. This is called "unidirectional prediction". Unidirectional prediction is available for P slices and B slices [2].

[0052] When signaling indicates that two reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bi-prediction." Bi-prediction is only available for B slices.

[0053] 1.1.1 Implementation of Candidates for Merge Mode

[0054] When using Merge mode to predict a PU, the index to the entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list can be summarized in the following steps:

[0055] Step 1: Initial candidate derivation

[0056] Step 1.1: Spatial Candidate Derivation

[0057] Step 1.2: Redundancy check of spatial candidates

[0058] Step 1.3: Time Domain Candidate Derivation

[0059] Step 2: Additional candidate insertions

[0060] Step 2.1: Create bidirectional prediction candidates

[0061] Step 2.2: Insert zero motion candidates

[0062] Figure 1An example of constructing a Merge candidate list based on the order of steps summarized above is shown. For spatial Merge candidate derivation, a maximum of four Merge candidates are selected from candidates located at five different positions. For temporal Merge candidate derivation, a maximum of one Merge candidate is selected from two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, binary unary truncation (TU) is used to encode the index of the best Merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

[0063] 1.1.2 Constructing Spatial Merge Candidates

[0064] In the derivation of spatial Merge candidates, Figure 2 Up to four Merge candidates are selected from the candidates in the positions depicted in . The order of derivation is A1, B1, B0, A0, and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is unavailable (for example, because the PU belongs to another slice or tile) or when intra coding. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates. The redundancy check ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency.

[0065] In order to reduce computational complexity, not all possible candidate pairs are considered in the proposed redundancy check. Instead, only pairs with Figure 3 The arrows in the list link pairs, and only if the corresponding candidate used for redundancy check has different motion information, then the candidate is added to the list. Another source of duplicate motion information is a "second PU" associated with a partition different from 2Nx2N. As an example, Figure 4A and 4B The second PU is depicted for the N×2N and 2N×N cases, respectively. When the current PU is partitioned into N×2N, the candidate at position A1 is not considered for list construction. In some embodiments, adding this candidate can result in two prediction units with the same motion information, which is redundant for having only one PU in the codec unit. Similarly, position B1 is not considered when the current PU is partitioned into 2N×N.

[0066] 1.1.3 Building Time Domain Merge Candidates

[0067] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located PU, which belongs to the picture with the smallest picture order count (POC) difference relative to the current picture in a given reference picture list. The reference picture list used for the derivation of the co-located PU is explicitly signaled in the slice header.

[0068] Figure 5 An example of the derivation of a scaled motion vector for a temporal merge candidate (shown as a dashed line) is shown, where the scaled motion vector is scaled from the motion vector of the collocated PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal merge candidate is set equal to zero. For B slices, two motion vectors are obtained and combined to produce a bi-predictive merge candidate, one for reference picture list 0 and the other for reference picture list 1.

[0069] like Figure 6 As shown, in a co-located PU (Y) belonging to a reference frame, the position for the temporal candidate is selected between candidates C0 and C1. If the PU at position C0 is not available, is intra-coded, or is outside the current CTU, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0070] 1.1.4 Building Merge Candidates of Additional Types

[0071] In addition to spatial-temporal merge candidates, there are two additional types of merge candidates: combined bi-predictive merge candidates and zero merge candidates. Combined bi-predictive merge candidates are generated by utilizing spatial-temporal merge candidates. Combined bi-predictive merge candidates are used only for B slices. Combined bi-predictive candidates are generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they will form a new bi-predictive candidate.

[0072] Figure 7An example of this process is shown where two candidates in the original list (710, on the left) with mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create a combined bi-predictive Merge candidate that is added to the final list (720, on the right).

[0073] Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list and thus reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional and bidirectional prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

[0074] 1.1.5 Example of Motion Estimation Regions for Parallel Processing

[0075] To speed up the encoding process, motion estimation can be performed in parallel, thereby deriving motion vectors for all prediction units in a given region at the same time. The derivation of Merge candidates from spatially adjacent PUs may interfere with parallel processing, because one prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is completed. To alleviate the trade-off between codec efficiency and processing latency, a motion estimation region (MER) can be defined. The size of the MER can be signaled in the picture parameter set (PPS) using the "log2_parallel_merge_level_minus2" syntax element. When a MER is defined, Merge candidates that fall into the same region are marked as unavailable and are therefore not considered in list construction.

[0076] 1.2 Advanced Motion Vector Prediction (AMVP) Implementation

[0077] AMVP utilizes the spatial-temporal correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. The motion vector candidate list is constructed by first checking the availability of neighboring PU positions in the time domain to the left and above, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to Merge index signaling, unary truncation is used to encode the index of the best motion vector candidate. The maximum value to be encoded in this case is 2 (see Figure 8 ). In the following sections, details on the derivation process of motion vector prediction candidates are provided.

[0078] 1.2.1 Example of Constructing Motion Vector Prediction Candidates

[0079] Figure 8 The derivation process for motion vector prediction candidates is summarized and can be implemented for each reference picture list with refidx as input.

[0080] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. Figure 2 As shown previously, for spatial motion vector candidate derivation, two motion vector candidates are ultimately derived based on the motion vectors of each PU located at five different positions.

[0081] For temporal motion vector candidate derivation, a motion vector candidate is selected from two candidates derived based on two different co-located positions. After creating the first list of spatial-temporal candidates, duplicate motion vector candidates are removed from the list. If the number of potential candidates is greater than two, motion vector candidates whose reference picture index within the associated reference picture list is greater than one are removed from the list. If the number of spatial-temporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0082] 1.2.2 Constructing Spatial Motion Vector Candidates

[0083] In the derivation of spatial motion vector candidates, a maximum of two candidates are considered among five potential candidates from the locations such as Figure 2 The PUs at the positions shown previously are the same as those of the motion merge. The derivation order for the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order for the top side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling and two of which use spatial scaling. The four different cases are summarized below.

[0084] No airspace scaling

[0085] (1) Same reference picture list and same reference picture index (same POC)

[0086] (2) Different reference picture lists, but same reference picture index (same POC)

[0087] Airspace scaling

[0088] (3) Same reference picture list, but different reference picture index (different POC)

[0089] (4) Different reference picture lists and different reference picture indexes (different POCs)

[0090] First, check for the case where no spatial scaling is performed, followed by the case where spatial scaling is allowed. Regardless of the reference picture list, spatial scaling is considered when the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU. If all PUs of the left candidate are unavailable or are intra-coded, scaling of the upper motion vector is allowed to facilitate the parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

[0091] like Figure 9 As shown in the example in , for the spatial scaling case, the motion vectors of the neighboring PUs are scaled in a similar way to the temporal scaling. The only difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the temporal scaling process.

[0092] 1.2.3 Constructing Temporal Motion Vector Candidates

[0093] Except for the reference picture index derivation, all the processes for derivation of temporal Merge candidates are the same as those for derivation of spatial motion vector candidates (e.g. Figure 6 In some embodiments, the reference picture index is signaled to the decoder.

[0094] 2. Example of inter-frame prediction method in Joint Exploration Model (JEM)

[0095] In some embodiments, a reference software called Joint Exploration Model (JEM) [3] [4] is used to explore future video codec technologies. In JEM, sub-block based prediction is adopted in several codec tools, such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), bidirectional optical flow (BIO), frame rate up-conversion (FRUC), locally adaptive motion vector resolution (LAMVR), overlapped block motion compensation (OBMC), local luminance compensation (LIC), and decoder-side motion vector refinement (DMVR).

[0096] 2.1 Example of motion vector prediction based on sub-CU

[0097] In JEM with quadtree plus binary tree (QTBT), each CU can have at most one set of motion parameters for each prediction direction. In some embodiments, two sub-CU level motion vector prediction methods are considered in the encoder by dividing the large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU. The optional temporal motion vector prediction (ATMVP) method allows each CU to obtain multiple sets of motion information from multiple blocks smaller than the current CU in the co-located reference picture. In the spatial-temporal motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using the temporal motion vector prediction and the spatial neighboring motion vectors. In some embodiments, and in order to preserve a more accurate motion field for sub-CU motion prediction, motion compression of the reference frame can be disabled.

[0098] 2.1.1 Example of Optional Temporal Motion Vector Prediction (ATMVP)

[0099] In the ATMVP method, the temporal motion vector prediction (TMVP) method is modified by retrieving multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU.

[0100] Figure 10 An example of the ATMVP motion prediction process for CU 1000 is shown. The ATMVP method predicts the motion vector of sub-CU 1001 within CU 1000 in two steps. The first step is to use the time domain vector to identify the corresponding block 1051 in reference picture 1050. Reference picture 1050 is also called the motion source picture. The second step is to divide the current CU 1000 into sub-CUs 1001 and obtain the motion vector and reference index of each sub-CU from the block corresponding to each sub-CU.

[0101] In the first step, the reference picture 1050 and the corresponding block are determined based on the motion information of the spatially neighboring blocks of the current CU 1000. To avoid repeated scanning of neighboring blocks, the first merge candidate in the merge candidate list of the current CU 1000 is used. The first available motion vector and its associated reference index are set to the temporal vector and the index of the motion source picture. This allows for more accurate identification of corresponding blocks compared to TMVP, where the corresponding block (sometimes referred to as a co-located block) is always located in the lower right or center position relative to the current CU.

[0102] In the second step, the corresponding blocks of the sub-CU 1051 are identified by the temporal vector in the motion source picture 1050 by adding the temporal vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (e.g., the minimum motion grid covering the center sample) is used to derive the motion information of the sub-CU. After the motion information of the corresponding N×N block is identified, it is converted into the motion vector and reference index of the current sub-CU in the same way as the TMVP of HEVC, where motion scaling and other processes are applied. For example, the decoder checks whether the low latency condition is met (e.g., the POC of all reference pictures of the current picture is less than the POC of the current picture) and may use the motion vector MVx (e.g., the motion vector corresponding to the reference picture list X) to predict the motion vector MVy of each sub-CU (e.g., where X is equal to 0 or 1 and Y is equal to 1-X).

[0103] 2.1.2 Example of Spatial-Temporal Motion Vector Prediction (STMVP)

[0104] In the STMVP method, the motion vectors of sub-CUs are recursively derived in raster scan order. Figure 11 An example of a CU with four sub-blocks and neighboring blocks is shown. Consider an 8×8 CU 1100 including four 4×4 sub-CUs A (1101), B (1102), C (1103), and D (1104). The neighboring 4×4 blocks in the current frame are labeled (1111), b (1112), c (1113), and d (1114).

[0105] The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU A1101 (block c 1113). If block c (1113) is not available or is intra-coded, the other N×N blocks above sub-CU A (1101) are checked (from left to right, starting from block c 1113). The second neighbor is the block to the left of sub-CU A 1101 (block b 1112). If block b (1112) is not available or is intra-coded, the other blocks to the left of sub-CU A 1101 are checked (from top to bottom, starting from block b 1112). The motion information obtained from the neighboring blocks for each list is scaled to the first reference frame of the given list. Next, the temporal motion vector prediction (TMVP) of sub-block A 1101 is derived by following the same process as the TMVP derivation specified in HEVC. The motion information of the co-located block at block D 1104 is retrieved and scaled accordingly. Finally, after extracting and scaling the motion information, all available motion vectors are averaged separately for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

[0106] 2.1.3 Example of Sub-CU Motion Prediction Mode Signaling

[0107] In some embodiments, sub-CU mode is enabled as an additional Merge candidate, and no additional syntax elements are required to signal the mode. Two additional Merge candidates are added to merge the candidate list of each CU to represent the ATMVP mode and the STMVP mode. In other embodiments, if the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven Merge candidates can be used. The encoding logic of the additional Merge candidates is the same as that of the Merge candidates in the HM, which means that for each CU in the P or B slice, the two additional Merge candidates may need to undergo two more RD checks. In some embodiments, such as JEM, all binary bits of the Merge index are context-coded by CABAC (context-based adaptive binary arithmetic coding). In other embodiments, such as HEVC, only the first binary number (bin) is context-coded, and the remaining binary numbers are context-bypass coded.

[0108] 2.2 Example of Adaptive Motion Vector Difference Resolution

[0109] In some embodiments, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the PU's motion vector and the predicted motion vector) is signaled in units of quarter luma samples. In JEM, Local Adaptive Motion Vector Resolution (LAMVR) is introduced. In JEM, MVD can be coded or decoded in units of quarter luma samples, integer luma samples, or four luma samples. The MVD resolution is controlled at the codec unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU with at least one non-zero MVD component.

[0110] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or four luma sample MV precision is used.

[0111] When the first MVD resolution flag of a CU is zero, or is not coded for the CU (meaning all MVDs in the CU are zero), a quarter luma sample MV resolution is used for the CU. When the CU uses integer luma sample MV precision or four luma sample MV precision, the MVP in the AMVP candidate list of the CU is rounded to the corresponding precision.

[0112] In the encoder, a CU-level RD check is used to determine which MVD resolution will be used for the CU. In other words, for each MVD resolution, the CU-level RD check is performed three times. In order to speed up the encoder, the following encoding scheme is applied in JEM.

[0113] During RD check of a CU with normal quarter luma sample MVD resolution, the motion information of the current CU (integer luma sample precision) is stored. For the same CU with integer luma samples and 4 luma sample MVD resolution, the stored motion information (after rounding) is used as a starting point for further small-scale motion vector refinement during RD check, so that the time-consuming motion estimation process is not repeated three times.

[0114] Conditionally call RD check for CUs with 4 luma sample MVD resolution. For a CU, when the RD cost of integer luma sample MVD resolution is much greater than the RD cost of quarter luma sample MVD resolution, skip the RD check of 4 luma sample MVD resolution for the CU.

[0115] 2.3 Example of Higher Motion Vector Storage Accuracy

[0116] In HEVC, motion vector precision is quarter-pixel (pel) (one-quarter luma sample and one-eighth chroma sample for 4:2:0 video). In JEM, the precision of internal motion vector storage and merge candidates is increased to 1 / 16 pixel. The higher motion vector precision (1 / 16 pixel) is used for motion compensated inter prediction of CUs coded in skip mode / merge mode. For CUs coded in normal AMVP mode, integer pixel or quarter-pixel motion is used.

[0117] The SHVC upsampling interpolation filter with the same filter length and normalization factor as the HEVC motion compensation interpolation filter is used as the motion compensation interpolation filter for the additional fractional pixel position. In JEM, the chroma component motion vector accuracy is 1 / 32 sample, and the additional interpolation filter for the 1 / 32 pixel fractional position is derived by using the average of the filters at two adjacent 1 / 16 pixel fractional positions.

[0118] 2.4 Example of Overlapped Block Motion Compensation (OBMC)

[0119] In JEM, OBMC can be switched on and off at the CU level using syntax elements. When OBMC is used in JEM, OBMC is performed on all motion compensation (MC) block boundaries except the right and bottom boundaries of the CU. In addition, it is applied to both luminance and chrominance components. In JEM, MC blocks correspond to codec blocks. When a CU is encoded and decoded using sub-CU modes (including sub-CU Merge, affine, and FRUC modes), each sub-block of the CU is an MC block. In order to uniformly handle CU boundaries, OBMC is performed on all MC block boundaries at the sub-block level when the sub-block size is set to 4x4, as shown in Figure 12A and Figure 12B shown.

[0120] Figure 12A The sub-blocks at the CU / PU boundary are shown, and the shaded sub-blocks are where OBMC is applied. Similarly, Figure 12B A sub-PU in ATMVP mode is shown.

[0121] When OBMC is applied to the current sub-block, in addition to the current MV, the vectors of the four neighboring sub-blocks (if available and not identical to the current motion vector) are also used to derive the prediction block for the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal for the current sub-block.

[0122] The prediction block based on the motion vectors of neighboring subblocks is denoted as PN, where N indicates the indices of the neighboring subblocks above, below, to the left, and to the right, and the prediction block based on the motion vector of the current subblock is denoted as PC. When the PN is based on the motion information of a neighboring subblock and this motion information is the same as that of the current subblock, OBMC is not performed on the PN. Otherwise, each sample of the PN is added to the same sample in the PC, i.e., four rows / columns of PN are added to the PC. Weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for the PN, and weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for the PC. The exception is that for small MC blocks (i.e., when the height or width of the codec block is equal to 4 or the CU is coded using sub-CU mode), only two rows / columns of PN are added to the PC. In this case, weighting factors {1 / 4, 1 / 8} are used for the PN, and weighting factors {3 / 4, 7 / 8} are used for the PC. For a PN generated based on the motion vectors of vertically (horizontally) adjacent sub-blocks, samples in the same row (column) of the PN are added to the PC with the same weighting factor.

[0123] In JEM, for CUs with a size less than or equal to 256 luma samples, a CU-level flag is signaled to indicate whether OBMC is applied to the current CU. For CUs with a size greater than 256 luma samples or not encoded in AMVP mode, OBMC is applied by default. At the encoder, when OBMC is applied to a CU, its impact is considered during the motion estimation stage. The prediction signal formed by OBMC using the motion information of the top and left neighboring blocks is used to compensate for the top and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0124] 2.5 Example of Local Luminance Compensation (LIC)

[0125] LIC is based on a linear model for luminance variation, using a scaling factor a and an offset b, and is adaptively enabled or disabled for each inter-mode coded coding unit (CU).

[0126] When LIC is applied to a CU, the least square error method is adopted to derive parameters a and b by using the neighboring samples of the current CU and its corresponding reference samples. Figure 13 An example of neighboring samples used to derive parameters for the IC algorithm is shown. Specifically, and as Figure 13 As shown, neighboring samples of the CU's subsampling (2:1 subsampling) and corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used. IC parameters are derived and applied to each prediction direction separately.

[0127] When the Merge mode is used to encode and decode a CU, the LIC flag is copied from the neighboring block in a manner similar to the motion information copying in the Merge mode; otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applicable.

[0128] When LIC is enabled for a picture, an additional CU-level RD check is required to determine whether LIC is applied to the CU. When LIC is enabled for a CU, the mean-removed sum of absolute diffefference (MR-SAD) and mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively.

[0129] In order to reduce the encoding complexity, the following encoding scheme is applied in JEM.

[0130] When there is no significant brightness change between the current picture and its reference pictures, LIC is disabled for the entire picture. To identify this situation, the encoder calculates the histogram of the current picture and each of its reference pictures. If the histogram difference between the current picture and each of its reference pictures is less than a given threshold, LIC is disabled for the current picture; otherwise, LIC is enabled for the current picture.

[0131] 2.6 Example of Affine Motion Compensated Prediction

[0132] In HEVC, only the translational motion model is applied to motion compensated prediction (MCP). However, the camera and objects can have various motions, such as zooming in / out, rotation, perspective motion, and other irregular motions. On the other hand, JEM applies a simplified affine transformation motion compensated prediction. Figure 14 An example of an affine motion field of a block 1400 described by two control point motion vectors V0 and V1 is shown. The motion vector field (MVF) of the block 1400 can be described by the following equation:

[0133]

[0134] like Figure 14 As shown, (v 0x ,v 0y ) is the motion vector of the left top corner control point, and (v 1x ,v 1y ) is the motion vector of the right top corner control point. To simplify motion compensation prediction, sub-block based affine transformation prediction can be applied. The sub-block size M×N is derived as follows:

[0135]

[0136] Where MvPre is the motion vector fractional precision (e.g. 1 / 16 in JEM). 2x ,v 2y ) is the motion vector of the lower left control point, calculated according to equation (1). If necessary, M and N can be adjusted downward to make them divisors of w and h respectively.

[0137] Figure 15 An example of an affine MVF for each sub-block of block 1500 is shown. To derive the motion vector for each M×N sub-block, the motion vector of the center sample of each sub-block can be calculated according to equation (1) and rounded to a fractional motion vector precision (e.g., 1 / 16 in JEM). A motion-compensated interpolation filter can then be applied to generate a prediction for each sub-block using the derived motion vector. After MCP, the high-precision motion vector for each sub-block is rounded and saved with the same precision as the normal motion vector.

[0138] 2.6.1 AF_INTER Mode Example

[0139] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs with width and height greater than 8, AF_INTER mode can be applied. The affine flag in the CU level is signaled in the bitstream to indicate whether AF_INTER mode is used. In AF_INTER mode, a pair of motion vectors {(v0,v1)|v0={v A ,v B ,v c},v1={v D ,v E}}'s candidate list.

[0140] Figure 16 An example of motion vector prediction (MVP) for block 1600 in AF_INTER mode is shown. Figure 16 As shown, v0 is selected from the motion vectors of sub-blocks A, B, or C. Motion vectors from neighboring blocks can be scaled based on a reference list. Motion vectors can also be scaled based on the relationship between the picture order count (POC) of the neighboring block's reference, the POC of the current CU's reference, and the POC of the current CU. The method for selecting v1 from neighboring sub-blocks D and E is similar. If the number of candidate lists is less than 2, the list is populated with motion vector pairs constructed by repeating each AMVP candidate. When the candidate list is greater than 2, the candidates can first be sorted based on neighboring motion vectors (e.g., based on the similarity of the two motion vectors in the candidate pair). In some implementations, the top two candidates are retained. In some implementations, a rate-distortion (RD) cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) for the current CU. An index indicating the position of the CPMVP in the candidate list can be signaled in the bitstream. After determining the CPMVP for the current affine CU, affine motion estimation is applied and the control point motion vector (CPMV) is found. The difference between the CPMV and the CPMVP is then signaled in the bitstream.

[0141] In AF_INTER mode, when using 4 / 6 parameter affine mode, 2 / 3 control points are required, and therefore 2 / 3 MVDs need to be encoded and decoded for these control points, as shown in Figure 17. In existing implementations [5], MVs can be derived as follows, for example, it predicts mvd1 and mvd2 from mvd0.

[0142]

[0143]

[0144]

[0145] In this article, mvd i and mv1 are the predicted motion vector, motion vector difference, and motion vector of the left top pixel (i=0), right top pixel (i=1), or left bottom pixel (i=2), respectively. Figure 18B In some embodiments, the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the sum of the two components individually. For example, newMV=mvA+mvB means that the two components of newMV are set to (xA+xB) and (yA+yB), respectively.

[0146] 2.6.2 Example of the Fast Affine ME Algorithm in AF_INTER Mode

[0147] In some embodiments of the affine mode, it is necessary to jointly determine the MVs of two or three control points. Direct joint search for multiple MVs is computationally complex. In the example, the fast affine ME algorithm [6] is proposed and adopted into the VTM / BMS.

[0148] For example, the fast affine ME algorithm is described for a 4-parameter affine model, and the idea can be extended to a 6-parameter affine model:

[0149]

[0150]

[0151] Replacing (a-1) with a' enables the motion vector to be rewritten as:

[0152]

[0153] If it is assumed that the motion vectors of the two control points (0, 0) and (0, w) are known, then according to equation (5), the affine parameters can be derived as:

[0154]

[0155] The motion vector can be rewritten in vector form as:

[0156]

[0157] In this paper, P = (x, y) is the pixel position,

[0158]

[0159] and

[0160]

[0161] In some embodiments, and at the encoder, the MVD of AF_INTER may be derived iteratively. Denote MVi(P) as the MV derived in the i-th iteration at position P, and denote dMV C i is represented as the increment updated for MVC in the i-th iteration. Then in the (i+1)-th iteration,

[0162]

[0163] Pic ref Indicate as reference picture and Pic cur Denote the current picture and Q = P + MV i (P). If MSE is used as the matching criterion, the function to be minimized can be written as:

[0164]

[0165] If you assume If it is small enough, it can be expanded based on the first-order Taylor expansion. Rewrite as an approximation, such as:

[0166]

[0167] In this article, If the notation E is used i+1 (P)=Pic cur (P)-Pic ref (Q), then

[0168]

[0169] the term This can be done by setting the derivative of the error function to zero and then according to The incremental MV of the control points (0, 0) and (0, w) is derived as follows:

[0170]

[0171]

[0172]

[0173]

[0174] In some embodiments, the MVD derivation process may be iterated n times, and the final MVD may be calculated as follows:

[0175]

[0176]

[0177]

[0178]

[0179] In the above implementation [5], predicting the incremental MV of the predicted control point (0, w) represented by mvd1 from the incremental MV of the control point (0, 0) represented by mvd0 results in Encoded only for mvd1.

[0180] 2.6.3 AF_MERGE Mode Example

[0181] When a CU is applied in AF_MERGE mode, it obtains the first block decoded using affine mode from valid neighboring reconstructed blocks. Figure 18A An example of the selection order of candidate blocks of the current CU 1800 is shown. Figure 18A As shown, the selection order may be from the left ( 1801 ), top ( 1802 ), top right ( 1803 ), bottom left ( 1804 ) to top left ( 1805 ) of the current CU 1800 . Figure 18B Another example of candidate blocks for the current CU 1800 in AF_MERGE mode is shown. Figure 18B As shown in FIG, if the adjacent lower left block 1801 is coded in affine mode, motion vectors v2, v3, and v4 are derived for the top left corner, top right corner, and bottom left corner of the CU containing the sub-block 1801. Based on v2, v3, and v4, the motion vector v0 of the top left corner of the current CU 1800 is calculated. The motion vector v1 of the top right corner of the current CU can be calculated accordingly.

[0182] After calculating the CPMV of the current CU v0 and v1 according to the affine motion model in equation (1), the MVF of the current CU can be generated. In order to identify whether the current CU is coded or decoded using the AF_MERGE mode, an affine flag can be signaled in the bitstream when there is at least one neighboring block coded or decoded in the affine mode.

[0183] 2.7 Example of Pattern Matched Motion Vector Derivation (PMMVD)

[0184] PMMVD mode is a special Merge mode based on the Frame Rate Up-Conversion (FRUC) method. In this mode, the motion information of the block is not signaled but derived at the decoder side.

[0185] When the Merge flag of a CU is true, the FRUC flag may be signaled to the CU. When the FRUC flag is false, the Merge index may be signaled and the normal Merge mode may be used. When the FRUC flag is true, an additional FRUC mode flag may be signaled to indicate which method (e.g., bilateral matching or template matching) will be used to derive motion information for the block.

[0186] On the encoder side, the decision on whether to use the FRUC Merge mode for a CU is based on the RD cost selection made for normal merge candidates. For example, multiple matching modes (e.g., bilateral matching and template matching) for a CU are checked using RD cost selection. The matching mode that results in the minimum cost is further compared with other CU modes. If the FRUC matching mode is the most efficient mode, the FRUC flag is set to true for the CU and the relevant matching mode is used.

[0187] Generally, the motion derivation process in FRUC Merge mode has two steps. First, CU-level motion search is performed, followed by sub-CU-level motion refinement. At the CU level, an initial motion vector is derived for the entire CU based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate that results in the minimum matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching is performed around the starting point. The MV that results in the minimum matching cost is taken as the MV of the entire CU. Subsequently, the motion information is further refined at the sub-CU level, with the derived CU motion vector as the starting point.

[0188] For example, the following derivation process is performed for W×HCU motion information derivation. In the first stage, the MV of the entire W×HCU is derived. In the second stage, the CU is further divided into M×M sub-CUs. The value of M is calculated as in Equation (3), where D is the predefined partition depth, which is set to 3 by default in JEM. The MV of each sub-CU is then derived.

[0189]

[0190] Figure 19An example of bilateral matching used in a frame rate up conversion (FRUC) method is shown. Bilateral matching is used to derive motion information of a current CU by finding the closest match between two blocks along the motion trajectory of the current CU (1900) in two different reference pictures (1910, 1911). Under the assumption of a continuous motion trajectory, the motion vectors MV0 (1901) and MV1 (1902) pointing to the two reference blocks are proportional to the temporal distance between the current picture and the two reference pictures, for example, TD0 (1903) and TD1 (1904). In some embodiments, when the current picture (1900) is temporally between the two reference pictures (1910, 1911) and the temporal distance from the current picture to the two reference pictures is the same, the bilateral matching becomes a mirror-based bidirectional MV.

[0191] Figure 20 An example of template matching used in the frame rate up conversion (FRUC) method is shown. Template matching can be used to derive motion information of the current CU 2000 by finding the closest match between a template in the current picture (e.g., the top neighboring block and / or the left neighboring block of the current CU) and a block in the reference picture 2010 (e.g., having the same size as the template). In addition to the above-mentioned FRUC Merge mode, template matching can also be applied to the AMVP mode. In both JEM and HEVC, there are two candidates for AMVP. Using the template matching method, new candidates can be derived. If the newly derived candidate by template matching is different from the first existing AMVP candidate, it is inserted at the beginning of the AMVP candidate list, and then the list size is set to 2 (e.g., by removing the second existing AMVP candidate). When applied to the AMVP mode, only CU level search is applied.

[0192] The CU-level MV candidate set may include: (1) the original AMVP candidate if the current CU is in AMVP mode, (2) all Merge candidates, (3) several MVs in the interpolated MV field (described later), and the top and left adjacent motion vectors.

[0193] When bilateral matching is used, each valid MV of a Merge candidate can be used as input to generate MV pairs assuming bilateral matching. For example, a valid MV of a Merge candidate is in reference list A (MVa, ref a ). Then, find the reference picture ref of its paired bilateral MV in other reference list B b , so that ref a and ref b Located on different sides of the current picture in the time domain. b If not available, refb was identified as ref a Different references, and ref b The temporal distance to the current image is the smallest one in list B. b Afterwards, based on the current image and ref a ,ref b The temporal distance between them is calculated by scaling MVa to derive MVb.

[0194] In some implementations, four MVs from the interpolated MV field can also be added to the CU-level candidate list. More specifically, the interpolated MVs at positions (0,0), (W / 2,0), (0,H / 2), and (W / 2,H / 2) of the current CU are added. When FRUC is applied to AMVP mode, the original AMVP candidate is also added to the CU-level MV candidate set. In some implementations, at the CU level, 15 MVs for the AMVP CU and 13 MVs for the Merge CU can be added to the candidate list.

[0195] The MV candidate set at the sub-CU level includes: (1) the MV determined from the CU level search, (2) the neighboring MVs at the top, left, left top, and right top, (3) scaled versions of the parallel MVs from the reference picture, (4) one or more ATMVP candidates (e.g., up to four), and (5) one or more STMVP candidates (e.g., up to four). The scaled MVs from the reference picture are derived as follows. The reference pictures in the two lists are traversed. The MVs at the parallel positions of the sub-CU in the reference picture are scaled to the reference of the starting CU level MV. The ATMVP and STMVP candidates can be the first four candidates. At the sub-CU level, one or more MVs (e.g., up to seventeen) are added to the candidate list.

[0196] Generation of interpolated MV fields. Before encoding or decoding a frame, an interpolated motion field is generated for the entire picture based on unilateral ME. The motion field can then be used as a CU-level or sub-CU-level MV candidate later.

[0197] In some embodiments, the motion field of each reference picture in both reference lists is traversed at a 4x4 block level. Figure 21 An example of unidirectional motion estimation (ME) 2100 in the FRUC method is shown. For each 4×4 block, if the motion associated with the block passes through a 4×4 block in the current picture and the block has not yet been assigned any interpolated motion, the motion of the reference block is scaled to the current picture according to the temporal distances TD0 and TD1 (same as the MV scaling of TMVP in HEVC), and the scaled motion is assigned to the block in the current frame. If a non-scaled MV is assigned to the 4×4 block, the motion of the block is marked as unavailable in the interpolated motion field.

[0198] Interpolation and matching costs. When motion vectors point to fractional sample locations, motion compensated interpolation is required. To reduce complexity, both bilateral matching and template matching can use bilinear interpolation instead of the conventional 8-tap HEVC interpolation.

[0199] The calculation of the matching cost is slightly different at different steps. When selecting a candidate from a CU-level candidate set, the matching cost can be the sum of absolute differences (SAD) of bilateral matching or template matching. After determining the starting MV, the matching cost C of the bilateral matching of the sub-CU level search is calculated as follows:

[0200] Where w is a weighting factor. In some embodiments, w is empirically set to 4, MV and MV s Indicates the current MV and the starting MV respectively. SAD can still be used as the matching cost of template matching in sub-CU level search.

[0201] In FRUC mode, MVs are derived using only luma samples. The derived motion is used for both luma and chroma inter-frame prediction of MC. After determining the MVs, final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0202] MV refinement is a pattern-based MV search with bilateral matching cost or template matching cost as the criterion. In JEM, two search modes are supported - unrestricted center-biased diamond search (UCBDS) and adaptive cross search for MV refinement at CU level and sub-CU level, respectively. For CU level and sub-CU level MV refinement, the MV is directly searched with a quarter luma sample MV precision and then refined with an eighth luma sample MV precision. The search range for MV refinement at CU step and sub-CU step is set to equal 8 luma samples.

[0203] In bilateral matching Merge mode, bidirectional prediction is applied because the motion information of a CU is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference pictures. In template matching Merge mode, the encoder can choose between unidirectional prediction from list 0, unidirectional prediction from list 1, or bidirectional prediction for the CU. The choice can be based on the template matching cost as follows:

[0204] If costBi <= factor * min (cost0, cost1)

[0205] Use bidirectional prediction;

[0206] Otherwise, if cost0 <= cost1

[0207] Use one-way prediction from list 0;

[0208] otherwise,

[0209] Use the one-way prediction from List 1;

[0210] Where cost0 is the SAD of list 0 template matching, cost1 is the SAD of list 1 template matching, and costBi is the SAD of bi-prediction template matching. For example, when the value of factor is equal to 1.25, it means that the selection process is biased towards bi-prediction. Inter prediction direction selection can be applied to the CU level template matching process.

[0211] 2.8 Example of Bidirectional Optical Flow (BIO)

[0212] The Bidirectional Optical Flow (BIO) method is a sample-by-sample motion refinement performed on top of block-by-block motion compensation for bidirectional prediction. In some implementations, sample-level motion refinement does not use signaling.

[0213] Let I (k) is the luminance value of reference k (k=0,1) after block motion compensation, and Represented as I (k) The horizontal and vertical components of the gradient. Assuming that the optical flow is valid, the motion vector field (v x ,v y ) is given by the following equation.

[0214]

[0215] Combining this optical flow equation with the Hermite interpolation of the motion trajectory for each sample results in a matching function value I at the end (k) and derivatives The value of this polynomial at t = 0 is the BIO prediction:

[0216]

[0217] Figure 22An example of an optical flow trajectory in the bidirectional optical flow (BIO) method is shown. Where τ0 and τ1 represent the distance to the reference frame. The distances τ0 and τ1 are calculated based on the POC of Ref0 and Ref1: τ0 = POC(current) - POC(Ref0), τ1 = POC(Ref1) - POC(current). If both predictions come from the same temporal direction (either from the past or from the future), the signs are different (e.g., τ0·τ1 < 0). In this case, if the predictions are not from the same moment (e.g., τ0≠τ1), BIO is applied. Both reference regions have non-zero motion (e.g., MVx0, MVy0, MVx1, MVy1≠0) and the block motion vector is proportional to the temporal distance (e.g., MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0218] The motion vector field (v) is determined by minimizing the difference Δ between the values ​​in point A and point B. x ,v y ). Figure 23 shows an example of the intersection of the motion trajectory and the reference frame plane. The model only uses the first linear term of the local Taylor expansion for Δ:

[0219]

[0220] All values ​​in the above equations depend on the sample position denoted as (i′, j′). Assuming that the motion is consistent in the local surrounding area, Δ can be minimized within a (2M+1)×(2M+1) square window Ω centered at the current prediction point (i, j), where M is equal to 2:

[0221]

[0222] For this optimization problem, JEM uses a simplified approach, first minimizing in the vertical direction and then minimizing in the horizontal direction. This results in

[0223]

[0224]

[0225] in,

[0226]

[0227] To avoid division by zero or very small values, regularization parameters r and m are introduced in Equations (28) and (29).

[0228] r=500·4 d-8 (12)

[0229] m=700·4d-8 (13) Where d is the bit depth of the video samples.

[0230] In order to keep the memory access of BIO the same as conventional bi-predictive motion compensation, all predictions and gradient values ​​I are calculated for the positions within the current block. (k) 、 Figure 23A An example of an access location outside of block 2300 is shown. Figure 23A As shown in equation (9), a (2M+1)×(2M+1) square window Ω centered at the current prediction point on the boundary of the prediction block needs to access the location outside the block. In JEM, the I outside the block (k) 、 The value of is set equal to the nearest available value within the block. For example, this can be implemented as filling area 2301, such as Figure 23B shown.

[0231] Using BIO, it is possible to refine the motion field for each sample. To reduce computational complexity, a block-based BIO design can be used in JEM. Motion refinement can be calculated based on 4×4 blocks. In block-based BIO, all samples in the 4×4 block can be aggregated to obtain the s in equation (9). n value, and then aggregate the s n The value is used for the derived BIO motion vector offset of the 4×4 block. More specifically, the following formula can be used for block-based BIO derivation:

[0232]

[0233] where b k represents the set of samples belonging to the kth 4×4 block of the prediction block. n,bk )>>4) Replace s in equations (28) and (29) n to derive the associated motion vector offset.

[0234] In some cases, the MV cluster of BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the magnitude of the MV cluster is clipped to a threshold. The threshold is determined based on whether the reference pictures of the current picture are all from one direction. For example, if all the reference pictures of the current picture are from one direction, the threshold is set to 12×2 14 -d ; otherwise, it is set to 12×2 13-d .

[0235] The gradient of the BIO can be calculated simultaneously with motion compensated interpolation using an operation consistent with the HEVC motion compensation process (e.g., a 2D separable finite impulse response (FIR)). In some embodiments, the input to the 2D separable FIR is the same reference frame samples as the fractional position (fracX, fracY) of the fractional part of the block motion vector and the motion compensation process. For horizontal gradients First, the signal is interpolated vertically using BIOfilterS corresponding to the fractional position fracY with a descaling offset of d-8. Then, a gradient filter BIOfilterG is applied horizontally corresponding to the fractional position fracX with a descaling offset of 18-d. For the vertical gradient First, a gradient filter is applied vertically using BIOfilterG corresponding to the fractional position fracY with a descaling offset of d-8. Then, a signal shift is performed horizontally using BIOfilterS corresponding to the fractional position fracX with a descaling offset of 18-d. The length of the interpolation filters used for the gradient calculation BIOfilterG and the signal shift BIOfilterF can be short (e.g., 6 taps) to maintain a reasonable complexity. Table 1 shows example filters that can be used for gradient calculation for different fractional positions of block motion vectors in BIO. Table 2 shows example interpolation filters that can be used for prediction signal generation in BIO.

[0236] Table 1 Example filters for gradient calculation in BIO

[0237]

[0238]

[0239] Table 2 Example interpolation filters for prediction signal generation in BIO

[0240] Fractional pixel position Interpolation filter for prediction signal (BIOfilterS) 0 {0,0,64,0,0,0} 1 / 16 {1,-3,64,4,-2,0} 1 / 8 {1,-6,62,9,-3,1} 3 / 16 {2,-8,60,14,-5,1} 1 / 4 {2,-9,57,19,-7,2} 5 / 16 {3,-10,53,24,-8,2} 3 / 8 {3,-11,50,29,-9,2} 7 / 16 {3,-11,44,35,-10,3} 1 / 2 {3,-10,35,44,-11,3}

[0241] In JEM, BIO can be applied to all bidirectionally predicted blocks when the two predictions come from different reference pictures. BIO can be disabled when local illumination compensation (LIC) is enabled for the CU.

[0242] In some embodiments, OBMC is applied to a block after the normal MC process. To reduce computational complexity, BIOs are not applied during the OBMC process. This means that BIOs are applied to the MC process of a block when using its own MV, but not when using the MVs of neighboring blocks in the OBMC process.

[0243] 2.9 Example of Decoder-Side Motion Vector Refinement (DMVR)

[0244] In bidirectional prediction, to predict a block region, two prediction blocks, formed using motion vectors (MVs) from list 0 and list 1, respectively, are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined through a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference picture to obtain the refined MVs without transmitting additional motion information.

[0245] like Figure 24 As shown, in DMVR, a bilateral template is generated from the initial MV0 of list 0 and MV1 of list 1 as a weighted combination (i.e., average) of the two prediction blocks. The template matching operation consists of calculating a cost metric between the generated template and the sample area in the reference picture (around the initial prediction block). For each of the two reference pictures, the MV that produces the minimum template cost is considered as the MV of the list

[0246] The updated MV is used to replace the original template. In JEM, for each list, nine MV candidates are searched.

[0247] The nine MV candidates include the original MV and 8 surrounding MVs, the surrounding MVs have relatively

[0248] The offset of one luma sample from the original MV in the horizontal or vertical direction or both. Figure 24 As shown, two new MVs, MV0' and MV1', are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as the cost metric.

[0249] DMVR is applied to the Merge mode of bi-prediction, using one MV from the past reference picture and another MV from the future reference picture without transmitting additional syntax elements. In JEM, DMVR is not applied when LIC, affine motion, FRUC or sub-CU Merge candidate is enabled for the CU.

[0250] 2.2.9 Example of Symmetric Motion Vector Difference

[0251] In [8], symmetric motion vector difference (SMVD) is proposed to encode MVD more efficiently.

[0252] First, at the stripe level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 are derived as follows:

[0253] Searches for the closest forward reference picture to the current picture in reference picture list 0. If found, sets RefIdxSymL0 equal to the reference index of the forward picture.

[0254] Searches for the backward reference picture closest to the current picture in reference picture list 1. If found, sets RefIdxSymL1 equal to the reference index of the backward picture.

[0255] If both forward and backward pictures are found, BiDirPredFlag is set equal to 1.

[0256] Otherwise, the following applies:

[0257] Searches for the backward reference picture closest to the current reference picture in reference picture list 0. If found, sets RefIdxSymL0 equal to the reference index of the backward picture.

[0258] Searches for the forward reference picture closest to the current reference picture in reference picture list 1. If found, sets RefIdxSymL1 equal to the reference index of the forward picture.

[0259] If both forward and backward pictures are found, BiDirPredFlag is set equal to 1. Otherwise, BiDirPredFlag is set equal to 0.

[0260] Second, at the CU level, if the prediction direction of the CU is bi-prediction and BiDirPredFlag is equal to 1, the symmetric mode flag is explicitly signaled, which indicates whether the symmetric mode is used.

[0261] When this flag is true, only mvp_l0_flag, mvp_l1_flag and MVD0 are explicitly signaled. For list 0 and list 1, the reference index is set equal to RefIdxSymL0, RefIdxSymL1 respectively.

[0262] Set MVD1 equal to -MVD0. The final motion vector is shown below.

[0263]

[0264] Figure 27 An example of a symmetric pattern is shown.

[0265] The modifications to the codec unit syntax are shown in Table 3.

[0266] Table 3: Modifications to the syntax of the codec unit

[0267]

[0268]

[0269] 2.10.1 Symmetric MVD for Affine Bidirectional Predictive Codec

[0270] Affine mode SMVD is proposed, which extends the symmetric MVD mode to affine bidirectional prediction. When the symmetric MVD mode is applied to affine bidirectional prediction codec, the MVD of the control point is not signaled but derived. Based on the assumption of linear motion, the MVD of the top left control point of List1 is derived from List0. The MVD of the other control points of List0 is set to 0.

[0271] 2.11 Context-Adaptive Binary Arithmetic Coding (CABAC)

[0272] 2.11.1 CABAC Design in HEVC

[0273] 2.11.1.1 Context Representation and Initialization Process in HEVC

[0274] In HEVC, for each context variable, two variables pStateIdx and valMps are initialized.

[0275] From the 8-bit table entry initValue, the two 4-bit variables slopeIdx and offsetIdx are derived as follows:

[0276]

[0277] The variables m and n used in the initialization of the context variables are derived from slopeIdx and offsetIdx as follows:

[0278]

[0279] The two values ​​assigned to pStateIdx and valMps for initialization are derived from the quantization parameter of the luma of the slice represented by SliceQpY. Given the variables m and n, the initialization is specified as follows:

[0280] preCtxState=Clip3(1,126,((m*Clip3(0,51,SliceQpY))>>4)+n)

[0281] valMps=(preCtxState<=63)? 0:1? (36)

[0282] pStateIdx=valMps? (preCtxState-64):(63-preCtxState)

[0283] 2.11.1.2 State Transition Process in HEVC

[0284] The inputs to this process are the current pStateIdx, decoded value binVal and valMps value of the context variables associated with ctxTable and ctxIdx.

[0285] The output of this process is the updated pStateIdx and updated valMps of the context variables associated with ctxIdx.

[0286] Depending on the decoded value binVal, the updates of the two variables pStateIdx and valMps associated with ctxIdx are derived as in (37):

[0287]

[0288] 2.11.2 CABAC Design in VVC

[0289] The context-based adaptive binary arithmetic codec (BAC) in VVC has been changed in VVC, which is different from that in HEVC in terms of context update process and arithmetic codec.

[0290] The following is an overview of the recently adopted proposal (JVET-M0473, CE Test 5.1.13).

[0291] Table 4: Overview of CABAC modifications in VVC

[0292]

[0293] 2.11.2.1 Context Initialization Process in VVC

[0294] In VVC, the two values ​​assigned to pStateIdx0 and pStateIdx1 for initialization are derived from SliceQpY. Given variables m and n, the initialization is specified as follows:

[0295] preCtxState=Clip3(0,127,((m*Clip3(0,51,SliceQpY))>>4)+n)

[0296] pStateIdx0=initStateIdxToState[preCtxState]>>4 (38)

[0297] pStateIdx1=initStateIdxToState[preCtxState]?

[0298] 2.11.2.2 State Transition Process in VVC

[0299] The input to this process is the current pStateIdx0 and the current pStateIdx1 and the decoded value binVal.

[0300] The output of this process is the updated pStateIdx0 and updated pStateIdx1 of the context variables associated with ctxIdx.

[0301] The variables shift0 (corresponding to variable a in the overview of CABAC modification in VVC Table 4) and shift1 (corresponding to variable b in the overview of CABAC modification in VVC Table 4) are derived from the shiftIdx values ​​associated with ctxTable and ctxInc.

[0302]

[0303] Depending on the decoded value binVal, the updates of the two variables pStateIdx0 and pStateIdx1 associated with ctxIdx are derived as follows:

[0304]

[0305] 3. Disadvantages of existing implementations

[0306] In some existing implementations, when an MV / MV difference (MVD) can be selected from a set of multiple MV / MVD precisions for an affine coded block, it is still unclear how to obtain a more accurate motion vector.

[0307] In other existing implementations, MV / MVD accuracy information also plays an important role in determining the overall codec gain of AMVR applied to affine mode, but achieving this goal remains uncertain.

[0308] 4. Example Method for MV Prediction (MVP) of Affine Mode with AMVR

[0309] Embodiments of the presently disclosed technology overcome the shortcomings of existing implementations, thereby providing video codecs with higher codec efficiency. Based on the disclosed technology, the derivation and signaling of motion vector prediction for affine mode with adaptive motion vector resolution (AMVR) can enhance existing and future video codec standards, as illustrated in the examples described below for various implementations. The examples of the disclosed technology provided below illustrate the general concepts and are not meant to be construed as limiting. In the examples, the various features described in these examples can be combined unless explicitly indicated to the contrary.

[0310] In some embodiments, when AMVR is applied, the following examples can be applied to either affine mode or normal mode. These examples assume that the precision Prec (i.e., MV has a precision of 1 / (2^Prec)) is used to encode the MVD in AF_INTER mode or to encode the MVD in normal inter mode. The motion vector prediction (e.g., inherited from the neighboring block MV) and its precision are respectively determined by MVPred (MVPred X ,MVPred Y ) and PredPrec represent.

[0311] In the following discussion, SatShift(x,n) is defined as

[0312]

[0313] Shift(x, n) is defined as Shift(x, n) = (x + offset0) >> n. In one example, offset0 and / or offset1 are set to (1 << n) >> 1 or (1 << (n-1)). In another example, offset0 and / or offset1 are set to 0. In another example, offset0 = offset1 = ((1 << n) >> 1) - 1 or ((1 << (n-1))) - 1.

[0314] In the following discussion, an operation between two motion vectors means that the operation will be applied to both components of the motion vector. For example, MV3 = MV1 + MV2 is equivalent to MV3x = MV1x + MV2x and MV3y = MV1y + MV2y. Alternatively, the operation can be applied only to the horizontal or vertical components of the two motion vectors.

[0315] Example 1. It is proposed that the final MV accuracy can remain unchanged, ie the same as the accuracy of the motion vector to be stored.

[0316] (a) In one example, the final MV accuracy can be set to 1 / 16 pixel or 1 / 8 pixel.

[0317] (b) In one example, the signaled MVD may be scaled first and then added to the MVP to form the final MV of a block.

[0318] Example 2. The MVP derived directly from neighboring blocks (eg, spatial or temporal) or a default MVP may be modified first and then added to the signaled MVD to form the final MV of the (current) block.

[0319] (a) Alternatively, for different Prec values, whether and how to apply the modification of MVP may be different.

[0320] (b) In one example, if Prec is greater than 1 (ie, the MVD has fractional precision), the precision of neighboring MVs is unchanged and no scaling is performed.

[0321] (c) In one example, if Prec is equal to 1 (i.e., the MVD has 1 pixel precision), the MV predictor (i.e., the MV of the neighboring block) needs to be scaled, for example, according to Example 4(b) of PCT application PCT / CN2018 / 104723.

[0322] (d) In one example, if Prec is less than 1 (i.e., the MVD has 4-pixel precision), the MV predictor (i.e., the MV of the neighboring block) needs to be scaled, for example, according to Example 4(b) of PCT application PCT / CN2018 / 104723.

[0323] Example 3. In one example, if the precision of the signaled MVD is the same as that of the stored MV, no scaling is required after the affine MV is reconstructed, otherwise, the MV is reconstructed with the precision of the signaled MVD and then scaled to the precision of the stored MV.

[0324] Example 4. In one example, the normal inter mode and the AF_INTER mode can be implemented based on the above different examples.

[0325] Example 5. In one example, the syntax element indicating the MV / MVD precision for affine mode may be signaled using the following semantics:

[0326] (a) In one example, syntax elements equal to 0, 1, and 2 indicate 1 / 4 pixel, 1 / 16 pixel, and 1 pixel MV precision, respectively.

[0327] (b) Alternatively, in affine mode, syntax elements equal to 0, 1, and 2 indicate 1 / 4 pixel, 1 pixel, and 1 / 16 pixel MV precision, respectively.

[0328] (c) Alternatively, in affine mode, syntax elements equal to 0, 1, and 2 indicate 1 / 16 pixel, 1 / 4 pixel, and 1 pixel MV precision, respectively.

[0329] Example 6. In one example, whether AMVR is enabled or disabled for affine mode may be signaled in the SPS, PPS, VPS, sequence / picture / slice header / slice, etc.

[0330] Example 7.In one example, the indication of allowed MV / MVD precision may be signaled in the SPS, PPS, VPS, sequence / picture / slice header / slice, etc.

[0331] (a) An indication of the selected MVD precision may be signaled per codec tree unit (CTU) and / or per region.

[0332] (b) The set of allowed MV / MVD precisions may depend on the codec mode (eg, affine or non-affine) of the current block.

[0333] (c) The set of allowed MV / MVD precisions may depend on the slice type / temporal layer index / low-delay check flag.

[0334] (d) The set of allowed MV / MVD precisions may depend on the block size and / or block shape of the current block or neighboring blocks.

[0335] (e) The set of allowed MV / MVD precisions may depend on the precision of the MV to be stored in the decoded picture buffer.

[0336] (i) In one example, if the stored MV is within X pixels, then the allowed MV / MVD precision set may have at least X pixels.

[0337] Improvements to support affine mode for AMVR

[0338] Example 8. The set of allowed MVD precisions may vary from picture to picture, from slice to slice, or from block to block.

[0339] a. In one example, the set of allowed MVD precisions may depend on codec information, such as block size, block shape, etc.

[0340] b. A set of allowed MV precisions may be predefined, such as {1 / 16, 1 / 4, 1}.

[0341] c. The indication of the allowed MV precision may be signaled in the SPS / PPS / VPS / sequence header / picture header / slice header / CTU group, etc.

[0342] d. The signaling of the selected MV precision from the set of allowed MV precisions further depends on the number of allowed MV precisions for the block.

[0343] Example 9. A syntax element is signaled to the decoder to indicate the MVD precision used in affine inter mode.

[0344] a. In one example, only one single syntax element is used to indicate the MVD precision that applies to both affine and AMVR modes.

[0345] i. In one example, the same semantics are used, ie, the same value of a syntax element is mapped to the same MVD precision for both AMVR and affine modes.

[0346] ii. Alternatively, the semantics of a single syntax element is different for AMVR mode and affine mode. In other words, the same value of a syntax element can be mapped to different MVD precisions for AMVR and affine modes.

[0347] b. In one example, when the affine mode uses the same set of MVD precisions as AMVR (eg, MVD precision is set to {1, 1 / 4, 4} pixels), the MVD precision syntax elements in AMVR are reused in the affine mode, i.e., only one single syntax element is used.

[0348] i. Alternatively, furthermore, when encoding / decoding this syntax element in a CABAC encoder / decoder, the same or different context models may be used for AMVR and affine modes.

[0349] ii. Alternatively, the syntax element may have different semantics in AMVR and affine modes. For example, syntax elements equal to 0, 1, and 2 indicate 1 / 4 pixel, 1 pixel, and 4 pixel MV precision, respectively, in AMVR, while syntax elements equal to 0, 1, and 2 indicate 1 / 4 pixel, 1 / 16 pixel, and 1 pixel MV precision, respectively, in affine mode.

[0350] c. In one example, when the affine mode uses the same number of MVD precisions as AMVR but a different set of MVD precisions (e.g., the MVD precision for AMVR is set to {1, 1 / 4, 4} pixels, while the MVD precision for affine is set to {1 / 16, 1 / 4, 1} pixels), the MVD precision syntax elements in AMVR are reused in the affine mode, i.e., only one single syntax element is used.

[0351] i. Alternatively, furthermore, when encoding / decoding this syntax element in a CABAC encoder / decoder, the same or different context models may be used for AMVR and affine modes.

[0352] ii. Alternatively, furthermore, this syntax element may have different semantics in AMVR and affine modes.

[0353] d. In one example, affine mode uses less MVD precision than AMVR, and MVD precision syntax elements from AMVR are reused in affine mode. However, only a subset of syntax element values ​​are valid for affine mode.

[0354] i. Alternatively, furthermore, when encoding / decoding this syntax element in a CABAC encoder / decoder, the same or different context models may be used for AMVR and affine modes.

[0355] ii. Alternatively, furthermore, this syntax element may have different semantics in AMVR and affine modes.

[0356] e. In one example, affine mode uses more MVD precision than AMVR, and the MVD precision syntax elements in AMVR are reused in affine mode. However, such syntax elements are extended to allow more values ​​in affine mode.

[0357] i. Alternatively, furthermore, when encoding / decoding this syntax element in a CABAC encoder / decoder, the same or different context models may be used for AMVR and affine modes.

[0358] ii. Alternatively, furthermore, this syntax element may have different semantics in AMVR and affine modes.

[0359] f. In one example, a new syntax element is used to encode and decode the MVD accuracy of the affine mode, namely,

[0360] Two different syntax elements are used to encode and decode the MVD accuracy of AMVR and affine modes.

[0361] g. The syntax for indicating the MVD precision for affine mode may be signaled under one or all of the following conditions:

[0362] i. The MVD of all control points is non-zero.

[0363] ii. The MVD of at least one control point is non-zero.

[0364] iii. The MVD of a control point (e.g., the first CPMV) is non-zero

[0365] In this case, when any or all of the above conditions fail, no signaling of MVD accuracy is required.

[0366] h. The syntax element for indicating the MVD precision of the affine mode or the AMVR mode may use context coding, and the context depends on the codec information.

[0367] i. In one example, when there is only a single syntax element, the context may depend on whether the current block is coded using affine mode.

[0368] i. In one example, the context may depend on block size / block shape / MVD accuracy of neighboring blocks / temporal layer index / prediction direction, etc.

[0369] j. Whether to enable or disable the use of multiple MVD precisions in affine mode can be signaled in the SPS / PPS / VPS / sequence header / picture header / slice header / CTU group, etc.

[0370] i. In one example, whether to signal the information enabling or disabling the use of multiple MVD precisions in the affine mode may depend on other syntax elements. For example, when the affine mode is enabled, the information enabling or disabling the use of multiple MVs and / or MVPs and / or MVD precisions in the affine mode is signaled; when the affine mode is disabled, the information enabling or disabling the use of multiple MVs and / or MVPs and / or MVD precisions in the affine mode is not signaled and is inferred to be 0.

[0371] k. Alternatively, multiple syntax elements may be signaled to indicate the MV and / or MVP and / or MVD precision used in affine inter mode (all referred to as "MVD precision" in the following discussion).

[0372] i. In one example, the syntax elements used to indicate the MVD precision used in affine inter mode and normal inter mode may be different.

[0373] 1. The number of syntax elements indicating the MVD precision used in affine inter mode and normal inter mode may be different.

[0374] 2. The semantics of the syntax element indicating the MVD precision used in affine inter mode and normal inter mode may be different.

[0375] 3. The context model used in arithmetic coding to encode a syntax element to indicate the MVD precision used in affine inter mode and normal inter mode may be different.

[0376] 4. The method of deriving a context model in arithmetic coding to encode a syntax element to indicate the MVD precision used in affine inter mode and normal inter mode may be different.

[0377] ii. In one example, a first syntax element (eg, amvr_flag) may be signaled to indicate whether AMVR is applied in an affine codec block.

[0378] 1. Conditionally signal the first syntax element.

[0379] a. In one example, when a certain mode (eg, CPR / IBC mode) is used to encode the current block, signaling of the first syntax element (amvr_flag) is skipped.

[0380] b. In one example, when the MVD (including horizontal and vertical components) of all CPMVs is zero, signaling of the first syntax element (amvr_flag) is skipped.

[0381] c. In one example, when the MVD (including both horizontal and vertical components) of a selected CPMV is zero, signaling of the first syntax element (amvr_flag) is skipped.

[0382] i. In one example, the MVD of the selected CPMV is the MVD of the first CPMV to be encoded / decoded.

[0383] d. In one example, when enable usage of multiple MVD precisions for affine codec blocks is false, signaling of the first syntax element (amvr_flag) is skipped.

[0384] e. In one example, the first syntax may be signaled under the following conditions

[0385] element:

[0386] i. Enable the use of multiple MVD precisions for affine codec blocks is true, and the current block uses affine mode codec;

[0387] ii. Alternatively, Enable use of multiple MVD precisions for affine coded blocks is true, the current block is coded using affine mode, and at least one component of the MVD of the CPMV is not equal to zero.

[0388] iii. Alternatively, Enable use of multiple MVD precisions for affine coded blocks is true, the current block is coded using affine mode, and at least one component of the MVD of the selected CPMV is not equal to zero.

[0389] 1. In one example, the MVD of the selected CPMV is the MVD of the first CPMV to be encoded / decoded.

[0390] 2. When AMVR is not applied to an affine codec block or the first syntax element does not exist, the default MV and / or MVD precision is used.

[0391] a. In one example, the default precision is 1 / 4 pixel (1 / 4-pel).

[0392] b. Alternatively, the default precision is set to the precision used in motion compensation for affine codec blocks.

[0393] 3. For example, if amvr_flag is equal to 0, the MVD precision of the affine mode is 1 / 4 pixel; otherwise, the MVD precision of the affine mode can be other values.

[0394] a. Alternatively, additional MVD precision may be further signaled via a second syntax element.

[0395] iii. In one example, a second syntax element (eg, amvr_coarse_precision_flag) may be signaled to indicate the MVD precision for the affine mode.

[0396] 1. In one example, whether to signal the second syntax element may depend on the first syntax element. For example, the second syntax element may be signaled only when the first syntax element is 1.

[0397] 2. In one example, if the second syntax element is 0, the MVD precision of the affine mode is 1 pixel; otherwise, the MVD precision of the affine mode is 1 / 16 pixel.

[0398] 3. In one example, if the second syntax element is 0, the MVD precision of the affine mode is 1 / 16 pixel; otherwise, the MVD precision of the affine mode is full pixel.

[0399] iv. In one example, the syntax element used to indicate the MVD precision used in affine inter mode shares the same context model as the syntax element with the same name but used to indicate the MVD precision used in normal inter mode.

[0400] 1. Alternatively, the syntax element used to indicate the MVD precision used in affine inter mode uses a different context model than the syntax element with the same name but used to indicate the MVD precision used in normal inter mode.

[0401] Example 10. Whether or how to apply AMVR on an affine coded block may depend on the reference picture of the current block.

[0402] aIn one example, if the reference picture is the current picture, AMVR is not applied, i.e.,

[0403] Apply intra block copy in the current block.

[0404] Fast Algorithm for AVMR in Affine Mode of Encoder

[0405] The RD costs (actual RD costs, or SATD / SSE / SAD costs plus bit costs) of the affine mode and AMVP mode are expressed as affineCosti and amvpCosti for IMV=i, where i=0, 1, or 2. In this article, IMV=0 means 1 / 4 pixel MV, and IMV=1 means integer MV for the AMVP mode and 1 / 16 pixel MV for the affine mode, and IMV=2 means 4 pixel MV for the AMVP mode and integer MV for the affine mode. The RD cost of the merge mode is expressed as mergeCost.

[0406] Example 11. If the best mode of its parent CU is not AF_INTER mode or AF_MERGE mode, it is proposed to disable AMVR for the affine mode of the current CU.

[0407] Alternatively, if the best mode of its parent CU is not AF_INTER mode, AMVR is disabled for the affine mode of the current CU.

[0408] Example 12. If affineCost0>th1*amvpCost0, where th1 is a positive threshold, it is proposed to disable AMVR for affine mode.

[0409] a. Alternatively, if min(affineCost0, amvpCost0)>th2*mergeCost, where th2 is a positive threshold, then disable AMVR for affine mode.

[0410] b. Alternatively, further, if affineCost0>th3*affineCost1, where th3 is a positive threshold, then integer MV is disabled for affine mode.

[0411] Example 12. If amvpCost0>th4*affineCost0, where th4 is the positive threshold, it is recommended to disable AMVR for AMVP mode.

[0412] a. Alternatively, if min(affineCost0, amvpCost0)>th5*mergeCost, where th5 is a positive threshold, then disable AMVR for AMVP mode.

[0413] Example 13. It is proposed that the 4 / 6 parameter affine model obtained in one MV accuracy can be used as a candidate starting search point for other MV accuracy.

[0414] a. In one example, the 4 / 6 parameter affine model obtained in 1 / 16 MV can be used as a candidate starting search point for other MV accuracies.

[0415] b. In one example, the 4 / 6 parameter affine model obtained in 1 / 4 MV can be used as a candidate open search point for other MV accuracies.

[0416] Example 14. If the parent block of the current block does not select the affine mode, the AMVR for the affine mode is not checked at the encoder for the current block.

[0417] Example 15. The rate-distortion calculation of the MV precision of the affine codec block in the current slice / slice / CTU row can be terminated early using statistics of usage of different MV precisions of the affine codec block in the previously coded frame / slice / slice / CTU row.

[0418] a. In one example, the percentage of affine codec blocks with a certain MV precision is recorded.

[0419] If the percentage is too low, skip checking the corresponding MV accuracy.

[0420] b. In one example, the previously coded frame with the same temporal layer is used to decide whether to skip a certain MV precision.

[0421] Context for encoding and decoding affine AMVR

[0422] Example 16. For each context used to encode and decode an affine AMVR code, it is proposed to set a variable (denoted by shiftIdx) to control the update speed of the two probabilities associated with this context.

[0423] a. In one example, a faster update speed is defined by (shiftIdx>>2)+2.

[0424] b. In one example, a slower update speed is defined by (shiftIdx&3)+3+shift0.

[0425] c. In one example, the conforming bitstream should follow the following rule: The derived faster update rate should be in the range [2, 5], inclusive.

[0426] d. In one example, the conforming bitstream shall follow the following rule: The derived faster update rate shall be in the range [3, 6], inclusive.

[0427] Example 17. When encoding or decoding the AMVR pattern of a block, it is proposed not to allow the affine AMVR pattern information of neighboring blocks to be used for context modeling.

[0428] a. In one example, the AMVR mode index of the neighboring block can be utilized, and the affine AMVR mode information of the neighboring block can be excluded. An example is shown in Table 5 (including Tables 5-1 and 5-2), where (xNbL, yNbL) and (xNbA, yNbA) represent the left and top neighboring blocks. In one example, the context index offset ctxInc = (condL && availableL) + (condA && availableA) + ctxSetIdx * 3.

[0429] Table 5-1 ctxInc specification using left and above syntax elements

[0430]

[0431] Table 5-2 ctxInc specification using left and above syntax elements

[0432]

[0433] b. Alternatively, the affine AMVR mode information of the neighboring block can be further utilized, but using a function instead of directly. In one example, when the amvr_mode[xNbL][yNbL] of the neighboring block decoded by affine code indicates a certain MV precision (e.g., 1 / 4 pixel MV precision), the function func described in Table 6-1 can return true. In one example, when the amvr_flag[xNbL][yNbL] of the neighboring block decoded by affine code indicates a certain MV precision (e.g., 1 / 4 pixel MV precision), the function func described in Table 6-2 can return true.

[0434] Table 6-1 ctxInc specification using left and above syntax elements

[0435]

[0436] Table 6-2 ctxInc specification using left and above syntax elements

[0437]

[0438] c. Alternatively, the affine AMVR mode information of the neighboring blocks can be further used to encode the first syntax element (eg, amvr_flag) of the AMVR mode (applied to normal inter mode). Tables 6-3 and 6-4 give some examples.

[0439] Table 6-3 ctxInc specification using left and above syntax elements

[0440]

[0441] Table 6-4 ctxInc specification using left and above syntax elements

[0442]

[0443] d. When the AMVR mode information is represented by multiple syntax elements (e.g., the first and second syntax elements, represented by amvr_flag, amvr_coarse_precision_flag), the above syntax amvr_mode can be replaced by any one of the multiple syntax elements, and the above method can still be applied.

[0444] Example 18. When encoding and decoding the affine AMVR mode, it is proposed that the AMVR mode information of neighboring blocks can be used for context encoding and decoding.

[0445] a. In one example, the AMVR mode information of the neighboring block is directly used. An example is shown in Table 7. Alternatively, in addition, the context index offset ctxInc = (condL && availableL) + (condA && availableA) + ctxSetIdx * 3.

[0446] Table 7 ctxInc specification using left and above syntax elements

[0447]

[0448] b. Alternatively, the AMVR pattern information of neighboring blocks is not allowed to be used for context modeling. An example is shown in Table 8.

[0449] Table 8: ctxInc specification using left and above syntax elements

[0450]

[0451] c. Alternatively, the AMVR mode information of the neighboring block can be further utilized, but using a function instead of directly using it. In one example, when the amvr_mode[xNbL][yNbL] of the neighboring block of the non-affine codec indicates a certain MV precision (such as 1 / 4 pixel MV precision), the function func described in Table 9 can return true.

[0452] Table 9: ctxInc specification using left and above syntax elements

[0453]

[0454]

[0455] d. When the affine AMVR mode information is represented by multiple syntax elements (e.g., the first and second syntax elements, represented by amvr_flag, amvr_coarse_precision_flag), the above syntax amvr_mode can be replaced by any one of the multiple syntax elements, and the above method can still be applied.

[0456] Fast algorithms for SMVD and affine SMVD

[0457] When verifying the SMVD mode, it is assumed that the currently selected best mode is CurBestMode, and the MVD accuracy of AMVR in AMVR is MvdPrec or the MVD accuracy of affine AMVR is MvdPrecAff.

[0458] Example 19. Depending on the currently selected best mode (i.e. CurBestMode), the MVD accuracy in AMVR, the SMVD mode may be skipped.

[0459] a. In one example, if CurBestMode is Merge mode or / and UMVE mode, SMVD mode may not be checked.

[0460] b. In one example, if the SMVD mode is not used to encode and decode CurBestMode, the SMVD mode may not be checked.

[0461] c. In one example, if CurBestMode is affine mode, SMVD mode may not be checked.

[0462] d. In one example, if CurBestMode is the sub-block Merge mode, the SMVD mode may not be checked.

[0463] e. In one example, if CurBestMode is affine SMVD mode, then SMVD mode may not be checked.

[0464] f. In one example, if CurBestMode is the affine Merge mode, the SMVD mode may not be checked.

[0465] g. In one example, the above fast method, ie, bullets 13.a-13.f, may be applied only to some MVD accuracies.

[0466] i. In one example, the above fast method can be applied only when the MVD precision is greater than or equal to the precision (eg, integer pixel precision).

[0467] ii. In one example, the above fast method can be applied only when the MVD precision is greater than the precision (eg, integer pixel precision).

[0468] iii. In one example, the above fast method can be applied only when the MVD precision is less than or equal to the precision (eg, integer pixel precision).

[0469] iv. In one example, the above fast method can be applied only when the MVD precision is less than the precision (eg, integer pixel precision).

[0470] Example 20. Depending on the currently selected best mode (i.e., CurBestMode), the MVD accuracy in the affine AMVR mode may be skipped.

[0471] a. In one example, if CurBestMode is Merge mode or / and UMVE mode, the affine SMVD mode may not be checked.

[0472] b. In one example, if the affine SMVD mode is not used to encode or decode CurBestMode, the affine SMVD mode may not be checked.

[0473] c. In one example, if CurBestMode is the sub-block Merge mode, the affine SMVD mode may not be checked.

[0474] d. In one example, if CurBestMode is SMVD mode, then the affine SMVD mode may not be checked.

[0475] e. In one example, if CurBestMode is the affine Merge mode, the affine SMVD mode may not be checked.

[0476] f. In one example, the above fast method, ie, bullets 20.a to 20.e, may be used only for some MVD accuracy.

[0477] i. In one example, the above fast method can be applied only when the affine MVD precision is greater than or equal to the precision (eg, integer pixel precision).

[0478] ii. In one example, the above fast method can be applied only when the affine MVD precision is greater than φ(n) precision (eg, integer pixel precision).

[0479] iii. In one example, the above fast method can be applied only when the affine MVD precision is less than or equal to the precision (eg, integer pixel precision).

[0480] iv. In one example, the above fast method can be applied only when the affine MVD precision is less than φ(V) precision (eg, integer pixel precision).

[0481] Example 21. The above proposed method may be applied under certain conditions, such as block size, slice / picture / slice type or motion information.

[0482] a. In one example, the proposed method is not allowed when the block size contains less than M*H samples (e.g., 16 or 32 or 64 luma samples).

[0483] b. Alternatively, the proposed method is not allowed when the minimum size of the width or / and height of the block is less than or not greater than X. In one example, X is set to 8.

[0484] c. Alternatively, the proposed method is not allowed when the minimum size of the width or / and height of the block is not less than X. In one example, X is set to 8.

[0485] d. Alternatively, when the width of the block is > th1 or >= th1 and / or the height of the block is > th2 or >= th2, the proposed method is not allowed. In one example, th1 and / or th2 is set to 8.

[0486] e. Alternatively, when the width of the block is < th1 or <= th1 and / or the height of the block is < th2 or <= th2, the proposed method is not allowed. In one example, th1 and / or th2 is set to 8.

[0487] f. Alternatively, whether to enable or disable the above methods and / or which method to apply may depend on block size, video processing data unit (VPDU), picture type, low delay check flag, codec information of the current block (such as reference picture, unidirectional or bidirectional prediction), or a previously coded block.

[0488] Example 22. The AMVR method for affine mode may be performed differently when intra block copy (IBC, also known as current picture reference (CPR)) is applied or not applied.

[0489] a. In one example, if the block is coded by IBC, AMVR for affine mode cannot be used.

[0490] b. In one example, if the block is coded by IBC, AMVR for affine mode may be used, but the candidate MV / MVD / MVP precision may be different from the precision for affine coded blocks for non-IBC codecs.

[0491] Example 23. All terms "slice" in this document can be replaced by "slice group" or "slice".

[0492] Example 24. In the VPS / SPS / PPS / slice header / slice group header, a syntax element (e.g., no_amvr_constraint_flag) equal to 1 specifies that the requirement for bitstream conformance is that the syntax element indicating whether AMVR is enabled (e.g., sps_amvr_enabled_flag) and the syntax element indicating whether affine AMVR is enabled (e.g., sps_affine_avmr_enabled_flag) should both be equal to 0. A syntax element (e.g., no_amvr_constraint_flag) equal to 0 does not impose a constraint.

[0493] Example 25. The syntax element (eg, no_affine_amvr_constraint_flag) may be signaled in a VPS / SPS / PPS / slice header / slice group header or other video data units.

[0494] a. In one example, no_affine_amvr_constraint_flag equal to 1 specifies that a requirement for bitstream conformance is that the syntax element indicating whether affine AMVR is enabled (eg, sps_affine_avmr_enabled_flag) should be equal to 0. A syntax element (eg, no_affine_amvr_constraint_flag) equal to 0 imposes no constraint.

[0495] Example 26. A second syntax element indicating coarse motion precision (eg, amvr_coarse_precision_flag) may be encoded and decoded with multiple contexts.

[0496] a. In one example, two contexts can be utilized.

[0497] b. In one example, the choice of context may depend on whether the current block is affine coded or not.

[0498] c. In one example, for the first grammar, only one context may be used to encode or decode it, and for the second grammar, only one context may also be used to encode or decode it.

[0499] d. In one example, for the first grammar, it can be encoded and decoded using only one context, and for the second grammar, it can also be bypassed and decoded.

[0500] e. In one example, the first grammar may be bypass coded and decoded, and the second grammar may also be bypass coded and decoded.

[0501] f. In one example, all syntax elements related to motion vector accuracy can be bypass coded.

[0502] Example 27. For example, only the first bin of the syntax element amvr_mode is coded using arithmetic codec context(s). All following bins of amvr_mode are coded as bypass codec.

[0503] a. The method disclosed above can also be applied to other grammatical elements.

[0504] b. For example, only the first binary of the syntax element SE is coded using (one or more) arithmetic codec contexts. All following binary digits of SE are coded as bypass codecs. SE can be

[0505] 1)alf_ctb_flag

[0506] 2)sao_merge_left_flag

[0507] 3)sao_merge_up_flag

[0508] 4)sao_type_idx_luma

[0509] 5)sao_type_idx_chroma

[0510] 6)split_cu_flag

[0511] 7)split_qt_flag

[0512] 8)mtt_split_cu_vertical_flag

[0513] 9)mtt_split_cu_binary_flag

[0514] 10)cu_skip_flag

[0515] 11)pred_mode_ibc_flag

[0516] 12)pred_mode_flag

[0517] 13) intra_luma_ref_idx

[0518] 14)intra_subpartitions_mode_flag

[0519] 15)intra_subpartition_split_flag

[0520] 16)intra_luma_mpm_flag

[0521] 17)intra_chroma_pred_mode

[0522] 18)merge_flag

[0523] 19)inter_pred_idc

[0524] 20)inter_affine_flag

[0525] 21)cu_affine_type_flag

[0526] 22)ref_idx_l0

[0527] 23)mvp_l0_flag

[0528] 24)ref_idx_l1

[0529] 25)mvp_l1_flag

[0530] 26)avmr_flag

[0531] 27)amvr_precision_flag

[0532] 28)gbi_idx

[0533] 29)cu_cbf

[0534] 30)cu_sbt_flag

[0535] 31)cu_sbt_quad_flag

[0536] 32)cu_sbt_horizontal_flag

[0537] 33)cu_sbt_pos_flag

[0538] 34)mmvd_flag

[0539] 35)mmvd_merge_flag

[0540] 36)mmvd_distance_idx

[0541] 37)ciip_flag

[0542] 38)ciip_luma_mpm_flag

[0543] 39)merge_subblock_flag

[0544] 40)merge_subblock_idx

[0545] 41)merge_triangle_flag

[0546] 42)merge_triangle_idx0

[0547] 43)merge_triangle_idx1

[0548] 44)merge_idx

[0549] 45)abs_mvd_greater0_flag

[0550] 46)abs_mvd_greater1_flag

[0551] 47)tu_cbf_luma

[0552] 48)tu_cbf_cb

[0553] 49)tu_cbf_cr

[0554] 50)cu_qp_delta_abs

[0555] 51)transform_skip_flag

[0556] 52)tu_mts_idx

[0557] 53)last_sig_coeff_x_prefix

[0558] 54)last_sig_coeff_y_prefix

[0559] 55)coded_sub_block_flag

[0560] 56)sig_coeff_flag

[0561] 57)par_level_flag

[0562] 58)abs_level_gt1_flag

[0563] 59)abs_level_gt3_flag

[0564] c. Alternatively, furthermore, if the syntax element SE is binary valued (ie it can only be equal to 0 or 1), it can be context coded.

[0565] i. Alternatively, furthermore, if the syntax element SE is binary-valued (ie it can only be equal to 0 or 1), it can be bypass-coded.

[0566] d. Alternatively, furthermore, only one context may be used to encode and decode the first binary number.

[0567] Example 28. The accuracy of the motion vector prediction (MVP) or motion vector difference (MVD) or reconstructed motion vector (MV) may vary depending on the motion accuracy that may be signaled.

[0568] a. In one example, if the original prediction of MVP is lower (or not higher) than the target precision, then MVP=MVP<<s. s is an integer and may depend on the difference between the original precision and the target precision.

[0569] ii. Alternatively, if the original precision of MVD is lower (or not higher) than the target precision, then MVD=MVD<<s. s is an integer and may depend on the difference between the original precision and the target precision.

[0570] iii. Alternatively, if the original precision of MV is lower (or not higher) than the target precision, then MV=MV<<s. s is an integer and may depend on the difference between the original precision and the target precision.

[0571] b. In one example, if the original prediction of MVP is higher (or not lower) than the target precision, then MVP=Shift(MVP, s). s is an integer and may depend on the difference between the original precision and the target precision.

[0572] i. Alternatively, if the original precision of MVD is higher (or not lower) than the target precision, then MVD=Shift(MVD, s). s is an integer and may depend on the difference between the original precision and the target precision.

[0573] ii. Alternatively, if the original precision of MV is higher (or not lower) than the target precision, then MV=Shift(MV, s). s is an integer and may depend on the difference between the original precision and the target precision.

[0574] c. In one example, if the original prediction of MVP is above (or not below) the target precision, then MVP=SatShift(MVP, s). s is an integer and may depend on the difference between the original precision and the target precision.

[0575] i. Alternatively, if the original precision of MVD is higher (or not lower) than the target precision, then MVD=SatShift(MVD, s). s is an integer and may depend on the difference between the original precision and the target precision.

[0576] ii. Alternatively, if the original precision of MV is higher (or not lower) than the target precision, then MV=SatShift(MV, s). s is an integer and may depend on the difference between the original precision and the target precision.

[0577] d. When the current block does not use affine mode encoding and decoding, the above disclosed method can be applied.

[0578] e. When the current block uses affine mode encoding and decoding, the above disclosed method can be applied.

[0579] 5. Examples

[0580] The highlighted sections show the modified specifications.

[0581] 5.1 Example 1: Instructions for the use of affine AMVR mode

[0582] This indication may be signaled in the SPS / PPS / VPS / APS / sequence header / picture header / slice group header, etc. This section describes the signaling in the SPS.

[0583] 5.1.1 SPS Syntax Table

[0584]

[0585]

[0586] Given the alternative SPS syntax table as follows:

[0587]

[0588]

[0589] Semantics:

[0590] sps_affine_amvr_enabled_flag equal to 1 specifies the use of adaptive motion vector difference resolution in motion vector coding and decoding in affine inter mode. amvr_enabled_flag equal to 0 specifies the use of adaptive motion vector difference resolution in motion vector coding and decoding in affine inter mode.

[0591] 5.2 AMVR Mode Information Parsing Process

[0592] The syntax of the affine AMVR mode information can be reused for the AMVR mode information (applied to normal inter mode). Alternatively, different syntax elements can be used.

[0593] The affine AMVR mode information can be signaled conditionally. The following different embodiments show some examples of conditions.

[0594] 5.2.1 Example #1: CU Syntax Table

[0595]

[0596]

[0597]

[0598] 5.2.2 Example 2: Alternative CU Syntax Table Design

[0599]

[0600]

[0601]

[0602] 5.2.3 Example 3: Design of the Third CU Syntax Table

[0603]

[0604]

[0605]

[0606] 5.2.4 Example 4: Syntax Table Design with Different Syntaxes in AMVR and Affine AMVR Modes

[0607]

[0608]

[0609]

[0610] In an example, conditionsA is defined as follows:

[0611] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1&&(MvdCpL0[x0][y0][0][0]!=0||MvdCpL0[x0] [y0][0][1]! =0||MvdCpL1[x0][y0][0][0]! =0||MvdCpL1[x0][y0][0][1]! =0||MvdCpL0[x0][y0][1][0]! |MvdCpL0[x0][y0][1][1]! =0||MvdCpL1[x0][y0][1][0]! =0||MvdCpL1[x0][y0][1][1]! =0||MvdCpL0[x0][y0][2][0]! =0||MvdCpL0[x0][y0][2][1]! =0||MvdCpL1[x0][y0][2][0]! =0||MvdCpL1[x0][y0][2][1]! =0))

[0612] Alternatively, conditionsA is defined as follows:

[0613] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1&&(MvdCpL0[x0][y0][0][0]!=0 ||MvdCpL0[x0][y0][0][1]! =0||MvdCpL1[x0][y0][0][0]! =0||MvdCpL1[x0][y0][0][1]!

[0614] Alternatively, conditionsA is defined as follows:

[0615] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1&&(MvdCpLX[x0][y0][0][0]!=0||MvdCpLX[x0][y0][0][1]!=0)

[0616] Where X is 0 or 1.

[0617] Alternatively, conditionsA is defined as follows:

[0618] (sps_affine_amvr_enabled_flag&&inter_affine_flag==1)

[0619] In an example, conditionsB is defined as follows:

[0620] ! sps_cpr_enabled_flag||! (inter_pred_idc[x0][y0]==PRED_L0

[0621] &&ref_idx_l0[x0][y0]==num_ref_idx_l0_active_minus1)

[0622] Alternatively, conditionsB is defined as follows:

[0623] ! sps_cpr_enabled_flag||! (pred_mode[x0][y0]==CPR)

[0624] Alternatively, conditionsB is defined as follows:

[0625] ! sps_ibc_enabled_flag||! (pred_mode[x0][y0]==IBC)

[0626] When different syntax elements are used to encode or decode AMVR or affine AMVR, the context modeling and / or context applied to affine AMVR used in the embodiment in 5.5 may be applied accordingly.

[0627] 5.2.5 Semantics

[0628] amvr_flag[x0][y0] specifies the resolution of the motion vector differences. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered codec block relative to the top-left luma sample of the picture. amvr_flag[x0][y0] equal to 0 specifies that the resolution of the motion vector differences is 1 / 4 of the luma samples. amvr_flag[x0][y0] equal to 1 specifies that the resolution of the motion vector differences is further specified by amvr_coarse_precisoin_flag[x0][y0].

[0629] When amvr_flag[x0][y0] is not present, the inference is as follows:

[0630] - If sps_cpr_enabled_flag is equal to 1, then amvr_flag[x0][y0] is inferred to be equal to 1.

[0631] Otherwise (sps_cpr_enabled_flag is equal to 0), amvr_flag[x0][y0] is inferred to be equal to 0.

[0632] amvr_coarse_precisoin_flag[x0][y0] equal to 1 specifies that the resolution of motion vector differences is 4 luma samples when inter_affine_flag is equal to 0, and that the resolution of motion vector differences is 1 luma sample when inter_affine_flag is equal to 1. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the codec block under consideration relative to the top-left luma sample of the picture.

[0633] When amvr_coarse_precisoin_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0634] If inter_affine_flag[x0][y0] is equal to 0, the variable MvShift is set equal to (amvr_flag[x0][y0] + amvr_coarse_precisoin_flag[x0][y0]) << 1, and the variables MvdL0[x0][y0][0], MvdL0[x0][y0][1], MvdL1[x0][y0][0], MvdL1[x0][y0][1] are modified as follows:

[0635] MvdL0[x0][y0][0]=MvdL0[x0][y0][0]<<(MvShift+2)(7-70)

[0636] MvdL0[x0][y0][1]=MvdL0[x0][y0][1]<<(MvShift+2)(7-71)

[0637] MvdL1[x0][y0][0]=MvdL1[x0][y0][0]<<(MvShift+2)(7-72)

[0638] MvdL1[x0][y0][1]=MvdL1[x0][y0][1]<<(MvShift+2)(7-73)

[0639] If inter_affine_flag[x0][y0] is equal to 1, the variable MvShift is set equal to (amvr_coarse_precisoin_flag ? (amvr_coarse_precisoin_flag << 1): (-(amvr_flag << 1))), and the variables MvdCpL0[x0][y0][0][0], MvdCpL0[x0][y0][0][1], MvdCpL0[x0][y0][1][0], MvdCpL0[x0][y0][1][1], MvdCpL0[x0][y0][2][0], MvdCpL0[x0][y0][2][1] are modified as follows:

[0640] MvdCpL0[x0][y0][0][0]=MvdCpL0[x0][y0][0][0]<<(MvShift+2)(7-73)

[0641] MvdCpL1[x0][y0][0][1]=MvdCpL1[x0][y0][0][1]<<(MvShift+2)(7-67)

[0642] MvdCpL0[x0][y0][1][0]=MvdCpL0[x0][y0][1][0]<<(MvShift+2)(7-66)

[0643] MvdCpL1[x0][y0][1][1]=MvdCpL1[x0][y0][1][1]<<(MvShift+2)(7-67)

[0644] MvdCpL0[x0][y0][2][0]=MvdCpL0[x0][y0][2][0]<<(MvShift+2)(7-66)

[0645] MvdCpL1[x0][y0][2][1]=MvdCpL1[x0][y0][2][1]<<(MvShift+2)(7-67)

[0646] Alternatively, if inter_affine_flag[x0][y0] is equal to 1, then the variable MvShift is set equal to (affine_amvr_coarse_precisoin_flag ? (affine_amvr_coarse_precisoin_flag << 1): (-(affine_amvr_flag << 1))).

[0647] 5.3 Motion Vector Rounding Process

[0648] Modified the rounding process so that when a given rightShift value is equal to 0 (which occurs at 1 / 16 pixel precision), the rounding offset is set to 0 instead of (1<<(rightShift-1)).

[0649] For example, the subclauses for the rounding process of MV are modified as follows:

[0650] The inputs to this process are:

[0651] – motion vector mvX,

[0652] – Right shift parameter rightShift for rounding,

[0653] – Left shift parameter leftShift for resolution increase.

[0654] The output of this process is the rounded motion vector mvX.

[0655] For rounding of mvX, the following applies:

[0656] offset=(rightShift==0)? 0:(1<<(rightShift-1)) (8-371) mvX[0]=(mvX[0]>=0?(mvX[0]+offset)>>

[0657] rightShift:-((-mvX[0]+offset)>>rightShift))<<leftShift (8-372)mvX[1]=(mvX[1]>=0?(mvX[1]+offset)>>

[0658] rightShift:-((-mvX[1]+offset)>>rightShift))<<leftShift (8-373)

[0659] 5.4 Decoding Process

[0660] The rounding process called during the affine motion vector derivation is performed using an input of (MvShift+2) instead of a fixed input of 2.

[0661] Derivation process of motion vector prediction for luminance affine control points

[0662] The inputs to this process are:

[0663] – The luminance position (xCb, yCb) of the top left sample of the current luminance codec block relative to the top left luminance sample of the current picture,

[0664] – Two variables cbWidth and cbHeight specifying the width and height of the current brightness codec block,

[0665] – The reference index of the current codec unit refIdxLX, where X is 0 or 1,

[0666] – The number of control point motion vectors numCpMv.

[0667] The output of this process is the luma affine control point motion vector prediction mvpCpLX[cpIdx], where X is 0 or 1 and cpIdx = 0..numCpMv-1.

[0668] To derive the control point motion vector prediction candidate list, cpMvpListLX, where X is 0 or 1, the following ordered steps are applied:

[0669] The number of control point motion vector prediction candidates in the list numCpMvpCandLX is set equal to 0.

[0670] The variables availableFlagA and availableFlagB are both set to FALSE.

[0671] …

[0672] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to cpMvpLX[cpIdx], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as input, and the rounded cpMvpLX[cpIdx] (where cpIdx = 0..numCpMv–1) as output.

[0673] …

[0674] The variable availableFlagA is set equal to TRUE

[0675] The derivation process of luma affine control point motion vectors from neighboring blocks specified in clause 8.4.4.5 is called with the luma codec block position (xCb, yCb), luma codec block width and height (cbWidth, cbHeight), neighboring luma codec block position (xNb, yNb), neighboring luma codec block width and height (nbW, nbH), and the number of control point motion vectors numCpMv as input, and the control point motion vector prediction candidate cpMvpLY[cpIdx] (where cpIdx = 0..numCpMv-1) as output.

[0676] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to cpMvpLY[cpIdx], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as inputs, and the rounded cpMvpLY[cpIdx] (where cpIdx=0..numCpMv-1) as output.

[0677] …

[0678] The derivation process of the luma affine control point motion vector from neighboring blocks specified in clause 8.4.4.5 is called, with the luma codec block position (xCb, yCb), the luma codec block width and height (cbWidth, cbHeight), the neighboring luma codec block position (xNb, yNb), the neighboring luma codec block width and height (nbW, nbH), and the number of control point motion vectors numCpMv as input, and the control point motion vector prediction candidate cpMvpLX[cpIdx] (where cpIdx = 0..numCpMv-1) as output.

[0679] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to cpMvpLX[cpIdx], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as input, and the rounded cpMvpLX[cpIdx] (where cpIdx = 0..numCpMv-1) as output.

[0680] Make the following allocations:

[0681] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[0] (8-618)

[0682] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[1] (8-619)

[0683] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[2] (8-620)

[0684] numCpMvpCandLX=numCpMvpCandLX+1 (8-621)

[0685] Otherwise, if PredFlagLY[xNbBk][yNbBk](withY=!X) is equal to 1 and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbBk][yNbBk]], RefPicListX[refIdxLX]) is equal to 0, then the following applies:

[0686] The variable availableFlagB is set to TRUE

[0687] The derivation process of luma affine control point motion vectors from neighboring blocks specified in clause 8.4.4.5 is called with the luma codec block position (xCb, yCb), luma codec block width and height (cbWidth, cbHeight), neighboring luma codec block position (xNb, yNb), neighboring luma codec block width and height (nbW, nbH), and the number of control point motion vectors numCpMv as input, and the control point motion vector prediction candidate cpMvpLY[cpIdx] (where cpIdx = 0..numCpMv-1) as output.

[0688] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to cpMvpLY[cpIdx], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as inputs, and the rounded cpMvpLY[cpIdx] (where cpIdx=0..numCpMv-1) as output.

[0689] Make the following allocations:

[0690] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLY[0] (8-622)

[0691] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLY[1] (8-623)

[0692] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLY[2] (8-624)

[0693] numCpMvpCandLX=numCpMvpCandLX+1 (8-625)

[0694] When numCpMvpCandLX is less than 2, the following applies:

[0695] The derivation process for constructing the affine control point motion vector prediction candidate specified in clause 8.4.4.8 is called, with the luma codec block position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight, and the reference index refIdxLX of the current codec unit as input, and the availability flag availableConsFlagLX and cpMvpLX[cpIdx] (where cpIdx = 0..numCpMv-1) as output.

[0696] When availableConsFlagLX is equal to 1 and numCpMvpCandLX is equal to 0, the following allocations are made:

[0697] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[0] (8-626)

[0698] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[1] (8-627)

[0699] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[2] (8-628)

[0700] numCpMvpCandLX=numCpMvpCandLX+1 (8-629)

[0701] For cpIdx = 0..numCpMv–1 the following applies:

[0702] When numCpMvpCandLX is less than 2 and availableFlagLX[cpIdx] is equal to 1, the following allocations are made:

[0703] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[cpIdx] (8-630)

[0704] cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[cpIdx] (8-631)

[0705] cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[cpIdx] (8-632)

[0706] numCpMvpCandLX=numCpMvpCandLX+1 (8-633)

[0707] When numCpMvpCandLX is less than 2, the following applies:

[0708] Invoke the derivation process of the temporal luma motion vector prediction specified in clause 8.4.2.11, with the luma codec block position (xCb, yCb), the luma codec block width cbWidth, the luma codec block height cbHeight, and refIdxLX as input, and where the output is the availability flag availableFlagLXCol and the temporal motion vector prediction mvLXCol.

[0709] When availableFlagLXCol is equal to 1, the following applies:

[0710] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to mvLXCol, rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as inputs, and the rounded mvLXCol as output.

[0711] Make the following allocations:

[0712] cpMvpListLX[numCpMvpCandLX][0]=mvLXCol (8-634)

[0713] cpMvpListLX[numCpMvpCandLX][1]=mvLXCol (8-635)

[0714] cpMvpListLX[numCpMvpCandLX][2]=mvLXCol (8-636)

[0715] numCpMvpCandLX=numCpMvpCandLX+1 (8-637)

[0716] When numCpMvpCandLX is less than 2, repeat the following steps until numCpMvpCandLX is equal to 2, where both mvZero[0] and mvZero[1] are equal to 0:

[0717] cpMvpListLX[numCpMvpCandLX][0]=mvZero (8-638)

[0718] cpMvpListLX[numCpMvpCandLX][1]=mvZero (8-639)

[0719] cpMvpListLX[numCpMvpCandLX][2]=mvZero (8-640)

[0720] numCpMvpCandLX=numCpMvpCandLX+1 (8-641) The affine control point motion vector prediction cpMvpLX (where X is 0 or 1) is derived as follows:

[0721] cpMvpLX=cpMvpListLX[mvp_lX_flag[xCb][yCb]](8-642)

[0722] The derivation process of the constructed affine control point motion vector prediction candidate

[0723] The inputs to this process are:

[0724] – specifies the luma position (xCb, yCb) of the top left sample of the current luma codec block relative to the top left luma sample of the current picture,

[0725] – Two variables cbWidth and cbHeight specifying the width and height of the current brightness codec block,

[0726] – The reference index refIdxLX of the current prediction unit partition, where X is 0 or 1,

[0727] The output of this process is:

[0728] – The availability flag of the constructed affine control point motion vector prediction candidate availableConsFlagLX, where X is 0 or 1,

[0729] – availability flag availableFlagLX[cpIdx], where cpIdx = 0..2 and X is 0 or 1,

[0730] – Constructed affine control point motion vector prediction candidate cpMvLX[cpIdx], where cpIdx=0..numCpMv-1 and X is 0 or 1.

[0731] The first (top left) control point motion vector cpMvLX[0] and the availability flag availableFlagLX[0] are derived in the following ordered steps:

[0732] The sample positions (xNbB2, yNbB2), (xNbB3, yNbB3), and (xNbA2, yNbA2) are set equal to (xCb-1, yCb-1), (xCb, yCb-1), and (xCb-1, yCb), respectively.

[0733] The availability flag availableFlagLX[0] is set equal to 0, and both components of cpMvLX[0] are set equal to 0.

[0734] For (xNbTL,yNbTL) apply the following steps, where TL is replaced by B2, B3, and A2:

[0735] The codec block availability derivation process specified in the clause is called with the luma codec block position (xCb, yCb), the luma codec block width cbWidth, the luma codec block height cbHeight, the luma position (xNbY, yNbY) set equal to (xNbTL, yNbTL) as input, and the output is assigned to the codec block availability flag availableTL.

[0736] When availableTL equals TRUE and availableFlagLX[0] equals 0, the following steps apply:

[0737] If PredFlagLX[xNbTL][yNbTL] is equal to 1, and DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbTL][yNbTL]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLX[xNbTL][yNbTL] is not the current picture, then availableFlagLX[0] is set equal to 1 and the following assignments are made:

[0738] cpMvLX[0]=MvLX[xNbTL][yNbTL] (8-643)

[0739] Otherwise, when PredFlagLY[xNbTL][yNbTL](withY=!X) is equal to 1, and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbTL][yNbTL]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbTL][yNbTL] is not the current picture, then availableFlagLX[0] is set equal to 1 and the following assignments are made:

[0740] cpMvLX[0]=MvLY[xNbTL][yNbTL] (8-644)

[0741] When availableFlagLX[0] is equal to 1, the motion vector rounding process specified in clause 8.4.2.14 is called, with mvX set equal to cpMvLX[0], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as inputs, and the rounded cpMvLX[0] as output.

[0742] The second (top right) control point motion vector cpMvLX[1] and the availability flag availableFlagLX[1] are derived in the following ordered steps:

[0743] Set the sample positions (xNbB1, yNbB1) and (xNbB0, yNbB0) equal to (xCb+cbWidth-1, yCb-1) and (xCb+cbWidth, yCb-1), respectively.

[0744] The availability flag availableFlagLX[1] is set equal to 0, and both components of cpMvLX[1] are set equal to 0.

[0745] For (xNbTR,yNbTR) apply the following steps, where TR is replaced by B1 and B0:

[0746] The codec block availability derivation procedure specified in clause 6.4.X is called with the luma codec block position (xCb, yCb), the luma codec block width cbWidth, the luma codec block height cbHeight, the luma position (xNbY, yNbY) set equal to (xNbTR, yNbTR) as input, and the output is assigned to the codec block availability flag availableTR.

[0747] When availableTR equals TRUE and availableFlagLX[1] equals 0, the following steps apply:

[0748] If PredFlagLX[xNbTR][yNbTR] is equal to 1, and DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbTR][yNbTR]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLX[xNbTR][yNbTR] is not the current picture, then availableFlagLX[1] is set equal to 1 and the following assignments are made:

[0749] cpMvLX[1]=MvLX[xNbTR][yNbTR] (8-645)

[0750] Otherwise, when PredFlagLY[xNbTR][yNbTR](withY=!X) is equal to 1, and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbTR][yNbTR]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbTR][yNbTR] is not the current picture, then availableFlagLX[1] is set equal to 1 and the following assignments are made:

[0751] cpMvLX[1]=MvLY[xNbTR][yNbTR] (8-646)

[0752] When availableFlagLX[1] is equal to 1, the motion vector rounding process specified in clause 8.4.2.14 is called, with mvX set equal to cpMvLX[1], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as inputs, and the rounded cpMvLX[1] as output.

[0753] The third (bottom left) control point motion vector cpMvLX[2] and the availability flag availableFlagLX[2] are derived in the following ordered steps:

[0754] Set the sample positions (xNbA1, yNbA1) and (xNbA0, yNbA0) equal to (xCb-1, yCb+cbHeight-1) and (xCb-1, yCb+cbHeight), respectively.

[0755] The availability flag availableFlagLX[2] is set equal to 0, and both components of cpMvLX[2] are set equal to 0.

[0756] For (xNbBL,yNbBL) apply the following steps, where BL is replaced by A1 and A0:

[0757] The codec block availability derivation process specified in clause 6.4.X is called with the luma codec block position (xCb, yCb), the luma codec block width cbWidth, the luma codec block height cbHeight, the luma position (xNbY, yNbY) set equal to (xNbBL, yNbBL) as input, and the output is assigned to the codec block availability flag availableBL.

[0758] When availableBL equals TRUE and availableFlagLX[2] equals 0, the following steps apply:

[0759] If PredFlagLX[xNbBL][yNbBL] is equal to 1, and DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbBL][yNbBL]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbBL][yNbBL] is not the current picture, then availableFlagLX[2] is set equal to 1 and the following assignments are made:

[0760] cpMvLX[2]=MvLX[xNbBL][yNbBL] (8-647)

[0761] Otherwise, when PredFlagLY[xNbBL][yNbBL](withY=!X) is equal to 1, and DiffPicOrderCnt(RefPicListY[RefIdxLY[xNbBL][yNbBL]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbBL][yNbBL] is not the current picture, then availableFlagLX[2] is set equal to 1 and the following assignments are made:

[0762] cpMvLX[2]=MvLY[xNbBL][yNbBL] (8-648)

[0763] When availableFlagLX[2] is equal to 1, the motion vector rounding process specified in clause 8.4.2.14 is called, with mvX set equal to cpMvLX[2], rightShift set equal to (MvShift+2), and leftShift set equal to (MvShift+2) as inputs, and the rounded cpMvLX[2] as output.

[0764] 5.5 Context Modeling

[0765] Assign ctxInc to the syntax element using the binary number of the context codec:

[0766]

[0767]

[0768] Specify ctxInc using the left and above syntax elements:

[0769] In one example, the context is increased by an offset ctxInc=(condL&&availableL)+(condA&&availableA)+ctxSetIdx*3.

[0770] Alternatively, ctxInc=((condL&&availableL)||(condA&&availableA))+ctxSetIdx*3.

[0771] ctxInc=(condL&&availableL)+M*(condA&&availableA)+ctxSetIdx*3.(For example, M=2)

[0772] ctxInc=M*(condL&&availableL)+(condA&&availableA)+ctxSetIdx*3.(For example, M=2)

[0773]

[0774] initValue value of ctxIdx of amvr_flag:

[0775] Different contexts are used when the current block is affine or non-affine.

[0776]

[0777] Alternatively,

[0778]

[0779] Alternatively, the same context can be used when the current block is affine or non-affine

[0780]

[0781] Instead, the amvr_flag is bypassed for coding.

[0782] initValue value of ctxIdx of amvr_coarse_precisoin_flag:

[0783] Different contexts are used when the current block is affine or non-affine.

[0784]

[0785] Alternatively,

[0786]

[0787] Alternatively, the same context may be used when the current block is affine or non-affine.

[0788]

[0789] Instead, the amvr_coarse_precisoin_flag is bypassed for coding.

[0790] The above examples may be incorporated in the context of the methods described below, eg, methods 2510 to 2540 , which may be implemented at a video decoder or a video encoder.

[0791] Figures 25A-25D A flow chart illustrating an exemplary method for video processing is shown. Figure 25A The illustrated method 2510 includes determining, at step 2512, that a conversion between a current video block of a video and a codec representation of the current video block is based on a non-affine inter AMVR mode. The method 2510 further includes, at step 2514, performing the conversion based on the determination. In some implementations, the codec representation of the current video block is based on context-based coding, and wherein the context for coding the current video block is modeled without using affine AMVR mode information of neighboring blocks during the conversion.

[0792] like Figure 25B The illustrated method 2520 includes determining, at step 2522, that a conversion between a current video block of a video and a codec representation of the current video block is based on an affine adaptive motion vector resolution (affine AMVR) mode. The method 2520 further includes performing the conversion based on the determination, at step 2524. In some implementations, the codec representation of the current video block is based on context-based coding, and wherein the variables control two probability update speeds of the context.

[0793] like Figure 25C The illustrated method 2530 includes, at step 2532, determining that a conversion between a current video block of a video and a codec representation of the current video block is based on an affine AMVR mode. The method 2530 further includes, at step 2534, performing the conversion based on the determination. In some implementations, the codec representation of the current video block is based on context-based coding, and wherein codec information of neighboring blocks is used to model a context for coding the current video block, the codec information of the neighboring blocks using both an affine inter mode and a normal inter mode AMVR mode during the conversion.

[0794] like Figure 25DThe illustrated method 2540 includes, at step 2542, determining use of multiple contexts for converting between a current video block of a video and a codec representation of the current video block. The method 2540 further includes performing the conversion based on the determination at step 2544. In some implementations, the multiple contexts are used to encode a syntax element indicating coarse motion precision.

[0795] like Figure 25E The method 2550 shown in FIG. 2 includes, at step 2552, determining whether to use a symmetric motion vector difference (SMVD) mode for converting between a current video block of a video and a codec representation of the current video block based on a currently selected best mode for the conversion. The method 2554 further includes performing the conversion based on the determination.

[0796] like Figure 25F The illustrated method 2560 includes, at step 2562, determining whether to use an affine SMVD mode for converting between a current video block of a video and a codec representation of the current video block based on a currently selected best mode for the conversion. The method 2564 further includes performing the conversion based on the determination.

[0797] 6. Example Implementations of the Disclosed Technology

[0798] Figure 26 2 is a block diagram of a video processing device 2600. Device 2600 can be used to implement one or more methods described herein. Device 2600 can be implemented in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Device 2600 may include one or more processors 2602, one or more memories 2604, and video processing hardware 2606. Processor(s) 2602 may be configured to implement one or more methods described herein (including but not limited to method 2500). Memory(s) 2604 may be used to store data and code used to implement the methods and techniques described herein. Video processing hardware 2606 may be used to implement some of the techniques described herein in hardware circuitry.

[0799] Figure 28 is another example of a block diagram of a video processing system that may implement the disclosed technology. Figure 28is a block diagram illustrating an example video processing system 2800 in which the various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 2800. System 2800 may include an input 2802 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or in a compressed or encoded format. Input 2802 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0800] System 2800 may include a codec component 2804 that can implement various coding or encoding methods described in this document. The codec component 2804 can reduce the average bit rate of the video from the input 2802 to the output of the codec component 2804 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of the codec component 2804 can be stored or transmitted via a connected communication, as shown in component 2806. The bitstream (or codec) representation of the storage or transmission of the video received at the input 2802 can be used by component 2808 to generate pixel values ​​or displayable video sent to the display interface 2810. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that the codec tools or operations are used at the encoder, and the corresponding decoding tools or the operation of reversing the codec results will be performed by the decoder.

[0801] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described in this document may be implemented in various electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of performing digital data processing and / or video display.

[0802] In some embodiments, the video encoding and decoding method can be used in Figure 26 or the device implemented on the hardware platform described in 28.

[0803] The following clause-based format can be used to describe various techniques and embodiments. These clauses can be implemented as preferred features of some embodiments.

[0804] The first set of clauses uses some of the techniques described in the previous section, including, for example, items 16-18, 23, and 26 in the previous section.

[0805] 1. A video processing method, comprising:

[0806] Determining that a transformation between a current video block of a video and a codec representation of the current video block is based on a non-affine inter AMVR mode; and

[0807] performing the converting based on the determination, and

[0808] The codec representation of the current video block is based on context-based coding, and the context for coding the current video block is modeled without using affine AMVR pattern information of neighboring blocks during the conversion.

[0809] 2. The method of clause 1, wherein a non-affine inter AMVR mode index of the neighboring blocks is coded using.

[0810] 3. A method according to clause 2, wherein the context index used to encode and decode the non-affine inter-frame AMVR mode index of the current video block depends on at least one of the following: an AMVR mode index, an affine mode flag, or the availability of left and above neighboring blocks, and the non-affine inter-frame AMVR mode index indicating a certain MVD precision.

[0811] 4. The method of clause 3, wherein offset values ​​are derived separately for the left neighboring block and the above neighboring block, and the context index is derived as the sum of the offset values ​​of the left neighboring block and the above neighboring block.

[0812] 5. A method according to clause 4, wherein at least one of the offset values ​​is derived as 1 if the corresponding neighboring block is available and is not encoded or decoded in affine mode and the AMVR mode index of the neighboring block is not equal to 0, otherwise at least one of the offset values ​​is derived as 0.

[0813] 6. The method of clause 1, wherein the affine AMVR pattern information of the neighboring blocks is not used directly but is used indirectly as a function of the affine AMVR pattern information.

[0814] 7. A method according to clause 6, wherein the function returns true if amvr_mode[xNbL][yNbL] or amvr_flag[xNbL][yNbL] of the neighbouring block cover (xNbL, yNbL) indicates a certain MVD precision.

[0815] 8. The method of clause 1, wherein the first syntax element of the non-affine inter AMVR mode of the current video block is encoded and decoded using the affine AMVR mode information of the neighboring block.

[0816] 9. A method according to clause 1, wherein, when the AMVR mode of the current video block is represented by multiple syntax elements, any one of the multiple syntax elements is encoded and decoded using the AMVR mode information of the neighboring block encoded and decoded in both non-affine inter mode and affine inter mode.

[0817] 10. The method of clause 1, wherein, when the AMVR mode of the current video block is represented by a plurality of syntax elements, the AMVR mode information of the neighboring block coded in affine inter mode is not utilized to code or decode any of the plurality of syntax elements.

[0818] 11. A method according to clause 1, wherein, when the AMVR mode of the current video block is represented by multiple syntax elements, the AMVR mode information of the neighboring block encoded and decoded in the affine inter-frame mode is not directly utilized but indirectly used to encode and decode any one of the multiple syntax elements.

[0819] 12. The method of any of clauses 9-11, wherein the AMVR mode of the current video block comprises an inter-affine AMVR mode and a non-inter-affine AMVR mode.

[0820] 13. A video processing method, comprising:

[0821] Determining that a conversion between a current video block of a video and a codec representation of the current video block is based on an affine adaptive motion vector resolution (affine AMVR) mode; and

[0822] performing the converting based on the determination, and

[0823] The codec representation of the current video block is based on context-based codec, and the variables control two probability update speeds of the context.

[0824] 14. The method of clause 13, wherein the two probability update rates include a faster update rate defined by (shiftIdx>>2)+2, shiftIdx indicating the variable.

[0825] 15. The method of clause 13, wherein the two probability update speeds include a slower update speed defined by (shiftIdx&3)+3+shift0, shift0 being defined by (shiftIdx>>2)+2, and shiftIdx indicating the variable.

[0826] 16. The method of clause 14, wherein the faster update rate is between 2 and 5.

[0827] 17. A video processing method, comprising:

[0828] Determining that a transformation between a current video block of a video and a codec representation of the current video block is based on an affine AMVR mode; and

[0829] performing the converting based on the determination, and

[0830] wherein the codec representation of the current video block is based on context-based coding, and wherein the context for coding the current video block is modeled using codec information of neighboring blocks, the codec information of the neighboring blocks using an AMVR mode of both an affine inter mode and a normal inter mode during the conversion.

[0831] 18. The method of clause 17, wherein the AMVR mode information of the neighboring blocks is directly used for the context-based coding and decoding.

[0832] 19. The method of clause 17, wherein AMVR mode information of the neighboring blocks coded in normal inter mode is not allowed to be used for the context-based coding.

[0833] 20. The method of clause 17, wherein the AMVR mode information of the neighboring blocks coded in normal inter mode is not used directly but is used indirectly as a function of the AMVR mode information.

[0834] 21. The method of clause 17, wherein the AMVR mode information of the neighboring block is utilized to encode and decode a first syntax element of the affine AMVR mode of the current video block.

[0835] 22. The method of clause 17, wherein, when the affine AMVR mode is represented by a plurality of syntax elements, any one of the plurality of syntax elements is encoded or decoded using AMVR mode information of the neighboring blocks encoded or decoded in both the affine inter mode and the normal inter mode.

[0836] 23. The method of clause 17, wherein, if the affine AMVR mode is represented by a plurality of syntax elements, none of the plurality of syntax elements is coded or decoded using AMVR mode information of the neighboring block coded or decoded in normal inter mode.

[0837] 24. A method according to clause 17, wherein, when the affine AMVR mode is represented by a plurality of syntax elements, AMVR mode information of the neighboring block coded in normal inter mode is not directly utilized but indirectly used to encode or decode any one of the plurality of syntax elements.

[0838] 25. A video processing method, comprising:

[0839] For a conversion between a current video block of a video and a codec representation of the current video block, determining use of a plurality of contexts for the conversion; and

[0840] performing the converting based on the determination, and

[0841] The multiple contexts are used to encode and decode syntax elements indicating coarse motion accuracy.

[0842] 26. The method of clause 25, wherein the plurality of contexts corresponds to exactly two contexts.

[0843] 27. The method of clause 26, wherein the plurality of contexts are selected based on whether the current video block is affine coded.

[0844] 28. A method according to clause 25, wherein a further syntax element is used to indicate that the conversion is based on an affine AMVR mode.

[0845] 29. A method according to clause 28, wherein the further syntax element is coded using only a first context and the syntax element is coded using only a second context.

[0846] 30. The method of clause 28, wherein the another syntax element is coded using only the first context and the syntax element is bypass coded.

[0847] 31. The method of clause 28, wherein the other syntax element is bypass coded and the syntax element is bypass coded.

[0848] 32. A method according to clause 25, wherein all syntax elements related to motion vector accuracy are bypass coded.

[0849] 33. A method according to clause 25 or 26, wherein, for normal inter mode, the syntax element indicates selection from a set consisting of 1 pixel or 4 pixel precision.

[0850] 34. A method according to clause 25 or 26, wherein, for affine mode, the syntax element indicates a selection between at least 1 / 16 pixel or 1 pixel precision.

[0851] 35. A method according to clause 34, wherein the syntax element is equal to 0, then the affine mode is 1 / 6 pixel precision.

[0852] 36. A method according to clause 34, wherein the syntax element is equal to 1, then the affine mode is 1 pixel precision.

[0853] 37. A method according to any of clauses 1 to 36, wherein performing the conversion comprises generating the codec representation from the current video block.

[0854] 38. A method according to any of clauses 1 to 36, wherein performing the conversion comprises generating the current video block from the codec representation.

[0855] 39. An apparatus in a video system comprising a processor and non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of one or more of clauses 1 to 38.

[0856] 40. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for implementing the method of one or more of clauses 1 to 38.

[0857] The second set of clauses uses some of the techniques described in the previous section, including, for example, items 19-21 and 23 in the previous section.

[0858] 1. A video processing method, comprising:

[0859] For a conversion between a current video block of a video and a codec representation of the current video block, determining whether to use a symmetric motion vector difference (SMVD) mode based on a currently selected best mode for the conversion; and

[0860] The converting is performed based on the determination.

[0861] 2. A method according to clause 1, wherein the SMVD mode is used without explicit signaling of at least one reference index of a reference list.

[0862] 3. The method of clause 2, wherein the reference index is derived based on a recursive picture order count (POC) calculation.

[0863] 4. The method of clause 1 , wherein, in a case where the currently selected best mode is Merge mode or UMVE mode, the determining disables use of the SMVD mode.

[0864] 5. The method of clause 4, wherein the UMVE mode applies motion vector offset to refine motion candidates derived from a Merge candidate list.

[0865] 6. The method of clause 1 , wherein the determining disables use of the SMVD mode if the currently selected best mode does not use the SMVD mode codec.

[0866] 7. The method of clause 1 , wherein the determining disables use of the SMVD mode if the currently selected best mode is an affine mode.

[0867] 8. The method of clause 1, wherein, in a case where the currently selected best mode is the Sub-Block Merge mode, the determining disables use of the SMVD mode.

[0868] 9. The method of clause 1 , wherein, in a case where the currently selected best mode is an affine SMVD mode, the determining disables use of the SMVD mode.

[0869] 10. The method of clause 1, wherein, in a case where the currently selected best mode is Affine Merge mode, the determining disables use of the SMVD mode.

[0870] 11. A method according to any of clauses 2 to 10, wherein the determining is applied only if the MVD (Motion Vector Difference) precision is greater than or equal to the precision.

[0871] 12. The method of any of clauses 2 to 10, wherein the determining is applied only if the MVD precision is greater than the precision.

[0872] 13. A method according to any of clauses 2 to 10, wherein the determining is applied only when the MVD precision is less than or equal to the precision.

[0873] 14. A method according to any of clauses 2 to 10, wherein the determining is applied only if the MVD precision is less than the precision.

[0874] 15. A video processing method, comprising:

[0875] For a conversion between a current video block of a video and a codec representation of the current video block, determining whether to use an affine SMVD mode based on a currently selected best mode for the conversion; and

[0876] The converting is performed based on the determination.

[0877] 16. The method of clause 15, wherein, in a case where the currently selected best mode is Merge mode or UMVE mode, the determining disables use of the affine SMVD mode.

[0878] 17. The method of clause 15, wherein, in the event that the currently selected best mode does not use an affine SMVD mode codec, the determining disables use of the affine SMVD mode.

[0879] 18. The method of clause 15, wherein, in a case where the currently selected best mode is Sub-block Merge mode, the determining disables use of the Affine SMVD mode.

[0880] 19. The method of clause 15, wherein, in a case where the currently selected best mode is SMVD mode, the determining disables use of the affine SMVD mode.

[0881] 20. The method of clause 15, wherein, in a case where the currently selected best mode is the Affine Merge mode, the determining disables use of the Affine SMVD mode.

[0882] 21. A method according to any of clauses 16 to 20, wherein the determining is applied only when the affine MVD precision is greater than or equal to the precision.

[0883] 22. The method of any of clauses 16 to 20, wherein the determining is applied only if the affine MVD precision is greater than the precision.

[0884] 23. A method according to any of clauses 16 to 20, wherein the determining is applied only when the affine MVD precision is less than or equal to the precision.

[0885] 24. A method according to any of clauses 16 to 20, wherein the determining is applied only when the affine MVD precision is less than the precision.

[0886] 25. The method according to clause 1 or 15, further comprising:

[0887] Determining characteristics of the current video block, the characteristics comprising one or more of: a block size of the video block, a type of slice or picture or slice of the video block, or motion information related to the video block, and wherein the determining is also based on the characteristics.

[0888] 26. A method according to any of clauses 1 to 25, wherein performing the conversion comprises generating the codec representation from the current block.

[0889] 27. A method according to any of clauses 1 to 25, wherein performing the conversion comprises generating the current block from the codec representation.

[0890] 28. An apparatus in a video system, comprising a processor and non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of one or more of clauses 1 to 27.

[0891] 29. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for implementing the method of one or more of clauses 1 to 27.

[0892] It will be appreciated from the foregoing that specific embodiments of the disclosed technology have been described herein for illustrative purposes, but various modifications may be made without departing from the scope of the invention. Accordingly, the disclosed technology is not to be limited except as set forth in the appended claims.

[0893] The subject matter and implementation of the functional operations described in this patent document can be implemented in various systems, in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more thereof. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, that is, one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials that effects a machine-readable propagated signal, or a combination of one or more thereof. The term "data processing unit" or "data processing apparatus" encompasses all devices, equipment, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof.

[0894] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or code portions). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0895] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0896] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include or be operably coupled to one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from or transfer data to the one or more mass storage devices, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0897] Intended to combine the instructions with the appendix Figure 1 The foregoing descriptions are to be considered as exemplary only, where exemplary means example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0898] Although this patent document contains many details, these details should not be interpreted as limitations on any invention or the scope of the claim, but rather as descriptions of features specific to particular embodiments of particular inventions. In this patent document, certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although the above features may be described as working in certain combinations and even initially claimed as such, in some cases, one or more features from the combination may be removed from the claimed combination, and the claimed combination may refer to a subcombination or a variation of the subcombination.

[0899] Similarly, while operations may be depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown, or in sequential order, or that all illustrated operations be performed, in order to achieve desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0900] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For a conversion between a current video block of a video and a bitstream of the current video block, determining whether to use a symmetric motion vector difference (SMVD) mode based on a currently selected best mode for the conversion; as well as performing said converting based on said determining, wherein the SMVD mode is used without explicit signaling of at least one reference index of a reference list, The determining is applied only when the motion vector difference MVD precision is greater than or equal to a first precision, wherein the first precision is integer pixel precision.

2. The method according to claim 1, wherein The reference index is derived based on a recursive picture order count (POC) calculation.

3. The method according to claim 1, wherein In case the currently selected best mode is the Merge mode or the UMVE mode, the determining disables use of the SMVD mode.

4. The method according to claim 3, wherein: The UMVE mode applies motion vector offset to refine motion candidates derived from the Merge candidate list.

5. The method according to claim 1, wherein In case the currently selected best mode does not use the SMVD mode codec, the determination disables the use of the SMVD mode.

6. The method according to claim 1, wherein In case the currently selected best mode is the affine mode, the determining disables the use of the SMVD mode.

7. The method according to claim 1, wherein In case the currently selected best mode is the Sub-Block Merge mode, the determining disables the use of the SMVD mode.

8. The method according to claim 1, wherein In case the currently selected best mode is the affine SMVD mode, the determining disables the use of the SMVD mode.

9. The method according to claim 1, wherein In case the currently selected best mode is the Affine Merge mode, the determining disables the use of the SMVD mode.

10. The method according to claim 1, further comprising: Determining characteristics of the current video block, the characteristics comprising one or more of: a block size of the video block, a type of slice or picture or slice of the video block, or motion information related to the video block, and wherein the determining is also based on the characteristics.

11. The method according to any one of claims 1 to 10, wherein: Performing the conversion includes generating the bitstream from the current video block.

12. The method according to any one of claims 1 to 10, wherein: Performing the conversion includes generating the current video block from the bitstream.

13. A video processing method, comprising: For a conversion between a current video block of a video and a bitstream of the current video block, determining whether to use an affine SMVD mode based on a currently selected best mode for the conversion; as well as performing said converting based on said determining, wherein, in a case where the currently selected best mode is the Merge mode or the UMVE mode, the determining disables the use of the affine SMVD mode; and The determining is applied only when the affine MVD precision is greater than or equal to a second precision, wherein the second precision is integer pixel precision.

14. The method according to claim 13, wherein In case the currently selected best mode does not use the affine SMVD mode for encoding and decoding, the determining disables the use of the affine SMVD mode.

15. The method according to claim 13, wherein In case the currently selected best mode is the Sub-Block Merge mode, the determining disables the use of the Affine SMVD mode.

16. The method according to claim 13, wherein: In case the currently selected best mode is the SMVD mode, the determining disables the use of the affine SMVD mode.

17. The method according to claim 13, wherein: In case the currently selected best mode is the affine Merge mode, the determining disables the use of the affine SMVD mode.

18. The method of claim 13, further comprising: Determining characteristics of the current video block, the characteristics comprising one or more of: a block size of the video block, a type of slice or picture or slice of the video block, or motion information related to the video block, and wherein the determining is also based on the characteristics.

19. The method according to any one of claims 13 to 18, wherein Performing the conversion includes generating the bitstream from the current video block.

20. The method according to any one of claims 13 to 18, wherein Performing the conversion includes generating the current video block from the bitstream.

21. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1 to 12 or claims 13 to 18.

22. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for implementing the method of any one of claims 1 to 12 or claims 13 to 18.

Images