Context for coding affine mode adaptive motion vector resolution - Patents.com

Adaptive motion vector resolution techniques enhance video coding efficiency by optimizing motion vector prediction in affine modes, addressing bandwidth challenges and improving compression performance in digital video standards.

JP7583084B2Active Publication Date: 2024-11-13DOUYIN VISION CO LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
JP2023027696
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-03-27
Filing Date
2023-02-24
Publication Date
2024-11-13
Estimated Expiration
2040-01-31

AI Technical Summary

Technical Problem

Existing video coding standards face challenges in achieving efficient compression of digital video due to increasing bandwidth demands and the complexity of motion vector prediction, particularly in affine modes, which are not adequately addressed by current adaptive motion vector resolution techniques.

Method used

The implementation of adaptive motion vector resolution (AMVR) methods for deriving and signaling motion vector predictors in affine modes, enhancing coding efficiency by improving motion vector accuracy and precision in video coding processes.

Benefits of technology

This approach leads to improved coding efficiency and reduced bandwidth requirements by optimizing motion vector prediction, particularly in affine modes, thereby enhancing the overall performance of video coding standards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007583084000054
    Figure 0007583084000054
  • Figure 0007583084000055
    Figure 0007583084000055
  • Figure 0007583084000056
    Figure 0007583084000056
Patent Text Reader

Abstract

A video processing method, device and system are provided. A video processing method includes determining that a transformation between a current video block of a video and a coding representation of the current video block is based on a non-affine inter-AMVR mode, and performing the transformation based on the determining, wherein the coding representation of the current video block is based on context-based coding, where a context used to code the current video block is modeled without using affine AMVR mode information of neighboring blocks during the transformation.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) Under applicable patent laws and / or regulations under the Paris Convention, this application is intended to timely claim priority to and the benefit of International Patent Application No. PCT / CN2019 / 074216, filed January 31, 2019, International Patent Application No. PCT / CN2019 / 074433, filed February 1, 2019, and International Patent Application No. PCT / CN2019 / 079962, filed March 27, 2019. For all purposes under U.S. law, the entire disclosures of the above applications are incorporated by reference as part of the disclosure herein.

[0002] This patent specification relates to image processing techniques, devices and systems. [Background technology]

[0003] Despite advances in video compression, digital video remains the largest bandwidth user on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demands for digital video use are expected to continue to grow. Summary of the Invention

[0004] Devices, systems, and methods are described that relate to digital video coding, and in particular to deriving and signaling motion vector predictors for affine modes with adaptive motion vector resolution (AMVR). The methods described may be applied to both existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards or video codecs.

[0005] In one representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining that a transformation between a current video block of a video and a coding representation of the current video block is based on a non-affine Inter AMVR mode and performing the transformation based on the determination, where the coding representation of the current video block is based on context-based coding, and a context used for coding the current video block is modeled without using affine AMVR mode information of neighboring blocks during the transformation.

[0006] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining that a conversion between a current video block of a video and a coding representation of the current video block is based on an affine adaptive motion vector resolution (affine AMVR) mode, and performing the conversion based on the determination, where the coding representation of the current video block is based on context-based coding, and variables control two probability update rates for the context.

[0007] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining that a transformation between a current video block of a video and a coding representation of the current video block is based on an affine AMVR mode, and performing the transformation based on the determination, where the coding representation of the current video block is based on context-based coding, and a context used to code the current video block is modeled using coding information of neighboring blocks in which both affine inter and normal inter AMVR modes are used during the transformation.

[0008] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining use of multiple contexts for a conversion between a current video block of a video and a coded representation of the current video block, and performing the conversion based on the determination, where the multiple contexts are used to code a syntax element indicating a coarse motion precision.

[0009] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining whether to use a symmetric motion vector difference (SMVD) mode for transforming between a current video block of a video and a coded representation of the current video block based on a currently selected best mode for the transform, and performing the transform based on the determination.

[0010] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining whether to use an affine SMVD mode for transforming between a current video block of a video and a coded representation of the current video block based on a currently selected best mode for the transform, and performing the transform based on the determination.

[0011] In another exemplary embodiment, the method is embodied in a processor executable code and stored on a computer readable program medium.

[0012] In yet another exemplary aspect, a device configured or operable to perform the above-described method is disclosed. The device may include a processing device programmed to implement the method.

[0013] In yet another representative aspect, a video decoder device may implement the methods as described herein.

[0014] These and other aspects and features of the disclosed technology are explained in more detail in the drawings, description and claims. [Brief description of the drawings]

[0015] [Figure 1] Diagram showing an example of building a merge candidate list [Diagram 2] FIG. 1 shows an example of spatial candidate locations. [Diagram 3] FIG. 1 shows an example of candidate pairs for which spatial merge candidate redundancy checks are performed. [Figure 4A] FIG. 1 illustrates an example of a second prediction unit (PU) location based on the size and shape of a current block. [Figure 4B] FIG. 1 illustrates an example of a second prediction unit (PU) location based on the size and shape of a current block. [Diagram 5] FIG. 1 illustrates an example of motion vector scaling for temporal merge candidates. [Figure 6] FIG. 13 is a diagram showing an example of candidate positions of temporal merge candidates; [Figure 7] FIG. 1 shows an example of generating joint bidirectional prediction merge candidates. [Figure 8] FIG. 1 shows an example of constructing motion vector prediction candidates. [Figure 9] FIG. 1 illustrates an example of motion vector scaling for spatial motion vector candidates. [Figure 10] FIG. 1 illustrates an example of motion prediction using an alternative temporal motion vector prediction (ATMVP) algorithm for a coding unit (CU). [Figure 11] FIG. 1 illustrates an example of a coding unit (CU) with sub-blocks and neighboring blocks used in the spatial-temporal motion vector prediction (STMVP) algorithm. [Figure 12A] FIG. 1 illustrates an exemplary snapshot of sub-blocks when using an overlapped block motion compensation (OBMC) algorithm. [Figure 12B] FIG. 1 illustrates an exemplary snapshot of sub-blocks when using an overlapped block motion compensation (OBMC) algorithm. [Figure 13] A diagram showing an example of the neighborhood of samples used to derive parameters for the Local Illumination Compensation (LIC) algorithm. [Figure 14] FIG. 1 shows an example of a simplified affine motion model. [Figure 15] FIG. 1 shows an example of an affine motion vector field (MVF) for each subblock. [Figure 16] A diagram showing an example of motion vector prediction (MVP) in AF_INTER affine motion mode. [Figure 17A] Diagram showing examples of 4-parameter and 6-parameter affine modes, respectively. [Figure 17B] Diagram showing examples of 4-parameter and 6-parameter affine modes, respectively. [Figure 18A] A diagram showing examples of AF_MERGE affine motion mode candidates. [Figure 18B] A diagram showing examples of AF_MERGE affine motion mode candidates. [Figure 19] A diagram showing an example of bilateral matching in the Pattern Matching Motion Vector Derivation (PMMVD) mode, a special merging mode based on the Frame Rate Up Conversion (FRUC) algorithm. [Figure 20] A diagram showing an example of template matching in the FRUC algorithm. [Figure 21] A diagram showing an example of one-sided motion estimation in the FRUC algorithm. [Figure 22] Diagram showing an example of an optical flow trajectory used in the Bidirectional Optical Flow (BIO) algorithm. [Figure 23A] FIG. 1 shows an example snapshot using the Bidirectional Optical Flow (BIO) algorithm without block enhancement. [Figure 23B] FIG. 1 shows an example snapshot using the Bidirectional Optical Flow (BIO) algorithm without block enhancement. [Figure 24] FIG. 1 illustrates an example of a decoder-side motion vector refinement (DMVR) algorithm based on bilateral template matching. [Figure 25A] FIG. 1 illustrates a flowchart of an exemplary method for video processing in accordance with some implementations of the disclosed technology. [Figure 25B] FIG. 1 illustrates a flowchart of an exemplary method for video processing in accordance with some implementations of the disclosed technology. [Figure 25C] FIG. 1 illustrates a flowchart of an exemplary method for video processing in accordance with some implementations of the disclosed technology. [Figure 25D] FIG. 1 illustrates a flowchart of an exemplary method for video processing in accordance with some implementations of the disclosed technology. [Figure 25E] FIG. 1 illustrates a flowchart of an exemplary method for video processing in accordance with some implementations of the disclosed technology. [Figure 25F] FIG. 1 illustrates a flowchart of an exemplary method for video processing in accordance with some implementations of the disclosed technology. [Figure 26] FIG. 1 is a block diagram illustrating an example of a hardware platform for implementing the visual media decoding or visual media coding techniques described in this patent specification. [Figure 27] Diagram showing an example of symmetric mode [Figure 28] FIG. 2 is another block diagram illustrating an example of a hardware platform for implementing the video processing system described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Due to the increasing demand for higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs generally include electronic circuits or software that compress or decompress digital video and are constantly being improved to provide higher coding efficiency. Video codecs convert uncompressed video to compressed formats or vice versa. There is a complex relationship between the quality of the video, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, the ease of editing, random access, and the end-to-end delay (latency). The compression format usually complies with a standard video compression standard, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2) [1], a general-purpose video coding standard to be completed, or other current and / or future video coding standards.

[0017] Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used herein to improve readability of the description and are not intended to limit the description or embodiments (and / or implementations) to only the respective sections.

[0018] 1. Example of Inter Prediction in HEVC / H.265

[0019] Video coding standards have improved significantly over the years and now, in part, achieve high coding efficiency and support higher resolutions. Recent standards such as HEVC and H.265 are based on a hybrid video coding structure in which temporal prediction plus transform coding is utilized.

[0020] 1.1 Prediction mode examples

[0021] Each inter-predicted PU (Prediction Unit) has motion parameters for one or two reference picture lists. In some embodiments, the motion parameters include a motion vector and a reference picture index. In other embodiments, the use of one of the two reference picture lists may be signaled using inter_pred_idc. In yet other embodiments, the motion vector may be explicitly coded as a delta to the predictor.

[0022] If one CU is coded in skip mode, one PU is associated with this CU, there are no significant residual coefficients, no coding motion vector deltas, and no reference picture indexes. Specify merge mode, which obtains motion parameters for the current PU from neighboring PUs, including spatial and temporal candidates. Merge mode can be applied to any inter-predicted PU, not just for skip mode. An alternative to merge mode is explicit transmission of motion parameters, explicitly signaling a motion vector for each reference picture list and a reference picture index corresponding to the use of the reference picture list, for each PU.

[0023] If the signaling indicates to use one of the two reference picture lists, the PU is generated from a block of one sample, which is called "uni-prediction". Uni-prediction is available for both P and B slices [2].

[0024] If the signaling indicates to use both reference picture lists, generate the PU from a block of two samples. This is called "bi-prediction." Bi-prediction is only available for B slices.

[0025] 1.1.1 Implementation of Constructing Merge Mode Candidates

[0026] When predicting a PU using merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to look up the motion information. The construction of this list can be summarized based on the following sequence of steps:

[0027] Step 1: Initial candidate selection Step 1.1: Spatial candidate derivation Step 1.2: Redundancy check of spatial candidates Step 1.3: Temporal candidate derivation Step 2: Additional candidate insertions Step 2.1: Creating bidirectional prediction candidates Step 2.2: Inserting zero motion candidates

[0028] Figure 1 shows an example of constructing a merge candidate list based on the sequence of steps above. For spatial merge candidate derivation, we select up to four merge candidates among candidates at five different positions. For temporal merge candidate derivation, we select up to one merge candidate among two candidates. Since we assume a constant number of candidates per PU on the decoder side, we generate additional candidates if the number of candidates does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, we use a shortened unary binarization (TU) to encode the index of the best merge candidate. If the size of a CU is equal to 8, all PUs of the current CU share one merge candidate list, which is the same as the merge candidate list of the 2N × 2N prediction units.

[0029] 1.1.2 Constructing spatial merging candidates

[0030] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any PU at positions A1, B1, B0, A0 is unavailable (e.g., because it belongs to another slice or tile) or is intra-coded. After adding the candidate at position A1, adding the remaining candidates is subject to a redundancy check, which can ensure that candidates with the same motion information are removed from the list, improving coding efficiency.

[0031] In order to reduce computational complexity, the above redundancy check does not consider all possible candidate pairs. Instead, only the pairs linked by arrows in FIG. 3 are considered, and a candidate is added to the list only if the corresponding candidate used in the redundancy check does not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a division different from 2N×2N. FIG. 4A and FIG. 4B depict the second PU for N×2N and 2N×N, respectively. When dividing the current PU into N×2N, the candidate at position A1 is not considered for list construction. In some embodiments, adding this candidate makes two prediction units have the same motion information, and it is redundant to have only one PU in one coding unit. Similarly, when dividing the current PU into 2N×N, position B1 is not considered.

[0032] 1.1.3 Building Temporal Merge Candidates

[0033] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the co-located PU belonging to the picture with the smallest POC difference with the current picture in a given reference picture list. In the slice header, the reference picture list used for deriving the co-located PU is explicitly signaled.

[0034] FIG. 5 shows an example of derivation of a scaled motion vector (dotted line) for a temporal merge candidate, where t is the POC difference between the reference picture of the current picture and the current picture, and t is the POC difference between the reference picture of the co-located picture and the co-located picture, scaled from the motion vector of the co-located PU using POC distances t, t. Set the reference picture index of the temporal merge candidate equal to zero. For a B slice, we obtain two motion vectors, one for reference picture list 0 and the other for reference picture list 1, and combine them to form a bi-predictive merge candidate.

[0035] In the collocation PU(Y) belonging to the reference frame, select a position of a temporal candidate between candidates C0 and C1 as shown in Fig. 6. If the PU at position C0 is not available, is intra-coded, or is outside the current CTU, position C1 is used. Otherwise, position C0 is used to derive the temporal merge candidate.

[0036] 1.1.4 Building Additional Types of Merge Candidates

[0037] Besides the spatio-temporal merge candidates, there are two additional types of merge candidates: joint bi-predictive merge candidates and zero merge candidates. The spatio-temporal merge candidates are utilized to generate joint bi-predictive merge candidates. The joint bi-predictive merge candidates are used only for B slices. The joint bi-predictive candidate is generated by combining the first reference picture list motion parameters of the first candidate with the second reference picture list motion parameters of another candidate. If these two tuples provide different motion hypotheses, these tuples form a new bi-predictive candidate.

[0038] FIG. 7 shows an example of this process, where two candidates in the original list (710, left) with mvL0, refIdxL0 or mvL1, refIdxL1 are used to generate a combined bidirectional prediction merge candidate that is added to the final list (720, right).

[0039] The MaxNumMergeCand capacity is hit by inserting zero motion candidates to fill the remaining entries in the merge candidate list. These candidates have a spatial displacement of zero and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is one for uni-prediction and two for bi-prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

[0040] 1.1.5 Example of motion estimation region for parallel processing

[0041] To speed up the encoding process, motion estimation can be done in parallel, thereby deriving motion vectors for all prediction units in a given region simultaneously. Derivation of merge candidates from spatial neighborhoods can interfere with parallel processing, since one prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is completed. To mitigate the tradeoff between coding efficiency and processing latency, a motion estimation region (MER) can be specified. The "log2_parallel_merge_level_minus2" syntax element may be used to signal the size of the MER in the picture parameter set (PPS). When specifying one MER, merge candidates in the same region are marked as unavailable and therefore not considered in list construction.

[0042] 1.2 Advanced Motion Vector Prediction (AMVP) Implementation

[0043] AMVP exploits the spatial-temporal correlation of motion vectors with neighboring PUs, which is used for unambiguous transmission of motion parameters. First, it builds a motion vector candidate list by checking the availability of the left and upper temporal neighboring PU positions, removing redundant candidates, and adding zero vectors to keep the length of the candidate list constant. Then, the encoder can select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the merge index signaling, the index of the best motion vector candidate is encoded using a shortened unary term. The maximum value to be coded in this case is 2 (see Figure 8). The following sections will explain the details of the motion vector prediction candidate derivation process.

[0044] 1.2.1 Example of constructing motion vector prediction candidates

[0045] FIG. 8 summarizes the process of deriving motion vector prediction candidates, which may be implemented for each reference picture list with refidx as input.

[0046] In motion vector prediction, two types of motion vector candidates can be considered: spatial motion vector candidates and temporal motion vector candidates. To derive spatial motion vector candidates, as shown in Figure 2 above, two motion vector candidates are ultimately derived based on the motion vectors of each PU at five different positions.

[0047] To derive a temporal motion vector candidate, select one motion vector candidate from two candidates derived based on two different co-located positions. After creating a first spatio-temporal candidate list, remove duplicate motion vector candidates in the list. If the number of candidates is greater than two, remove motion vector candidates whose reference picture index in the associated reference picture list is greater than 1 from the list. If the number of spatio-temporal motion vector candidates is less than two, add an additional zero motion vector candidate to the list.

[0048] 1.2.2 Construction of spatial motion vector candidates

[0049] In deriving spatial motion vector candidates, we consider up to two candidates among five potential candidates derived from PUs located as shown in Fig. 2 above, which are located at the same position as the motion merge. The order of derivation for the left side of the current PU is specified as A0, A1, scaled A0, scaled A1. The order of derivation for the top side of the current PU is specified as B0, B1, B2, scaled B0, scaled B1, scaled B2. So, for each side, there are four possible cases for motion vector candidates: two cases where spatial scaling does not need to be used and two cases where spatial scaling is used. The four different cases can be summarized as follows:

[0050] -No spatial scaling (1) The same reference picture list and the same reference picture index (same POC) (2) Different reference picture lists, but the same reference picture (same POC) -Spatial Scaling (3) Same reference picture list, but different reference pictures (different POC) (4) Different reference picture lists and different reference pictures (different POCs)

[0051] First, check the case of non-spatial scaling, then check the case of enabling spatial scaling. Regardless of the reference picture list, consider spatial scaling if the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU. If all PUs of the left candidate are not available or are intra-coded, scaling of the top motion vector is useful for parallel derivation of the left and top MV candidates. Otherwise, spatial scaling is not allowed for the top motion vector.

[0052] As shown in the example of Figure 9, in the case of spatial scaling, the motion vectors of neighboring PUs are scaled in the same manner as in temporal scaling. One difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as in temporal scaling.

[0053] 1.2.3 Construction of Temporal Motion Vector Candidates

[0054] Other than deriving the reference picture index, all the processes for deriving the temporal merge candidates are the same as the processes for deriving the spatial motion vector candidates (shown in the example of FIG. 6). In some embodiments, the reference picture index is signaled to the decoder.

[0055] 2. Example of Inter-Prediction Method in Joint Exploration Model (JEM)

[0056] In some embodiments, future video coding techniques are explored using a reference software known as the Joint Exploration Model (JEM) [3][4]. In JEM, sub-block based prediction is applied in several coding tools, such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), bidirectional optical flow (BIO), frame rate up-conversion (FRUC), locally adaptive motion vector resolution (LAMVR), overlapping block motion compensation (OBMC), local illumination compensation (LIC), and decoder-side motion vector refinement (DMVR).

[0057] 2.1 Example of motion vector prediction based on sub-CU

[0058] In the quadtree plus binary tree JEM (QTBT), each CU can have at most one set of motion parameters for each prediction direction. In some embodiments, the encoder considers two sub-CU level motion vector prediction methods by splitting the large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU. The alternative temporal motion vector prediction (ATMVP) method allows each CU to retrieve multiple sets of motion information from multiple blocks smaller than the current CU in the aligned reference picture. In the spatio-temporal motion vector prediction (STMVP) method, the temporal motion vector predictor and spatial neighbor motion vectors are used to recursively derive the motion vectors of the sub-CUs. In some embodiments, the motion compression of the reference frame may be disabled to maintain a more accurate motion field for sub-CU motion prediction.

[0059] 2.1.1 Alternative Temporal Motion Vector Prediction (ATMVP) Example

[0060] In the ATMVP method, the temporal motion vector prediction (TMVP) method is modified by retrieving multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU.

[0061] 10 shows an example of ATMVP motion prediction processing in a CU 1000. The ATMVP method predicts a motion vector of a sub-CU 1001 in a CU 1000 in two steps. In the first step, a corresponding block 1051 in a reference picture 1050 is identified by a temporal vector. The reference picture 1050 is also called a motion source picture. In the second step, the current CU 1000 is divided into sub-CUs 1001, and the motion vector and reference index of each sub-CU are obtained from the block corresponding to each sub-CU.

[0062] In the first step, the reference picture 1050 and corresponding blocks are determined according to the motion information of the spatially neighboring blocks of the current CU 1000. The first merging candidate in the merging candidate list of the current CU 1000 is used to avoid the repeated scanning process of the neighboring blocks. The first available motion vector and its associated reference index are set to the temporal vector and the index of the motion source picture. In this way, the corresponding blocks can be identified more accurately compared to TMVP, and the corresponding blocks (sometimes called aligned blocks) are always in the bottom-right or center position with respect to the current CU.

[0063] In a second step, the corresponding block of the sub-CU 1051 is identified by the temporal vector in the motion source picture 1050 by adding the temporal vector to the coordinate of the current CU. For each sub-CU, the motion information of its corresponding block (e.g., the smallest motion grid covering the center sample) is used to derive the motion information of the sub-CU. After identifying the motion information of the corresponding N×N block, it is converted into the motion vector and reference index of the current sub-CU, and motion scaling and other procedures are applied, similar to TMVP of HEVC. For example, the decoder checks whether a low-delay condition (e.g., the POC of all reference pictures of the current picture is smaller than the POC of the current picture) is met, and possibly predicts the motion vector MVy (e.g., X is equal to 0 or 1, and Y is equal to 1-X) of each sub-CU using the motion vector MVx (e.g., the motion vector corresponding to the reference picture list X).

[0064] 2.1.2 Spatial-Temporal Motion Vector Prediction (STMVP) Example

[0065] In the STMVP method, the motion vectors of sub-CUs are derived recursively along the raster scan order. Figure 11 shows an example of one CU with four sub-blocks and neighboring blocks. Consider an 8x8 CU 1100, which includes four 4x4 sub-CUs, A (1101), B (1102), C (1103), and D (1104). The neighboring 4x4 blocks in the current frame are labeled a (1111), b (1112), c (1113), and d (1114).

[0066] The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU, A 1101 (block c 1113). If this block c (1113) is not available or is intra-coded, check the other N×N blocks above sub-CU A (1101) (starting from block c 1113 and going from left to right). The second neighbor is the block to the left of sub-CU A 1101 (block b 1112). If block b (1112) is not available or is intra-coded, check the other blocks to the left of sub-CU A 1101 (starting from block b 1112 and going from top to bottom). The motion information obtained from the neighboring blocks in each list is scaled to the first reference frame of a given list. Then, derive the temporal motion vector predictor (TMVP) of sub-block A 1101 following the same procedure as the TMVP derivation specified in HEVC. The motion information of the ordered blocks in block D1104 is fetched and scaled accordingly. Finally, after retrieving and scaling the motion information, average all available motion vectors for each reference list separately. This averaged motion vector is the motion vector of the current sub-CU.

[0067] 2.1.3 Example of Sub-CU Motion Prediction Mode Signaling

[0068] In some embodiments, sub-CU modes are enabled as additional merge candidates, and no additional syntax elements are required to signal the mode. Add two additional merge candidates to the merge candidate list of each CU to represent ATMVP and STMVP modes. In other embodiments, up to seven merge candidates may be used if the sequence parameter set indicates that ATMVP and STMVP are enabled. The encoding logic of the additional merge candidates is the same as that of the merge candidates in HM, that is, for each CU in a P or B slice, two or more RD checks may be required for the two additional merge candidates. In some embodiments, such as JEM, all bins of the merge index are context coded by context-based adaptive binary arithmetic coding (CABAC). In other embodiments, such as HEVC, only the first bin is context coded, and the remaining bins are context bypass coded.

[0069] 2.2 Example of Adaptive Motion Vector Difference Resolution

[0070] In an embodiment of the present invention, when use_integer_mv_flag is 0 in the slice header, the motion vector difference (MVD) (between the motion vector and the predicted motion vector of the PU) is signaled in units of quarter luma samples. In JEM, locally adaptive motion vector resolution (LAMVR) is introduced. In JEM, MVD can be coded in units of quarter luma samples, integer luma samples or four luma samples. MVD resolution is controlled at coding unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU that has at least one non-zero MVD module.

[0071] For a CU with at least one non-zero MVD module, a first flag is signaled to indicate whether ¼ luma sample MV precision is used in the CU. If the first flag (equal to 1) indicates that ¼ luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or 4 luma sample MV precision is used.

[0072] If the first MVD resolution flag of a CU is zero or not coded for the CU (i.e., all MVDs in the CU are zero), then 1 / 4 luma sample MV resolution is used for the CU. If the CU uses integer luma sample MV precision or 4 luma sample MV precision, round the MVP in the CU's AMVP candidate list to the corresponding precision.

[0073] In the encoder, the CU-level RD check is used to determine which MVD resolution to use for the CU. That is, the CU-level RD check is performed three times for each MVD resolution. To increase the speed of the encoder, the following coding method is applied in JEM.

[0074] --During RD check of a CU with normal 1 / 4 luma sample MVD resolution, the motion information (integer luma sample precision) of the current CU is stored. During RD check of the same CU with integer luma sample and 4 luma sample MVD resolution, the stored motion information (after rounding) is used as the starting point for further small range motion vector refinement, so that the time-consuming motion estimation process is not duplicated three times.

[0075] --Conditionally invoke RD check for CUs with 4 luma sample MVD resolution. For a CU, if the RD cost of the integer luma sample MVD resolution is much larger than that of the quarter luma sample MVD resolution, the RD check of the 4 luma sample MVD resolution for the CU is skipped.

[0076] 2.3 Example of improving the accuracy of motion vector storage

[0077] In HEVC, the precision of motion vectors is 1 / 4 pixel (1 / 4 luma sample and 1 / 8 chroma sample for 4:2:0 video). In JEM, the precision of internal motion vector storage and merge candidates is increased to 1 / 16 pixel. The higher motion vector precision (1 / 16 pixel) is used for motion compensated inter prediction for CUs coded in skip / merge mode. For CUs coded in regular AMVP mode, either integer pixel or quarter pixel motion is used.

[0078] The SHVC upsampling interpolation filter, which has the same filter length and normalization coefficient as the HEVC motion compensation interpolation filter, is used as the motion compensation interpolation filter for the additional fractional pixel positions. In JEM, the accuracy of the motion vector of the chroma component is 1 / 32 samples, and the average of the two neighboring 1 / 16th pixel fractional position filters is used to derive the additional 1 / 32th pixel fractional position interpolation filter.

[0079] 2.4 Example of Overlapped Block Motion Compensation (OBMC)

[0080] In JEM, OBMC can be switched on / off using CU-level syntax. When using OBMC in JEM, OBMC is performed for all motion compensation (MC) block boundaries except the bottom-right boundary of the CU. It also applies to both luma and chroma components. In JEM, an MC block corresponds to a coding block. If the CU is coded in a sub-CU mode (including sub-CU merge, affine, and FRUC modes), each sub-block of the CU is one MC block. In order to handle CU boundaries uniformly, OBMC is performed at the sub-block level for all MC block boundaries, where the sub-block size is set equal to 4×4, as shown in Figures 12A and 12B.

[0081] Figure 12A shows the sub-blocks at the CU / PU boundary, where the shaded sub-blocks are where OBMC is applied. Similarly, Figure 12B shows the sub-PUs in ATMVP mode.

[0082] When OBMC is applied to the current subblock, in addition to the current motion vector, the motion vectors of the four connected neighboring subblocks are also available and used to derive a prediction block for the current subblock if they are not the same as the current motion vector. These multiple prediction blocks based on multiple motion vectors are combined to generate a final prediction signal for the current subblock.

[0083] Let PN (N is the index of the neighboring subblocks above, below, left, and right) be the prediction block based on the motion vector of the neighboring subblock, and PC be the prediction block based on the motion vector of the current subblock. If PN is based on the motion information of the neighboring subblocks that contain the same motion information as the current subblock, OBMC is not performed from PN. Otherwise, add all samples of PN to the same samples in PC, i.e., add four rows / columns of PN to PC. Use weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} for PN and weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} for PC. The exception is small MC blocks (i.e., the height or width of the coding block is equal to 4 or one CU is coded in sub-CU mode), in which case only two rows / columns of PN are added to PC. In this case, weighting factors {1 / 4, 1 / 8} are used for PN and weighting factors {3 / 4, 7 / 8} are used for PC. For a PN generated based on the motion vector of a neighboring subblock in the vertical (horizontal) direction, a sample in the same row (column) of the PN is added to the PC with the same weighting coefficient.

[0084] In JEM, for CUs whose size is 256 luma samples or less, a CU level flag is signaled to indicate whether OBMC is applied to the current CU. For CUs whose size is larger than 256 luma samples or not coded in AMVP mode, OBMC is applied by default. In the encoder, if OBMC is applied to a CU, its influence is considered during the motion estimation stage. The prediction signal formed by OBMC using the motion information of the upper neighboring block and the left neighboring block is used to compensate the upper and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0085] 2.5 Local Illumination Compensation (LIC) Example

[0086] LIC is based on a linear model of illumination changes, with a scale factor a and an offset b, and is adaptively enabled or disabled for each inter-mode coding unit (CU).

[0087] When LIC is applied to a CU, a least square error method is used to derive parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. Figure 13 shows an example of the neighboring samples used to derive the parameters of the IC algorithm. Specifically, as shown in Figure 13, the subsampled (2:1 subsampled) neighboring samples of the CU and the corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used. The IC parameters are derived and applied separately for each prediction direction.

[0088] If a CU is coded in merge mode, it copies the LIC flag from the neighboring block, similar to copying motion information in merge mode, otherwise it signals the LIC flag to the CU to indicate whether LIC is applied or not.

[0089] When LIC is enabled for a picture, an additional CU-level RD check is required to determine whether LIC applies for a CU. If LIC is enabled for a CU, the mean removed sum of absolute differences (MR-SAD) and mean removed sum of absolute Hadamard transform differences (MR-SATD) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively.

[0090] In order to reduce the coding complexity, the following coding scheme is applied in JEM:

[0091] --If there is no clear illumination change between the current picture and its reference pictures, LIC is disabled for the entire picture. To identify this situation, in the encoder, we calculate the histograms of the current picture and all of its reference pictures. If the histogram difference between the current picture and all of its reference pictures is less than a given threshold, we disable LIC for the current picture, otherwise we enable LIC for the current picture.

[0092] 2.6 Example of Affine Motion Compensation Prediction

[0093] In HEVC, only a translational motion model is applied for motion compensated prediction (MCP). However, the camera and the object may have various kinds of motion, such as zoom in / out, rotation, perspective motion, and / or other irregular motion. On the other hand, JEM applies a simplified affine transformation motion compensated prediction. Figure 14 shows an example of an affine motion field of a block 1400 described by two control point motion vectors V0 and V1. The motion vector field (MVF) of the block 1400 can be expressed as the following equation:

[0094]

number

[0095] As shown in Figure 14, (v 0x ,v 0y ) is the motion vector of the control point in the upper left corner, and (v 1x ,v 1y ) is the motion vector of the control point in the upper right corner. To simplify the motion compensation prediction, sub-block based affine transformation prediction can be applied. The size of the sub-block M×N is derived as follows:

[0096]

number

[0097] where MvPre is the fractional precision of the motion vector (e.g., 1 / 16 in JEM). 2x ,v 2y ) is the motion vector of the bottom-left control point calculated according to equation (1). If necessary, M and N can be adjusted downwards to be divisors of w and h, respectively.

[0098] FIG. 15 shows an example of per-subblock affine MVF for block 1500. To derive motion vectors for each M×N subblock, the motion vector of the center sample of each subblock is calculated according to equation (1) and rounded to the fractional accuracy of the motion vector (e.g., 1 / 16 for JEM). A motion compensation interpolation filter can then be applied to generate a prediction for each subblock using the derived motion vector. After MCP, the high-precision motion vector for each subblock is rounded and stored with the same precision as the normal motion vector.

[0099] 2.6.1 AF_INTER mode implementation

[0100] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs with both width and height greater than 8, AF_INTER mode can be applied. A CU-level affine flag is signaled in the bitstream to indicate whether AF_INTER mode is used. In AF_INTER mode, the motion vector pair {(v0,v1)|v0={v A ,v B ,v C},v1={v D ,v E}} to build a candidate list.

[0101] FIG. 16 shows an example of motion vector prediction (MVP) for block 1600 in AF_INTER mode. As shown in FIG. 16, v0 is selected from the motion vectors of subblocks A, B, or C. The motion vectors from the neighboring blocks can be scaled according to the reference list. The motion vectors may also be scaled based on the relationship between the picture order count (PIC) of the references of the neighboring blocks and the POC of the current CU. The method of selecting v1 from the neighboring subblocks D and E is similar. If the number of candidate lists is less than two, fill the list with motion vector pairs that duplicate each of the AMVP candidates. If the candidate list is greater than two, first sort the candidates based on the neighboring motion vectors (e.g., based on the similarity of the two motion vectors in the pair candidate). In some implementations, keep the first two candidates. In some embodiments, use a distortion ratio (RD) cost check to determine which motion vector pair candidate to select as the control point motion vector prediction (CPMVP) of the current CU. An index indicating the position of the CPMVP in the candidate list can be signaled in the bitstream. After determining the CPMVP of the current affine CU, affine motion estimation is applied to obtain a control point motion vector (CPMV). Then, the difference between the CPMV and the CPMVP is signaled in the bitstream.

[0102] In AF_INTER mode, if a 4 / 6 parameter affine mode is used, 2 / 3 control points are needed, and therefore it is necessary to code 2 / 3 MVDs for these control points, as shown in Figure 17. In an existing implementation [5], the MVs may be derived, for example, by predicting mvd1 and mvd2 from mvd0, as follows:

[0103] mv0 = m ̄v ̄0 + mvd0 mv1 = mv1 + mvd1 + mvd0 mv2 = m̄v̄2 + mvd2 + mvd0

[0104] Here, m̂v̂ i , mvd i , mv1 are the predicted motion vector, the motion vector difference, and the motion vector of the top left pixel (i=0), the top right pixel (i=1), and the bottom left pixel (i=2), respectively, as shown in FIG. 18B. In some embodiments, the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equivalent to summing the two components separately. For example, newMV=mvA+mvB means setting the two components of newMV to (xA+xB) and (yA+yB), respectively.

[0105] 2.6.2 Example of the fast affine ME algorithm in AF_INTER mode

[0106] In some embodiments of the affine mode, the MVs of two or three control points need to be determined jointly. Directly searching multiple MVs jointly is computationally complex. In one example, a fast affine ME algorithm [6] is proposed and adopted for VTM / BMS.

[0107] For example, we describe a fast affine ME algorithm for a four-parameter affine model, and the ideas can be extended to a six-parameter affine model.

[0108]

number

[0109]

number

[0110] By replacing (a-1) with a', the motion vector can be rewritten as follows:

[0111]

number

[0112] Assuming the motion vectors of two control points (0,0) and (0,w) are known, From equation (5), the affine parameters may be derived as follows:

[0113]

number

[0114] The motion vector can be rewritten in vector form as follows:

[0115]

number

[0116] In some embodiments, at the encoder, the MVD of AF_INTER can be derived iteratively. i Let (P) be the MV derived at the ith iteration of position P, and dMV C i MV at the ith iteration C Then, in the (i+1)th iteration, Music Video i+1 (P)=A(P)*((MV c i ) T+(dMV c i ) T ) =A(P)*(MV c i ) T +A(P)*(dMV c i ) T =MV i (P)+A(P)*(dMV c i ) T .Equation (10)

[0117] Pic ref Let Pic be the reference picture. cur Let be the current picture, and Q = P + MV i (P). When using MSE as the matching criterion, the function that needs to be minimized may be expressed as:

[0118] minΣ P (Pic cur (P)-Pic ref (P+MV i+1 (P))) 2 =minΣ P (Pic cur (P)-Pic ref (Q+A(P)*(dMV C i ) T )) 2 Formula (11)

[0119] (dMV C i ) T Assuming that is sufficiently small, Pic ref (Q+A(P)*(dMV C i ) T ) may be rewritten as an approximation based on a first-order Taylor expansion as follows:

[0120] Pic ref (Q+A(P)*(dMV c i ) T )≒Picref (Q)+Pic ref '(Q)*A(P)*(dMV c i ) T Formula (12)

[0121] Here, Pic ref '(Q)=[dPic ref (Q) / dx dPic ref (Q) / dy]. E i+1 (P)=Pic cur (P)-Pic ref If we adopt the notation (Q), it becomes as follows.

[0122] minΣ P (Pic cur (P)-Pic ref (Q)-Pic ref '(Q)*A(P)*(dMV C i ) T ) 2 =minΣ P (E i+1 (P)-Pic ref '(Q)*A(P)*(dMV C i ) T ) 2 Formula (13)

[0123] Set the derivative of the error function to zero, then A(P)*(dMV C i ) T Based on this, the delta MV of the controls (0,0) and (0,w) is calculated as follows: C i can be derived.

[0124] dMV (0,0) h =dMV C i [0] Formula (14)

[0125] dMV (0,w) h =dMV C i[1]*w+dMV C i [2] Equation (15)

[0126] dMV (0,0) v =dMV C i [2] Equation (16)

[0127] dMV (0,w) v =-dMV C i [3]*w+dMV C i [2] Equation (17)

[0128] In some embodiments, this MVD derivation process can be repeated n times and the final MVD calculated as follows:

[0129] fdMV (0,0) h =Σ n-1 i=0 dMV C i [0] Formula (18)

[0130] fdMV (0,w) h =Σ n-1 i=0 dMV C i [1]*w+Σ n-1 i=0 dMV C i [0] Formula (19)

[0131] fdMV (0,0) v =Σ n-1 i=0 dMV C i [2] Formula (20)

[0132] fdMV (0,w) v =Σ n-1 i=0 -dMV C i [3]*w+Σ n-1i=0 dMV C i [2] Formula (21)

[0133] In the above implementation [5], predicting the delta MV of the control point (0,w) represented by mvd1 from the delta MV of the control point (0,0) represented by mvd0 is possible only for mvd1 (Σ n-1 i=0 dMV C i [1]*w,-Σ n-1 i=0 -dMV C i [3]*w) will be encoded.

[0134] 2.6.3 Implementation of AF_MERGE Mode

[0135] When applying a CU in AF_MERGE mode, the CU obtains the first block coded in affine mode from the valid neighboring reconstructed blocks. FIG. 18A shows an example of a selection order of candidate blocks for a current CU 1800. As shown in FIG. 18A, the selection order can be the left (1801), top (1802), right (1803), bottom left (1804), and top left (1805) of the current CU 1800. FIG. 18B shows another example of candidate blocks for the current CU 1800 in AF_MERGE mode. As shown in FIG. 18B, when the bottom-left neighbor block 1801 is coded in affine mode, motion vectors v2, v3, and v4 of the top left corner, top right corner, and bottom left corner of the CU including the sub-block 1801 are derived. Based on v2, v3, and v4, the motion vector v0 of the top left corner in the current CU 1800 is calculated. Thus, the motion vector v1 of the top right of the current CU can be calculated.

[0136] After calculating the CPMV of the current CU v0, v1 according to the affine motion model in equation (1), the MVF of the current CU can be generated. To identify whether the current CU is coded in AF_MERGE mode, an affine flag can be signaled in the bitstream if there is at least one neighboring block coded in affine mode.

[0137] 2.7 Pattern Matching Motion Vector Derivation (PMMVD) Example

[0138] The PMMVD mode is a special merging mode based on the Frame Rate Up-Conversion (FRUC) method, in which the motion information of the blocks is not signaled but is derived at the decoder side.

[0139] The FRUC flag may be signaled to the CU if its merge flag is true. If the FRUC flag is false, a merge index may be signaled and normal merge mode is used. If the FRUC flag is true, an additional FRUC mode flag may be signaled to indicate which method (e.g., bilateral matching or template matching) is used to derive motion information for the block.

[0140] On the encoder side, the decision to use the FRUC merging mode for a CU is based on the RD cost selection, just as it is done for regular merging candidates. For example, we use the RD cost selection to check multiple matching modes (e.g., bilateral matching and template matching) for one CU. The one that leads to the minimum cost is compared with the other CU modes. If the FRUC matching mode is the most efficient one, we set the FRUC flag to true for the CU and use the associated matching mode.

[0141] Generally, the motion derivation process in FRUC merge mode has two steps: first, CU-level motion search, and then sub-CU-level motion refinement. At the CU level, an initial motion vector for the entire CU is derived based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate that leads to the minimum matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching is performed near the starting point. The MV result at the minimum matching cost is taken as the MV for the entire CU. Then, the derived CU motion vector is used as the starting point to further refine the motion information at the sub-CU level.

[0142] For example, to derive W×H CU motion information, the following derivation process is performed: In the first stage, an MV for the entire W×H CU is derived. In the second stage, the CU is further divided into M×M sub-CUs. The value of M is calculated as in Equation (3), where D is a predefined division depth, which is set to 3 by default in JEM. Then, an MV for each sub-CU is derived.

[0143]

number

[0144] FIG. 19 shows an example of bilateral matching used in the frame rate up-conversion (FRUC) method. This bilateral matching is used to derive the motion information of the current CU (1900) by finding the closest match between two blocks along the motion trajectory of the current CU (1900) in two different reference pictures (1910, 1911). Assuming a continuous motion trajectory, the motion vectors MV0 (1901), MV1 (1902) pointing to the two reference blocks are proportional to the temporal distance between the current picture and the two reference pictures, e.g., TD0 (1903), TD1 (1904). In some embodiments, when the current picture 1900 is between the two reference pictures (1910, 1911) and the temporal distance from the current picture to the two reference pictures is the same, the bilateral matching becomes a bidirectional MV based on the mirror.

[0145] FIG. 20 shows an example of template matching used in the Frame Rate Up Conversion (FRUC) method. Using template matching, motion information of the current CU 2000 can be derived by finding the closest match between a template in the current picture (e.g., a block in the upper and / or left neighbor of the current CU) and a block in the reference picture 2010 (e.g., the same size as the template). Besides the FRUC merge mode mentioned above, template matching can also be applied to the AMVP mode. In both JEM and HEVC, AMVP has two candidates. A new candidate can be derived by using the template matching method. If the newly derived candidate by template matching is different from the first existing AMVP candidate, it is inserted at the beginning of the AMVP candidate list, and then the list size is set to 2 (e.g., by removing the second existing AMVP candidate). When applied to the AMVP mode, only the CU level search is applied.

[0146] The CU-level MV candidate set may include: (1) the original AMVP candidates if the current CU is in AMVP mode, (2) all merge candidates, (3) multiple MVs in the interpolated MV field (described below), and the motion vectors of the top-left neighbor.

[0147] When using bilateral matching, each valid MV of the merge candidate can be used as input to generate MV pairs assuming bilateral matching. For example, one valid MV of the merge candidate is (MVa,ref a ) and its corresponding bilateral MV reference picture ref b is found in another reference list B, ref a and ref b are on different sides of the current picture in time. b If not available, ref b ref a The ref is determined as a different reference from the current picture, and its temporal distance from the current picture is the smallest value in list B. b After determining the current picture and ref a ,ref b MVb is derived by scaling MVa based on the time distance between

[0148] In some implementations, four MVs from the interpolated MV field may be added to the CU level candidate list. Specifically, the interpolated MVs at positions (0,0), (W / 2,0), (0,H / 2), and (W / 2,H / 2) of the current CU are added. When applying FRUC in AMVP mode, the original AMVP candidates are also added to the CU level MV candidate set. In some implementations, at the CU level, 15 MVs for the AMVP CU and 13 MVs for the merged CU can be added to the candidate list.

[0149] The sub-CU level MV candidate set includes the MV determined by the CU level search, (2) the MVs of the top, left, top-left, and top-right neighbors, (3) the scaled version of the aligned MV from the reference picture, (4) one or more (e.g., up to four) ATMVP candidates, and (5) one or more (e.g., up to four) STMVP candidates. The scaled MV from the reference picture is derived as follows: Traverse the reference pictures in both lists. The MV at the aligned position of the sub-CU in the reference picture is scaled with respect to the reference of the starting CU level MV. The candidates for ATMVP and STMVP may be the first four candidates. At the sub-CU level, one or more (e.g., up to 17) MVs are added to the candidate list.

[0150] Generate Interpolated MV Fields

[0151] Before coding a frame, an interpolated motion field is generated for the whole picture based on one-sided ME, and the motion field may be used as a CU-level or sub-CU-level MV candidate later.

[0152] In some embodiments, the motion field of each reference picture in both reference lists is traversed at the 4x4 block level. Figure 21 shows an example of unilateral motion estimation (ME) 2100 in the FRUC method. For each 4x4 block, if the motion related to the block passing through the 4x4 block of the current picture is not yet assigned an interpolated motion, the motion of the reference block is scaled to the current picture based on the temporal distances TD0 and TD1 (similar to the MV scaling of TMVP in HEVC), and the scaled motion is assigned to the block of the current frame. If the 4x4 block is not assigned a scaled MV, the motion of the block is marked as unavailable in the interpolated motion field.

[0153] Interpolation and Matching Costs

[0154] When one motion vector points to one fractional sample position, motion compensated interpolation is necessary. To reduce complexity, bilinear interpolation can be used for both bilateral and template matching instead of the usual 8-tap HEVC interpolation.

[0155] The calculation of the matching cost is slightly different in different steps. When selecting a candidate from the CU-level candidate set, the matching cost may be the sum of absolute differences (SAD) of bilateral matching or template matching. After determining the starting MV, the matching cost C of bilateral matching in the sub-CU level search is calculated as follows:

[0156] C = SAD + w (|MV x -MV x s |+|MV y -MV y s |) Formula (4)

[0157] where w is a weighting factor. In some embodiments, w may be empirically set to 4. MV and MV S denote the current MV and the start MV, respectively. The SAD may still be used as the matching cost for template matching in the sub-CU level search.

[0158] In FRUC mode, the MV is derived by using only luma samples. The derived motion is used for both luma and chroma for MC inter prediction. After determining the MV, the final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0159] MV refinement is a pattern-based MV search with the criteria of bilateral matching cost or template matching cost. JEM supports two search patterns, namely Unlimited Center Biased Diamond Search (UCBDS) and Adaptive Cross Search for MV refinement at CU level and sub-CU level, respectively. For both CU and sub-CU level MV refinement, MV is directly searched with the accuracy of 1 / 4 luma sample MV, followed by 1 / 8 luma sample MV refinement. The search range of MV refinement for CU and sub-CU steps is set equal to 8 luma samples.

[0160] In the bilateral matching merge mode, bidirectional prediction is applied because the motion information of the CU is derived based on the closest matching between two blocks along the motion trajectory of the current CU in two different reference pictures. In the template matching merge mode, the encoder can select from single prediction from list0, single prediction from list1, or bidirectional prediction for the CU. The selection can be made based on the template matching cost as follows:

[0161] If costBi<=factor*min(cost0,cost1) Bidirectional prediction is used. Otherwise, if cost0<=cost1 Use a single prediction from list0. If not, Use a single prediction from list1.

[0162] Here, cost0 is the SAD of list0 template matching, cost1 is the SAD of list1 template matching, and costBi is the SAD of bidirectional prediction template matching. For example, a factor value of 1.25 means that the selection process is biased toward bidirectional prediction. This inter-prediction direction selection can be applied to the template matching process at the CU level.

[0163] 2.8 Bidirectional Optical Flow (BIO) Example

[0164] The Bi-directional Optical Flow (BIO) method is a sample-by-sample motion refinement performed on top of block-by-block motion compensation for bi-prediction. In some embodiments, the sample-level motion refinement does not use signaling.

[0165] After block motion compensation, the luminance from reference k (k=0,1) is expressed as I (k) , and ∂I (k) / ∂x and ∂I (k) / ∂y, respectively, I (k) Let the horizontal and vertical components of the gradient be the motion vector field (v x ,v y ) is given by:

[0166]

number

[0167] Combining this optical flow equation with Hermite interpolation of the motion trajectory of each specimen, we obtain the function value I (k) and the derivative ∂I (k) / ∂x,∂I (k) We obtain a unique 3rd order polynomial that fits both ends of / ∂y. The value of this polynomial at t=0 is the BIO prediction.

[0168]

number

[0169] FIG. 22 shows an example of optical flow trajectory in the bidirectional optical flow (BIO) method. Here, τ0 and τ1 indicate the distance to the reference frame. The distances τ0 and τ1 are calculated based on the POCs of Ref0 and Ref1 as follows: τ0=POC(current)-POC(Ref0), τ1=POC(Ref1)-POC(current). If both predictions come from the same time direction (either both from the past or or both come from the future) or have different signs (e.g. τ0·τ1<0). In this case, BIO is applied if the predictions are not from the same temporal moment (e.g. τ0 ≠ τ1). Both reference regions have non-zero motion (e.g. MVx0,MVy0,MVx1,MVy1 ≠ 0) and the block motion vectors are proportional to the temporal distance (e.g. MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0170] The motion vector field (v x ,v y 9A and 9B show examples of the intersection of a motion trajectory with a reference frame plane. The model uses only the first linear term of the local Taylor expansion for Δ.

[0171]

number

[0172] All values ​​in the above equation depend on the sample position, denoted as (i',j'). Assuming that the motion is consistent in the local surrounding area, Δ can be minimized inside a (2M+1)×(2M+1) square window Ω centered on the current prediction point (i,j), where M is equal to 2.

[0173]

number

[0174] For this optimization problem, JEM uses a simple approach that first minimizes vertically and then horizontally, resulting in:

[0175]

number

[0176] To avoid division by zero or very small values, regularization parameters r and m can be introduced into equations (9) and (10).

[0177] r=500·4 d-8 Formula (12)

[0178] r=700·4 d-8 Formula (13)

[0179] where d is the bit depth of the video samples.

[0180] To make the memory access of BIO the same as that of normal bipredictive motion compensation, all the prediction values ​​and gradient values ​​I (k) , ∂I (k) / ∂x,∂I (k) y is calculated for a position within the current block. FIG. 23A shows an example of access positions outside the block 2300. As shown in FIG. 23A, in equation (9), (2M+1)×(2M+1) square windows centered on the current prediction point on the boundary of the prediction block need to access positions outside the block. In JEM, I (k) , ∂I (k) / ∂x,∂I (k) The value of / ∂y is set equal to the closest available value inside the block. For example, this can be implemented as a padding region 2301, as shown in Figure 23B.

[0181] Using BIO, we can refine the motion field on a sample-by-sample basis. To reduce the computational complexity, a block-based BIO design is used in JEM. The motion refinement can be calculated based on a 4×4 block. In block-based BIO, we use s in equation (9) for all samples in the 4×4 block. n The value of s is integrated and then n The integrated value of in is used to derive the BIO motion vector offset for the 4x4 block. Specifically, the following equation can be used for block-based BIO derivation:

[0182]

number

[0183] where b k In Equations (9) and (10), s denotes the set of samples belonging to the k-th 4×4 block of the prediction block. n ((s n , bk )>>4) and derive the associated motion vector offset.

[0184] In some scenarios, the MV regimen of BIO may be unreliable due to noise and irregular motion. Therefore, in BIO, the magnitude of the MV regimen is clipped to a threshold value. The threshold value is determined based on whether the reference pictures of the current picture are all from one direction or not. If the reference pictures of the current picture are all from one direction, the value of the threshold value is 12 × 2 14-d otherwise, 12 × 2 13-d is set to.

[0185] The gradients of BIOs may be calculated simultaneously with the motion compensation interpolation using arithmetic compatible with the HEVC motion compensation process (e.g., 2D separable finite impulse response (FIR)). In some embodiments, the input for the 2D separable FIR is the same reference frame sample as for the motion compensation process and fractional positions (fracX, fracY) based on the fractional part of the block motion vector. For the horizontal gradient ∂I / ∂x, the signal is first interpolated vertically using BIOfilterS corresponding to fractional position fracY with a scaling shift of d-8. Then, the gradient filter BIOfilterG is applied horizontally corresponding to fractional position fracX and the descaling is shifted by 18-d. For the vertical gradient ∂I / ∂y, the signal is gradient filtered vertically using BIOfilterG corresponding to fractional position fracY with a scaling shift of d-8. Then, BIOFilterS is used to perform a horizontal signal displacement corresponding to the fractional position fracX and shift the scaling by 18-d. To maintain reasonable complexity, the length of the interpolation filters for the gradient calculation BIOFilterG and the signal displacement BIOFilterF may be short (e.g., 6 taps). Table 1 shows exemplary filters that can be used for gradient calculation of different fractional positions of the block motion vector in BIO. Table 2 shows exemplary interpolation filters that can be used for generating the prediction signal in BIO.

[0186] [Table 1]

[0187] [Table 2]

[0188] In this JEM, BIO can be applied to all bidirectionally predicted blocks when the two predictions are from different reference pictures. BIO can be disabled when CU's Local Lighting Compensation (LIC) is enabled.

[0189] In some embodiments, OBMC is applied to one block after normal MC processing. In order to reduce computational complexity, BIO may not be applied during OBMC processing. That is, BIO is applied in the MC processing of one block when it uses its own MV, and is not applied in the MC processing when it uses the MV of a neighboring block in the OBMC processing.

[0190] 2.9 Decoder-side Motion Vector Refinement (DMVR) Example

[0191] In the bidirectional prediction operation, the bidirectionally predicted blocks constructed by using the motion vectors (MVs) in list0 and MVs in list1 respectively to predict one block region are combined to form one prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined by a bilateral template matching process. To obtain the refined MVs without transmitting additional motion information, bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference picture.

[0192] In DMVR, we generate a bilateral template from the first MV0 in list0 and MV1 in list1 as a weighted combination (i.e., average) of two prediction blocks, respectively, as shown in Fig. 24. The template matching operation consists of calculating a cost measure between the generated template and a sample region (near the first prediction block) in the reference picture. For each of the two reference pictures, the MV with the smallest template cost is considered as the updated MV of that list and replaces the original MV. In JEM, we search for nine MV candidates for each list. The nine MV candidates include the original MV and eight surrounding MVs with one luma sample offset with respect to the original MV in either horizontal or vertical direction or both directions. Finally, we use two new MVs, i.e., MV0' and MV1' as shown in Fig. 24, to generate the final bidirectional prediction result. We use the sum of absolute differences (SAD) as the cost measure.

[0193] DMVR is applied to the bi-prediction merge mode between one MV from a past reference picture and one MV from a future reference picture without transmitting additional syntax elements. In JEM, DMVR is not applied when LIC, affine motion, FRUC, or sub-CU merge candidates are enabled for a CU.

[0194] 2.2.9 Symmetric Motion Vector Difference Example

[0195] In [8], we propose Symmetric Motion Vector Difference (SMVD) to code MVD more efficiently.

[0196] First, at the slice level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 are derived as follows.

[0197] Search for the forward reference picture in reference picture list 0 that is closest to the current picture. If found, set RefIdxSymL0 equal to the reference index of the forward picture.

[0198] Search for the backward reference picture in reference picture list 1 that is closest to the current picture. If found, set RefIdxSymL1 equal to the reference index of the backward picture.

[0199] If both forward and backward pictures are found, BiDirPredFlag is set equal to one.

[0200] Otherwise the following applies:

[0201] Search for the backward reference picture in reference picture list 0 that is closest to the current reference picture. If found, set RefIdxSymL0 equal to the reference index of the backward picture.

[0202] Search for the forward reference picture in reference picture list 1 that is closest to the current reference picture. If found, set RefIdxSymL1 equal to the reference index of the forward picture.

[0203] If both a backward and a forward picture are found, BiDirPredFlag is set equal to 1. Otherwise, BiDirPredFlag is set equal to 0.

[0204] Second, at the CU level, if the prediction direction of the CU is bi-predictive and BiDirPredFlag is equal to 1, then we explicitly signal a symmetric mode flag indicating whether to use symmetric mode.

[0205] If this flag is true, then only mvp_l0_flag, mvp_l1_flag, and MVD0 are explicitly signaled. The reference indices are set equal to RefIdxSymL0, RefIdxSymL1 for list 0 and list 1, respectively. MVD1 is simply set to -MVD0. The final motion vectors are given by the following equations:

[0206]

number

[0207] FIG. 27 shows an example of a symmetric mode.

[0208] Table 3 shows the changes in coding unit syntax.

[0209] [Table 3]

[0210] 2.2.11 Symmetric MVD for Affine Bidirectional Predictive Coding

[0211] SMVD for affine modes was proposed in [9].

[0212] 2.3 Context-Adaptive Binary Arithmetic Coding (CABAC)

[0213] 2.3.1 CABAC Design in HEVC

[0214] 2.3.1.1 Context Representation and Initialization in HEVC

[0215] In HEVC, for each context variable, we initialize two variables: pStateIdx and valMps.

[0216] From the 8-bit table entry initValue, two 4-bit variables slopeIdx and offsetIdx are derived as follows:

[0217] slopeIdx=initValue>>4 offsetIdx=initValue&15 (34)

[0218] The variables m and n used to initialize the context variables are derived from slopeIdx and offsetIdx as follows: m=slopeIdx*5-45 n=(offsetIdx<<3)-16 (35)

[0219] The two values ​​assigned to pStateIdx and valMps for initialization are derived from the luma quantization parameter of the slice represented by SliceQpY. Given the variables m and n, the initialization is specified as follows:

[0220] preCtxState=Clip3(1,126,((m*Clip3(0,51,SliceQpY))>>4)+n) valMps=(preCtxState<=63)?0:1 (36) pStateIdx=valMps?(preCtxState-64):(63-preCtxState)

[0221] 2.3.1.2 State Transition Process in HEVC

[0222] The input to this process is the current pStateIdx, ctxTable and the decoded values ​​of the context variables binVal and valMps associated with ctxIdx.

[0223] The output of this process is the updated pStateIdx and valMps of the context variables associated with ctxIdx.

[0224] Based on the decoded value binVal, updates to the two variables pStateIdx and valMps associated with ctxIdx are derived as follows (37):

[0225] if(binVal==valMps) pStateIdx=transIdxMps(pStateIdx) else { (37) if(pStateIdx==0)valMps=1□valMps pStateIdx=transIdxLps(pStateIdx) }

[0226] 2.3.2 CABAC Design in VVC

[0227] The context-adaptive binary arithmetic coder (BAC) in VVC has been improved in a way that is different from that in HEVC in terms of the context update process and the arithmetic coder.

[0228] The outline of the recently adopted proposal (JVET-M0473, CE test 5.1.13) is given below.

[0229] [Table 4]

[0230] 2.3.2.1 Context initialization process in VVC

[0231] In VVC, the two values ​​assigned to pStateIdx0 and pStateIdx1 for initialization are derived from SliceQpY. Given variables m and n, the initialization is specified as follows:

[0232] preCtxState=Clip3(0,127,((m*Clip3(0,51,SliceQpY))>>4)+n) pStateIdx0=initStateIdxToState[preCtxState]>>4 pStateIdx1=initStateIdxToState[preCtxState] (38)

[0233] 2.3.2.2 State transition processing in VVC

[0234] The inputs to this process are the current pStateIdx0 and pStateIdx1, and the decoded value binVal.

[0235] The output of this process is the updated pStateIdx0 and pStateIdx1 of the context variables associated with ctxIdx.

[0236] The variables shift0 (corresponding to variable a in the CABAC modification summary in VVCTable 4) and shift1 (corresponding to variable b in the CABAC modification summary in VVCTable 4) are derived from the shiftIdx values ​​associated with ctxTable and ctxInc.

[0237] shift0=(shiftIdx>>2)+2 shift1=(shiftIdx&3)+3+shift0 (39)

[0238] Depending on the decoded value binVal, the updates of the two variables pStateIdx0 and pStateIdx1 associated with ctxIdx are derived as follows:

[0239] pStateIdx0=pStateIdx0□(pStateIdx0>>shift0)+(1023*binVal>>shift0) pStateIdx1=pStateIdx1□(pStateIdx1>>shift1)+(16383*binVal>>shift1) (40)

[0240] 3. Shortcomings of existing implementations

[0241] In some existing implementations, when the MV / MV difference (MVD) can be selected from a set of multiple MV / MVD precisions for an affine coding block, it is still unclear how a more accurate motion vector can be obtained.

[0242] In other existing implementations, MV / MVD precision information also plays an important role in determining the overall coding gain of AMVR applied to affine mode, but whether it achieves this goal remains uncertain.

[0243] 4. Exemplary Method for MV Predictor for Affine Modes Using AMVR

[0244] The embodiments of the disclosed technology overcome the shortcomings of existing implementations, thereby providing video coding with higher coding efficiency. Based on the disclosed technology, deriving and signaling a motion vector predictor for affine mode with adaptive motion vector resolution (AMVR) can improve both existing and future video coding standards, as will be demonstrated in the following examples that illustrate various implementations. The examples of the disclosed technology provided below are intended to illustrate the general concept and should not be construed as limiting. In an example, various features described in these examples can be combined, unless expressly indicated to the contrary.

[0245] In some embodiments, when AMVR is applied, the following examples can be applied to affine or normal modes. These examples assume that precision Prec (i.e., MV has precision 1 / (2^Prec)) is used to code MVD in AF_INTER mode or to code MVD in normal inter mode. Let the motion vector predictor (e.g., inherited from neighboring block MV) and its precision be denoted by MVPred (MVPred X ,MVPred Y ) and PredPrec.

[0246] In the following description, SatShift(x,n) is defined as follows:

[0247]

number

[0248] Shift(x,n) is defined as Shift(x,n)=(x+offset0)>>n. In one example, offset0 and / or offset1 are (1<<n)> >1 or (1<<(n-1)). In another example, offset0 and / or offset1 are set to 0. In another example, offset0=offset1=((1<<n)> >1)-1 or ((1<<(n-1)))-1.

[0249] In the following description, an operation between two motion vectors means applying this operation to both of the two components of the motion vector. For example, MV3=MV1+MV2 is equivalent to MV3x=MV1x+MV2x and MV3y=MV1y+MV2y. Alternatively, this operation may be applied only to the horizontal or vertical components of the two motion vectors.

[0250] Example 1 It is proposed that the final MV precision remains unchanged, ie it is kept the same as the precision of the motion vectors to be stored. (a) In one example, the final MV precision may be set to 1 / 16 pixel or 1 / 8 pixel. (b) In one example, the signaled MVD is first scaled and then added to the MVP to form the final MV for one block.

[0251] Example 2 First, an MVP derived directly from neighboring blocks (e.g., spatial or temporal) or a default MVP can be modified and then added to the signaled MVD to form the final MV for the (current) block. (a) Alternatively, whether and how to apply the MVP correction may be different for different values ​​of Prec. (b) In one example, if Prec is greater than 1 (i.e., the MVD is fractional precision), the precision of the neighboring MVs is not changed and no scaling is performed. (c) In one example, when Prec is equal to 1 (i.e., the MVD has an accuracy of 1 pixel), the MV predictor (i.e., the MV of the neighboring blocks) needs to be scaled, e.g., see Example 4(b) of PCT Application No. PCT / CN2018 / 104723. (d) In one example, if Prec is less than 1 (i.e., the MVD is 4-pixel accurate), the MV predictor (i.e., the MV of neighboring blocks) needs to be scaled, e.g., Example 4(b) of PCT Application No. PCT / CN2018 / 104723.

[0252] Example 3 In one example, if the precision of the MVD signal is the same as the precision of the stored MV, then no scaling is required after reconstructing the affine MV; otherwise, the MV is reconstructed with the precision of the signaled MVD and then scaled to the precision of the stored MV.

[0253] Example 4 In one example, the normal inter mode and the AF_INTER mode may select an implementation form based on the different examples described above.

[0254] Example 5 In one example, a syntax element to indicate the MV / MVD precision for affine mode may be signaled with the following semantics: (a) In one example, syntax elements equal to 0, 1, and 2 indicate MV precision of ¼ pixel, 1 / 16 pixel, and 1 pixel, respectively. (b) Alternatively, in affine mode, syntax elements equal to 0, 1, and 2 indicate MV precision of 1 / 4 pixel, 1 pixel, and 1 / 16 pixel, respectively. (c) Alternatively, in affine mode, syntax elements equal to 0, 1, and 2 indicate MV precision of 1 / 16 pixel, 1 / 4 pixel, and 1 pixel, respectively.

[0255] Example 6 In one example, enabling or disabling AMVR for affine mode may be signaled in the SPS, PPS, VPS, sequence / picture / slice header / tile, etc.

[0256] Example 7 In one example, an indication of the allowed MV / MVD precision may be signaled in the SPS, PPS, VPS, sequence / picture / slice header / tile, etc. (a) An indication of the selected MVD precision may be signaled for each coding tree unit (CTU) and / or region. (b) The set of allowed MV / MVD precisions may depend on the coding mode of the current block (e.g., affine or non-affine). (c) The set of allowed MV / MVD precisions may depend on slice type / temporal layer index / low latency check flag. (d) The set of allowed MV / MVD precisions may depend on the block size and / or block shape of the current block or neighboring blocks. (e) The set of allowed MV / MVD precisions may depend on the precision of the MVs to be stored in the decoded picture buffer. (i) In one example, if the stored MV is X pixels (X-pels), then the allowed MV / MVD precision set may have at least X pixels (X-pels).

[0257] Improved affine mode with AMVR support

[0258] Example 8 The set of allowed MVD precisions may vary from picture to picture, slice to slice, or block to block. In one example, the set of allowed MVD precisions may depend on coded information such as block size, block shape, etc. b. The set of allowed MV precisions may be predefined, such as {1 / 16, 1 / 4, 1}. c. An indication of the allowed MV precision may be signaled in the SPS / PPS / VPS / sequence header / picture header / slice header / group etc. of the CTU. d. Signaling the selected MV precision from the set of allowed MV precisions further depends on the number of allowed MV precisions for one block.

[0259] Example 9 One syntax element is signaled to the decoder to indicate the precision of the MVD used in affine inter mode. In one example, only one single syntax element is used to indicate the MVD precision that applies to affine and AMVR modes. i. In one example, the same semantics are used, i.e. syntax elements with the same value are mapped to the same MVD precision for AMVR and affine modes. Alternatively, the semantics of a single syntax element may differ for AMVR and affine modes, i.e., the values ​​of the same syntax element may be mapped to different MVD precisions for AMVR and affine modes. b. In one example, if the affine mode uses the same set of MVD precisions as the AMVR (e.g., the MVD precision set is {1,1 / 4,4}-pixel), the MVD precision syntax element in the AMVR is reused in the affine mode, i.e., only a single syntax element is used. i. Alternatively, in addition, in the CABAC encoder / decoder, the same or different context models for AMVR and affine modes may be used when encoding / decoding this syntax element. Alternatively, this syntax element may have different semantics in AMVR and affine mode: for example, in AMVR, syntax elements equal to 0, 1, and 2 indicate MV precisions of ¼ pixel, 1 pixel, and 4 pixels, respectively, while in affine mode, syntax elements equal to 0, 1, and 2 indicate MV precisions of ¼ pixel, 1 / 16 pixel, and 1 pixel, respectively. c. In one example, if the affine mode uses the same number of MVD precisions as the AMVR, but a different set of MVD precisions (e.g., the MVD precision set in the AMVR is {1, ¼, 4}-pixels, and for affine it is {1 / 16, ¼, 1}-pixels}-pixels), the MVD precision syntax element in the AMVR is reused in the affine mode, i.e., only one syntax element is used. i. Alternatively, in addition, in the CABAC encoder / decoder, the same or different context models for AMVR and affine modes may be used when encoding / decoding this syntax element. ii. Alternatively, this syntax element may have different semantics in AMVR and affine modes. In one example, the affine mode uses a smaller MVD precision than the AMVR, and the MVD precision syntax elements in the AMVR are reused in the affine mode, however, only a subset of the syntax element values ​​are valid for the affine mode. i. Alternatively, in addition, in the CABAC encoder / decoder, the same or different context models for AMVR and affine modes may be used when encoding / decoding this syntax element. ii. Alternatively, this syntax element may have different semantics in AMVR and affine modes. In one example, the affine mode uses a higher MVD precision than the AMVR, and the MVD precision syntax elements in the AMVR are reused in the affine mode, however, such syntax elements are extended to allow more values ​​in the affine mode. i. Alternatively, in addition, in the CABAC encoder / decoder, the same or different context models for AMVR and affine modes may be used when encoding / decoding this syntax element. ii. Alternatively, this syntax element may have different semantics in AMVR and affine modes. f. In one example, a new syntax element is used to code the MVD precision for affine mode, i.e., two different syntax elements are used to code the MVD precision for AMVR and for affine mode. g. Syntax for indication of affine mode MVD accuracy may be signaled under one or all of the following conditions are true: i. The MVD of all control points is non-zero. ii. The MVD for at least one control point is non-zero. iii. The MVD of one control point (e.g., the first CPMV) is not zero. In this case, there is no need to signal the MVD accuracy if one or all of the above conditions are not met. h. The syntax elements for indicating the MVD precision for either the affine or AMVR modes may be coded with a context, where the context depends on the coded information. i. In one example, if there is only one syntax element, the context may depend on whether the current block is coded in affine mode. i. In one example, the context may depend on neighboring blocks / temporal layer index / block size of prediction direction / block shape / MVD precision, etc. j. Enabling or disabling the use of multiple MVD precisions in affine mode may be signaled in the SPS / PPS / VPS / sequence header / picture header / slice header / CTUs, etc. In one example, signaling information to allow or disallow the use of multiple MVD precisions for affine mode may depend on other syntax elements, for example, information to enable or disable the use of multiple MV and / or MVP and / or MVD precisions in affine mode is signaled when affine mode is enabled and is not signaled and inferred to be 0 when affine mode is disabled. k. Alternatively, multiple syntax elements may be signaled to indicate the MV and / or MVP and / or MVD precision (all referred to as "MVD precision" in the following description) to be used in affine inter mode. i. In one example, the syntax elements for indicating the MVD precision used in affine inter mode and normal inter mode may be different. 1. The number of syntax elements to indicate the MVD precision used in affine inter mode and normal inter mode may be different. 2. The meaning of the syntax elements for indicating the accuracy of the MVD used in affine inter mode and normal inter mode may be different. 3. The context model in arithmetic coding for coding one syntax element and indicating the accuracy of the MVD used in affine inter mode and normal inter mode may be different. 4. The method of deriving a context model in arithmetic coding to code one syntax element to indicate the MVD precision used in affine inter mode and normal inter mode may be different. ii. In one example, a first syntax element (eg, amvr_flag) may be signaled to indicate whether to apply AMVR in the affine coding block. 1. The first syntax element is conditionally signaled. In one example, if the current block is coded in a particular mode (eg, CPR / IBC mode), the signaling of the first syntax element (amvr_flag) is skipped. b. In one example, if the MVDs (including both horizontal and vertical components) of all CPMVs are all zero, the signaling of the first syntax element (amvr_flag) is skipped. c. In one example, if the MVD (including both horizontal and vertical components) of the selected CPMV is all zeros, the signaling of the first syntax element (amvr_flag) is skipped. i. In one example, the MVD of the selected CPMV is the MVD of the first CPMV to be encoded / decoded. d. In one example, if the use of enabling multiple MVD precision for affine coding blocks is false, the signaling of the first syntax element (amvr_flag) is skipped. e. In one example, the first syntax element may be signaled under the following conditions: i. The use of allowing multiple MVD precisions for affine coding blocks is true and the current block is coded in affine mode. ii. Alternatively, the use of enabling multiple MVD precisions for affine coding blocks is true, the current block is coded in affine mode, and at least one component of the MVD of the CPMV is not equal to 0. iii. Alternatively, the use of enabling multiple MVD precisions for affine coding blocks is true, the current block is coded in affine mode, and at least one component of the MVD of the selected CPMV is not equal to 0. 1. In one example, the MVD of the selected CPMV is the MVD of the first CPMV to be encoded / decoded. 2. If AMVR is not applied to the affine coding block or the first syntax element is not present, the default MV and / or MVD precision is used. f. In one example, the default precision is 1 / 4 pixel. g. Alternatively, the default precision is set to the precision used for motion compensation of the affine coding block. 3. For example, if amvr_flag is 0, then the MVD precision of the affine mode is 1 / 4 pixel, otherwise the MVD precision of the affine mode may be other values. h. Alternatively, the additional MVD precision may be further signaled via a second syntax element. iii. In one example, a second syntax element (eg, amvr_coarse_precision_flag) may be signaled to indicate the MVD precision of the affine mode. 1. In one example, whether the second syntax element is signaled may depend on the first syntax element. For example, the second syntax element is signaled only when the first syntax element is 1. 2. In one example, if the second syntax element is 0, then the MVD precision of the affine mode is 1 pixel, otherwise, the MVD precision of the affine mode is 1 / 16 pixel. 3. In one example, if the second syntax element is 0, then the MVD precision of the affine mode is 1 / 16 pixel, otherwise the MVD precision of the affine mode is full pixel. iv. In one example, a syntax element used to indicate the MVD precision used in affine inter mode shares the same context model as a syntax element that has the same name but is used to indicate the MVD precision used in normal inter mode. 4. Alternatively, the syntax element used to indicate the MVD precision used in affine inter mode uses a different context model as the syntax element having the same name but used to indicate the MVD precision used in regular inter mode.

[0260] Example 10 Whether or how to apply AMVR to an affine coding block may depend on the reference pictures of the current block. In one example, if the reference picture is the current picture, i.e., if intra block copy is applied in the current block, AMVR is not applied.

[0261] A Fast Algorithm of AVMR in Affine Mode for Encoders

[0262] The RD cost of affine mode and AMVP mode (real RD cost, or SATD / SSE / SAD cost plus rough bit cost) is denoted as affineCosti and amvpCosti when IMV=i, where i=0,1,2. Here, IMV=0 means 1 / 4 pixel MV, IMV=1 means integer MV in AMVP mode and 1 / 16 pixel MV in affine mode, and IMV=2 means 4 pixel MV in AMVP mode and integer MV in affine mode. The RD cost of merge mode is mergeCost.

[0263] Example 11 For the current CU's affine mode, if the best mode of its parent CU is neither AF_INTER nor AF_MERGE mode, it is suggested to disable AMVR. Alternatively, AMVR is disabled for affine mode of the current CU if its parent CU's best mode is not AF_INTER mode Example 12. It is suggested that AMVR is disabled for affine mode if affineCost0>th1*amvpCost0, where th1 is a positive threshold. a. Alternatively, if min(affineCost0, amvpCost0)>th2*mergeCost, where th2 is a positive threshold, then AMVR is disabled for affine mode. b. Alternatively, if affineCost0>th3*affineCost1, where th3 is a positive threshold, integer MVs are disabled for affine modes.

[0264] Example 12 If amvpCost0>th4*affineCost0, where th4 is a positive threshold, it is suggested to disable AMVR in AMVP mode. a. Alternatively, in AMVP mode, if min(affineCost0, amvpCost0)>th5*mergeCost, where th5 is a positive threshold, AMVR is disabled.

[0265] Example 13 It is proposed that the 4 / 6 parameter affine models obtained at one MV precision may be used as candidate starting search points for the other MV precision. b. In one example, the 4 / 6 parameter affine model obtained with 1 / 16 MV may be used as a candidate starting search point for other MV accuracies. c. In one example, the 4 / 6 parameter affine model obtained with 1 / 4 MV may be used as a candidate starting search point for other MV accuracy.

[0266] Example 14 In the encoder, if the parent block of the current block does not select affine mode, the AMVR of the affine mode is not checked.

[0267] Example 15 By utilizing statistics of the usage of different MV precisions for affine coding blocks in the aforementioned coded frame / slice / tile / CTU row, the distortion rate calculation of the MV precision for the affine coding blocks in the current slice / tile / CTU row can be terminated early. d. In one example, the percentage of affine coding blocks with a particular MV precision is recorded. If the percentage is too small, the check for the corresponding MV precision is skipped. e. In one example, a previously coded frame having the same temporal layer is utilized to determine whether to skip a certain MV precision.

[0268] Affine AMVR coding context

[0269] Example 16 For each context used for coding an affine AMVR code, it is proposed to set a variable (denoted shiftIdx) to control the two probability update rates associated with this context. In one example, the faster update rate is defined as (shiftIdx>2)+2. b. In one example, the slower update rate is defined by (shiftIdx&3)+3+shift0. c. In one example, a conforming bitstream follows the rule that the derived faster update rate should be contained within [2,5]. d. In one example, a conforming bitstream follows the rule that the derived faster update rate should be contained within [3,6].

[0270] Example 17 It is proposed that when coding the AMVR mode of one block, the affine AMVR mode information of neighboring blocks is not allowed for context modeling. In one example, the AMVR mode index of the neighboring block may be used, eliminating the affine AMVR mode information of the neighboring block. Table 5 (Table 5-1 and Table 5-2) shows an example of (xNbL, yNbL), (xNbA, yNbA) as the top-left neighboring block. In one example, the context index offset ctxInc=(condL && availableL)+(condA && availableA)+ctxSetIdx*3.

[0271] [Table 5]

[0272] [Table 6]

[0273] b. Alternatively, the affine AMVR mode information of the neighboring blocks may be further used using a function instead of being used directly. In one example, the function func listed in Table 6-1 may return true if the amvr_mode[xNbL][yNbL] of the affine coded neighboring blocks indicates a constant MV precision (e.g., 1 / 4 pixel MV precision). In one example, the function func listed in Table 3-2 may return true if the amvr_flag[xNbL][yNbL] of the affine coded neighboring blocks indicates a constant MV precision (e.g., 1 / 4 pixel MV precision).

[0274] [Table 7]

[0275] [Table 8]

[0276] Alternatively, the affine AMVR mode information of neighboring blocks may be further used to code the first syntax element (e.g., amvr_flag) of the AMVR mode (applied to normal inter mode). Examples are shown in Table 6-3 and Table 6-4.

[0277] [Table 9]

[0278] [Table 10]

[0279] d. If the AMVR mode information is represented by multiple syntax elements (e.g., a first and second syntax element represented by amvr_flag, amvr_coarse_precision_flag), the above syntax amvr_mode may be replaced by any of the multiple syntax elements and the above method may still apply.

[0280] Example 18 When coding an affine AMVR mode, the AMVR mode information of neighboring blocks may be used for context coding. In one example, the AMVR mode information of neighboring blocks is used directly. An example is shown in Table 7. Alternatively, in addition, the context index offset ctxInc=(condL&availableL)+(condA&availableA)+ctxSetIdx*3.

[0281] [Table 11]

[0282] Alternatively, the AMVR mode information of neighboring blocks is not allowed for context modeling. An example is shown in Table 8.

[0283] [Table 12]

[0284] Alternatively, the affine AMVR mode information of the neighboring blocks may be further used using a function instead of being used directly. In one example, the function func listed in Table 9 may return true if the amvr_mode[xNbL][yNbL] of the non-affine coded neighboring blocks indicates a particular MV precision (e.g., 1 / 4 pixel MV precision).

[0285] [Table 13]

[0286] d. If the affine AMVR mode information is represented by multiple syntax elements (e.g., a first and second syntax element represented by amvr_flag, amvr_coarse_precision_flag), the above syntax amvr_mode may be replaced by any of the multiple syntax elements and the above method may still apply.

[0287] Fast Algorithms for SMVD and Affine SMVD

[0288] When checking the SMVD mode, assume that the currently selected best mode is CurBestMode and the AMVR MVD precision in the AMVR is MvdPrec or the MVD precision of the affine AMVR is MvdPrecAff.

[0289] Example 19 The SMVD mode may be skipped based on the currently selected best mode (ie, CurBestMode), i.e., the MVD accuracy in the AMVR. In one example, if CurBestMode is merge mode and / or UMVE mode, SMVD mode may not be checked. b. In one example, if CurBestMode is not coded with SMVD mode, then SMVD mode may not be checked. c. In one example, if CurBestMode is affine mode, SMVD mode may not be checked. d. In one example, if CurBestMode is sub-block merging mode, SMVD mode may not be checked. e. In one example, if CurBestMode is affine SMVD mode, then SMVD mode may not be checked. f. In one example, if CurBestMode is affine merge mode, SMVD mode may not be checked. g. In one example, the above fast methods, ie bullets 13.a-13.f, may be applied only to a certain MVD precision. i. In one example, the above fast method may be applied only if the MVD precision is greater than or equal to the precision (eg, integer pixel precision). ii. In one example, the above fast method may be applied only when the MVD precision is greater than the precision (eg, integer pixel precision). iii. In one example, the above fast method may be applied only when the MVD precision is less than or equal to the precision (eg, integer pixel precision). iv. In one example, the above fast method may be applied only when the MVD precision is less than the precision (eg, integer pixel precision).

[0290] Example 20 The affine SMVD mode may be skipped based on the currently selected best mode (i.e., CurBestMode), the MVD accuracy in the affine AMVR. In one example, if CurBestMode is merge mode and / or UMVE mode, affine SMVD mode may not be checked. b. In one example, if CurBestMode is not coded in affine SMVD mode, then affine SMVD mode may not be checked. c. In one example, if CurBestMode is sub-block merging mode, then affine SMVD mode may not be checked. d. In one example, if CurBestMode is SMVD mode, then affine SMVD mode may not be checked. e. In one example, if CurBestMode is affine merge mode, then affine SMVD mode may not be checked. f. In one example, the above fast methods, ie, bullets 20.a-20.e, may be applied only to a certain MVD precision. i. In one example, the above fast method may be applied only when the affine MVD accuracy is equal to or higher than the accuracy (e.g., integer pixel accuracy). ii. In one example, the above fast method may be applied only when the affine MVD accuracy is greater than the accuracy (e.g., integer pixel accuracy). iii. In one example, the above fast method may be applied only when the affine MVD accuracy is equal to or lower than the accuracy (e.g., integer pixel accuracy). iv. In one example, the above fast method may be applied only when the affine MVD accuracy is less than the accuracy (e.g., integer pixel accuracy).

[0291] Example 21 The proposed method may be applied under specific conditions such as block size, slice / picture / tile type, or motion information. a. In one example, when the block size is smaller than M*H samples, for example, including 16 or 32 or 64 luminance samples, the proposed method is not permitted. b. Alternatively, when the minimum size of the width and / or height of the block is smaller than or equal to X, the proposed method is not permitted. In one example, X is set to 8. c. Alternatively, when the minimum size of the width and / or height of the block is not smaller than X, the proposed method is not permitted. In one example, X is set to 8. d. Alternatively, when the width of the block > th1 or >= th1 and / or the height of the block > th2 or >= th2, the proposed method is not permitted. In one example, th1 and / or th2 are set to 8. e. Alternatively, when the width of the block < th1 or <= th1 and / or the height of the block < th2 or <= th2, the proposed method is not permitted. In one example, th1 and / or th2 are set to 8. f. Alternatively, whether the above methods are enabled or disabled and / or which method is applied may depend on the block dimensions, video processing data unit (VPDU), picture type, low latency check flag, coded information of the current block (e.g., reference picture, uni- or bi-prediction) or previously coded blocks.

[0292] Example 22 The AMVR method for affine mode can be implemented in different ways depending on whether or not it applies intra block copy (IBC, also known as current picture reference (CPR)). In one example, if a block is coded by IBC, AMVR for affine mode cannot be used. b. In one example, AMVR for affine mode may be used, but if one block is coded by IBC, the candidate MV / MVD / MVP precision may be different from that used for non-IBC coding affine coding blocks.

[0293] Example 23 Any use of the term "slice" in the documentation may be replaced with "tile group" or "tile".

[0294] Example 24 In a VPS / SPS / PPS / slice header / tile group header, a syntax element equal to 1 (e.g., no_amvr_constraint_flag) specifies that it is a bitstream conformance requirement that the syntax element indicating whether AMVR is enabled (e.g., sps_amvr_enabled_flag) and the syntax element indicating whether affine AMVR is enabled (e.g., sps_affine_avmr_enabled_flag) are both equal to 0. A syntax element equal to 0 (e.g., no_amvr_constraint_flag) imposes no constraint.

[0295] Example 25 A syntax element (e.g., no_affine_amvr_constraint_flag) can be signaled in the VPS / SPS / PPS / slice header / tile group header or other video data unit. In one example, no_affine_amvr_constraint_flag equal to 1 specifies that it is a bitstream conformance requirement that a syntax element to indicate whether affine AMVR is enabled (e.g., sps_affine_avmr_enabled_flag) is equal to 0. When the syntax element (e.g., no_affine_amvr_constraint_flag) is equal to 0, no constraint is imposed.

[0296] Example 26 Multiple contexts may be used to code a second syntax element that indicates the coarse motion precision (eg, amvr_coarse_precision_flag). In one example, two contexts may be used. b. In one example, the selection of the context may depend on whether the current block is affine coded or not. c. In one example, a first syntax may be coded in only one context and a second syntax may be coded in only one context. d. In one example, a first syntax may be coded in only one context, and a second syntax may be bypass coded. e. In one example, for a first syntax, it may be bypass coded, and for a second syntax, it may be bypass coded. f. In one example, for all syntax elements related to motion vector precision, they may be bypass coded.

[0297] Example 27 For example, only the first bin of the syntax element amvr_mode is coded in the arithmetic coding context. All following bins of amvr_mode are bypass coded. The methods disclosed above may be applied to other syntax elements. For example, only the first bin of a syntax element SE is coded in an arithmetic coding context. All following bins of SE are coded as bypass coding. SE may be: 1) alf_ctb_flag 2) sao_merge_left_flag 3) sao_merge_up_flag 4) sao_type_idx_luma 5) sao_type_idx_chroma 6) split_cu_flag 7) split_qt_flag 8) mtt_split_cu_vertical_flag 9) mtt_split_cu_binary_flag 10) cu_skip_flag 11) pred_mode_ibc_flag 12) pred_mode_flag 13) intra_luma_ref_idx 14) intra_subpartitions_mode_flag 15) intra_subpartition_split_flag 16) intra_luma_mpm_flag 17) intra_chroma_pred_mode 18) merge_flag 19) inter_pred_idc 20) inter_affine_flag 21) cu_affine_type_flag 22) ref_idx_l0 23) mvp_l0_flag 24) ref_idx_l1 25) mvp_l1_flag 26) avmr_flag 27) amvr_precision_flag 28) gbi_idx 29) cu_cbf 30) cu_sbt_flag 31) cu_sbt_quad_flag 32) cu_sbt_horizontal_flag 33) cu_sbt_pos_flag 34) mmvd_flag 35) mmvd_merge_flag 36) mmvd_distance_idx 37) ciip_flag 38) ciip_luma_mpm_flag 39) merge_subblock_flag 40) merge_subblock_idx 41) merge_triangle_flag 42) merge_triangle_idx0 43) merge_triangle_idx1 44) merge_idx 45) abs_mvd_greater0_flag 46) abs_mvd_greater1_flag 47) tu_cbf_luma 48) tu_cbf_cb 49) tu_cbf_cr 50) cu_qp_delta_abs 51) transform_skip_flag 52) tu_mts_idx 53) last_sig_coeff_x_prefix 54) last_sig_coeff_y_prefix 55) coded_sub_block_flag 56) sig_coeff_flag 57) par_level_flag 58) abs_level_gt1_flag 59) abs_level_gt3_flag c. Further alternatively, if the syntax element SE is a binary value (i.e., can only be equal to 0 or 1), it may be context-coded. i. Further alternatively, if the syntax element SE is a binary value (i.e., can only be equal to 0 or 1), it may be bypass-coded. d. Further alternatively, only one context may be used to code the first bin.

[0298] Example 28 Based on the motion accuracy that can be signaled, the accuracy of motion vector prediction (MVP) or motion vector difference (MVD), or the reconstructed motion vector (MV) may be changed. a. In one example, if the original prediction of MVP is lower (or not higher) than the target accuracy, MVP = MVP<s. s is an integer, and this integer may depend on the difference between the original accuracy and the target accuracy. iii. Alternatively, if the original accuracy of MVD is lower (or not higher) than the target accuracy, MVD = MVD<s. s is an integer, and it may depend on the difference between the original accuracy and the target accuracy. iv. Alternatively, if the original accuracy of MV is lower (or not higher) than the target accuracy, MV = MV<s. s is an integer, and it may depend on the difference between the original accuracy and the target accuracy. b. In one example, if the original prediction of MVP is higher (or not lower) than the target accuracy, MVP = Shift(MVP, s). s is an integer, and this integer may depend on the difference between the original accuracy and the target accuracy. v. Alternatively, if the original precision of the MVD is higher (or not lower) than the target precision, then MVD=Shift(MVD,s), where this integer may depend on the difference between the original precision and the target precision. Alternatively, if the native precision of MV is higher (or lower) than the target precision, then MV=Shift(MV,s), where s is an integer and may depend on the difference between the native precision and the target precision. c. In one example, if the original prediction of the MVP is higher (or not lower) than the target accuracy, then MVP=SatShift(MVP,s), where s is an integer, which may depend on the difference between the original accuracy and the target accuracy. Alternatively, if the native precision of the MVD is higher (or lower) than the target precision, then MVD=SatShift(MVD,s), where s is an integer and may depend on the difference between the native precision and the target precision. viii. Alternatively, if the native precision of MV is higher (or lower) than the target precision, then MV=SatShift(MV,s), where s is an integer and may depend on the difference between the native precision and the target precision. d. The above method may be applied if the current block is not coded in affine mode. e. If the current block is coded in affine mode, the method disclosed above may be applied.

[0299] 1. Embodiment The highlighted parts indicate the revised specifications.

[0300] [ka]

[0301] [Table 14]

[0302] Alternatively, the SPS syntax table is given as follows:

[0303] [Table 15]

[0304] Semantics: sps_affine_amvr_enabled_flag equal to 1 specifies that adaptive motion vector differential resolution is used for affine inter mode motion vector coding. amvr_enabled_flag equal to 0 specifies that adaptive motion vector differential resolution is not used for affine inter mode motion vector coding.

[0305] [ka]

[0306] [Table 16] [Table 17] [Table 18]

[0307] [Table 19] [Table 20]

[0308] [Table 21] [Table 22]

[0309] [Table 23] [Table 24]

[0310] In one example, conditionsA is defined as follows: (sps_affine_amvr_enabled_flag && inter_affine_flag==1&& (MvdCpL0[x0][y0][0][0]!=0||MvdCpL0[x0][y0][0][1]!=0|| MvdCpL1[x0][y0][0][0]!=0||MvdCpL1[x0][y0][0][1]!=0|| MvdCpL0[x0][y0][1][0]!=0||MvdCpL0[x0][y0][1][1]!=0|| MvdCpL1[x0][y0][1][0]!=0||MvdCpL1[x0][y0][1][1]!=0|| MvdCpL0[x0][y0][2][0]!=0||MvdCpL0[x0][y0][2][1]!=0|| MvdCpL1[x0][y0][2][0]!=0||MvdCpL1[x0][y0][2][1]!=0))

[0311] Alternatively, conditionsA can be defined as follows: (sps_affine_amvr_enabled_flag && inter_affine_flag==1&& (MvdCpL0[x0][y0][0][0]!=0||MvdCpL0[x0][y0][0][1]!=0|| MvdCpL1[x0][y0][0][0]!=0||MvdCpL1[x0][y0][0][1]!=0||)

[0312] Alternatively, conditionsA can be defined as follows: (sps_affine_amvr_enabled_flag && inter_affine_flag==1&& (MvdCpLX[x0][y0][0][0]!=0||MvdCpLX[x0][y0][0][1]!=0) Here, X is 0 or 1.

[0313] Alternatively, conditionsA can be defined as follows: (sps_affine_amvr_enabled_flag && inter_affine_flag==1)

[0314] In one example, conditionsB is defined as follows: !sps_cpr_enabled_flag||!(inter_pred_idc[x0][y0]==PRED_L0 && ref_idx_l0[x0][y0]==num_ref_idx_l0_active_minus1)

[0315] Alternatively, conditionsB can be defined as follows: !sps_cpr_enabled_flag||!(pred_mode[x0][y0]==CPR).

[0316] Alternatively, conditionsB can be defined as follows: !sps_ibc_enabled_flag||!(pred_mode[x0][y0]==IBC).

[0317] If different syntax elements are used to code AMVR or Affine AMVR, the context modeling and / or context used in Example 5.5 applied to Affine AMVR can be applied accordingly.

[0318] Semantics

[0319] amvr_flag[x0][y0] specifies the resolution of the motion vector difference. The array index x0,y0 specifies the position (x0,y0) of the top-left luma sample of the coding block relative to the top-left luma sample of the picture. amvr_flag[x0][y0] equal to 0 specifies that the resolution of the motion vector difference is 1 / 4 of the luma sample, and amvr_flag[x0][y0] equal to 1 specifies that the resolution of the motion vector difference is further specified by amvr_coarse_precisoin_flag[x0][y0].

[0320] If amvr_flag[x0][y0] is not present, it is inferred as follows: -If sps_cpr_enabled_flag is equal to 1, amvr_flag[x0][y0] is inferred to be equal to 1. - Otherwise (sps_cpr_enabled_flag is equal to 0), amvr_flag[x0][y0] is inferred to be equal to 0.

[0321] amvr_coarse_precisoin_flag[x0][y0] equal to 1 specifies that the resolution of the motion vector differential is 4 luma samples if inter_affine_flag is 0, and 1 luma sample if inter_affine_flag is 1. The array index x0,y0 specifies the position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If amvr_coarse_precisoin_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0322] If inter_affine_flag[x0][y0] is 0, the variable MvShift is set equal to (amvr_flag[x0][y0] + amvr_coarse_precisoin_flag[x0][y0]) << 1, and the variables MvdL0[x0][y0][0], MvdL0[x0][y0][1], MvdL1[x0][y0][0], MvdL1[x0][y0][1] are modified as follows:

[0323] MvdL0[x0][y0][0]=MvdL0[x0][y0][0]<<(MvShift+2) (7-70) MvdL0[x0][y0][1]=MvdL0[x0][y0][1]<<(MvShift+2) (7-71) MvdL1[x0][y0][0]=MvdL1[x0][y0][0]<<(MvShift+2) (7-72) MvdL1[x0][y0][1]=MvdL1[x0][y0][1]<<(MvShift+2) (7-73)

[0324] If inter_affine_flag[x0][y0] is equal to 1, the variable MvShift is set equal to (amvr_coarse_precisoin_flag?(amvr_coarse_precisoin_flag<<1):(-(amvr_flag<<1))) and the variables MvdCpL0[x0][y0][0][0], MvdCpL0[x0][y0][0][1], MvdCpL0[x0][y0][1][0], MvdCpL0[x0][y0][1][1], MvdCpL0[x0][y0][2][0], MvdCpL0[x0][y0][2][1] are modified as follows:

[0325] MvdCpL0[x0][y0][0][0]=MvdCpL0[x0][y0][0][0]<<(MvShift+2) (7-73) MvdCpL1[x0][y0][0][1]=MvdCpL1[x0][y0][0][1]<<(MvShift+2) (7-67) MvdCpL0[x0][y0][1][0]=MvdCpL0[x0][y0][1][0]<<(MvShift+2)(7-66) MvdCpL1[x0][y0][1][1]=MvdCpL1[x0][y0][1][1]<<(MvShift+2) (7-67) MvdCpL0[x0][y0][2][0]=MvdCpL0[x0][y0][2][0]<<(MvShift+2) (7-66) MvdCpL1[x0][y0][2][1]=MvdCpL1[x0][y0][2][1]<<(MvShift+2) (7-67)

[0326] Alternatively, if inter_affine_flag[x0][y0] is equal to 1, then the variable MvShift is set equal to (affine_amvr_coarse_precisoin_flag?(affine_amvr_coarse_precisoin_flag<<1):(-(affine_amvr_flag<<1))).

[0327] [ka]

[0328] The rounding process is modified such that if a given rightShift value is 0 (which occurs with 1 / 16 pixel accuracy), then the rounding offset is set to 0 instead of (1<<(rightShift-1)). For example, the rounding subclause for MV is modified as follows:

[0329] The inputs to this process are: - Motion Vector mvX - Right shift of the right shift parameter when rounding, - Left shift parameter leftShift to increase resolution

[0330] The output of this process is the rounded motion vector mvX.

[0331] When rounding mvX, the following applies: offset=(rightShift==0)?0:(1<<(rightShift-1)) (8-371) mvX[0]=(mvX[0]>=0?(mvX[0]+offset)>>rightShift: -((-mvX[0]+offset)>>rightShift))< <leftshift (8-372) mvx[1]="(mvX[1]">=0?(mvX[1]+offset)>>rightShift:-((-mvX[1]+offset)>>rightShift))< <leftShift (8-373)

[0332] [ka]

[0333] The rounding process invoked in the affine motion vector derivation process is done with an input of (MvShift+2) instead of being fixed at 2.

[0334] Derivation of luminance affine control point motion vector predictor

[0335] The inputs to this process are: - Luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture - Two variables cbWidth and cbHeigh that specify the width and height of the current luminance coding block. - Reference index of the current coding unit refIdxLX, where X is 0 or 1. - number of control point motion vectors numCpMv

[0336] The output of this process is the intensity affine control point motion vector predictor mvpCpLX[cpIdx], where X is either 0 or 1, cpIdx=0..numCpMv-1.

[0337] To derive the control point motion vector predictor candidate list, which is cpMvpListLX, where X is 0 or 1, the following ordered steps are applied: The number of control point motion vector predictor candidates in the list numCpMvpCandLX is set equal to 0.

[0338] The variables availableFlagA and availableFlagB are both set equal to FALSE. ...

[0339] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to cpMvpLX[cpIdx], rightShift set equal to (MvShift+2), leftShift set equal to (MvShift+2) as input, and cpMvpLX[cpIdx] rounded by cpIdx=0..numCpMv-1 as output. ...

[0340] The variable availableFlagA is set equal to TRUE.

[0341] The derivation of luminance affine control point motion vectors from neighboring blocks process specified in subclause 8.4.4.5 is called with the luminance coding block position (xCb, yCb), the luminance coding block width and height (cbWidth, cbHeight), the neighboring luminance coding block position (xNb, yNb), the neighboring luminance coding block width and height (nbW, nbH), and the number of control point motion vectors numCpMv as inputs, and the control point motion vector predictor candidate cpMvpLY[cpIdx] (cpIdx=0..numCpMv-1) as output.

[0342] The motion vector rounding process specified in clause 8.4.2.14 is called with input mvX set equal to cpMvpLY[cpIdx], rightShift set equal to (MvShift+2), leftShift set equal to (MvShift+2), and output cpMvpLY[cpIdx] rounded by cpIdx=0..numCpMv-1. ...

[0343] The derivation process of a luminance affine control point motion vector from neighboring blocks specified in subclause 8.4.4.5 is called with the luminance coding block position (xCb, yCb), the luminance coding block width and height (cbWidth, cbHeight), the neighboring luminance coding block position (xNb, yNb), the neighboring luminance coding block width and height (nbW, nbH), and the number of control point motion vectors numCpMv as input, and the control point motion vector predictor candidate cpMvpLX[cpIdx] (cpIdx=0..numCpMv-1) as output.

[0344] The motion vector rounding process specified in clause 8.4.2.14 is called with mvX set equal to cpMvpLX[cpIdx], rightShift set equal to (MvShift+2), leftShift set equal to (MvShift+2) as input, and cpMvpLX[cpIdx] rounded by cpIdx=0..numCpMv-1 as output.

[0345] Make the following assignments: cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[0] (8-618) cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[1] (8-619) cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[2] (8-620) numCpMvpCandLX=numCpMvpCandLX+1 (8-621)

[0346] Alternatively, if PredFlagLY[xNbBk][yNbBk] (with Y=!X) is equal to 1 and DiffPicORDerCnt(RefPicListY[RefIdxLY[xNbBk][yNbBk]],RefPicListX[refIdxLX]) is equal to 0 then the following applies:

[0347] The variable availableFlagB is set to TRUE.

[0348] The derivation of a luminance affine control point motion vector from neighboring blocks process specified in subclause 8.4.4.5 is called with the luminance coding block position (xCb, yCb), the luminance coding block width and height (cbWidth, cbHeight), the neighboring luminance coding block position (xNb, yNb), the neighboring luminance coding block width and height (nbW, nbH), and the number of control point motion vectors numCpMv as input, and the control point motion vector predictor candidate cpMvpLY[cpIdx] (cpIdx=0..numCpMv-1) as output.

[0349] The motion vector rounding process specified in clause 8.4.2.14 is called with input mvX set equal to cpMvpLY[cpIdx], rightShift set equal to (MvShift+2), leftShift set equal to (MvShift+2), and output cpMvpLY[cpIdx] rounded by cpIdx=0..numCpMv-1.

[0350] Make the following assignments: cpMvpListLX[numCpMvpCandLX][0]=cpMvpLY[0] (8-622) cpMvpListLX[numCpMvpCandLX][1]=cpMvpLY[1] (8-623) cpMvpListLX[numCpMvpCandLX][2]=cpMvpLY[2] (8-624) numCpMvpCandLX=numCpMvpCandLX+1 (8-625)

[0351] If numCpMvpCandLX is less than 2, the following applies:

[0352] The affine control point motion vector prediction candidate derivation process specified in Section 8.4.4.8 is called with the luma coding block position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight, and the reference index of the current coding unit refIdxLX as input, and the availability flag availableConsFlagLX, the availability flag availableFlagLX[cpIdx], and cpMvpLX[cpIdx] (cpIdx=0..numCpMv-1) as output.

[0353] If availableConsFlagLX is equal to 1 and numCpMvpCandLX is equal to 0, the following allocation is made:

[0354] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[0] (8-626) cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[1] (8-627) cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[2] (8-628) numCpMvpCandLX=numCpMvpCandLX+1 (8-629)

[0355] The following applies for cpIdx=0..numCpMv-1. If numCpMvpCandLX is less than 2 and availableFlagLX[cpIdx] is equal to 1, the following allocation takes place:

[0356] cpMvpListLX[numCpMvpCandLX][0]=cpMvpLX[cpIdx] (8-630) cpMvpListLX[numCpMvpCandLX][1]=cpMvpLX[cpIdx] (8-631) cpMvpListLX[numCpMvpCandLX][2]=cpMvpLX[cpIdx] (8-632) numCpMvpCandLX=numCpMvpCandLX+1 (8-633)

[0357] If numCpMvpCandLX is less than 2, the following applies:

[0358] The temporal luma motion vector prediction derivation process described in subclause 8.4.2.11 is called using the luma coding block position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight, refIdxLX as input, and the availability flag availableFlagLXCol, and the temporal motion vector predictor mvLXCol as output.

[0359] If availableFlagLXCol is equal to 1, the following applies: The motion vector rounding process described in clause 8.4.2.14 is called with mvX set equal to mvLXCol, rightShift set equal to (MvShift+2), leftShift set equal to (MvShift+2) as input, and the rounded mvLXCol as output.

[0360] Make the following assignments: cpMvpListLX[numCpMvpCandLX][0]=mvLXCol (8-634) cpMvpListLX[numCpMvpCandLX][1]=mvLXCol (8-635) cpMvpListLX[numCpMvpCandLX][2]=mvLXCol (8-636) numCpMvpCandLX=numCpMvpCandLX+1 (8-637)

[0361] If numCpMvpCandLX is less than 2, then both mvZero[0] and mvZero[1] become 0 and the following is repeated until numCpMvpCandLX becomes 2.

[0362] cpMvpListLX[numCpMvpCandLX][0]=mvZero (8-638) cpMvpListLX[numCpMvpCandLX][1]=mvZero (8-639) cpMvpListLX[numCpMvpCandLX][2]=mvZero (8-640) numCpMvpCandLX=numCpMvpCandLX+1 (8-641)

[0363] The affine control point motion vector prediction module cpMvpLX, where X is 0 or 1, is derived as follows. cpMvpLX=cpMvpListLX[mvp_lX_flag[xCb][yCb]] (8-642)

[0364] Derivation process of motion vector prediction candidates based on constructed affine control points

[0365] The inputs to this process are: - A luma position (xCb, yCb) that specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture. - Two variables cbWidth and cbHeigh that specify the width and height of the current luminance coding block. - Reference index of the current prediction unit partition refIdxLX (X is 0 or 1)

[0366] The output of this process is: The availability flag of the constructed affine control point motion vector prediction candidate is represented as availableConsFlagLX, where X is 0 or 1. - Availability flag availableLX[cpIdx] (cpIdx=0..2, X is 0 or 1) - Constructed affine control point motion vector prediction candidate cpMvLX[cpIdx] (cpIdx=0..numCpMv-1, X is 0 or 1)

[0367] The first (top-left) control point motion vector cpMvLX[0] and availability flag availableFlagLX[0] are derived in the following sequential steps.

[0368] Sample positions (xNbB2, yNbB2), (xNbB3, yNbB3) and (xNbA2, yNbA2) are set equal to (xCb-1, yCb-1), (xCb, yCb-1) and (xCb-1, yCb), respectively.

[0369] The availability flag availableFlagLX[0] is set to 0, and both components of cpMvLX[0] are set to 0.

[0370] The following applies when (xNbTL, yNbTL) is replaced by B2, B3, and A2.

[0371] The coding block availability derivation process described in paragraph 2 is called with inputs the position of the luminance coding block (xCb, yCb), the width of the luminance coding block cbWidth, the height of the luminance coding block cbHeight, and the luminance position (xNbY, yNbY) set equal to (xNbTL, yNbTL), and the output is assigned to the coding block availability flag availableTL.

[0372] If availableTL is equal to TRUE and availableFlagLX[0] is equal to 0, the following applies:

[0373] If PredFlagLX[xNbTL][yNbTL] is equal to 1, DiffPicORDerCnt(RefPicListX[RefIdxLX[xNbTL][yNbTL]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLX[xNbTL][yNbTL] is not the current picture, then availableFlagLX[0] is set equal to 1 and the following allocation is made: cpMvLX[0]=MvLX[xNbTL][yNbTL] (8-643)

[0374] Otherwise, if PredFlagLY[xNbTL][yNbTL] (Y=!X) is equal to 1, and DiffPicORDerCnt(RefPicListY[RefIdxLY[xNbTL][yNbTL]],RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbTL][yNbTL] is not the current picture, set availableFlagLX[0] equal to 1 and make the assignment as follows: cpMvLX[0]=MvLY[xNbTL][yNbTL] (8-644)

[0375] If availableFlagLX[1]=1, the motion vector rounding process as specified in clause 8.4.2.14 is invoked with the mvX set equal to cpMvLX[1], the rightShift set equal to (MvShift+2), the leftShift set equal to (MvShift+2), and the rounded cpMvLX as output.

[0376] The second (top right) control point motion vector cpMvLX[1] and availability flag availableFlagLX[1] are derived in the following sequential steps.

[0377] Sample positions (xNbB1, yNbB1) and (xNbB0, yNbB0) are set equal to (xCb+cbWidth-1, yCb-1) and (xCb+cbWidth, yCb-1), respectively.

[0378] The availability flag availableFlagLX[0] is set to 0, and both components of cpMvLX[0] are set to 0.

[0379] The following applies when (xNbTR, yNbTR) is replaced by B1 and B0.

[0380] As specified in section 6.4.X, the coding block availability derivation process is invoked with inputs the luma coding block position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight, and the luma position (xNbY, yNbY) set equal to (xNbTR, yNbTR), and the output is assigned to the coding block availability flag availableTR.

[0381] If availableTR is equal to TRUE and availableFlagLX[1] is equal to 0, the following applies:

[0382] If PredFlagLX[xNbTR][yNbTR] is equal to 1, DiffPicOrderCnt(RefPicListX[RefIdxLX[xNbTR][yNbTR]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLX[xNbTR][yNbTR] is not the current picture, then availableFlagLX[1] is set equal to 1 and the following allocation is made: cpMvLX[1]=MvLX[xNbTR][yNbTR] (8-645)

[0383] Otherwise, if PredFlagLY[xNbTR][yNbTR] (Y=!X) is equal to 1, and DiffPicORDerCnt(RefPicListY[RefIdxLY[xNbTR][yNbTR]],RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbTR][yNbTR] is not the current picture, set availableFlagLX[1] equal to 1 and make the allocation as follows: cpMvLX[1]=MvLY[xNbTR][yNbTR] (8-646)

[0384] If availableFlagLX[1]=1, then the motion vector rounding process as specified in clause 8.4.2.14 is invoked with the mvX set equal to cpMvLX[1], the rightShift set equal to (MvShift+2), the leftShift set equal to (MvShift+2), and the rounded cpMvLX as output.

[0385] The third (bottom-left) control point motion vector cpMvLX[2] and availability flag availableFlagLX[2] are derived in the following sequential steps.

[0386] Sample positions (xNbA1, yNbA1) and (xNbA0, yNbA0) are set equal to (xCb-1, yCb+cbHeight-1) and (xCb-1, yCb+cbHeight), respectively.

[0387] The availability flag availableFlagLX[2] is set to 0, and both components of cpMvLX[2] are set to 0.

[0388] The following applies to (xNbBL, yNbBL) where BL is replaced by A1 and A0:

[0389] The coding block availability derivation process described in Section 6.4.X is invoked with the luminance coding block position (xCb, yCb), the luminance coding block width cbWidth, the luminance coding block height cbHeight, and the luminance position (xNbY, yNbY) = (xNbBL, yNbBL) as inputs, and its output is assigned to the coding block availability flag availableBL.

[0390] If availableBL is equal to TRUE and availableFlagLX[2] is equal to 0, the following applies:

[0391] If PredFlagLX[xNbBL][yNbBL] is equal to 1, DiffPicORDerCnt(RefPicListX[RefIdxLX[xNbBL][yNbBL]], RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbBL][yNbBL] is not the current picture, set availableFlagLX[2] to 1 and make the following allocation: cpMvLX[2]=MvLX[xNbBL][yNbBL] (8-647)

[0392] Otherwise, if PredFlagLY[xNbBL][yNbBL] (Y=!X) is equal to 1, and DiffPicORDerCnt(RefPicListY[RefIdxLY[xNbBL][yNbBL]],RefPicListX[refIdxLX]) is equal to 0, and the reference picture corresponding to RefIdxLY[xNbBL][yNbBL] is not the current picture, set availableFlagLX[2] equal to 1 and make the following allocation: cpMvLX[2]=MvLY[xNbBL][yNbBL] (8-648)

[0393] If availableFlagLX[2] is equal to 1, the motion vector rounding process as specified in Section 8.4.2.14 is invoked with the mvX set equal to cpMvLX[2] as input, the rightShift set equal to (MvShift+2), the leftShift set equal to (MvShift+2), and the rounded cpMvLX[2] as output.

[0394] [ka]

[0395] [Table 25]

[0396] A specification for ctxInc using the syntax elements to the left and above: An example is the context increment offset ctxInc=(condL && availableL)+(condA && availableA)+ctxSetIdx*3. Alternatively, ctxInc=((condL && availableL)||(condA && availableA))+ctxSetIdx*3.+ctxSetIdx*3. ctxInc=(condL && availableL)+M*(condA&&availableA)+ctxSetIdx*3.(Example: M=2)ctxInc=M*(condL && availableL)+(condA&&availableA)+ctxSetIdx*3.(Example: M=2)

[0397] [Table 26]

[0398] Value of initValue of ctxIdx in amvr_flag: Different contexts are used if the current block is affine or non-affine.

[0399] [Table 27]

[0400] Or: [Table 28]

[0401] Alternatively, the same context is used if the current block is affine or non-affine.

[0402] [Table 29]

[0403] Alternatively, amvr_flag is bypass coded. Value of initValue of ctxIdx in amvr_coarse_precisoin_flag: Different contexts are used if the current block is affine or non-affine.

[0404] [Table 30]

[0405] Or:

[0406] [Table 31]

[0407] Alternatively, the same context is used if the current block is affine or non-affine.

[0408] [Table 32]

[0409] Alternatively, amvr_coarse_precision_flag is bypass coded.

[0410] The examples described above may be incorporated in conjunction with the methods described below, such as methods 2510 through 2540, which may be implemented in a video decoder or a video encoder.

[0411] 25A and 25D are flow charts illustrating an example of a video processing method. As shown in FIG. 25A, the method 2510 includes determining, at step 2512, that a transformation between a current video block of a video and a coding representation of the current video block is based on a non-affine inter-AMVR mode. The method 2510 includes performing the transformation based on the determination, at step 2514. In some implementations, the coding representation of the current video block is a context-based coding, and the context used to code the current video block is modeled without using affine AMVR mode information of neighboring blocks during the transformation.

[0412] The method 2520 includes, as shown in FIG. 25B, determining at step 2522 that a conversion between a current video block of a video and a coding representation of the current video block is based on an affine adaptive motion vector resolution (affine AMVR) mode. The method 2520 includes performing the conversion based on the determination at step 2524. In some implementations, the coding representation of the current video block is based on context-based coding, and the variables control two probability update rates for the context.

[0413] 25C, the method 2530 includes determining, at step 2532, that a transformation between a current video block of a video and a coding representation of the current video block is based on an affine AMVR mode. The method 2530 includes performing the transformation based on the determination, at step 2534. In some implementations, the coding representation of the current video block is based on context-based coding, where a context used to code the current video block is modeled using coding information of neighboring blocks in which both affine inter and normal inter AMVR modes are used during the transformation.

[0414] The method 2540 includes determining use of multiple contexts for converting between a current video block of a video and a coded representation of the current video block at operation 2542, as shown in FIG. 25D. The method 2540 includes performing the conversion based on the determination at step 2544. In some implementations, the multiple contexts are used to code a syntax element indicating coarse motion precision.

[0415] Method 2550 includes, as shown in FIG. 25E, determining whether to use a symmetric motion vector differential (SMVD) mode based on a currently selected best mode for converting between the current video block and a coded representation of the current video block at step 2552. Method 2554 includes performing the conversion based on the determination.

[0416] Method 2560 includes, as shown in FIG. 25F, determining whether to use an affine SMVD mode for transforming between the current video block and a coding representation of the current video block based on a currently selected best mode for the transform at step 2562. Method 2564 includes performing the transform based on the determination.

[0417] 5. Exemplary implementations of the disclosed technology

[0418] FIG. 26 is a block diagram of a video processing device 2600. The device 2600 may be used to implement one or more of the methods described herein. The device 2600 may be implemented in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 2600 may include one or more processing devices 2602, one or more memories 2604, and video processing hardware 2606. The one or more processing devices 2602 may be configured to implement one or more methods described herein, including but not limited to the method 2500. The memory(s) 2604 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 2606 may be used to implement the techniques described herein in a hardware circuit.

[0419] FIG. 28 is another example of a block diagram of a video processing system in which the disclosed technology may be implemented. FIG. 28 is a block diagram illustrating an example video processing system 2800 in which various technologies disclosed herein may be implemented. Various implementations may include some or all of the modules of the system 2800. The system 2800 may include an input unit 2802 for receiving video content. The video content may be received in a raw or uncompressed format, for example, 8 or 10 bit multi-module pixel values, or may be received in a compressed or encoded format. The input unit 2802 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0420] The system 2800 may include a coding module 2804 that may implement various coding or encoding methods described herein. The coding module 2804 may reduce the average bit rate of the video from the input unit 2802 to the output of the coding module 2804, generating a coded representation of the video. Thus, this coding technique may be referred to as a video compression or video transcoding technique. The output of the coding module 2804 may be stored or transmitted via a connected communication, as represented by module 2806. The bitstream (or coded) representation of the video received at the input unit 2902, stored or communicated, may be used by module 2808 to generate pixel values ​​or displayable video that are transmitted to the display interface unit 2810. The process of generating a user-viewable video from the bitstream representation may be referred to as video decompression. Additionally, although certain video processing operations are referred to as "coding" operations or tools, it will be understood that the coding tool or operation is performed by an encoder, and the corresponding decoding tool or operation that reverses the results of the decoding is performed by a decoder.

[0421] Examples of peripheral bus interface units or display interface units may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.

[0422] In some embodiments, the video coding method may be performed using an apparatus implemented on a hardware platform, such as those described with reference to FIG. 26 or FIG.

[0423] Various techniques and embodiments may be described using a format based on the following sections, which may be implemented as preferred features of some embodiments.

[0424] A first set of the multiple clauses uses some of the techniques described in the previous chapter, including, for example, clauses 16 to 18, 23 and 26 of the previous chapter.

[0425] 1. A video processing method comprising: determining that a transformation between a current video block of a video and a coding representation of the current video block is based on a non-affine inter-AMVR mode; and performing the transformation based on the determination, wherein the coding representation of the current video block is based on context-based coding, and a context used in coding the current video block is modeled without using affine AMVR mode information of neighboring blocks during the transformation.

[0426] 2. The method of claim 1, using non-affine Inter AMVR mode indices of neighboring coded blocks.

[0427] 3. The method of claim 2, wherein the context index used to code the non-affine inter AMVR mode of the current video block depends on at least one of an AMVR mode index, an affine mode flag, or the availability of an upper-left neighboring block, and the non-affine inter AMVR mode index indicates a particular MVD precision.

[0428] 4. The method of claim 3, further comprising deriving an offset value for each of the left neighbouring block and the neighboring block, and deriving the context index as a sum of the offset values ​​of the left neighbouring block and the neighboring block.

[0429] 5. The method of claim 4, wherein if a corresponding neighboring block is available, is not coded in affine mode, and has an AMVR mode index of the neighboring block not equal to 0, then deriving at least one of the offset values ​​as 1, otherwise deriving at least one of the offset values ​​as 0.

[0430] 6. The method according to claim 1, wherein the affine AMVR mode information of the neighboring blocks is not used directly, but indirectly with a function of the affine AMVR mode information.

[0431] 7. The method of claim 6, wherein the function returns true if amvr_mode[xNbL][yNbL] or amvr_flag[xNbL][yNbL] of the neighboring block containing (xNbL,yNbL) indicates a certain MVD precision.

[0432] 8. The method of claim 1, wherein affine AMVR mode information of a neighboring block is used to code a first syntax element of a non-affine Inter AMVR mode of the current video block.

[0433] 9. The method of claim 1, wherein if the AMVR mode of the current video block is represented by multiple syntax elements, coding any one of the multiple syntax elements using AMVR mode information of neighboring blocks coded in both non-affine inter mode and affine inter mode.

[0434] 10. The method of claim 1, wherein when the AMVR mode of a current video block is represented by multiple syntax elements, the AMVR mode information of neighboring blocks coded in affine inter mode is not used for coding any one of the multiple syntax elements.

[0435] 11. The method of claim 1, wherein when the AMVR mode of a current video block is represented by multiple syntax elements, the AMVR mode information of a neighboring block coded in affine inter mode is used indirectly, rather than directly, in the coding of any one of the multiple syntax elements.

[0436] 12. The method according to any one of clauses 9 to 11, wherein the AMVR mode of the current video block includes affine inter AMVR mode and non-affine inter AMVR mode.

[0437] 13. A method of video processing comprising: determining that a conversion between a current video block of a video and a coding representation of the current video block is based on an affine adaptive motion vector resolution (affine AMVR) mode; and performing the conversion based on the determination, wherein the coding representation of the current video block is based on context-based coding, and variables control two probability update rates for the context.

[0438] 14. The method of claim 13, wherein the two probability update rates include a faster update rate defined as (shiftIdx>>2)+2, where shiftIdx denotes a variable.

[0439] 15. The method of claim 13, wherein the two probability update rates include a slower update rate defined as (shiftIdx&3)+3+shift0, and shift0 defined as (shiftIdx>>2)+2, with shiftIdx indicating a variable.

[0440] 16. The method of claim 14, wherein the faster update rate is between 2 and 5.

[0441] 17. A video processing method, comprising: determining that a transformation between a current video block of a video and a coding representation of the current video block is based on an affine AMVR mode; and performing the transformation based on the determination, wherein the coding representation of the current video block is based on context-based coding, and the context used to code the current video block is modeled using coding information of neighboring blocks in which both affine inter mode and normal inter mode AMVR modes are used during the transformation.

[0442] 18. The method of claim 17, wherein the AMVR mode information of neighboring blocks is used directly for context-based coding.

[0443] 19. The method of claim 17, wherein AMVR mode information of neighboring blocks coded in normal inter mode is not allowed for context-based coding.

[0444] 20. The method according to clause 17, wherein the AMVR mode information of neighboring blocks normally coded in inter mode is not used directly, but indirectly with a function of the AMVR mode information.

[0445] 21. The method of claim 17, wherein AMVR mode information of a neighboring block is used to code a first syntax element of an affine AMVR mode of the current video block.

[0446] 22. The method of claim 17, wherein, when an affine AMVR mode is represented by multiple syntax elements, the method uses AMVR mode information of neighboring blocks coded in both affine inter mode and normal inter mode to code any one of the multiple syntax elements.

[0447] 23. The method of claim 17, wherein when an affine AMVR mode is represented by multiple syntax elements, the AMVR mode information of neighboring blocks coded in normal inter mode is not used to code any one of the multiple syntax elements.

[0448] 24. The method of claim 17, wherein, when an affine AMVR mode is represented by multiple syntax elements, the AMVR mode information of a neighboring block coded in normal inter mode is used indirectly, rather than directly, to code one of the multiple syntax elements.

[0449] 25. A method of video processing, comprising: determining the use of multiple contexts for a conversion between a current video block of a video and a coded representation of the current video block; and performing the conversion based on the determination, wherein the multiple contexts are used to code a syntax element indicating a coarse motion precision.

[0450] 26. The method of claim 25, wherein the multiple contexts correspond to exactly two contexts.

[0451] 27. The method of clause 26, wherein the multiple contexts are selected based on whether the current video block is affine coded.

[0452] 28. The method of claim 25, wherein another syntax element is used to indicate that the transformation is based on an affine AMVR mode.

[0453] 29. The method of claim 28, wherein the another syntax element is coded only in a first context and the syntax element is coded only in a second context.

[0454] 30. The method of claim 28, wherein the other syntax element is coded in a first context only, and the syntax element is bypass coded.

[0455] 31. The method of claim 28, wherein the another syntax element is bypass coded and the syntax element is bypass coded.

[0456] 32. The method of claim 25, wherein all syntax elements related to motion vector precision are bypass coded.

[0457] 33. The method of claim 25 or 26, wherein the syntax element indicates a selection from a set including 1 pixel or 4 pixel precision for normal inter mode.

[0458] 34. The method of claim 25 or 26, wherein the syntax element indicates that in the affine mode, an accuracy selected from at least 1 / 16 pixel or 1 pixel accuracy.

[0459] 35. The method of claim 34, wherein the syntax elements are equal to 0, 1 / 6 pixel precision in the affine mode.

[0460] 36. The method of claim 34, wherein the syntax elements are 1,1 pixel precision in the affine mode.

[0461] 37. The method of any one of clauses 1 to 36, wherein performing the transformation includes generating the coding representation from a current video block.

[0462] 38. The method of any one of clauses 1 to 36, wherein performing the transformation includes generating the current video block from the coding representation.

[0463] 39. A video system comprising a processing device and a non-transitory memory having instructions stored therein, the instructions being implemented by the processing device to cause the processing device to implement a method as recited in any one of clauses 1 to 36.

[0464] 40. A computer program product stored on a non-transitory computer readable medium, comprising program code for carrying out the method according to any one of clauses 1 to 38.

[0465] The second set of clauses uses some of the techniques described in the previous chapter, including, for example, clauses 19-21 and 23 of that chapter.

[0466] 1. A video processing method, comprising: determining whether to use a symmetric motion vector difference (SMVD) mode for a conversion between a current video block of a video and a coding representation of the current video block based on a currently selected best mode for the conversion; and performing the conversion based on the determining.

[0467] 2. The method of claim 1, wherein the SMVD mode is used without explicitly signaling a reference list of at least one reference index.

[0468] 3. The method of claim 2, wherein the reference index is derived based on a recursive Picture Order Count (POC) calculation.

[0469] 4. The method of claim 1, wherein said determining disables use of said SMVD mode if said currently selected best mode is a merge mode or an UMVE mode.

[0470] 5. The method of claim 4, wherein the UMVE mode applies a motion vector offset to refine motion candidates derived from a merge candidate list.

[0471] 6. The method of claim 1, wherein if the currently selected best mode is not coded in the SMVD mode, the determining disables the use of the SMVD mode.

[0472] 7. The method of claim 1, wherein if the currently selected best mode is an affine mode, the determining disables the use of the SMVD mode.

[0473] 8. The method of claim 1, wherein if the currently selected best mode is a sub-block merging mode, the determining disables the use of the SMVD mode.

[0474] 9. The method of claim 1, wherein if the currently selected best mode is an affine SMVD mode, the determining disables the use of the SMVD mode.

[0475] 10. The method of claim 1, wherein if the currently selected best mode is an affine merge mode, the determination disables the use of the SMVD mode.

[0476] 11. The method according to any one of clauses 2 to 10, wherein said determining is applied only if MVD (motion vector difference) accuracy is greater than or equal to accuracy.

[0477] 12. The method of any one of claims 2 to 10, wherein the determining is applied only if the MVD accuracy is greater than the accuracy.

[0478] 13. The method of any one of claims 2 to 10, wherein the determining is applied only if the MVD accuracy is less than or equal to the accuracy.

[0479] 14. The method of any one of claims 2 to 10, wherein the determining is applied only if the MVD accuracy is less than the accuracy.

[0480] 15. A video processing method comprising: determining whether to use an affine SMVD mode for transforming between a current video block of a video and a coding representation of the current video block based on a currently selected best mode for the transform; and performing the transform based on the determining.

[0481] 16. The method of claim 15, wherein the determining disables the use of the affine SMVD mode if the currently selected best mode is a merge mode or an UMVE mode.

[0482] 17. The method of claim 15, wherein if the currently selected best mode is not coded in the affine SMVD mode, the determining disables the use of the affine SMVD mode.

[0483] 18. The method of claim 15, wherein if the currently selected best mode is a sub-block merging mode, the determining disables the use of the affine SMVD mode.

[0484] 19. The method of claim 15, wherein if the currently selected best mode is an SMVD mode, the determining disables the use of the affine SMVD mode.

[0485] 20. The method of claim 15, wherein if the currently selected best mode is an affine merge mode, the determining disables the use of the affine SMVD mode.

[0486] 21. The method of any one of clauses 16 to 20, wherein the determining is applied only if the affine MVD accuracy is greater than or equal to accuracy.

[0487] 22. The method of any one of clauses 16 to 20, wherein the determining is applied only if the affine MVD precision is greater than the accuracy.

[0488] 23. The method of any one of clauses 16 to 20, wherein the determining is applied only if the affine MVD accuracy is less than or equal to the accuracy.

[0489] 24. The method of any one of clauses 16 to 20, wherein the determining is applied only if the affine MVD precision is less than the precision.

[0490] 25. The method of claim 1 or 15, further comprising determining characteristics of the current video block, the characteristics including one or more of a block size of the video block, a slice or picture or tile type of the video block, or motion information associated with the video block, and further comprising: determining characteristics of the current video block, the characteristics including one or more of a block size of the video block, a slice or picture or tile type of the video block, or motion information associated with the video block, and the determining being based on the characteristics.

[0491] 26. A method according to any one of claims 1 to 25, wherein performing the transformation includes generating the coding representation from a current block.

[0492] 27. The method of any one of claims 1 to 25, wherein performing the conversion includes generating the current block from the coding representation.

[0493] 28. A video system comprising a processing device and a non-transitory memory having instructions stored therein, the instructions being implemented by the processing device to cause the processing device to implement a method as recited in any one of clauses 1 to 27.

[0494] 29. A computer program product stored on a non-transitory computer readable medium, comprising program code for executing the method according to any one of clauses 1 to 27.

[0495] While specific embodiments of the disclosed technology have been described above for purposes of illustration, it will be understood that various modifications may be made without departing from the scope of the invention. Accordingly, the disclosed technology is not to be limited except as by the appended claims.

[0496] Implementations of the subject matter and functional operations described in this patent specification may be implemented in various systems, digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. Implementations of the subject matter described herein may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible, non-transportable computer-readable medium for execution by or for controlling the operation of a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that provides a machine-readable propagated signal, or one or more combinations thereof. The term "data processing unit" or "data processing device" includes all apparatus, devices, and machines for processing data, including, for example, a programmable processing device, a computer, or multiple processing devices or computers. In addition to hardware, the apparatus may include code that creates an environment for the execution of the computer program, such as code that constitutes a processing device firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0497] A computer program (also called a program, software, software application, script, or code) can be written in any type of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), may be stored in a single file dedicated to the program, or may be stored in multiple coordinated files (e.g., a file that stores one or more modules, subprograms, or code portions). A computer program can be deployed to run on one computer located at one site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0498] The processes and logic flows described herein may be performed by one or more programmable processing devices executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0499] Processors suitable for executing a computer program include, for example, both general purpose and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Typically, a computer may include one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or may be operatively coupled to receive data from or transfer data to these mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all types of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, flash memory devices, and the like. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0500] This specification, together with the drawings, are for illustrative purposes only, and illustrative is intended to mean example. In this specification, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0501] Although this patent specification contains many details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather as descriptions of features that may be specific to certain embodiments of a particular invention. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in one example. Conversely, various features described in the context of one example may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0502] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desired results, nor should the separation of various system modules in the examples described in this patent document be understood as requiring such separation in all embodiments.

[0503] Only some implementations and examples have been described; other embodiments, extensions and variations are possible based on what is described and illustrated in this patent document.< / leftshift>

Claims

1. 1. A video processing method, comprising: determining at least one syntax element, for conversion between a current video block of a video and a bitstream of the video, wherein at least one bin is context coded with respect to the current video block; determining a context increment value for a first bin of the at least one bin; determining a context variable based on the context increment value; performing said transformation based on at least said context variables; Including, the context variables control two probability update rates associated with the context variables; the at least one syntax element of the bitstream is used to indicate a motion precision in a motion precision set; the context variable is selected from a plurality of context variables for the at least one syntax element based on a coding mode of the current video block; The two probability update rates include a first update rate defined by (shiftIdx>>2)+2, where shiftIdx denotes the context variable; the first update rate is less than or equal to 5; method.

2. a plurality of syntax elements are selectively represented in the bitstream to indicate the motion precision of the motion precision set; the plurality of syntax elements includes a first syntax element and a second syntax element; the second syntax element is present in the bitstream if the first syntax element is present in the bitstream with a particular value; the at least one syntax element includes the first syntax element and / or the second syntax element; The method of claim 1.

3. the context increment value is selected from a plurality of context increment values ​​based on whether the coding mode of the current video block is a first mode; In the first mode, a control point motion vector is derived and further used to derive a sub-region partition motion vector from the current video block. The method of claim 1.

4. the context increment value is selected without using coding information of neighboring video blocks of the current video block; the coding information of the neighboring video block includes a coding mode of the neighboring video block.

4. The method according to any one of claims 1 to 3.

5. The two probability update rates include a second update rate defined by (shiftIdx&3)+3+shift0, where shift0 indicates the first update rate and shiftIdx indicates the context variable.

5. The method according to any one of claims 1 to 4.

6. the first update rate is greater than or equal to 2; The method according to claim 5.

7. the second syntax element indicates a selection from 1-pel, ½-pel, or 4-pel precision for normal inter mode; the second syntax element indicates a selection between 1 / 16-pel or 1-pel precision for a first mode; In the first mode, a control point motion vector is derived and further used to derive a sub-region partition motion vector from the current video block. The method of claim 2.

8. the second syntax element is equal to 0 and the motion precision is 1 / 16-pel precision for a first mode; the second syntax element is equal to 1 and the motion precision is 1-pel precision for a first mode; In the first mode, a control point motion vector is derived and further used to derive a motion vector of a sub-region partition from the current video block. The method of claim 2.

9. performing the conversion includes generating the bitstream based on the current video block.

9. The method according to any one of claims 1 to 8.

10. performing the conversion includes generating the current video block from the bitstream.

9. The method according to any one of claims 1 to 8.

11. 1. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions stored thereon, the apparatus comprising: The instructions, when executed by the processing device, cause the processing device to: determining at least one syntax element, for conversion between a current video block of a video and a bitstream of the video, wherein at least one bin is context coded with respect to the current video block; determining a context increment value for a first bin of the at least one bin; determining a context variable based on the context increment value; performing said transformation based on at least said context variables; Run the command, the context variables control two probability update rates associated with the context variables; the at least one syntax element of the bitstream is used to indicate a motion precision in a motion precision set; the context variable is selected from a plurality of context variables for the at least one syntax element based on a coding mode of the current video block; The two probability update rates include a first update rate defined by (shiftIdx>>2)+2, where shiftIdx denotes the context variable; the first update rate is less than or equal to 5; Device.

12. A non-transitory computer-readable storage medium storing instructions, comprising: The instructions may be for causing a processor to: determining at least one syntax element, for conversion between a current video block of a video and a bitstream of the video, wherein at least one bin is context coded with respect to the current video block; determining a context increment value for a first bin of the at least one bin; determining a context variable based on the context increment value; performing said transformation based on at least said context variables; Run the command, the context variables control two probability update rates associated with the context variables; the at least one syntax element of the bitstream is used to indicate a motion precision in a motion precision set; the context variable is selected from a plurality of context variables for the at least one syntax element based on a coding mode of the current video block; The two probability update rates include a first update rate defined by (shiftIdx>>2)+2, where shiftIdx denotes the context variable; the first update rate is less than or equal to 5; A non-transitory computer-readable storage medium.

13. A method for storing a video bitstream, comprising the steps of: determining, for a current video block of the video, at least one syntax element in which at least one bin is context coded with respect to the current video block; determining a context increment value for a first bin of the at least one bin; determining a context variable based on the context increment value; generating the bitstream based on at least the context variables; storing the bitstream on a non-transitory computer readable recording medium; Including, the context variables control two probability update rates associated with the context variables; the at least one syntax element of the bitstream is used to indicate a motion precision in a motion precision set; the context variable is selected from a plurality of context variables for the at least one syntax element based on a coding mode of the current video block; The two probability update rates include a first update rate defined by (shiftIdx>>2)+2, where shiftIdx denotes the context variable; the first update rate is less than or equal to 5; method.

Citation Information

Cited By

  • Syntax reuse for affine mode with adaptive motion vector resolution

    US12278985B2