Motion vector derivation using higher bit depth precision

Abstract

A method for video processing is provided, comprising: deriving a chroma motion vector for a chroma block associated with a current video block of a video based on unclipped previously calculated luma motion vectors for a plurality of luma blocks; and performing a conversion between the current video block and a bitstream representation of the video based on the chroma motion vector.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a Chinese national phase application of International Patent Application No. PCT / CN2019 / 127010 filed on December 20, 2019, and this application timely claims priority and benefits of International Patent Application No. PCT / CN2018 / 122740 filed on December 21, 2018. The entire disclosure of the above application is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to video coding and decoding technology, equipment and systems. Background Art

[0004] Despite advances in video compression, digital video consumes the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention

[0005] Devices, systems and methods related to digital video coding are described, and in particular, devices, systems and methods related to selecting and applying motion vector ranges for video coding are described. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Codec (HEVC)) and future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: deriving a chroma motion vector for a chroma block associated with a current video block of a video based on unclipped previously calculated luma motion vectors for a plurality of luma blocks; and performing a conversion between the current video block and a bitstream representation of the video based on the chroma motion vector.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: determining a motion vector precision of one or more motion vectors associated with a current video block of a video; selecting a motion vector range based on the motion vector precision; and performing conversion between the current video block and a bitstream representation of the video based on the motion vector range.

[0008] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: determining an internal bit depth using one or more motion vectors associated with a current video block of a video, wherein the internal bit depth has M bits, M is greater than 16, and the current video block uses temporal motion vector prediction; and performing conversion between the current video block and a bitstream representation of the video based on at least the M-bit motion vector.

[0009] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: determining the precision of a motion vector for conversion between a current video block of a video unit of the video and a codec of the video using a motion vector precision value; and performing the conversion based on the precision of the motion vector according to the motion vector precision value, wherein a syntax element at the video unit level indicates the motion vector precision value.

[0010] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: determining parameters for conversion between a current video block of a video and a bitstream representation of the current video block; and performing the conversion based on the parameters, wherein the determined parameters are based on a height or a width of a current picture including the current video block.

[0011] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: clipping a temporary motion vector to generate a clipped motion vector; deriving a motion vector associated with a current video block of a video based at least on the clipped motion vector; and performing a conversion between the current video block of the video and a bitstream representation of the video based on the motion vector associated with the current video block, wherein the clipped temporary motion vector is based on at least one of a height (H) or a width (W) of a reference picture of the current video block, an upper left position (x0, y0) of the current video block, or a motion vector precision or an internal bit depth (M) of the motion vector.

[0012] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: performing a conversion between a current video block of a video and a bitstream representation of the video, wherein the conversion uses a scaling process to obtain a motion vector difference of a first reference picture from a motion vector difference from a second reference picture, and wherein the scaling process uses a shearing operation to restrict the motion vector difference of the first reference picture to fall within a range of [-((1<<(M-1))),(1<<(M-1))-1], where M is a number greater than 16.

[0013] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: determining that a conversion between a current video block of a video unit of a video and a bitstream representation of the video uses an internal bit depth of one or more motion vectors; and performing the conversion using the one or more motion vectors, wherein a syntax element at the video unit level indicates the internal bit depth.

[0014] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: determining that a conversion between a current video block of a video unit of a video and a bitstream representation of the video uses an internal bit depth of affine motion parameters; and performing the conversion based on the determination, wherein the internal bit depth of the affine motion parameters has the same range as non-affine motion parameters.

[0015] In yet another representative aspect, the above method is implemented in the form of processor executable code and stored in a computer readable program medium.

[0016] In yet another representative aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

[0017] In yet another representative aspect, a video decoder device may implement a method as described herein.

[0018] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 An example of constructing a Merge candidate list is shown.

[0020] Figure 2 Examples of locations of spatial candidates are shown.

[0021] Figure 3 An example of candidate pairs that are subject to redundancy checking for spatial-domain Merge candidates is shown.

[0022] Figure 4A and 4B An example of the location of the second prediction unit (PU) based on the size and shape of the current block is shown.

[0023] Figure 5 An example of motion vector scaling of a temporal merge candidate is shown.

[0024] Figure 6 An example of candidate positions of time-domain Merge candidates is shown.

[0025] Figure 7 An example of generating a combined bi-prediction Merge candidate is shown.

[0026] Figure 8 An example of constructing motion vector prediction candidates is shown.

[0027] Fig. 9 An example of motion vector scaling of spatial motion vector candidates is shown.

[0028] Fig.10 An example of motion prediction using an optional temporal motion vector prediction (ATMVP) algorithm for a codec unit (CU) is shown.

[0029] Fig.11 An example of a codec unit (CU) with sub-blocks and neighboring blocks used by a spatio-temporal motion vector prediction (STMVP) algorithm is shown.

[0030] Fig. 12A and 12B Examples of a simplified 4-parameter affine model and a simplified 6-parameter affine model are shown respectively.

[0031] Fig.13 An example of an affine motion vector field (MVF) for each sub-block is shown.

[0032] Fig.14A and 14B Examples are shown for 4-parameter and 6-parameter affine models, respectively.

[0033] Fig.15 An example of motion vector prediction of AF_INTER inheriting affine candidates is shown.

[0034] Fig.16 An example of motion vector prediction of AF_INTER constructing affine candidates is shown.

[0035] Fig.17A and 17B Example candidate blocks and CPMV predictor derivation for AF_Merge mode are shown respectively.

[0036] Fig.18 An example of candidate positions for the affine merge mode is shown.

[0037] Fig.19 An example of a UMVE search process is shown.

[0038] Fig. 20 An example of a UMVE search point is shown.

[0039] Fig.21 A flow chart illustrating an example method of video processing based on some implementations of the disclosed technology is shown.

[0040] Figures 22A to 22E A flow chart illustrating an example method of video processing based on some implementations of the disclosed technology is shown.

[0041] Figures 23A to 23C A flow chart illustrating an example method of video processing based on some implementations of the disclosed technology is shown.

[0042] Fig.24A and24B An example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described herein is shown. DETAILED DESCRIPTION

[0043] Due to the increasing demand for high-resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs generally consist of electronic circuits or software that compress or decompress digital video and are constantly being improved to provide higher coding efficiency. Video codecs convert uncompressed video to a compressed format or vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bitrate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). The compression format usually conforms to a standard video compression specification, such as the High Efficiency Video Codec (HEVC) standard (also known as H.265 or MPEG-H Part 2) [1], the pending Versatile Video Codec (VVC) standard, or other current and / or future video codec standards.

[0044] Embodiments of the disclosed technology can be applied to existing video codec standards (e.g., HEVC, H.265) and future standards to improve runtime performance. Section headings are used herein to improve the readability of the description and do not in any way limit the discussion or embodiments (and / or implementations) to only individual sections.

[0045] 1. Example of inter-frame prediction in HEVC / H.265

[0046] Video codec standards have improved significantly over the years and now partially provide high codec efficiency and support for higher resolutions. Recent standards such as HEVC and H.265 are based on a hybrid video codec structure where temporal prediction plus transform coding is used.

[0047] 1.1 Example of prediction mode

[0048] Each inter-predicted PU (prediction unit) has motion parameters for one or two reference picture lists. In some embodiments, the motion parameters include motion vectors and reference picture indices. In other embodiments, the use of one of the two reference picture lists may also be signaled using inter_pred_idc. In other embodiments, the motion vectors may be explicitly encoded as deltas relative to the predictor.

[0049] When a CU is coded in skip mode, a PU is associated with the CU and has no significant residual coefficients, no coded motion vector increments or reference picture indices. A Merge mode is specified, by which the motion parameters of the current PU can be obtained from neighboring PUs (including spatial and temporal candidates). Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to Merge mode is explicit transmission of motion parameters, where the motion vectors, the corresponding reference picture indices for each reference picture list, and the use of the reference picture lists are explicitly signaled per PU.

[0050] When the signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of samples. This is called "unidirectional prediction". Unidirectional prediction is available for both P slices and B slices [2].

[0051] When signaling indicates that two reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bidirectional prediction". Bidirectional prediction is only available for B slices.

[0052] Reference Image List

[0053] In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of a reference picture instead of the current decoded picture. As in H.264 / AVC, a picture can be predicted from multiple reference pictures. Reference pictures used for inter prediction are organized in one or more reference picture lists. A reference index identifies which reference picture in the list should be used to create a prediction signal.

[0054] A single reference picture list (list 0) is used for P slices, and two reference picture lists (list 0 and list 1) are used for B slices. It should be noted that the reference pictures included in lists 0 / 1 can come from both past and future pictures in capture / display order.

[0055] 1.1.1 Example of Merge Mode Construction Candidate

[0056] When predicting a PU using Merge mode, an index pointing to an entry in the Merge candidate list is parsed from the bitstream and the motion information is retrieved using the index. The construction of this list can be summarized in the following order of steps:

[0057] Step 1: Initial candidate derivation

[0058] Step 1.1: Spatial Candidate Derivation

[0059] Step 1.2: Spatial Candidate Redundancy Check

[0060] Step 1.3: Time Domain Candidate Derivation

[0061] Step 2: Additional candidate insertions

[0062] Step 2.1: Create bidirectional prediction candidates

[0063] Step 2.2: Insert zero motion candidates

[0064] Figure 1 An example of constructing a Merge candidate list based on the order of steps summarized above is shown. For spatial Merge candidate derivation, at most four Merge candidates are selected from candidates located at five different positions. For temporal Merge candidate derivation, at most one Merge candidate is selected from two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates does not reach the maximum number of Merge candidates (maxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share a Merge candidate list, which is the same as the Merge candidate list for a 2N×2N prediction unit.

[0065] 1.1.2 Constructing Spatial Merge Candidates

[0066] In the derivation of spatial merge candidates, Figure 2 A maximum of four Merge candidates are selected from the candidates at the positions shown. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is unavailable (for example, because it belongs to another slice or piece) or is intra-coded. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency.

[0067] In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only pairs with Figure 3 Only pairs linked by arrows in are considered, and candidates are added to the list only if the corresponding candidates for redundancy checking do not have the same motion information. Another source of duplicate motion information is the "second PU" associated with a different partition of 2Nx2N. For example, Figure 4A and 4BThe second PU in the N×2N and 2N×N cases is described respectively. When the current PU is split into N×2N, the candidate at position A1 is not considered for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant for having only one PU in the codec unit. Similarly, when the current PU is split into 2N×N, position B1 is not considered.

[0068] 1.1.3 Constructing Time Domain Merge Candidates

[0069] In this step, only one candidate is added to the list. In particular, in the derivation of this temporal Merge candidate, a scaled motion vector is derived based on the collocated PU belonging to the picture with the minimum POC difference with the current picture in a given reference picture list. The reference picture list used to derive the collocated PU is explicitly signaled in the slice header.

[0070] Figure 5 An example of the derivation of the scaled motion vector of the temporal Merge candidate is shown (as dotted line). It is scaled from the motion vector of the collocated PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set to zero. For B slices, two motion vectors are obtained (one for reference picture list 0 and the other for reference picture list 1) and combined to make it a bidirectional prediction Merge candidate.

[0071] In the collocated PU(Y) belonging to the reference frame, the position of the temporal candidate is selected between candidates C0 and C1, such as Figure 6 As shown. If the PU at position C0 is not available, is intra-coded, or is outside the current CTU, position C1 is used. Otherwise, position C0 is used for the derivation of the time domain Merge candidate.

[0072] 1.1.4 Constructing Merge Candidates of Additional Types

[0073] In addition to the space-time Merge candidates, there are two additional types of Merge candidates: combined bi-directional prediction Merge candidates and zero Merge candidates. Combined bi-directional prediction Merge candidates are generated using space-time Merge candidates. Combined bi-directional prediction Merge candidates are only used for B slices. A combined bi-directional prediction candidate is generated by combining the first reference picture list motion parameters of the initial candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they will form a new bi-directional prediction candidate.

[0074] Figure 7 An example of this process is shown where two candidates in the original list (710, on the left) with mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create a combined bi-predictive Merge candidate that is added to the final list (720, on the right).

[0075] Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list to the capacity of MaxNumMergeCand. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional prediction and bidirectional prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

[0076] 1.2 Embodiment of Advanced Motion Vector Prediction (AMVP)

[0077] AMVP exploits the spatio-temporal correlation of motion vectors with neighboring PUs for explicit transmission of motion parameters. A motion vector candidate list is first constructed by checking the availability of the top left temporal neighboring PU positions, removing redundant candidates, and adding zero vectors to make the candidate list length constant. The encoder can then select the best predictor from the candidate list and send the corresponding index indicating the selected candidate. Similar to Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary. In this case, the maximum value to be encoded is 2 (see Figure 8 ). In the following sections, details on the derivation process of motion vector prediction candidates are provided.

[0078] 1.2.1 Example of deriving AMVP candidates

[0079] Figure 8 The derivation process of motion vector prediction candidates is generalized and can be implemented for each reference picture taking refidx as input.

[0080] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, based on the previously Figure 2 The motion vectors of each PU at the five different positions shown ultimately lead to two motion vector candidates.

[0081] For the derivation of temporal motion vector candidates, one motion vector candidate is selected from two candidates that are derived based on two different collocated positions. After making the first list of spatio-temporal candidates, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates with reference picture index greater than 1 in the associated reference picture list are removed from the list. If the number of spatio-temporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0082] 1.2.2 Constructing spatial motion vector candidates

[0083] When deriving spatial motion vector candidates, at most two candidates are considered among five potential candidates, which are from the previous Figure 2 PUs at the positions shown, which are the same as the positions of the motion merge. The derivation order on the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order on the upper side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, there are four cases on each side that can be used as motion vector candidates, two of which do not require spatial scaling, and two use spatial scaling. The four different cases are summarized as follows:

[0084] --No airspace scaling

[0085] (1) Same reference picture list and same reference picture index (same POC)

[0086] (2) Different reference picture lists, but the same reference picture (same POC)

[0087] --Airspace scaling

[0088] (3) Same reference picture list, but different reference pictures (different POC)

[0089] (4) Different reference picture lists and different reference pictures (different POC)

[0090] First check the case without spatial scaling, then check the case with spatial scaling allowed. Spatial scaling is considered whenever the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, regardless of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, then scaling of the above motion vectors is allowed to help parallel derivation of the left and above MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

[0091] like Fig. 9As shown in the example in , for the case of spatial scaling, the motion vectors of neighboring PUs are scaled in a similar way to temporal scaling. One difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the temporal scaling process.

[0092] 1.2.3 Constructing temporal motion vector candidates

[0093] Except for the derivation of the reference picture index, all derivation processes of the temporal Merge candidate are the same as those of the spatial motion vector candidate (e.g. Figure 6 In some embodiments, the reference picture index is signaled to the decoder.

[0094] 2. Example of inter prediction method in Joint Exploration Model (JEM)

[0095] In some embodiments, a reference software called Joint Exploration Model (JEM) [3][4] is used to explore future video codec technologies. In JEM, sub-block based prediction is used in various codec tools such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), bidirectional optical flow (BIO), frame rate up-conversion (FRUC), local adaptive motion vector resolution (LAMVR), overlapped block motion compensation (OBMC), local illumination compensation (LIC), and decoder-side motion vector refinement (DMVR).

[0096] 2.1 Example of motion vector prediction based on sub-CU

[0097] In JEM with quadtree plus binary tree (QTBT), each CU can have at most one set of motion parameters for each prediction direction. In some embodiments, two sub-CU level motion vector prediction methods are considered in the encoder by dividing a large CU into sub-CUs and deriving motion information of all sub-CUs of the large CU. The optional time domain motion vector prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. The ATMVP method is also called SbTMVP (sub-block time domain motion vector prediction). In the space-time motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using a time domain motion vector predictor and a spatial domain neighboring motion vector. In some embodiments, in order to maintain a more accurate motion field for sub-CU motion prediction, motion compression of the reference frame can be disabled.

[0098] 2.1.1 Example of Alternative Temporal Motion Vector Prediction (ATMVP)

[0099] In the ATMVP method, the temporal motion vector prediction (TMVP) method is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU.

[0100] Fig.10 An example of ATMVP motion prediction processing for CU 1000 is shown. The ATMVP method predicts the motion vector of sub-CU 1001 within CU 1000 in two steps. The first step is to identify the corresponding block 1051 in the reference picture 1050 using the time domain vector. The reference picture 1050 is also called the motion source picture. The second step is to divide the current CU 1000 into sub-CUs 1001 and obtain the motion vector and reference index of each sub-CU from the block corresponding to each sub-CU.

[0101] In the first step, the reference picture 1050 and the corresponding block are determined by the motion information of the spatially adjacent blocks of the current CU 1000. In order to avoid repeated scanning processing of adjacent blocks, the first Merge candidate in the Merge candidate list of the current CU 1000 is used. The first available motion vector and its associated reference index are set to the index of the temporal vector and the motion source picture. In this way, the corresponding block can be identified more accurately compared to TMVP, where the corresponding block (sometimes called a collocated block) is always located at the lower right corner or center relative to the current CU.

[0102] In the second step, the corresponding block of the sub-CU 1051 is identified by the time domain vector in the motion source picture 1050 by adding the time domain vector to the coordinates of the current CU. For each sub-CU, the motion information of the sub-CU is derived using the motion information of its corresponding block (e.g., the minimum motion grid covering the center sample point). After the motion information of the corresponding N×N block is identified, it is converted into the motion vector and reference index of the current sub-CU, the same as the TMVP method of HEVC, where motion scaling and other processing are applied. For example, the decoder checks whether the low latency condition is met (e.g., the POC of all reference pictures of the current picture is less than the POC of the current picture), and may use the motion vector MVx (e.g., the motion vector corresponding to the reference picture list X) to predict the motion vector MVy (e.g., X is equal to 0 or 1 and Y is equal to 1-X) for each sub-CU.

[0103] 2.1.2 Example of Space-Temporal Motion Vector Prediction (STMVP)

[0104] In the STMVP method, the motion vectors of the sub-CUs are recursively derived in raster scan order. Fig.11An example of a CU with four sub-blocks and adjacent blocks is shown. Consider an 8×8 CU 1100, which includes four 4×4 sub-CUs A (1101), B (1102), C (1103) and D (1104). The adjacent 4×4 blocks in the current frame are labeled a (1111), b (1112), c (1113) and d (1114).

[0105] The motion derivation of sub-CU A starts with identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU A1101 (block c 1113). If block c (1113) is not available or is internally coded, check the other N×N blocks above sub-CU A (1101) (from left to right, starting from block c 1113). The second neighbor is a block on the left side of sub-CU A 1101 (block b 1112). If block b (1112) is not available or is internally coded, check the other blocks on the left side of sub-CU A 1101 (from top to bottom, starting from block b 1112). The motion information obtained from the adjacent blocks in each list is scaled to the first reference frame of the given list. Next, the temporal motion vector prediction (TMVP) of sub-block A 1101 is derived according to the same procedure as TMVP specified in HEVC. The motion information of the collocated block at block D 1104 is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors are averaged for each reference list separately. The average motion vector is assigned as the motion vector of the current sub-CU.

[0106] 2.1.3 Example of Sub-CU Motion Prediction Mode Signaling

[0107] In some embodiments, the sub-CU mode is enabled as an additional Merge candidate, and no additional syntax elements are required to signal the mode. Two additional Merge candidates are added to the Merge candidate list of each CU to indicate the ATMVP mode and the STMVP mode. In other embodiments, if the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven Merge candidates can be used. The encoding logic of the additional Merge candidates is the same as that of the Merge candidates in the HM, which means that for each CU in the P or B slice, two additional RD checks may be required for the two additional Merge candidates. In some embodiments, such as JEM, all blocks (bins) of the Merge index are context-coded by CABAC (text-based adaptive binary arithmetic coding). In other embodiments, such as HEVC, only the first block is context-coded, and the remaining blocks are context bypass coded.

[0108] 2.2 Example of inter-frame prediction method in VVC

[0109] There are several new codec tools for inter prediction improvements, such as Adaptive Motion Vector Difference Resolution (AMVR) for signaling MVD, Affine Prediction Mode, Triangular Prediction Mode (TPM), ATMVP, Generalized Bidirectional Prediction (GBI), Bidirectional Optical Flow (BIO).

[0110] 2.2.1 Example of Adaptive Motion Vector Difference Resolution

[0111] In some embodiments, when use_integer_MV_flag is equal to 0 in the slice header, the motion vector difference (MVD) (between the motion vector of the PU and the predicted motion vector) is signaled in units of quarter luma samples. In JEM, local adaptive motion vector resolution (LAMVR) is introduced. In JEM, MVD can be coded and decoded in units of quarter luma samples, integer luma samples, or four luma samples. The MVD resolution is controlled at the codec unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU with at least one non-zero MVD component.

[0112] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or quad luma sample MV precision is used.

[0113] When the first MVD resolution flag of a CU is zero or is not encoded for the CU (meaning that all MVDs in the CU are zero), the CU uses a quarter luma sample MV resolution. When a CU uses integer luma sample MV precision or four luma sample MV precision, the MVP in the AMVP candidate list of the CU will be rounded to the corresponding precision.

[0114] 2.2.2 Example of Affine Motion Compensated Prediction

[0115] In HEVC, only the translation motion model is applied to motion compensation prediction (MCP). However, the camera and the object may have various motions, such as zooming in / out, rotation, perspective motion, and / or other irregular motions. In VVC, a simplified affine transformation motion compensation prediction with a 4-parameter affine model and a 6-parameter affine model is adopted. Fig. 12A and 12B As shown, the affine motion field of the block is described by two (in a 4-parameter affine model using variables a, b, e and f) or three (in a 6-parameter affine model using variables a, b, c, d, e and f) control point motion vectors, respectively.

[0116] The motion vector field (MVF) of a block is described by the following equations with a 4-parameter affine model and a 6-parameter affine model respectively:

[0117]

[0118]

[0119] Here, (mv h 0,mv h 0) is the motion vector of the upper left control point (CP), and (mv h 1,mv h 1) is the motion vector of the upper right control point, and (mv h 2,mv h 2) is the motion vector of the lower left control point, and (x, y) represents the coordinates of the representative point relative to the upper left sample point in the current block. The CP motion vector can be signaled (such as in the affine AMVP mode) or derived on the fly (such as in the affine Merge mode). w and h are the width and height of the current block. In practice, the division is implemented by a right shift operation with rounding. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample point in the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2). For each sub-block (for example, 4×4 in VTM), the representative point is used to derive the motion vector of the entire sub-block.

[0120] Fig.13 An example of an affine MVF for each sub-block of block 1300 is shown, where in order to further simplify motion compensated prediction, a sub-block based affine transform prediction is applied. To derive the motion vector of each M×N sub-block, the motion vector of the center sample of each sub-block can be calculated according to equations (1) and (2) and rounded to the motion vector fractional precision (e.g., 1 / 16 in JEM). A motion compensated interpolation filter is then applied to generate predictions for each sub-block using the derived motion vectors. An interpolation filter of 1 / 16 pixels is introduced through the affine mode. After MCP, the high-precision motion vector of each sub-block is rounded and saved to the same precision as the regular motion vector.

[0121] 2.2.3 Example of signaling for affine prediction

[0122] Similar to the translational motion model, affine prediction also has two modes for signaling incidental information. They are AFFINE_INTER mode and AFFINE_MERGE mode.

[0123] 2.2.4 Example of AF_INTER mode

[0124] AF_INTER mode can be applied for CUs with width and height both greater than 8. In the bitstream, the affine flag at the CU level is signaled to indicate whether AF_INTER mode is used.

[0125] In this mode, for each reference picture list (list 0 or list 1), an affine AMVP candidate list is constructed with three types of affine motion predictors in the following order, where each candidate includes the estimated CPMV of the current block. The difference between the best CPMV found at the encoder side (e.g. Fig.16 In addition, the index of the affine AMVP candidate from which the estimated CPMV is derived is further signaled.

[0126] 1) Inherited affine motion predictor

[0127] The checking order is similar to that of the spatial MVP in the HEVC AMVP list construction. First, the left-inherited affine motion predictor is derived from the first block in {A1, A0}, which is affine-coded and has the same reference picture as the current block. Second, the above-inherited affine motion predictor is derived from the first block in {B1, B0, B2}, which is affine-coded and has the same reference picture as the current block. Fig.15 Five blocks A1, A0, B1, B0, B2 are depicted.

[0128] Once the neighboring block is found to be affine coded, the CPMV of the codec covering the neighboring block is used to derive the predictor of the CPMV of the current block. For example, if A1 is coded in non-affine mode and A0 is coded in 4-parameter affine mode, the left-inherited affine MV predictor will be derived from A0. In this case, the CPMV of the CU covering A0 (e.g. Fig. 17B For the upper left CPMV Indicates that, and for the upper right CPMV denoted) is used to derive the estimated CPMV of the current block (for the top left (coordinates (x0, y0), top right (coordinates (x1, y1) and bottom right (coordinates (x2, y2) positions of the current block, denoted as )).

[0129] 2) Constructed affine motion predictor

[0130] like Fig.16As shown, the constructed affine motion predictor consists of control point motion vectors (CPMVs) derived from adjacent inter-frame codec blocks with the same reference picture. If the current affine motion model is 4-parameter affine, the number of CPMVs is 2; otherwise, if the current affine motion model is 6-parameter affine, the number of CPMVs is 3. Top left CPMV Derived from the MV at the first block in the group {A, B, C}, which is inter-coded and has the same reference picture as the current block. Top right CPMV Derived from the MV at the first block in group {D, E}, which is inter-coded and has the same reference picture as the current block. Bottom left CPMV Derived by the MV at the first block in group {F, G}, which is inter-coded and has the same reference picture as the current block.

[0131] – If the current affine motion model is 4-parameter affine, only when and Both are found (that is, and When used as the estimated CPMV of the upper left (coordinates (x0, y0)) and upper right (coordinates (x1, y1)) positions of the current block, the constructed affine motion predictor is inserted into the candidate list.

[0132] – If the current affine motion model is a 6-parameter affine, only when and are found (that is, and When used as the estimated CPMV of the upper left (coordinate (x0, y0)), upper right (coordinate (x1, y1)) and lower right (coordinate (x2, y2)) positions of the current block, the constructed affine motion predictor is inserted into the candidate list.

[0133] When inserting the constructed affine motion predictor into the candidate list, no pruning process is applied.

[0134] 3) Conventional AMVP motion predictor

[0135] The following applies until the number of affine motion predictors reaches a maximum value.

[0136] 1) By setting all CPMVs equal to (if available) to derive the affine motion predictor.

[0137] 2) By setting all CPMVs equal to (if available) to derive the affine motion predictor.

[0138] 3) By setting all CPMVs equal to (if available) to derive the affine motion predictor.

[0139] 4) Derive the affine motion predictor by setting all CPMVs equal to HEVC TMVP (if available).

[0140] 5) Derive the affine motion predictor by setting all CPMVs equal to zero MV.

[0141] Note that we have derived the affine motion predictor in the construction

[0142] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required, and thus 2 / 3 MVDs need to be encoded and decoded for these control points, such as Fig.14A and 14B In the existing implementation [5], MV can be derived as follows, for example, predicting mvd1 and mvd0 from mvd1

[0143]

[0144] in, mvd i and mv1 are the predicted motion vector, motion vector difference and motion vector of the upper left pixel (i=0), upper right pixel (i=1) or lower left pixel (i=2), respectively. Fig. 14B In some embodiments, the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the sum of the two components. For example, newMV=mvA+mvB means setting the two components of newMV to (xA+xB) and (yA+yB), respectively.

[0145] 2.2.5 Example of AF_MERGE mode

[0146] When a CU is applied in AF_MERGE mode, it obtains the first block encoded and decoded in affine mode from the valid adjacent reconstructed blocks. And the selection order of candidate blocks is from left, top, top right, bottom left to top left, such as Fig.17A As shown in (in turn represented as A, B, C, D, E). For example, if the adjacent lower left block is encoded and decoded using the affine mode (such as Fig. 17B The control point (CP) motion vector mv0 of the upper left, upper right and lower left of the adjacent CU / PU containing block A is extracted. N 、mv1 N and mv2 N . And based on mv0 N 、mv1 N and mv2N Calculate the current CU / PU upper left / upper right / lower left motion vector mv0 C 、mv1 C and mv2 C (Only for 6-parameter affine model), it should be noted that in VTM-2.0, if the current block is affine coded, the sub-block at the upper left corner (for example, 4×4 block in VTM) stores mv0, and the sub-block at the upper right corner stores mv1. If the current block is coded with a 6-parameter affine model, the sub-block at the lower left corner stores mv2; otherwise (using a 4-parameter affine model), LB stores mv2'. Other sub-blocks store MVs for MC.

[0147] After calculating the CPMV v0 and v1 of the current CU, the MVF of the current CU is generated according to the affine motion model in equations (1) and (2). In order to identify whether the current CU is encoded or decoded in AF_MERGE mode, when at least one neighboring block is encoded or decoded in affine mode, the affine flag is signaled in the bitstream.

[0148] In some embodiments (e.g., JVET-L0142 and JVET-L0632), the following steps are used to construct an affine merge candidate:

[0149] 1) Insert inherited affine candidates

[0150] An inherited affine candidate is one that is derived from the affine motion model of its valid neighboring affine codec block. The two largest affine candidates are derived from the affine motion model of the neighboring block and inserted into the candidate list. For the left predictor, the scanning order is {A0, A1}; for the top predictor, the scanning order is {B0, B1, B2}.

[0151] 2) Insert the constructed affine candidate

[0152] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (set to 5 in this paper), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate means that the candidate is constructed by combining the adjacent motion information of each control point.

[0153] a) First, Fig.18 The prescribed spatial and temporal neighbors shown derive the motion information of the control point. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial positions used to predict CPk (k = 1, 2, 3); T is the temporal position used to predict CP4.

[0154] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.

[0155] Get the motion information for each control point in the following order of priority:

[0156] For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is not available, use B3. If both B2 and B3 are not available, use A2. If none of the three candidates are available, the motion information of CP1 cannot be obtained.

[0157] For CP2, the check priority is B1->B0;

[0158] For CP3, the check priority is A1->A0;

[0159] For CP4, use T.

[0160] b) Secondly, the combination of control points is used to construct affine merge candidates.

[0161] I. Motion information of three control points is required to construct a 6-parameter affine candidate. These three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted to a 6-parameter motion model represented by the top left, top right and bottom left control points.

[0162] II. Motion information of two control points is required to construct a 4-parameter affine candidate. The two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). The combination {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4} will be converted to a 4-parameter motion model represented by the top left and top right control points.

[0163] III. Insert the combinations of constructed affine candidates into the candidate list in the following order:

[0164] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}.

[0165] i. For a combined reference list X (X is 0 or 1), select the reference index with the highest usage rate in the control point as the reference index of list X, and scale the motion vectors pointing to different reference pictures.

[0166] After deriving the candidates, a full pruning process is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate is discarded.

[0167] 3) Fill zero motion vectors

[0168] If the number of candidates in the affine merge candidate list is less than 5, zero motion vectors with zero reference index are inserted into the candidate list until the list is full.

[0169] More specifically, for the sub-block Merge candidate list, the MV of the 4-parameter Merge candidate is set to (0, 0) and the prediction direction is set to unidirectional prediction (for P slices) and bidirectional prediction (for B slices) from list 0.

[0170] 2.2.6 Example of sub-block sizes for chroma affine codec blocks

[0171] In the proposal JVET-L0265 adopted by VTM 3.0, the size of the chroma subblock is modified from 2x2 to 4x4. The MV of a 4×4 chroma subblock is calculated as the average of the MVs of the four corresponding 4×4 luma subblocks. With the extended subblock size, the worst-case bandwidth is reduced; for example,

[0172] MV_Chroma=Average(clipped(MV_luma0)+clipped(MV_luma1)+clipped(MV_luma2)+clipped(MV_luma3))

[0173] 2.4 Example of Merge with Motion Vector Difference (MMVD)

[0174] In JVET-L0054, the final motion vector representation (UMVE, also called MMVD) is proposed. UMVE and the proposed motion vector representation method are used in Skip or Merge mode.

[0175] UMVE reuses the same Merge candidates as those contained in the regular Merge candidate list in VVC. Among the Merge candidates, a base candidate can be selected and further extended by the proposed motion vector expression method.

[0176] UMVE provides a new method to represent motion vector difference (MVD), in which the starting point, motion amplitude and motion direction are used to represent the MVD.

[0177] The proposed technique uses the Merge candidate list as is. However, only candidates of the default Merge type (MRG_TYPE_DEFAULT_N) are considered for UMVE extension.

[0178] The base candidate index defines the starting point. The base candidate index indicates the best candidate among the candidates in the list, as shown below.

[0179] Table 1 Basic candidate IDX

[0180]

[0181] If the number of base candidates is equal to 1, the base candidate IDX is not signaled.

[0182] The distance index is the motion amplitude information. The distance index indicates the predefined distance from the start point information. The predefined distances are as follows:

[0183] Table 2 Distance IDX

[0184]

[0185] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions as shown below.

[0186] Table 3 Direction IDX

[0187]

[0188]

[0189] In some embodiments, the UMVE flag is signaled immediately after the Skip flag or Merge flag is sent. If the Skip or Merge flag is true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. However, if it is not 1, the Affine flag is parsed. If the Affine flag is equal to 1, it is Affine mode, but if it is not 1, the Skip / Merge index is parsed for the Skip / Merge mode of the VTM.

[0190] Additional line buffers due to UMVE candidates are not needed, since software's Skip / Merge candidates are used directly as base candidates. By using the input UMVE index, the complement of the MV is determined before motion compensation. No long line buffers need to be kept for this.

[0191] Under the current common test conditions, the first or second Merge candidate in the Merge candidate list can be selected as the basic candidate.

[0192] 2.5 Details of MVD derivation process

[0193] The following text is from JVET-L1001-v6.

[0194] The brightness Merge motion vector differences mMvdL0 and mMvdL1 are derived as follows:

[0195] – If both predFlagL0 and predFlagL1 are equal to 1, the following applies:

[0196] currPocDiffL0=DiffPicOrderCnt(currPic, RefPicList0[refIdxL0])(8-272)

[0197] currPocDiffL1=DiffPicOrderCnt(currPic, RefPicList1[refIdxL1])(8-273)

[0198] – If -currPocDiffL0*currPocDiffL1 is greater than 0, the following applies:

[0199] mMvdL0[0]=MmvdOffset[0] (8-274)

[0200] mMvdL0[1]=MmvdOffset[1] (8-275)

[0201] mMvdL1[0]=-MmvdOffset[0] (8-276)

[0202] mMvdL1[1]=-MmvdOffset[1] (8-277)

[0203] – Otherwise (–currPocDiffL0*currPocDiffL1 is less than 0), the following applies:

[0204] mMvdL0[0]=MmvdOffset[0] (8-278)

[0205] mMvdL0[1]=MmvdOffset[1] (8-279)

[0206] mMvdL1[0]=MmvdOffset[0] (8-280)

[0207] mMvdL1[1]=MmvdOffset[1] (8-281)

[0208] – If Abs(currPocDiffL0) is greater than Abs(currPocDiffL1), then the following applies:

[0209] td=Clip3(-128,127,currPocDiffL0) (8-282)

[0210] tb=Clip3(-128,127,currPocDiffL1) (8-283)

[0211] tx=(16384+(Abs(td)>>1)) / td (8-284)

[0212] distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6)(8-285)

[0213] mMvdL1[0]=Clip3(-2 15 ,2 15 -1,Sign(distScaleFactor*

[0214] mMvdL1[0])* ((Abs(distScaleFactor*mMvdL1[0])+127)>>8))(8-286)

[0215] mMvdL1[1]=Clip3(-2 15 ,2 15 -1,Sign(distScaleFactor*

[0216] mMvdL1[1])* ((Abs(distScaleFactor*mMvdL1[1])+127)>>8))(8-287)

[0217] – Otherwise, if Abs(currPocDiffL0) is less than Abs(currPocDiffL0), then the following applies:

[0218] td=Clip3(-128,127,currPocDiffL1) (8-288)

[0219] tb=Clip3(-128,127,currPocDiffL0) (8-289)

[0220] tx=(16384+(Abs(td)>>1)) / td (8-290)

[0221] distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6)(8-291)

[0222] mMvdL0[0]=Clip3(-2 15 ,2 15 -1,Sign(distScaleFactor*

[0223] mMvdL0[0])*

[0224] ((Abs(distScaleFactor*mMvdL0[0])+127)>>8))

[0225] (8-292)

[0226] mMvdL0[1]=Clip3(-2 15 ,2 15 -1,Sign(distScaleFactor*

[0227] mMvdL0[1])*

[0228] ((Abs(distScaleFactor*mMvdL0[1])+127)>>8))

[0229] (8-293)

[0230] – Otherwise (predFlagL0 or predFlagL1 equals 1), the following applies if X is 0 or 1:

[0231] mMvdLX[0]=(predFlagLX==1)? MmvdOffset[0]:0(8-294)

[0232] mMvdLX[1]=(predFlagLX==1)? MmvdOffset[1]:0(8-295)

[0233] 3. Defects of existing implementation

[0234] In existing implementations (e.g., VTM3.0), the original limit on the MV range is meaningless for 8K resolution video due to the increase in MV precision to 1 / 16 pixel. In addition, for 4x4 sub-PU prediction (i.e., affine mode), the minimum chroma sub-block size is not limited to 4x4. The four luma motion vectors are averaged together to form a single motion vector for the corresponding chroma 4x4 sub-block. By clipping the luma motion vector before averaging to form the chroma vector (clipping then averaging), each decoder must implement this clipping to obtain a match even for most cases where the MV does not exceed the restricted range.

[0235] 4. Example Method for Selecting and Applying Motion Vector Ranges for Video Codecs

[0236] Embodiments of the presently disclosed technology overcome the shortcomings of existing implementations, thereby providing a video codec with higher codec efficiency. Based on the disclosed technology, the selection and application of motion vector ranges for video codecs that can enhance both existing and future video codec standards are illustrated in the following examples described for various implementations. The examples of the disclosed technology provided below illustrate the general concepts and are not intended to be interpreted as limiting. In the examples, the various features described in these examples may be combined unless expressly indicated to the contrary. It should be noted that some of the proposed techniques may be applied to existing candidate list construction processes.

[0237] Example 1 .It is proposed to use the initially derived luma motion vectors (e.g., unclipped) of multiple luma blocks to derive the chroma motion vector of a chroma block. A "block" can be defined as a prediction unit, a codec unit, a sub-block used in a prediction / codec unit encoded and decoded using sub-block based techniques (e.g., affine, ATMVP). In some examples, "luminance" and "chroma" can be interpreted as the Y color component and the Cb / Cr color component. In some examples, "luminance" and "chroma" can be interpreted as the G color component and the B / R color component. In the following examples, "sub-block" can also be replaced by the term "block".

[0238] (a) In one example, how the motion vectors of the chroma components of a sub-block are derived depends on the color format such as 4:2:0, 4:2:2, or 4:4:4.

[0239] i. In one example, the motion vectors of the four M x N luma blocks are used to derive the motion vectors of the M x N chroma blocks. Alternatively, in addition, it can be called under the condition of 4:2:0 color subsampling format.

[0240] ii. In one example, the motion vector of the M x N chroma block is derived using the motion vectors of the two M x N luminance blocks. Alternatively, in addition, it can be called under the condition of 4:2:2 color subsampling format.

[0241] iii. In one example, the motion vector of one M x N luma block is used as the motion vector of the M x N chroma block. Alternatively, in addition, it can be called in 4:4:4 color format.

[0242] (b) In one example, a linear function of multiple luma motion vectors may be applied to derive chroma motion vectors.

[0243] (c) Alternatively, additionally, the derived chroma motion vectors may be further clipped before being used for motion compensation of the chroma blocks.

[0244] (d) The above method may be applied to sub-block based techniques such as affine, ATMVP, STMVP, etc. Alternatively, further, M x N may represent a sub-block size, and multiple luma / chroma blocks may represent multiple luma sub-blocks and one chroma sub-block.

[0245] (e) In one example, for affine mode, MV_Chroma may be set to the average of four corresponding luma blocks, such as Average(MV_luma0+MV_luma1+MV_luma2+MV_luma3).

[0246] (f) The above-mentioned luminance / chrominance components may refer to Y / Cb / Cr or other color representations, such as G / R / B.

[0247] Example 2 .It is proposed to limit the MV range based on MV accuracy.

[0248] (a) In one example, different MV ranges may be defined based on the use of AMVR, and it may be different for different profiles / levels.

[0249] (b) For different modes (such as affine or non-affine), the MV range can be different.

[0250] (c) When the restriction range is different, clipping of motion vectors can be applied using different input parameters (ie, based on the range).

[0251] Example 3 .The internal bit depth of motion vectors can be increased to M bits (e.g., M>16).

[0252] (a) In one example, for AMVP processing, when a motion vector prediction candidate is derived from a motion vector associated with a reference picture different from a target reference picture, each component of the motion vector may be scaled to the target reference picture in the range [-((1<<(M-1)), (1<<(M-1))-1]. In one implementation, M is set to 18, and the range [-((1<<(M-1)), (1<<(M-1))-1] is [-131072, 131071].

[0253] (b) In one example, for TMVP processing of either AMVP / Merge mode and / or ATMVP / STMVP, when deriving a motion vector prediction candidate from a motion vector associated with a temporal reference picture from a target reference picture, the motion vector can be scaled to a target reference picture in the range of [-((1<<(M-1))),(1<<(M-1))-1].

[0254] (c) In one example, for the MVD derivation process of MMVD, when the MVD of one reference picture list is scaled to another reference picture list, the scaled MVD may be limited to the range of [-((1<<(M-1))),(1<<(M-1))-1].

[0255] (d) In one example, the above method may be called when affine is enabled for a sequence / picture / strip / tile.

[0256] Example 4 .MV precision (e.g., for MV storage, internal MV precision) can be signaled from the encoder to the decoder in VPS / PPS / slice header / slice group / slice / CTU row / CTU.

[0257] (a) In different standard profiles / levels / layers, MV accuracy can be different.

[0258] Example 5. The internal bit depth of the MV may be signaled from the encoder to the decoder in VPS / PPS / slice header / slice group / slice / CTU row / CTU.

[0259] (a) In different standard profiles / levels / layers, the internal bit depth of the MV can be different.

[0260] Example 6 .MV precision can depend on picture width and / or height.

[0261] Example 7. The internal bit depth of a MV may depend on the picture width and / or height.

[0262] Example 8. In one example, the clipping of the MV component may depend on the dimensions of the reference picture, the top-left position of the current block, the MV precision, and the internal bit depth of the MV.

[0263] (a) For example, assume that the width and height of the reference picture are W and H, and the top-left position of the current block is (x0, y0), and the MV precision is 1 / (1<<Shift) sub-pixels (e.g., 1 / 16 sub-pixels when Shift equals 4). CW and CH are the maximum width and maximum height of the CU / PU / video data processing unit, respectively. Then

[0264] i. PicMinH = (-x0 + 1 + Offset1) << Shift; or PicMinH = (-x0 + Offset1) << Shift;

[0265] ii. PicMaxH = (W - 1 - x0 + Offset2) << Shift; or PicMaxH = (W - x0 + Offset2) << Shift;

[0266] iii. PicMinV = (-y0 + 1 + Offset3) << Shift; or PicMinV = (-y0 + Offset3) << Shift;

[0267] iv. PicMaxV = (H - 1 - y0 + Offset4) << Shift. or PicMaxV = (H - y0 + Offset4) << Shift.

[0268] Here, Offset1, Offset2, Offset3, and Offset4 are offsets that may depend on the padding method. For example, Offset1 = -CW - 8, Offset3 = -CH - 8, Offset2 = Offset4 = 8.

[0269] Then

[0270] MVMinH = max(PicMinH, -((1<<(M - 1)))),

[0271] MVMinV = max(PicMinV, -((1<<(M - 1)))),

[0272] MVMaxH = min(PicMaxH, ((1<<(M - 1))) - 1), and

[0273] MVMaxV = min(PicMaxV, ((1<<(M - 1))) - 1).

[0274] The x-component of MV may be clipped to be within the range [MVMinH, MVMaxH], and the y-component of MV may be clipped to be within the range [MVMinV, MVMaxV].

[0275] Example 9. The internal bit depth of affine motion vectors / affine motion vector predictors / control point motion vectors can be restricted to the same range as non-affine motion vectors.

[0276] (a) In one example, the control point motion vectors can be constrained to be in the range [-2 15 ,2 15 –1].

[0277] The above examples may be incorporated into the context of the methods described below, for example, methods 2100 and 2200 , which may be implemented at a video decoder or a video encoder.

[0278] Fig.21 A flow chart of an exemplary method for video processing is shown. The method 2100 includes: at step 2110, deriving a chroma motion vector for a chroma block associated with a current video block of a video based on unclipped previously calculated luma motion vectors for a plurality of luma blocks. The method 2100 includes: at step 2120, performing a conversion between the current video block and a bitstream representation of the video based on the chroma motion vector.

[0279] Fig.22A A flow chart of an exemplary method for video processing is shown. The method 2200 includes: at step 2202, determining a motion vector accuracy of one or more motion vectors associated with a current video block of a video. The method 2200 includes: at step 2204, selecting a motion vector range based on the motion vector accuracy. The method 2200 includes: at step 2206, performing a conversion between the current video block and a bitstream representation of the video based on the motion vector range.

[0280] Fig. 22B A flow chart of an exemplary method for video processing is shown. The method 2210 includes: at step 2212, determining an internal bit depth using one or more motion vectors associated with a current video block of a video, wherein the internal bit depth has M bits, M is greater than 16, and the current video block uses temporal motion vector prediction. The method 2210 also includes: at step 2214, performing conversion between the current video block and a bitstream representation of the video based on at least the M-bit motion vector.

[0281] Fig. 22CA flow chart of an exemplary method for video processing is shown. The method 2220 includes: at step 2222, determining the precision of a motion vector for conversion between a current video block of a video unit of the video and an encoded representation of the video using a motion vector precision value. The method 2220 also includes: at step 2224, performing the conversion based on the precision of the motion vector according to the motion vector precision value. In some implementations, a syntax element at the video unit level indicates the motion vector precision value.

[0282] Fig.22D A flow chart of an exemplary method for video processing is shown. The method 2230 includes: at step 2232, determining parameters for conversion between a current video block of a video and a bitstream representation of the current video block. The method 2230 also includes: at step 2234, performing the conversion based on the parameters. In some implementations, determining the parameters is based on the height or width of a current picture that includes the current video block.

[0283] Fig.22E A flow chart of an exemplary method for video processing is shown. The method 2240 includes: at step 2242, clipping a temporary motion vector to generate a clipped motion vector. The method 2240 also includes: at step 2244, deriving a motion vector associated with a current video block of the video based on at least the clipped motion vector. The method 2240 also includes: at step 2246, performing a conversion between the current video block of the video and a bitstream representation of the video based on the motion vector associated with the current video block. In some implementations, the clipped temporary motion vector is based on at least one of a height (H) or a width (W) of a reference picture of the current video block, an upper left position (x0, y0) of the current video block, or a motion vector precision or an internal bit depth (M) of the motion vector.

[0284] Fig.23A A flow chart of an exemplary method for video processing is shown. The method 2300 includes: at step 2302, performing a conversion between a current video block of a video and a bitstream representation of the video, wherein the conversion uses a scaling process to obtain a motion vector difference of a first reference picture from a motion vector difference from a second reference picture, and wherein the scaling process uses a shearing operation to limit the motion vector difference of the first reference picture to fall within the range [-((1<<(M-1))),(1<<(M-1))-1], where M is a number greater than 16.

[0285] Fig. 23BA flow chart of an exemplary method for video processing is shown. The method 2310 includes: at step 2312, determining that a conversion between a current video block of a video unit of a video and a bitstream representation of the video uses an internal bit depth of one or more motion vectors. The method 2310 also includes: at step 2314, performing the conversion using the one or more motion vectors. In some implementations, a syntax element at the video unit level indicates the internal bit depth.

[0286] Fig.23C A flow chart of an exemplary method for video processing is shown. The method 2320 includes: at step 2322, determining that a conversion between a current video block of a video unit of a video and a bitstream representation of the video uses an internal bit depth of affine motion parameters. The method 2320 also includes: at step 2324, performing the conversion based on the determination, wherein the internal bit depth of the affine motion parameters has the same range as the non-affine motion parameters.

[0287] 5. Example Implementation of Disclosed Technology

[0288] In some embodiments, modifications in the JVET-L1001-v6 specification are shown below, with \begin{DELETE} and \end{ADD} markers indicating deletions and additions, respectively.

[0289] 8.3.3 Derivation of sub-block motion vector components and reference indices

[0290] 8.3.3.1 Overview

[0291] The input to this process is:

[0292] – The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0293] – The variable cbWidth specifies the width of the current codec block in luma samples,

[0294] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0295] The output of this processing is:

[0296] – reference indices refIdxL0 and refIdxL1,

[0297] – Number of luminance codec sub-blocks numSbX and numSbY in the horizontal and vertical directions,

[0298] – The prediction list uses the flag arrays predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx], where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbX-1,

[0299] – Luma sub-block motion vector arrays mvL0[xSbIdx][ySbIdx] and mvL1[xSbIdx][ySbIdx] with 1 / 16 fractional sampling accuracy, where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1,

[0300] – Chroma sub-block motion vector arrays mvCL0[xSbIdx][ySbIdx] and mvCL1[xSbIdx][ySbIdx] with 1 / 32 fractional sampling precision, where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1,

[0301] – Bidirectional prediction weighted index gbiIdx.

[0302] For the derivation of the variables mvL0[xSbIdx][ySbIdx], mvL1[xSbIdx][ySbIdx], mvCL0[xSbIdx][ySbIdx], and mvCL1[xSbIdx][ySbIdx]refIdxL0refIdxL1numSbXnumSbYpredFlagL0[xSbIdx][ySbIdx], and predFlagL1[xSbIdx][ySbIdx], the following applies:

[0303] – If merge_subblock_flag[xCb][yCb] is equal to 1, the luminance codec block position (xCb, yCb), luminance codec block width cbWidth and luminance codec block height cbHeight are used as input, the number of luminance codec sub-blocks numSbX and numSbY in the horizontal and vertical directions, the reference index refIdxL0, refIdxL1, the prediction list uses the flag array predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbId x], luminance sub-block motion vector arrays mvL0[xSbIdx][ySbIdx] and mvL0[xSbIdx][ySbIdx], and chrominance sub-block motion vector arrays mvCL0[xSbIdx][ySbIdx] and mvCL1[xSbIdx][ySbIdx] (where xSbIdx = 0..numBx-1, ySbIdx = 0..numSbY-1), and bidirectional prediction weighted index gbiIdx as output, calling the derivation process of motion vector and reference index in the sub-block Merge mode specified in 8.3.3.2.

[0304] – Otherwise (merge_subblock_flag[xCb][yCb] is equal to 0), since X in the variables predFlagLX, cpMvLX, MvdCpLX, and refIdxLX, and in the syntax element ref_idx_LX is replaced by 0 or 1, the following sequence of steps applies:

[0305] – For the derivation of the number of control point motion vectors numCpMv, the control point motion vectors cpMvLX[cpIdx] (where cpIdx is in the range from 0 to numCpMv–1), refIdxLX, predFlagLX[0][0], the following applies:

[0306] 1. Set the number of control point motion vectors numCpMv equal to MotionModelIdc[xCb][yCb]+1.

[0307] 2. The variables refIdxLX and predFlagLX are derived as follows:

[0308] - If inter_pred_idc[xCb][yCb] is equal to PRED_LX or PRED_BI, then

[0309] refIdxLX=ref_idx_lX[xCb][yCb] (8-347)

[0310] predFlagLX[0][0]=1 (8-348)

[0311] Otherwise, the variables refIdxLX and predFlagLX are specified by:

[0312] refIdxLX=-1 (8-349)

[0313] predFlagLX[0][0]=0 (8-350)

[0314] 3. The variable cpMvLX[cpIdx] (where cpIdx is in the range from 0 to numCpMv–1) is derived as follows:

[0315] mvdCpLX[cpIdx][0]=MvdCpLX[xCb][yCb][cpIdx][0](8-351)

[0316] mvdCpLX[cpIdx][1]=MvdCpLX[xCb][yCb][cpIdx][1](8-352)

[0317] 4. When predFlagLX[0][0] is equal to 1, the luma codec block position (xCb, yCb), as well as the variables cbWidth, cbHeight, refIdxLX and the number of control point motion vectors numCpMv are used as input, and the output is mvpCpLX[cpIdx], where cpIdx is in the range from 0 to numCpMv-1, and the derivation process of the luma affine control point motion vector predictor specified in Section 8.3.3.7 is called.

[0318] 5. When predFlagLX[0][0] is equal to 1, the luma motion vector mvpCpLX[cpIdx] (where cpIdx is in the range from 0 to numCpMv-1) is derived as follows:

[0319]

[0320]

[0321] – The variables numSbX and numSbY are derived as follows:

[0322] numSbX=(cbWidth>>2) (8-357)

[0323] numSbY=(cbHeight>>2) (8-358)

[0324] – For xSbIdx=0..numSbX-1, ySbIdx=0..numSbY-1, the following applies:

[0325] predFlagLX[xSbIdx][ySbIdx]=predFlagLX[0][0] (8-359)

[0326] – When predFlagLX[0][0] is equal to 1, the process of deriving motion vector arrays from affine control point motion vectors as specified in subclause 8.3.3.9 is called with as input the luma codec block position (xCb, yCb), the luma codec block width cbWidth, the luma prediction block height cbHeight, the number of control point motion vectors numCpMv, the control point motion vector cpMvLX[cpIdx] (cpIdx is 0..2), the number of luma codec sub-blocks numSbX and numSbY in the horizontal and vertical directions, and as output the luma motion vector array mvLX[xSbIdx][ySbIdx] and the chroma motion vector array mvCLX[xSbIdx][ySbIdx] (where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1).

[0327] – Set the bidirectional prediction weight index gbiIdx equal to gbi_idx[xCb][yCb].

[0328] Fig.24A 2400 is a block diagram of a video processing device 2400. The device 2400 may be used to implement one or more methods described herein. The device 2400 may be implemented in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 2400 may include one or more processors 2402, one or more memories 2404, and video processing hardware 2406. The processor 2402 may be configured to implement the methods described herein. Figures 21 to 23C Memory 2404 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 2406 may be used to implement some of the techniques described herein in hardware circuits.

[0329] Fig. 24B is another example of a block diagram of a video processing system that may implement the disclosed technology. Fig. 24B3100 is a block diagram illustrating an exemplary video processing system 3100 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 3100. System 3100 may include an input 3102 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be received in a compressed or encoded format. Input 3102 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0330] System 3100 may include a codec component 3104 that can implement various codecs or encoding methods described herein. Codec component 3104 can reduce the average bit rate of the video from input 3102 to the output of codec component 3104 to generate the codec of the video. Therefore, encoding technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 3104 can be stored or transmitted through connected communication, as shown in component 3106. The storage or communication bitstream (or codec) of the video received at input 3102 represents a displayable video that can be used by component 3108 to generate pixel values ​​or send to display interface 3110. The process of generating a user-viewable video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations of the inverse codec results will be performed by the decoder.

[0331] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Display Port, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described herein may be implemented in a variety of electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and / or video display.

[0332] In some embodiments, the video encoding and decoding method can use Fig.24A Or it can be implemented by the device implemented on the hardware platform described in 24B.

[0333] Various techniques and embodiments may be described using the following clause-based format.

[0334] The first group of clauses describes certain features and aspects of the disclosed technology listed in the previous sections, including, for example, Examples 1 and 3(a).

[0335] 1. A method for video processing, comprising:

[0336] deriving a chroma motion vector for a chroma block associated with a current video block of the video based on unclipped previously calculated luma motion vectors for a plurality of luma blocks; and

[0337] A conversion between the current video block and a bitstream representation of the video is performed based on the chroma motion vector.

[0338] 2. The method of clause 1, wherein the method for deriving the chroma motion vector is also based on a color format of the current video block.

[0339] 3. The method of clause 2, wherein the color format of the current video block is 4:2:0, 4:2:2, or 4:4:4.

[0340] 4. A method according to clause 1, wherein the deriving of the chrominance motion vector is based on four luma motion vectors of the unclipped previously calculated luma motion vector.

[0341] 5. The method of clause 4, wherein the color format is 4:2:0.

[0342] 6. A method according to clause 1, wherein said deriving said chrominance motion vector is based on two luma motion vectors of said unclipped previously calculated luma motion vector.

[0343] 7. A method according to clause 6, wherein the color format is 4:2:2.

[0344] 8. A method according to clause 1, wherein said deriving said chrominance motion vector is based on a luma motion vector of said unclipped previously calculated luma motion vector.

[0345] 9. The method of clause 8, wherein the color format is 4:4:4.

[0346] 10. The method of clause 1, wherein said deriving said chrominance motion vector is based on a linear function of a plurality of luma motion vectors.

[0347] 11. The method according to clause 1, further comprising:

[0348] clipping the chroma motion vector to generate a clipped chroma motion vector,

[0349] Wherein, performing the conversion comprises motion compensation based on the clipped chroma motion vector.

[0350] 12. A method according to any one of clauses 1 to 11, wherein the current video block comprises sub-blocks, and wherein performing the conversion is based on at least one of affine prediction, sub-block temporal motion vector prediction (SbTMVP), or spatio-temporal motion vector prediction (STMVP).

[0351] 13. A method according to clause 12, wherein said performing said conversion is based on said affine prediction and setting said chrominance motion vector to an average of motion vectors of four corresponding luminance blocks.

[0352] 14. A method according to any one of clauses 1 to 11, wherein the chrominance block and the plurality of luminance blocks correspond to a Y / Cb / Cr color space.

[0353] 15. A method according to any one of clauses 1 to 11, wherein the chrominance block and the plurality of luma blocks correspond to an R / G / B color space.

[0354] 16. A method for video processing, comprising:

[0355] determining an internal bit depth using one or more motion vectors associated with a current video block of the video, the internal bit depth being M bits, where M is greater than 16; and

[0356] Conversion between the current video block and a bitstream representation of the video is performed based on an advanced motion vector prediction (AMVP) process and the M-bit motion vector.

[0357] 17. A method according to clause 16, wherein at least one of the one or more motion vectors is derived from a motion vector predictor, and wherein the motion vector predictor is scaled to the range [—(1<<(M-1)), (1<<(M-1))-1].

[0358] 18. A method according to clause 17, wherein M is 18 and the range is [-131072, 131071].

[0359] 19. The method of clause 17, wherein the motion vector predictor is associated with a block having a reference picture different from a target reference picture for the current video block.

[0360] 20. A method according to any of clauses 1 to 19, wherein the converting comprises generating pixel values ​​of the current video block from the bitstream representation.

[0361] 21. A method according to any of clauses 1 to 19, wherein the converting comprises generating the bitstream representation from pixel values ​​of the current video block.

[0362] 22. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein when the instructions thereon are executed by the processor, the processor is caused to implement one or more of the methods described in clauses 1 to 21.

[0363] 23. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for implementing one or more of the methods of clauses 1 to 21.

[0364] The second group of clauses describes certain features and aspects of the disclosed technology listed in the previous sections, including, for example, Examples 2, 3(b), 4, 6, and 8.

[0365] 1. A method for video processing, comprising:

[0366] determining a motion vector accuracy of one or more motion vectors associated with a current video block of a video;

[0367] selecting a motion vector range based on the motion vector accuracy; and

[0368] A conversion between the current video block and a bitstream representation of the video is performed based on the motion vector range.

[0369] 2. The method of clause 1, wherein said performing said converting uses an Adaptive Motion Vector Difference Resolution (AMVR) codec.

[0370] 3. The method of clause 1, wherein said selecting said motion vector range is further based on a profile or level of an AMVR codec.

[0371] 4. The method of clause 1, wherein the selecting the motion vector range is further based on a codec mode of the current video block.

[0372] 5. The method according to clause 1, further comprising:

[0373] Prior to said performing said converting, said one or more motion vectors are clipped based on said motion vector range.

[0374] 6. A method according to clause 5, wherein clipping of one or more motion vectors is performed using input parameters determined based on a range of motion vectors.

[0375] 7. A method for video processing, comprising:

[0376] determining an internal bit depth using one or more motion vectors associated with a current video block of a video, wherein the internal bit depth has M bits, M is greater than 16, and the current video block uses temporal motion vector prediction; and

[0377] Converting between the current video block and a bitstream representation of the video is performed based on at least an M-bit motion vector.

[0378] 8. The method of clause 7, wherein the converting is performed based on temporal motion vector prediction of an advanced motion vector prediction (AMVP) mode of the current video block.

[0379] 9. The method of clause 7, wherein the converting is performed based on temporal motion vector prediction of Merge mode of the current video block.

[0380] 10. A method according to clause 7, wherein the conversion is performed based on a temporal motion vector prediction of an optional temporal motion vector prediction (ATMVP) mode of the current video block, wherein the ATMVP mode allows at least one set of motion information to be derived from a block smaller than the current codec unit.

[0381] 11. The method of clause 7, wherein the converting is performed based on temporal motion vector prediction of a spatial-temporal motion vector prediction (STMVP) mode of the current video block.

[0382] 12. A method according to clause 7, wherein the one or more motion vectors are derived from a motion vector prediction candidate, the motion vector prediction candidate is derived based at least on a motion vector associated with a temporal reference picture, the motion vector is scaled to a target reference picture, and the scaled motion vector is clipped to be within the range of [-((1<<(M-1))),(1<<(M-1))-1].

[0383] 13. A method according to clause 12, wherein M is 18 and the range is [-131072, 131071].

[0384] 14. A video processing method, comprising:

[0385] determining a precision of motion vectors for conversion between a current video block of a video unit of a video and a codec representation of the video using a motion vector precision value; and

[0386] The converting is performed based on a precision of the motion vector according to the motion vector precision value, wherein the video unit level syntax element indicates the motion vector precision value.

[0387] 15. The method of clause 14, wherein the video unit corresponds to a video sequence, a video picture, a slice, a codec tree unit row, or a codec tree unit.

[0388] 16. The method of clause 14, wherein the syntax element is located in a video parameter set, a picture parameter set, a slice header, a slice group header, a slice header, a codec tree unit header, or a codec tree unit header.

[0389] 17. A method according to any of clauses 14 to 16, wherein said determining to use said motion vector accuracy value is based on a profile, level or layer used for said conversion.

[0390] 18. A method for video processing, comprising:

[0391] determining parameters for conversion between a current video block of a video and a bitstream representation of the current video block; and

[0392] performing the conversion based on the parameters,

[0393] The determining of the parameter is based on a height or a width of a current picture including the current video block.

[0394] 19. A method according to clause 18, wherein the parameter is a motion vector accuracy of one or more motion vectors used during the conversion.

[0395] 20. The method of clause 18, wherein the parameter is an internal bit depth of one or more motion vectors used during the conversion.

[0396] 21. A method for video processing, comprising:

[0397] clipping the temporary motion vector to generate a clipped motion vector;

[0398] deriving a motion vector associated with a current video block of the video based at least on the clipped motion vector; and

[0399] performing a conversion between the current video block of video and a bitstream representation of the video based on the motion vector associated with the current video block,

[0400] The cutting of the temporary motion vector is based on the height (H) or width (W) of the reference picture of the current video block, the upper left position (x0, y0) of the current video block, or at least one of the motion vector accuracy or internal bit depth (M) of the motion vector.

[0401] 22. The method according to clause 21, wherein the motion vector precision is 1 / (1<<Shift) sub-pixels, where Shift is a positive integer, where CW and CH are the maximum width and maximum height of the coding and decoding unit of the current video block respectively, wherein the x component of the motion vector is clipped to the range [MVMinH, MVMaxH], and wherein the y component of the one or more motion vectors is clipped to the range [MVMinV, MVMaxV],

[0402] where MVMinH = max(PicMinH, -(1<<(M - 1))),

[0403] MVMinV = max(PicMinV, -(1<<(M - 1))),

[0404] MVMaxH = min(PicMaxH, (1<<(M - 1)) - 1), and

[0405] MVMaxV = min(PicMaxV, (1<<(M - 1)) - 1),

[0406] where PicMinH = (-x0 + 1 + Offset1)<<Shift,

[0407] PicMaxH = (W - 1 - x0 + Offset2)<<Shift,

[0408] PicMinV = (-y0 + 1 + Offset3)<<Shift, and

[0409] PicMaxV = (H - 1 - y0 + Offset4)<<Shift,

[0410] where Offset1, Offset2, Offset3, and Offset4 are non-negative integers based on one or more padding methods.

[0411] 23. The method according to clause 22, wherein Offset1 = -CW - 8, Offset3 = -CH - 8, Offset2 = 8, and Offset4 = 8.

[0412] 24. The method according to any one of clauses 1 to 23, wherein the conversion includes generating pixel values of the current video block from the bitstream representation.

[0413] 25. The method according to any one of clauses 1 to 23, wherein the conversion includes generating the bitstream representation from the pixel values of the current video block.

[0414] 26. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein when the instructions thereon are executed by the processor, the processor is caused to implement one or more of the methods described in clauses 1 to 25.

[0415] 27. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for implementing the method of one or more of clauses 1 to 25.

[0416] The third group of clauses describes certain features and aspects of the disclosed technology listed in the previous sections, including, for example, Examples 3(c)-(d), 5, 7, and 9.

[0417] 1. A method for video processing, comprising:

[0418] performing a conversion between a current video block of a video and a bitstream representation of said video,

[0419] wherein the conversion uses a scaling process to obtain a motion vector difference of a first reference picture from a motion vector difference from a second reference picture, and wherein the scaling process uses a shearing operation to limit the motion vector difference of the first reference picture to fall within a range of [-((1<<(M-1))),(1<<(M-1))-1], where M is a number greater than 16.

[0420] 2. A method according to clause 1, wherein the first reference picture is associated with a first reference picture list.

[0421] 3. A method according to clause 1, wherein the first reference picture is associated with a second reference picture list.

[0422] 4. A method according to clause 3, wherein the first reference picture list is different from the second reference picture list.

[0423] 5. The method of clause 1, wherein an affine codec tool is enabled for the conversion.

[0424] 6. A method as described in clause 5, wherein a field in the bitstream representation indicates that the affine codec tool is enabled.

[0425] 7. A method according to clause 1, wherein the conversion is performed based on a motion vector difference (MVD) derivation process of a motion vector difference Merge (MMVD) mode, and the MMVD mode includes a motion vector expression, which includes a start point, a motion vector distance and a motion direction of the current video block.

[0426] 8. The method of clause 7, wherein at least one of the motion vector differences of the first reference picture or the second reference picture is derived based on at least one of the motion vector distance or the motion direction of the current video block.

[0427] 9. A method according to clause 8, wherein at least one Merge candidate is selected based on the motion vector expression, and the at least one Merge candidate is further refined by the at least one of the motion vector differences.

[0428] 10. The method of clause 1, wherein the internal bit depth of the one or more motion vectors is signaled in a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group, a codec tree unit (CTU), or a CTU row.

[0429] 11. A method for video processing, comprising:

[0430] determining an internal bit depth for conversion between a current video block of a video unit of a video and a bitstream representation of the video using one or more motion vectors; and

[0431] performing said converting using said one or more motion vectors,

[0432] The syntax element at the video unit level indicates the internal bit depth.

[0433] 12. The method of clause 11, wherein the video unit corresponds to a video sequence, a video picture, a slice, a codec tree unit row, or a codec tree unit.

[0434] 13. The method according to clause 12 further includes: accordingly, the syntax element is located in a video parameter set, a picture parameter set, a slice header, a slice group header, a slice header, a codec tree unit header or a codec tree unit header.

[0435] 14. A method according to clause 11 or 12, wherein the determining to use the internal bit depth is based on a profile, or a level or layer used for the conversion.

[0436] 15. A method according to clause 11 or 12, wherein the internal bit depth is based on the height or width of a current picture containing the current video block.

[0437] 16. A method for video processing, comprising:

[0438] determining an internal bit depth for a conversion between a current video block of a video unit of a video and a bitstream representation of the video using affine motion parameters; and

[0439] performing said converting based on said determining,

[0440] Therein, the internal bit depth of the affine motion parameters has the same range as that of non-affine motion parameters.

[0441] 17. The method according to clause 16, wherein the range is [-2 15 ,2 15 -1].

[0442] 18. The method according to clause 16, wherein the range is [-2 17 ,2 17 -1].

[0443] 19. The method of clause 16, wherein the affine motion parameters comprise at least one of an affine motion vector, an affine motion vector predictor, or a control point motion vector (CPMV).

[0444] 20. A method according to any of clauses 1 to 19, wherein the converting comprises generating pixel values ​​of the current video block from the bitstream representation.

[0445] 21. A method according to any of clauses 1 to 19, wherein the converting comprises generating the bitstream representation from pixel values ​​of the current video block.

[0446] 22. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein when the instructions thereon are executed by the processor, the processor is caused to implement one or more of the methods described in clauses 1 to 21.

[0447] 23. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for implementing one or more of the methods of clauses 1 to 21.

[0448] It should be understood that, for the purpose of illustration, specific embodiments of the disclosed technology are described herein, but various modifications may be made without departing from the scope of the invention. Therefore, the disclosed technology is not limited except as set forth in the appended claims.

[0449] The implementation of the subject matter and functional operations described in this patent document can be implemented in various systems, digital electronic circuits or computer software, firmware or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, that is, one or more computer program instruction modules encoded on a tangible and non-transitory computer-readable medium, for execution by a data processing device or controlling the operation of a data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a combination of substances that affect machine-readable propagation signals, or a combination of one or more of them. The term "data processing unit" or "data processing device" includes all devices, apparatuses and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0450] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0451] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0452] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be coupled to one or more mass storage devices by operation to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices. The processor and memory can be supplemented by, or incorporated into, a dedicated logic circuit.

[0453] The present specification, together with the drawings, is to be regarded as exemplary only, wherein exemplary refers to examples.As used herein, the use of "or" is intended to include "and / or" unless the context clearly dictates otherwise.

[0454] Although this patent document contains many details, they should not be interpreted as limitations on the scope of any invention or claim, but rather as descriptions of features of specific embodiments of specific inventions. Some features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in some combinations, or even initially required to be so, in some cases, one or more features in the claim combination can be removed from the combination, and the combination of claims can be directed to subcombinations or variations of subcombinations.

[0455] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0456] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for video processing, comprising: deriving a chroma motion vector for a chroma block associated with a current video block of the video based on unclipped previously calculated luma motion vectors for a plurality of luma blocks, wherein the luma motion vector has an accuracy of 1 / 16 pixel; clipping the chroma motion vector to generate a clipped chroma motion vector; as well as performing conversion between the current video block and a bitstream of the video based on the clipped chroma motion vector; Wherein, performing the conversion comprises motion compensation based on the clipped chroma motion vector.

2. The method according to claim 1, wherein: The method for deriving the chroma motion vector is also based on a color format of the current video block.

3. The method according to claim 2, wherein: The color format of the current video block is 4:2:0, 4:2:2, or 4:4:

4.

4. The method according to claim 2, wherein: The deriving of the chrominance motion vector is based on four luma motion vectors of the unclipped previously calculated luma motion vector.

5. The method according to claim 4, wherein: The color format is 4:2:

0.

6. The method according to claim 2, wherein: The deriving of the chrominance motion vector is based on two luma motion vectors of the unclipped previously calculated luma motion vector.

7. The method according to claim 6, wherein: The color format is 4:2:

2.

8. The method according to claim 2, wherein: The deriving of the chrominance motion vector is a luminance motion vector based on the unclipped previously calculated luminance motion vector.

9. The method according to claim 8, wherein: The color format is 4:4:

4.

10. The method according to claim 1, wherein: The deriving of the chrominance motion vector is based on a linear function of multiple luminance motion vectors.

11. The method according to claim 1, wherein: The current video block comprises sub-blocks, and wherein the performing the conversion is based on at least one of affine prediction, sub-block temporal motion vector prediction SbTMVP, or spatio-temporal motion vector prediction STMVP.

12. The method according to claim 11, wherein: The performing of the conversion is based on the affine prediction, and the chrominance motion vector is set to the average of the motion vectors of four corresponding luminance blocks.

13. The method according to claim 1, wherein: The chrominance block and the plurality of luminance blocks correspond to a Y / Cb / Cr color space.

14. The method according to claim 1, wherein: The chrominance block and the plurality of luminance blocks correspond to an R / G / B color space.

15. The method according to claim 1, wherein: The method further comprises: determining an internal bit depth using one or more motion vectors associated with the current video block, the internal bit depth being M bits, where M is greater than 16; and The performing the conversion further includes performing conversion between the current video block and a bitstream of the video based on an Advanced Motion Vector Prediction (AMVP) process and the M-bit motion vector.

16. The method according to claim 15, wherein: At least one of the one or more motion vectors is derived from a motion vector predictor, and wherein the motion vector predictor is scaled to a range of [—(1<<(M−1)), (1<<(M−1))−1].

17. The method according to claim 16, wherein: M is 18, and the range is [-131072, 131071].

18. The method according to claim 16, wherein: The motion vector predictor is associated with a block having a reference picture that is different than a target reference picture for the current video block.

19. The method according to any one of claims 1 to 18, wherein: The converting includes generating pixel values ​​for the current video block from the bitstream.

20. The method according to any one of claims 1 to 18, wherein: The converting includes generating the bitstream from pixel values ​​of the current video block.

21. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein: When the instructions thereon are executed by the processor, the processor is caused to implement the method according to any one of claims 1 to 20.

22. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code, which, when executed by a processor, causes the processor to implement the method of any one of claims 1 to 20.

Images