Motion Accuracy in Subblock-Based Inter Prediction

Abstract

An apparatus, system and method for digital video coding and decoding are described, which include a sub-block based inter-frame prediction method. An exemplary method for video processing includes: for a conversion between a current block of a video and a bitstream representation of the video, based on whether temporal motion vector prediction (TMVP) is enabled during the conversion or whether a current picture reference (CPR) coding mode is used for the conversion, determining a maximum number of candidates in a sub-block based Merge candidate list and / or whether to add a sub-block based temporal motion vector prediction (SbTMVP) candidate to the sub-block based Merge candidate list; and performing the conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a Chinese national phase application of International Patent Application No. PCT / CN2020 / 108805 filed on August 13, 2020, and this application timely claims priority and benefits of International Patent Application No. PCT / CN2019 / 100396 filed on August 13, 2019. The entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] The patent document relates to image and video encoding and decoding. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth required for digital video usage is expected to continue to grow. Summary of the invention

[0005] Devices, systems and methods related to digital video coding are described, including sub-block based inter-frame prediction methods. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Codec (HEVC) and / or Versatile Video Codec (VVC)) and future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a video and a bitstream representation of the video, based on whether temporal motion vector prediction (TMVP) is enabled during the conversion or whether a current picture reference (CPR) codec mode is used for the conversion, determining a maximum number of candidates (ML) in a sub-block-based Merge candidate list and / or whether to add a sub-block-based temporal motion vector prediction (SbTMVP) candidate to a sub-block-based Merge candidate list; and performing the conversion based on the determination.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a video and a bitstream representation of the video, based on whether temporal motion vector prediction (TMVP), sub-block-based temporal motion vector prediction (SbTMVP) and an affine codec mode are enabled during the conversion, determining a maximum number of candidates (ML) in a sub-block-based Merge candidate list; and performing the conversion based on the determination.

[0008] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a first video segment of a video and a bitstream representation of the video, determining that a sub-block based motion vector prediction (SbTMVP) mode is disabled for the conversion because a temporal motion vector prediction (TMVP) mode is disabled at the first video segment level; and performing the conversion based on the determination, wherein the bitstream representation conforms to a format, the format specifies whether to include an indication of the SbTMVP mode and / or the position of the indication of the SbTMVP mode in a Merge candidate list relative to the indication of the TMVP mode.

[0009] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: performing a conversion between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool or a temporal motion vector prediction (TMVP) tool and a bitstream representation of the video, wherein coordinates of corresponding positions of the current block or a sub-block of the current block are selectively masked using a mask based on compression of motion vectors associated with the SbTMVP tool or the TMVP tool, and wherein application of the mask includes calculating a bitwise AND operation between values ​​of the coordinates and values ​​of the mask.

[0010] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing, the method comprising: applying a sub-block based motion vector prediction (SbTMVP) tool to a current block of a video segment of a video based on one or more characteristics of the current block, determining a valid corresponding region of the current block; and performing conversion between the current block and a bitstream representation of the video based on the determination.

[0011] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: determining a default motion vector for a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool; and based on the determination, performing a conversion between the current block and a bitstream representation of the video, wherein the default motion vector is determined without obtaining a motion vector from a block covering a corresponding position in a collocated picture associated with a center position of the current block.

[0012] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a current block of a video segment of a video, when a current picture of the current block is a reference picture whose index is set to M in a reference picture list X, inferring that a sub-block based temporal motion vector prediction (SbTMVP) tool or a temporal motion vector prediction (TMVP) tool is disabled for the video segment, where M and X are integers, and where X=0 or X=1; and based on the inference, performing conversion between the current block and a bitstream representation of the video.

[0013] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for a current block of a video, if a current picture of the current block is a reference picture whose index is set to M in a reference picture list X, determining to enable application of a sub-block based temporal motion vector prediction (SbTMVP) tool, where M and X are integers; and based on the determination, performing conversion between the current block and a bitstream representation of the video.

[0014] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: performing a conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded and decoded using a sub-block based encoding and decoding tool, and wherein performing the conversion includes encoding and decoding a sub-block Merge index using a plurality of binary bits (N) by a unified method when a sub-block based temporal motion vector prediction (SbTMVP) tool is enabled or disabled.

[0015] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool, determining a motion vector used by the SbTMVP tool to locate a corresponding block in a picture different from a current picture including the current block; and based on the determination, performing conversion between the current block and a bitstream representation of the video.

[0016] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a video and a bitstream representation of the video, determining whether to insert a zero-motion affine Merge candidate into a sub-block Merge candidate list based on whether affine prediction is enabled for the conversion of the current block; and performing the conversion based on the determination.

[0017] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a video using a sub-block Merge candidate list and a bitstream representation of the video, if the sub-block Merge candidate list is not full, inserting a zero-motion non-affine filling candidate into the sub-block Merge candidate list; and performing the conversion after the insertion.

[0018] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a current block of a video and a bitstream representation of the video, determining a motion vector using a rule that the motion vector is derived from one or more motion vectors of a block covering a corresponding position in a collocated picture; and performing the conversion based on the motion vector.

[0019] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a video and a bitstream representation of the video, when a temporal block associated with the current block or a subblock of the current block in a collocated picture is encoded and decoded using a codec mode, in which a video unit is reconstructed based on previously encoded and decoded samples in the same picture, determining to use a subblock-based temporal motion vector prediction (sbTMVP) tool with a default motion candidate for conversion; and performing the conversion based on the default motion candidate.

[0020] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for a sub-block based temporal motion vector prediction (sbTMVP) process as part of a conversion between a current block of a video and a bitstream representation of the video, deriving default motion information for the sbTMVP process based on a location used in a sub-block motion information derivation process for a sub-block of the current block; and performing the conversion based on the default motion information.

[0021] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a current block of a video encoded and decoded using a sub-block-based temporal motion vector prediction (sbTMVP) tool and a bitstream representation of the video, determining a modified motion vector for locating a corresponding block in a picture different from the current picture of the current block, wherein the modified motion vector is generated by right-shifting the predicted motion vector in the sbTMVP tool to integer precision, and performing the conversion based on the determination. In yet another example aspect, a video encoder device is disclosed. The video encoder device includes a processor configured to implement the method described herein.

[0022] In yet another example aspect, a video decoder apparatus is disclosed. The video decoder apparatus includes a processor configured to implement the methods described herein.

[0023] In another aspect, a computer readable medium having code stored thereon is disclosed. When executed by a processor, the code causes the processor to implement the method described in this document.

[0024] These and other aspects are described in this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is an example of the derivation process used for Merge candidate list construction.

[0026] Figure 2 Example locations of spatial merge candidates are shown.

[0027] Figure 3 An example of candidate pairs considered for redundancy checking of spatial merge candidates is shown.

[0028] Figure 4A and Figure 4B Example locations of the second prediction units (PUs) for Nx2N and 2NxN partitions are shown.

[0029] Figure 5 is an example illustration of motion vector scaling for temporal merge candidates.

[0030] Figure 6 Example candidate positions C0 and C1 of the time-domain Merge candidates are shown.

[0031] Figure 7 An example of a combined bi-predictive Merge candidate is shown.

[0032] Figure 8 An example derivation process for motion vector prediction candidates is shown.

[0033] Fig. 9 is an example illustration of motion vector scaling for spatial motion vector candidates.

[0034] Fig.10 An example of optional temporal motion vector prediction (ATMVP) motion prediction for a CU is shown.

[0035] Fig.11 An example of one CU having four sub-blocks (AD) and their neighboring blocks (ad) is shown.

[0036] Fig.12 This is a flowchart of an example of encoding with different MV precisions.

[0037] Fig.13A and Fig. 13B A 135 degree segmentation type (a segmentation from the upper left corner to the lower right corner) and a 45 degree segmentation pattern are shown respectively.

[0038] Fig.14 An example of the positions of neighboring blocks is shown.

[0039] Fig.15 Examples of upper and left blocks are shown.

[0040] Fig.16A and 16B Examples of 2 control point motion vectors (CPMV) and 3 CPMV are shown respectively.

[0041] Fig.17 An example of an affine motion vector field (MVF) for each sub-block is shown.

[0042] Fig.18A and 18B Examples are shown for 4 and 6 parameter affine models, respectively.

[0043] Fig.19 An example of MVP of AF_INTER of inherited affine candidate is shown.

[0044] Fig. 20 An example of constructing an affine motion predictor in AF_INTER is shown.

[0045] Fig.21A and 21B An example of control point motion vectors in affine codec in AF_MERGE is shown.

[0046] Fig. 22 An example of candidate positions for the affine merge mode is shown.

[0047] Fig.23 An example of an intra-picture block copy operation is shown.

[0048] Fig.24 Examples of valid corresponding areas in collocated pictures are shown.

[0049] Fig.25 An example flow chart for history-based motion vector prediction is shown.

[0050] Fig.26 The modified Merge list construction process is shown.

[0051] Fig. 27 An example embodiment of the proposed valid region when the current block is within the base region is shown.

[0052] Fig.28 An example embodiment of a valid region when a current block is not within a base region is shown.

[0053] Fig.29A and 29B Existing examples and proposed examples for identifying the location of default motion information are shown, respectively.

[0054] Figure 30-45 is a flow chart of an example of a video processing method.

[0055] Fig.46 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described herein.

[0056] Fig.47 is a block diagram of an example video processing system in which the disclosed techniques may be implemented. DETAILED DESCRIPTION

[0057] This document provides various techniques that can be used by a decoder of a video bitstream to improve the quality of a decompressed or decoded digital video or image. In addition, a video encoder can also implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0058] The section headings used in this document are for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. Thus, the embodiments of one section can be combined with the embodiments of other sections.

[0059] 1. Overview

[0060] This patent document relates to video codec technology. Specifically, it relates to motion vector coding in video codec. It can be applied to existing video codec standards (such as HEVC), or to a pending standard (multifunctional video codec). It may also be applicable to future video codec standards or video codecs.

[0061] 2. Introduction

[0062] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations worked together to produce the H.262 / MPEG-2 Video and H.264 / MPEG-4 Enhanced Video Codec (AVC) and H.265 / HEVC standards. Starting with H.262, video codec standards are based on a hybrid video codec structure that utilizes temporal prediction and transform codecs. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software called the "Joint Exploration Model" (JEM). In April 2018, a Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was established to work on the VVC standard with the goal of reducing bit rate by 50% compared to HEVC.

[0063] The latest version of the VVC draft, Versatile Video Codec (Draft 3), can be found at:

[0064] http: / / phenix.it-

[0065] sudparis.eu / jvet / doc_end_user / documents / 12_Macao / wg11 / JVET-L1001- v2.zip

[0066] The latest reference software for VVC, called VTM, can be found at:

[0067] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-3.0rc1

[0068] 2.1 Inter-frame prediction in HEVC / H.265

[0069] Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include motion vectors and reference picture indices. The use of one of the two reference picture lists can also be signaled using inter_pred_idc. Motion vectors can be explicitly encoded and decoded as deltas relative to the predictor.

[0070] When a CU is coded in skip mode, a PU is associated with the CU and has no significant residual coefficients, no coded motion vector increments or reference picture indices. A Merge mode is specified, by which the motion parameters of the current PU can be obtained from neighboring PUs (including spatial and temporal candidates). Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to Merge mode is the explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference (MVD) compared to the motion vector predictor), the corresponding reference picture index of each reference picture list, and the use of the reference picture list are explicitly signaled per PU. In the present disclosure, such a mode is named Advanced Motion Vector Prediction (AMVP).

[0071] When signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of samples. This is called "unidirectional prediction". Unidirectional prediction is available for both P slices and B slices.

[0072] When signaling indicates that two reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bidirectional prediction". Bidirectional prediction is only available for B slices.

[0073] The following provides details about the inter prediction modes specified in HEVC. The description will start with the Merge mode.

[0074] 2.1.1 Reference Image List

[0075] In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. As in H.264 / AVC, a picture can be predicted from multiple reference pictures. Reference pictures used for inter prediction are organized in one or more reference picture lists. A reference index identifies which reference pictures in the list should be used to create the prediction signal.

[0076] A single reference picture list, List 0, is used for P slices, and two reference picture lists, List 0 and List 1, are used for B slices. It should be noted that the reference pictures contained in Lists 0 / 1 may be from both past and future pictures in terms of capture / display order.

[0077] 2.1.2 Merge Mode

[0078] 2.1.2.1 Merge Mode Candidate Derivation

[0079] When predicting a PU using Merge mode, an index pointing to an entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized according to the following steps in order:

[0080] Step 1: Original candidate derivation

[0081] Step 1.1: Spatial Candidate Derivation

[0082] Step 1.2: Redundancy check of spatial candidates

[0083] οStep 1.3: Time Domain Candidate Derivation

[0084] Step 2: Insert additional candidates

[0085] οStep 2.1: Create bidirectional prediction candidates

[0086] οStep 2.2: Insert zero motion candidates

[0087] These steps are also Figure 1 As schematically shown in . For spatial Merge candidate derivation, up to four Merge candidates are selected from candidates located at five different positions. For temporal Merge candidate derivation, up to one Merge candidate is selected from two candidates. Since a constant number of candidates is assumed at the decoder for each PU, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, truncated unary binarization (TU) is used to encode the index of the best Merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

[0088] Hereinafter, operations associated with the aforementioned steps are described in detail.

[0089] 2.1.2.2 Derivation of Spatial Candidates

[0090] In the derivation of spatial merge candidates, Figure 2 Up to four Merge candidates are selected from the candidates at the depicted position. The order of derivation is A1, B1, B0, A0 and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is unavailable (for example, because it belongs to another stripe or slice) or is intra-coded. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving the coding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only the possible candidate pairs are considered. Figure 3 The pairs connected by arrows in , and the candidate is added to the list only if the corresponding candidate for redundancy check has different motion information. Another source of repeated motion information is a "second PU" associated with a partition other than 2N×2N. As an example, Figure 4A and 4B The second PU is depicted for the N×2N and 2N×N cases, respectively. When the current PU is partitioned as N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate would result in two prediction units with the same motion information, which is redundant for having only one PU in the codec unit. Similarly, when the current PU is partitioned as 2N×N, position B1 is not considered.

[0091] 2.1.2.3 Derivation of time domain candidates

[0092] In this step, only one candidate is added to the list. Specifically, in the derivation of the temporal Merge candidate, the scaled motion vector is derived based on the collocated PU, which belongs to the picture with the smallest POC difference with the current picture in a given reference picture list. The reference picture list to be used for deriving the collocated PU is explicitly signaled in the slice header. Figure 5 As shown by the dotted line, a scaled motion vector for the temporal Merge candidate is obtained, which is scaled from the motion vector of the collocated PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC specification. For B slices, two motion vectors are obtained, one for reference picture list 0 and the other for reference picture list 1, and the two motion vectors are combined to obtain a bidirectional prediction Merge candidate.

[0093] Figure 5 This is an explanation of the derivation of motion vector scaling for temporal merge candidates.

[0094] In the collocated PU(Y) belonging to the reference frame, the position of the temporal candidate is selected between candidates C0 and C1, such as Figure 6 As shown. If the PU at position C0 is not available, is intra-coded, or is outside the current codec tree unit (CTU aka. LCU, largest codec unit) row, position C1 is used. Otherwise, position C0 is used for the derivation of the time domain Merge candidate.

[0095] Figure 6 Examples C0 and C1 of candidate positions of the time-domain Merge candidate are shown.

[0096] 2.1.2.4 Additional candidate insertion

[0097] In addition to spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bidirectional prediction Merge candidates and zero Merge candidates. Combined bidirectional prediction Merge candidates are generated by utilizing spatial and temporal Merge candidates. Combined bidirectional prediction Merge candidates are only used for B slices. Combined bidirectional prediction candidates are generated by combining the first reference picture list motion parameters of the original candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they will form a new bidirectional prediction candidate. For example, Figure 7 It shows the case when two candidates in the original list (left) have mvL0 and refIdxL0 or mvL1 and refIdxL1, which are used to create a combined bi-predictive Merge candidate added to the final list (right). There are many rules about combining, which are considered to generate these additional Merge candidates.

[0098] Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list to reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list.

[0099] More specifically, the following steps are performed in sequence until the Merge list is full:

[0100] 1. Set the variable numRef to the number of reference pictures associated with list 0 for P slices, or to the minimum number of reference pictures in both lists for B slices;

[0101] 2. Add non-repeating zero-motion candidates:

[0102] For variable i being 0...numRef-1, for list 0 (if P slice) or both lists (if B slice), add a default motion candidate (set MV to (0, 0) and set reference picture index to i).

[0103] 3. Add repeated zero motion candidates, where MV is set to (0, 0), the reference picture index of list 0 is set to 0 (if it is a P slice), and the reference picture indexes of both lists are set to 0 (if it is a B slice).

[0104] Finally, no redundant checks are performed on these candidates.

[0105] 2.1.3 Advanced Motion Vector Prediction (AMVP)

[0106] AMVP exploits the spatiotemporal correlation of motion vectors with neighboring PUs for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is constructed by first checking the availability of the PU positions above and to the left in the temporal domain, removing redundant candidates and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and send a corresponding index indicating the selected candidate. Similar to Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary. The maximum value to be encoded in this case is 2 (see Figure 8 ). In the following sections, details about the derivation process of motion vector prediction candidates are provided.

[0107] 2.1.3.1 Derivation of AMVP Candidates

[0108] Figure 8 The derivation process of motion vector prediction candidates is summarized.

[0109] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For spatial motion vector candidate derivation, based on the Figure 2 The motion vectors of each PU at the five different positions shown in ultimately derive two motion vector candidates.

[0110] For temporal motion vector candidate derivation, a motion vector candidate is selected from two candidates, which are derived based on two different collocated positions. After generating the first list of spatio-temporal candidates, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than 2, motion vector candidates whose reference picture index within the associated reference picture list is greater than 1 are removed from the list. If the number of spatio-temporal motion vector candidates is less than 2, an additional zero motion vector candidate is added to the list.

[0111] 2.1.3.2 Spatial Motion Vector Candidates

[0112] In the derivation of spatial motion vector candidates, at most two candidates are considered among five potential candidates, which are located at Figure 2 The derivation order of the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order of the upper side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling and two use spatial scaling. The four different cases are summarized as follows.

[0113] No airspace scaling

[0114] -(1) Same reference picture list and same reference picture index (same POC)

[0115] -(2) Different reference picture lists, but same reference pictures (same POC)

[0116] Airspace scaling

[0117] -(3) Same reference picture list, but different reference pictures (different POC)

[0118] -(4) Different reference picture lists, and different reference pictures (different POC)

[0119] First check the case without spatial scaling, then check spatial scaling. Consider spatial scaling when the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU regardless of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, then scaling of the above motion vectors is allowed to help parallel derivation of the left and above MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

[0120] In the spatial domain scaling process, the motion vectors of neighboring PUs are scaled in a similar way to the temporal domain scaling. Fig. 9 As shown, the main difference is that the reference picture list and the index of the current PU are given as input; the actual scaling process is the same as the time domain scaling process.

[0121] 2.1.3.3 Temporal Motion Vector Candidates

[0122] Except for the reference picture index derivation, all the processes used to derive the temporal merge candidate are the same as those used to derive the spatial motion vector candidate (see Figure 6). The reference picture index is signaled to the decoder.

[0123] 2.2 Motion vector prediction method based on sub-CU in JEM

[0124] In JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. Two sub-CU level motion vector prediction methods are considered in the encoder by dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU. The Alternative Temporal Motion Vector Prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the Spatial-Temporal Motion Vector Prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using the temporal motion vector prediction value and the spatial neighboring motion vector.

[0125] In order to preserve a more accurate motion field for sub-CU motion prediction, motion compression of reference frames is currently disabled.

[0126] Fig.10 An example of ATMVP motion prediction for a CU is shown.

[0127] 2.2.1 Optional temporal motion vector prediction

[0128] In the optional temporal motion vector prediction (ATMVP) method, the motion vector temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. In some embodiments, the sub-CU is a square N×N block (N is set to 4 by default).

[0129] ATMVP predicts the motion vector of a sub-CU within a CU in two steps. The first step is to identify the corresponding block in the reference picture using the so-called time domain vector. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU.

[0130] In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. In order to avoid repeated scanning of neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set to the index of the temporal vector and the motion source picture. In this way, in ATMVP, the corresponding block can be identified more accurately than in TMVP, where the corresponding block (sometimes called the collocated block) is always located at the lower right or center position relative to the current CU.

[0131] In the second step, the corresponding block of the sub-CU is identified by the time domain vector in the motion source picture by adding the time domain vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (the minimum motion grid covering the center sample point) is used to derive the motion information of the sub-CU. After identifying the motion information of the corresponding N×N block, it is converted into the reference index and motion vector of the current sub-CU in the same way as the TMVP of HEVC, where motion scaling and other processes also apply. For example, the decoder checks whether the low latency condition is met (that is, the POC of all reference pictures of the current picture is less than the POC of the current picture) and may use the motion vector MV x (corresponding to the motion vector of reference picture list X) to predict the motion vector MV of each sub-CU y (where X equals 0 or 1 and Y equals 1-X).

[0132] 2.2.2 Spatial-Temporal Motion Vector Prediction (STMVP)

[0133] In this method, the motion vector of a sub-CU is recursively derived in raster scan order. Fig.11 The concept is illustrated in Figure 1. Let us consider an 8×8 CU containing four 4×4 sub-CUs A, B, C and D. The neighboring 4×4 blocks in the current frame are labeled a, b, c and d.

[0134] The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block (block c) above sub-CU A. If block c is not available or is intra-coded, then check the other N×N blocks above sub-CU A (starting from block c, from left to right). The second neighbor is the block (block b) on the left side of sub-CU A. If block b is not available or is intra-coded, then check the other blocks on the left side of sub-CU A (starting from block b, from top to bottom). The motion information obtained from the adjacent blocks of each list is scaled to the first reference frame of the given list. Next, the temporal motion vector prediction (Temporal Motion Vector Predictor, TMVP) of sub-block A is derived by following the same process as the TMVP derivation specified in HEVC. The motion information of the collocated block at position D is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

[0135] 2.2.3 Sub-CU motion prediction mode signaling notification

[0136] Sub-CU mode is enabled as an additional Merge candidate, and no additional syntax elements are required to signal the mode. Two additional Merge candidates are added to the Merge candidate list of each CU to represent ATMVP mode and STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, a maximum of seven Merge candidates are used. The encoding logic of the additional Merge candidates is the same as that of the Merge candidates in HM, which means that for each CU in a P or B slice, two additional Merge candidates require an additional two RD checks.

[0137] In JEM, all bins of the merge index are context-coded by CABAC, whereas in HEVC, only the first bin is context-coded, while the remaining bins are context-bypass coded.

[0138] 2.3 Inter-frame prediction method in VVC

[0139] There are several new codec tools for inter prediction improvements, such as Adaptive Motion Vector Differential Resolution (AMVR) for signaling MVD, Affine Prediction Mode, Triangular Prediction Mode (TPM), ATMVP, Generalized Bidirectional Prediction (GBI), Bidirectional Optical Flow (BIO).

[0140] 2.3.1 Adaptive Motion Vector Differential Resolution

[0141] In HEVC, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the PU's motion vector and the predicted motion vector) is signaled in units of quarter luma samples. In VVC, Locally Adaptive Motion Vector Resolution (LAMVR) is introduced. In VVC, MVD can be encoded and decoded in units of quarter luma samples, integer luma samples, or four luma samples (i.e., 1 / 4 pixel, 1 pixel, 4 pixels). The MVD resolution is controlled at the codec unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU with at least one non-zero MVD component.

[0142] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or quad luma sample MV precision is used.

[0143] When the first MVD resolution flag of a CU is zero or is not encoded for a CU (meaning that all MVDs in the CU are zero), a quarter luma sample MV resolution is used for the CU. When a CU uses integer luma sample MV precision or four luma sample MV precision, the MVP in the CU's AMVP candidate list is rounded to the corresponding precision.

[0144] In the encoder, the CU-level RD check is used to determine which MVD resolution is used for the CU. That is, for each MVD resolution, the CU-level RD check is performed three times. In order to speed up the encoder, the following encoding scheme is applied in JEM.

[0145] During RD check of a CU with regular quarter luma sample MVD resolution, the motion information of the current CU (integer luma sample accuracy) is stored. The stored motion information (after rounding) is used as a starting point for further small-scale motion vector refinement during RD check for the same CU with integer luma sample and 4 luma sample MVD resolution, so that the time-consuming motion estimation process is not repeated three times.

[0146] · Conditionally call RD check for CUs with 4 luma sample MVD resolution. For a CU, when the RD cost integer luma sample MVD resolution is much larger than a quarter luma sample MVD resolution, skip RD check for the CU's 4 luma sample MVD resolution.

[0147] The encoding process is as follows Fig.12 as shown. First, the 1 / 4 pixel MV is tested, and the RD cost is calculated and denoted as RDCost0. Then, the integer MV is tested, and the RD cost is denoted as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive value), then the 4 pixel MV is tested; otherwise, the 4 pixel MV is skipped. Basically, when checking the integer or 4 pixel MV, since the motion information and RD cost, etc. are known for the 1 / 4 pixel MV, they can be reused to speed up the encoding process of the integer or 4 pixel MV.

[0148] 2.3.2 Triangular Prediction Mode

[0149] The concept of the Triangular Prediction Mode (TPM) is to introduce a new triangular partition for motion compensation prediction. As shown in the figure. Refer to Figures 13A-13B , which divides the CU into two triangular prediction units along the diagonal direction or the reverse diagonal direction. Each triangular prediction unit in the CU uses the reference frame index derived from a single unidirectional prediction candidate list and its own unidirectional prediction motion vector for inter-frame prediction. After predicting the triangular prediction units, adaptive weighting processing is performed on the diagonal edges. Then, the transform and quantization processes are applied to the entire CU. Note that this mode is only applied to the Merge mode (Note: The skip mode is regarded as a special Merge mode).

[0150] Figures 13A-13B is a diagram showing the division of the CU into two triangular prediction units (two division modes): 13A: 135-degree division type (divided from the upper left corner to the lower right corner); Fig. 13B : 45-degree division mode.

[0151] 2.3.2.1 Unidirectional Prediction Candidate List of TPM

[0152] The unidirectional prediction candidate list, which is called the TPM motion candidate list, consists of five unidirectional prediction motion vector candidates. As Fig.14 shown, it is derived from seven neighboring blocks including five spatial neighboring blocks (1 to 5) and two temporal collocated blocks (6 to 7). According to the order of the unidirectional prediction motion vector, the L0 motion vector of the bidirectional prediction motion vector, the L1 motion vector of the bidirectional prediction motion vector, and the average motion vector of the L0 and L1 motion vectors of the bidirectional prediction motion vector, the motion vectors of the seven neighboring blocks are collected and put into the unidirectional prediction candidate list. If the number of candidates is less than five, zero motion vectors are added to the list. The motion candidates added to the list for TPM are called TPM candidates, and the motion information derived from the spatial / temporal blocks is called conventional motion candidates.

[0153] More specifically, the following steps are involved:

[0154] 1) When adding regular motion candidates from spatially adjacent blocks, the full cropping operation From A1, B1, B0, A0, B2, Col and Col2 (corresponding to Fig.14 Obtain regular motion candidates in blocks 1-7).

[0155] 2) Set the variable numCurrMergeCand = 0

[0156] 3) For each normal motion candidate derived from A1, B1, B0, A0, B2, Col and Col2 (if not cropped and numCurrMergeCand is less than 5), if the normal motion candidate is unidirectionally predicted (from list 0 or list 1), it is added directly to the Merge list as a TPM candidate and numCurrMergeCand is increased by 1. Such a TPM candidate is called an "original unidirectionally predicted candidate".

[0157] application Fully cropped .

[0158] 4) For each motion candidate derived from A1, B1, B0, A0, B2, Col and Col2 (if not cropped and numCurrMergeCand is less than 5), if the regular motion candidate is bidirectionally predicted, the motion information in list 0 is added to the TPM Merge list as a new TPM candidate (i.e., modified to unidirectional prediction from list 0), and numCurrMergeCand is increased by 1. Such a TPM candidate is called a "truncated list 0 predicted candidate".

[0159] application Fully cropped .

[0160] 5) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2 (if not cropped and numCurrMergeCand is less than 5), if the regular motion candidate is bi-directionally predicted, the motion information in list 1 is added to the TPM Merge list (i.e., modified to be uni-directionally predicted from list 1), and numCurrMergeCand is increased by 1. Such a TPM candidate is called a "truncated list 1 predicted candidate".

[0161] application Fully cropped .

[0162] 6) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2 (if not cropped and numCurrMergeCand is less than 5), if the regular motion candidate is bi-predicted,

[0163] – If the slice QP of list 0 reference picture is less than the slice QP of list 1 reference picture, the motion information of list 1 is first scaled to list 0 reference picture, and the average of two MVs (one from original list 0 and the other is the scaled MV from list 1) is added to the TPMMerge list, such a candidate is called an average unidirectional prediction from list 0 motion candidate, and numCurrMergeCand is increased by 1.

[0164] – Otherwise, first scale the motion information of list 0 to the reference picture of list 1, then add the average of the two MVs (one from the original list 1 and the other is the scaled MV from list 0) to the TPM Merge list, such a TPM candidate is called the average unidirectional prediction from list 1 motion candidate, and numCurrMergeCand is increased by 1.

[0165] application Fully cropped .

[0166] 7) If numCurrMergeCand is less than 5, add a zero motion vector candidate.

[0167] When a candidate is inserted into the list, if it must be compared to all previously added candidates to see if it is identical to one of them, this process is called full pruning.

[0168] 2.3.2.2 Adaptive Weighting Process

[0169] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangular prediction units to derive the final prediction for the entire CU. Two groups of weighting factors are defined as follows:

[0170] The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luma and chroma samples respectively;

[0171] · Second set of weighting factors: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} for luma and chroma samples respectively.

[0172] The weighting factor set is selected based on the comparison of the motion vectors of the two triangular prediction units. When the reference pictures of the two triangular prediction units are different from each other or the difference of their motion vectors is greater than 16 pixels, the second weighting factor set is used. Otherwise, the first weighting factor set is used.

[0173] 2.3.2.3 Signaling of Triangular Prediction Mode (TPM)

[0174] First, a one-bit flag indicating whether to use TPM can be signaled. Then, two partitioning mode indications (such as Figures 13A-13B ), and the Merge index selected for each of the two splits.

[0175] 2.3.2.3.1 TPM Flag Signaling

[0176] Let's denote the width and height of a luma block by W and H, respectively. If W*H<64, triangular prediction mode is disabled.

[0177] When encoding or decoding a block in affine mode, triangular prediction mode is also disabled.

[0178] When encoding or decoding a block in Merge mode, a one-bit flag may be signaled to indicate whether triangular prediction mode is enabled or disabled for the block.

[0179] The flags are encoded and decoded with 3 contexts based on the following equations:

[0180] Ctx index = ((left block L is available && L is encoded and decoded with TPM?) 1:0) + ((upper block A is available && A is encoded and decoded with TPM?) 1:0);

[0181] Fig.15 An example of neighboring blocks (A and L) used for context selection in TPM flag encoding and decoding is shown.

[0182] 2.3.2.3.2 Signaling the indication of the two partitioning modes (as shown in FIG. 13 ) and the Merge index selected for each of the two partitions

[0183] Note that the partition mode and the Merge index of the two partitions are jointly coded. In the existing implementation, the two partitions are restricted from using the same reference index. Therefore, there are 2 (partition mode) * N (maximum number of Merge candidates) * (N-1) possibilities, where N is set to 5. An indication is coded and the mapping between the partition mode, the two Merge indexes and the coding indication is derived from the array defined below:

[0184] const uint8_t g_TriangleCombination[TRIANGLE_MAX_NUM_CANDS][3]={{0,1,0},{1,0,1},{1,0,2},{0,0,1},{0,2,0},{1 ,0,3},{1,0,4},{1,1,0},{0,3,0},{0,4,0},{0,0,2},{0,1,2},{1,1,2},{0,0,4},{0,0,3},{0,1,3},{0,1,4} ,{1,1,4},{1,1,3},{1,2,1},{1,2,0},{0,2,1},{0,4,3},{1,3,0},{1,3,2},{1,3,4},{1,4,0},{1,3,1},{1,2,3},{1,4,1},{0,4,1},{0,2,3},{1,4,2},{0,3,2},{1,4,3},{0,3,1},{0,2,4},{1,2,4},{0,4,2},{0,3,4}};

[0185] Division mode (45 degrees or 135 degrees) = g_TriangleCombination [indication notified by signaling] [0];

[0186] Merge index of candidate A = g_TriangleCombination[indication notified by signaling][1];

[0187] Merge index of candidate B = g_TriangleCombination [indication notified by signaling] [2];

[0188] Once two motion candidates A and B are derived, the motion information of the two partitions (PU1 and PU2) can be set from A or B. Whether PU1 uses the motion information of Merge candidate A or B depends on the prediction direction of the two motion candidates. Table 1 shows the relationship between the two derived motion candidates A and B with two partitions.

[0189] Table 1: Motion information derived from two derived Merge candidates (A, B)

[0190] The predicted direction of A The predicted direction of B PU1 motion information PU2 motion information L0 L0 A(L0) B(L0) L1 L1 B(L1) A(L1) L0 L1 A(L0) B(L1) L1 L0 B(L0) A(L1)

[0191] 2.3.2.3.3 Entropy encoding and decoding of indicator (represented by merge_triangle_idx)

[0192] merge_triangle_idx is in the range of [0, 39], including the endpoints. K-order exponential Golomb (EG) code is used for binarization of merge_triangle_idx, where K is set to 1.

[0193] Kth Stage EG

[0194] To encode larger numbers with fewer bits (at the expense of using more bits to encode smaller numbers), we can generalize using a non-negative integer parameter k. Encode a non-negative integer x with an exp-Golomb code of order k:

[0195] 1. Use the above 0th order exp-Golomb code pair Encode, then

[0196] 2. x mod 2 k Encoded as binary bits.

[0197] Table 1: Exp-Golomb-k encoding and decoding example

[0198]

[0199]

[0200] 2.3.3 Affine Motion Compensated Prediction

[0201] In HEVC, motion compensated prediction (MCP) only applies the translation motion model. However, in the real world, there may be a variety of motions, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, the simplified affine transformation motion compensated prediction is applied to the 4-parameter affine model and the 6-parameter affine model. Figures 16A-16B As shown, by using a 4-parameter affine model ( Fig.16A ) and the two control point motion vectors (CPMV) for the 6-parameter affine model ( Fig. 16B ) to describe the affine motion field of the block.

[0202] The motion vector field (MVF) of a block is described by the following equation with the 4-parameter affine model in equation (1) (where the 4 parameters are defined as variables a, b, e and f) and the 6-parameter affine model in equation (2) (where the 6 parameters are defined as variables a, b, c, d, e and f):

[0203]

[0204]

[0205] Where (mv h0,mv h 0) is the motion vector of the upper left control point, (mv h 1,mv h 1) is the motion vector of the upper right control point, (mv h 2,mv h 2) is the motion vector of the motion vector control point in the lower left corner. All three motion vectors are called control point motion vectors (CPMVs), (x, y) represents the coordinates of the representative point relative to the upper left corner sample point in the current block, and (mv h (x,y),mv v (x,y)) is the motion vector derived for the sample at (x,y). The CP motion vector can be signaled (e.g. in affine AMVP mode) or derived on the fly (e.g. in affine Merge mode). w and h are the width and height of the current block. In practice, division is performed by rounding and right shifting. In VTM, the representative point is defined as the center position of the sub-block, for example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample in the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2). For each sub-block (i.e. 4x4 in VTM), the representative point is used to derive the motion vector for the entire sub-block.

[0206] To further simplify motion compensation prediction, sub-block based affine transformation prediction is applied. To derive the motion vector of each M×N (in the current VVC, both M and N are set to 4) sub-block, as Fig.17 As shown, the motion vector of the center sample of each sub-block is calculated according to equations (1) and (2) and rounded to a fractional precision of 1 / 16. Then, a motion compensation interpolation filter suitable for 1 / 16 pixels is applied to generate a prediction for each sub-block with a derived motion vector. The affine mode introduces an interpolation filter of 1 / 16 pixels.

[0207] After MCP, the high-precision motion vector of each sub-block is rounded and saved with the same precision as the regular motion vector.

[0208] 2.3.3.1 Signaling of Affine Prediction

[0209] Similar to the translational motion model, due to affine prediction, there are also two modes for signaling auxiliary information. They are AFFINE_INTER and AFFINE_MERGE modes.

[0210] 2.3.3.2AF_INTER mode

[0211] AF_INTER mode can be applied for CUs with width and height both greater than 8. A CU-level affine flag is signaled in the bitstream to indicate whether AF_INTER mode is used.

[0212] In this mode, for each reference picture list (list 0 or list 1), the affine AMVP candidate list consists of three types of affine motion predictors in the following order, where each candidate includes the estimated CPMV of the current block. Fig. 20 The difference between the best CPMV found and the estimated CPMV is calculated by using mv0, mv1, mv2 in mv1. In addition, the index of the affine AMVP candidate from which the estimated CPMV is derived is further signaled.

[0213] 1) Inherited affine motion predictor

[0214] The checking order is similar to the spatial MVP in HEVC AMVP list construction. First, the left-inherited affine motion predictor is derived from the first block in {A1, A0}, which is affine-encoded and has the same reference picture as the current block. Second, the above-inherited affine motion predictor is derived from the first block in {B1, B0, B2}, which is affine-encoded and has the same reference picture as the current block. Fig.19 Five blocks A1, A0, B1, B0, B2 are shown in FIG.

[0215] Once a neighboring block is found to be coded in affine mode, the CPMV of the codec covering the neighboring block is used to derive the predictor of the CPMV of the current block. For example, if A1 is coded in non-affine mode and A0 is coded in 4-parameter affine mode, the left-inherited affine MV predictor will be derived from A0. In this case, the CPMV of the CU covering A0, such as Fig. 21B CPMV in the upper left corner and CPMV in the upper right corner As shown, the estimated CPMV for deriving the current block is composed of the upper left (coordinate (x0, y0)), upper right (coordinate (x1, y1)) and lower right (coordinate (x2, y2)) positions of the current block. express.

[0216] 2) Constructing an affine motion predictor

[0217] The constructed affine motion predictor consists of control point motion vectors (CPMVs) derived from adjacent inter-codec blocks with the same reference picture, as Fig. 20 If the current affine motion model is 4-parameter affine, the number of CPMVs is 2. Otherwise, if the current affine motion model is 6-parameter affine, the number of CPMVs is 3. The CPMV in the upper left corner It is derived from the MV at the first block in the group {A, B, C} that is inter-coded and has the same reference picture as the current block. It is derived from the MV at the first block in the {D, E} group, which is inter-coded and has the same reference picture as the current block. is derived from the MV at the first block in the {F, G} group that is inter-coded and has the same reference picture as the current block.

[0218] - If the current affine motion model is 4-parameter affine, only when and When all are known, the constructed affine motion predictor is inserted into the candidate list. Used as the estimated CPMV of the upper left (coordinate (x0, y0)) and upper right (coordinate (x1, y1)) positions of the current block.

[0219] - If the current affine motion model is 6-parameter affine, only when and When all are known, the constructed affine motion predictor is inserted into the candidate list, that is, and Used as the estimated CPMV of the upper left (coordinate (x0, y0)), upper right (coordinate (x1, y1)), and lower right (coordinate (x2, y2)) positions of the current block.

[0220] When inserting the constructed affine motion predictor into the candidate list, no clipping process is applied.

[0221] 3) Conventional AMVP motion predictor

[0222] Apply the following until the number of affine motion predictors reaches a maximum.

[0223] 1) By setting all CPMVs equal to (if available) to derive an affine motion predictor.

[0224] 2) By setting all CPMVs equal to (if available) to derive an affine motion predictor.

[0225] 3) By setting all CPMVs equal to (if available) to derive an affine motion predictor.

[0226] 4) An affine motion predictor is derived by setting all CPMVs equal to HEVC TMVP (if available).

[0227] 5) An affine motion predictor is derived by setting all CPMVs to zero MV.

[0228] Notice, It has been derived in the constructed affine motion predictor.

[0229] Fig.18A An example of a 4-parameter affine model is shown. Fig.18B An example of a 6-parameter affine model is shown.

[0230] Fig.19 An example of MVP of AF_INTER of inherited affine candidate is shown.

[0231] Fig. 20 An example of MVP of AF_INTER of the constructed affine candidate is shown.

[0232] In AF_INTER mode, when using 4 / 6 parameter affine mode, 2 / 3 control points are required, and therefore, 2 / 3 MVDs need to be encoded and decoded for these control points, as shown in Figure 18. In the existing implementation, it is proposed to derive MVs as follows, i.e., predict mvd1 and mvd2 from mvd0.

[0233]

[0234]

[0235]

[0236] in mvd i and mv1 are the predicted motion vector, motion vector difference and motion vector of the upper left pixel (i=0), upper right pixel (i=1) or lower left pixel (i=2), respectively. Fig.18B Please note that the sum of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the sum of the two components, i.e., newMV=mvA+mvB, and the two components of newMV are set to (xA+xB) and (yA+yB), respectively.

[0237] 2.3.3.3 AF_MERGE Mode

[0238] When a CU is applied in AF_MERGE mode, it obtains the first block encoded and decoded in affine mode from the valid neighboring reconstructed blocks. And the selection order of candidate blocks is from left, top, top right, bottom left to top left (represented by A, B, C, D, E in sequence), such as Fig.21A For example, if the neighboring lower left block is encoded and decoded in affine mode, such as Fig. 21B As shown in A0 in the figure, the control point (CP) motion vector mv0 of the upper left corner, upper right corner and lower left corner of the adjacent CU / PU containing block A is obtained. N 、mv1 N and mv2N . Based on mv0 N 、mv1 N and mv2 N Calculate the motion vector mv0 of the upper left corner / upper right / lower left corner of the current CU / PU C 、mv1 C and mv2 C (Only for 6-parameter affine model). It should be noted that in VTM-2.0, if the current block is affine coded, the sub-block located at the upper left corner (e.g., 4×4 block in VTM) stores mv0, and the sub-block located at the upper right corner stores mv1. If the current block is coded using the 6-parameter affine model, the sub-block located at the lower left corner stores mv2; otherwise (the current block is coded using the 4-parameter affine model), LB stores mv2'. The other sub-blocks store the MV for MC.

[0239] In deriving the CPMVmv0 of the current CU C 、mv1 C and mv2 C Afterwards, the MVF of the current CU is generated according to the simplified affine motion model equations (1) and (2). In order to identify whether the current CU is coded or decoded in AF_MERGE mode, an affine flag is signaled in the bitstream when at least one neighboring block is coded or decoded in affine mode.

[0240] In some existing implementations, the affine merge candidate list is constructed by the following steps:

[0241] 1) Insert inherited affine candidates

[0242] An inherited affine candidate is one that is derived from the affine motion model of its valid neighbor affine codec block. Up to two inherited affine candidates are derived from the affine motion model of the neighboring blocks and inserted into the candidate list. For the left predictor, the scan order is {A0, A1}; for the above predictors, the scan order is {B0, B1, B2}.

[0243] 2) Insert the constructed affine candidate

[0244] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (for example, 5), the constructed affine candidate is inserted into the candidate list. Constructing an affine candidate means constructing a candidate by combining neighbor motion information of each control point.

[0245] a) First, Fig. 22The motion information of the control point is derived from the specified spatial and temporal neighbors shown. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial positions used to predict CPk (k = 1, 2, 3); T is the temporal position used to predict CP4.

[0246] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.

[0247] The motion information for each control point is obtained according to the following priority order:

[0248] – For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is not available, use B3. If both B2 and B3 are not available, use A2. If none of these three candidates are available, the motion information of CP1 cannot be obtained.

[0249] – For CP2, the check priority is B1→B0;

[0250] – For CP3, the inspection priority is A1→A0;

[0251] – For CP4, use T.

[0252] b) Secondly, the combination of control points is used to construct affine merge candidates.

[0253] I. Motion information of three control points is required to construct a 6-parameter affine candidate. Three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the top left, top right, and bottom left control points.

[0254] II. Motion information of two control points is required to construct a 4-parameter affine candidate. Two control points can be selected from one of two combinations ({CP1, CP2}, {CP1, CP3}). These two combinations will be converted into a 4-parameter motion model represented by the top left and top right control points.

[0255] III. Insert the constructed combinations of affine candidates into the candidate list in the following order:

[0256] {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}

[0257] i. For each combination, check the reference index of list X for each CP, if they are all the same, then this combination has a valid CPMV for list X. If the combination does not have a valid CPMV for both list 0 and list 1, then the combination is marked as invalid. Otherwise, it is valid and the CPMV is put into the sub-block Merge list.

[0258] 3) Fill zero motion vectors

[0259] If the number of candidates in the affine merge candidate list is less than 5, zero motion vectors with zero reference index are inserted into the candidate list until the list is full.

[0260] More specifically, for the sub-block Merge candidate list, there are 4-parameter Merge candidates for unidirectional prediction (for P slices) and bidirectional prediction (for B slices) with MV set to (0, 0) and prediction direction set to list 0.

[0261] 2.3.4 Current Image Reference

[0262] Intra-block copying (also known as IBC or intra-picture block compensation), also known as current picture reference (CPR), is used in the HEVC Screen Content Codec extension (SCC). This tool is very effective for the encoding and decoding of screen content videos, because repeated patterns of text and graphic-rich content often appear in the same picture. Using previously reconstructed blocks with the same or similar patterns as predictors can effectively reduce the prediction error and thus improve the codec efficiency. An example of intra-block compensation is shown in Fig.23 Shown in.

[0263] Similar to the design of CRP in HEVC SCC, in VVC, the use of IBC mode is signaled at the sequence and picture levels. When IBC mode is enabled on the sequence parameter set (SPS), IBC mode can be enabled at the picture level. When IBC mode is enabled at the picture level, the current reconstructed picture will be considered as a reference picture. Therefore, no syntax changes at the block level are required on top of the existing VVC inter-frame mode to signal the use of IBC mode.

[0264] Key Features:

[0265] – is considered as a regular inter-frame mode. Therefore, Merge and Skip modes can also be used in IBC mode. The Merge candidate list construction is unified and contains Merge candidates from neighboring positions that are coded or decoded in IBC mode or HEVC inter-frame mode. Depending on the selected Merge index, the current block in Merge or Skip mode can be merged to a neighbor coded in IBC mode or to one of the regular inter-frame mode codes with different pictures as reference pictures.

[0266] – The block vector prediction and coding scheme for IBC mode reuses the scheme used for motion vector prediction and coding in HEVC inter mode (AMVP and MVD coding).

[0267] – Motion vectors (also called block vectors) in IBC mode are encoded and decoded with integer pixel precision, but are stored in memory with 1 / 16 pixel precision after decoding because quarter pixel precision is required during the interpolation and deblocking stages. When used for motion vector prediction in IBC mode, the stored vector predictor is right-shifted by 4.

[0268] – Search range: limited to the current CTU.

[0269] – CPR is not allowed when affine mode / triangle mode / GBI / weighted prediction is enabled.

[0270] 2.3.5 Merge List Design in VVC

[0271] VVC supports three different Merge list construction processes:

[0272] 1) Sub-block Merge candidate list: It includes ATMVP and affine merge candidates. Affine mode and ATMVP mode share a merge list construction process. Here, ATMVP and affine merge candidates can be added in sequence. The size of the sub-block merge list is signaled in the slice header, and the maximum value is 5.

[0273] 2) Unidirectional prediction TPM Merge list: For triangular prediction mode, even though the two partitions can choose their own Merge candidate indexes, they share one Merge list construction process for the two partitions. When constructing the Merge list, the spatial neighbors and two temporal blocks of the block are checked. The motion information derived from the spatial neighbors and temporal blocks is called regular motion candidates in our IDF. These regular motion candidates are further used to derive multiple TPM candidates. Note that the transform is performed at the whole block level, even though the two partitions may use different motion vectors to generate their own prediction blocks. The bidirectional prediction TPM Merge list size is fixed to 5.

[0274] 3) Conventional Merge List: For the remaining codec blocks, a Merge list construction process is shared. Here, spatial / temporal / HMVP, paired bidirectional prediction Merge candidates and zero motion candidates can be inserted in sequence. The size of the conventional Merge list is signaled in the slice header, and the maximum value is 6.

[0275] 2.3.5.1 Sub-block Merge Candidate List

[0276] It is recommended to put all sub-block related motion candidates into a separate Merge list except for the regular Merge list with non-sub-block Merge candidates.

[0277] The motion candidates related to the sub-blocks are placed in a separate Merge list, named "Sub-block Merge Candidate List".

[0278] In one example, the sub-block Merge candidate list includes affine Merge candidates, ATMVP candidates, and / or sub-block-based STMVP candidates.

[0279] 2.3.5.1.1 Another ATMVP Implementation

[0280] In this contribution, the ATMVP Merge candidate in the regular Merge list is moved to the first position of the affine Merge list. In this way, all Merge candidates in the new list (i.e., the sub-block based Merge candidate list) are based on the sub-block codec tool.

[0281] 2.3.5.1.2 ATMVP in VTM-3.0

[0282] In VTM-3.0, in addition to the regular Merge candidate list, a special Merge candidate list is added, called the sub-block Merge candidate list (also known as the affine Merge candidate list). The sub-block Merge candidate list is filled with candidates in the following order:

[0283] b. ATMVP candidate (may be available or not);

[0284] c. Inherited affine candidates;

[0285] d. constructed affine candidates;

[0286] e. Fill with zero MV

[0287] The maximum number of candidates in the sub-block Merge candidate list (denoted as ML) is derived as follows:

[0288] 1) If the ATMVP usage flag (e.g., the flag may be named “sps_sbtmvp_enabled_flag”) is on (equal to 1), but the affine usage flag (e.g., the flag may be named “sps_affine_enabled_flag”) is off (equal to 0), then ML is set equal to 1.

[0289] 2) If the ATMVP usage flag is off (equal to 0) and the affine usage flag is off (equal to 0), then ML is set equal to 0. In this case, the sub-block Merge candidate list is not used.

[0290] 3) Otherwise (affine use flag is on (equal to 1), ATMVP use flag is on or off), send ML from encoder to decoder with new number. Valid ML is 0<=ML<=5.

[0291] When constructing the sub-block Merge candidate list, first check the ATMVP candidate. If any of the following conditions is true, skip the ATMVP candidate and do not put it into the sub-block Merge candidate list.

[0292] 1) The ATMVP usage flag is off;

[0293] 2) Any TMVP usage flag (e.g., when signaled at slice level, the flag may be named “slice_temporal_mvp_enabled_flag”);

[0294] 3) The reference picture with reference index 0 in reference list 0 is the same as the current picture (CPR)

[0295] ATMVP in VTM-3.0 is much simpler than in JEM. When generating ATMVP Merge candidates, the following process is applied:

[0296] a. Fig. 22 As shown, check the adjacent blocks A1, B1, B0, A0 to find the first inter-frame coded but not CPR coded block, which is denoted as block X;

[0297] b. Initialize TMV=(0,0). If there is an MV (denoted as MV') for block X, referring to the collocated reference picture (as signaled in the slice header), then set TMV equal to MV'.

[0298] c. Assuming the center point of the current block is (x0, y0), locate the corresponding position of the juxtaposed image pair (x0, y0) according to M = (x0 + MV'x, y0 + MV'y). Find the block Z covering M.

[0299] i. If Z is intra-coded, ATMVP is not available;

[0300] ii. If Z is inter-coded, the MVZ_0 and MVZ_1 of the two lists of block Z are scaled to (Reflist0 index 0) and (Reflist 1 index 0) as MVdefault0 and MVdefault1, and stored.

[0301] d. For each 8x8 sub-block, assuming its center point is (x0S, y0S), locate the corresponding position of (x0S, y0S) in the collocated image according to MS = (x0S + MV'x, y0S + MV'y). Find the block ZS covering MS.

[0302] i. If ZS is intra-frame coded and decoded, MVdefault0 and MVdefault1 are assigned to the sub-block;

[0303] ii. If ZS is inter-frame coded, the MVZS_0 and MVZS_1 of the two lists of block ZS are scaled to (Reflist 0 index 0) and (Reflist 1 index 0) and assigned to the sub-blocks;

[0304] MV cropping and masking in ATMVP:

[0305] When a corresponding position such as M or MS is located in a collocated picture, it is cropped into a predetermined area. The CTU size is S×S, and in VTM-3.0, S=128. Assuming that the upper left corner position of the collocated CTU is (xCTU, yCTU), the corresponding position M or MS at (xN, yN) will be cropped into the valid area xCTU<=xN <xCTU+S+4;yCTU<=yN<yCTU+S。

[0306] In addition to clipping, (xN, yN) is also masked as xN = xN & MASK, yN = yN & MASK, where MASK is equal to ~(2 N -1), and N=3, so that the lowest 3 bits are set to 0. Therefore, xN and yN must be multiples of 8 ("~" represents the bitwise complement operator).

[0307] Fig.24 Examples of valid corresponding areas in collocated pictures are shown.

[0308] 2.3.5.1.3 Syntax Design in Stripe Header

[0309]

[0310]

[0311] 2.3.5.2 Conventional Merge List

[0312] Different from the Merge list design, in VVC, a history-based motion vector prediction (HMVP) method is adopted.

[0313] In HMVP, the motion information of the previously coded blocks is stored. The motion information of the previously coded blocks is defined as HMVP candidates. Multiple HMVP candidates are stored in a table called HMVP table, and the table is maintained during the real-time encoding / decoding process. When encoding / decoding of a new slice starts, the HMVP table is cleared. Whenever there is an inter-coded block, the associated motion information is added to the last item of the table as a new HMVP candidate. The overall encoding and decoding process is described in Fig.25 Shown in.

[0314] HMVP candidates can be used in the AMVP and Merge candidate list construction process. Fig.26 The modified Merge candidate list construction process is shown (highlighted in blue). When the Merge candidate list is not full after inserting the TMVP candidate, the HMVP candidates stored in the HMVP table can be used to fill the Merge candidate list. Considering that a block usually has a higher correlation with the nearest neighboring block in terms of motion information, the HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first and the first entry is added last. Similarly, redundancy removal is applied to the HMVP candidates. Once the total number of available Merge candidates reaches the maximum number of Merge candidates allowed to be signaled, the Merge candidate list construction process terminates.

[0315] 2.4 MV rounding

[0316] In VVC, when MV is right-shifted, it is required to be rounded toward zero. For the MV (MVx, MVy) to be right-shifted by N bits, the result MV' (MVx', MVy') will be derived in a formulaic manner as follows:

[0317] MVx'=(MVx+((1<<N)> >1)-(MVx>=0?1:0))>>N;

[0318] MVy'=(MVy+((1<<N)> >1)-(MVy>=0?1:0))>>N;

[0319] 2.5 Example of Reference Picture Resampling (RPR)

[0320] ARC, also known as Reference Picture Resampling (RPR), has been incorporated into several existing and upcoming video standards.

[0321] In some embodiments of RPR, if the collocated picture has a different resolution than the current picture, TMVP is disabled. In addition, when the reference picture has a different resolution than the current picture, BDOF and DMVR are disabled.

[0322] To handle regular MC when the reference picture has a different resolution than the current picture, the interpolation part is defined as follows:

[0323] 8.5.6.3 Fractional Sample Interpolation Process

[0324] 8.5.6.3.1 Overview

[0325] The inputs to this process are:

[0326] – Luma position (xSb, ySb), specifies the upper left corner sample of the current codec subblock relative to the upper left corner luminance sample of the current picture;

[0327] –Variable sbWidth specifies the width of the current codec sub-block.

[0328] –Variable sbHeight specifies the height of the current codec sub-block.

[0329] – motion vector offset mvOffset,

[0330] – the refined motion vector refMvLX,

[0331] – The selected reference picture sample array refPicLX,

[0332] – Half-sample interpolation filter index hpelIfIdx,

[0333] – bidirectional optical flow flag bdofFlag,

[0334] –Variable cIdx specifies the color component index of the current block.

[0335] The output of this process is:

[0336] – The (sbWidth+brdExtSize)x(sbHeight+brdExtSize) array of predicted sample values, predSamplesLX.

[0337] The prediction block boundary extension size brdExtSize is derived as follows:

[0338] brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))? 2:0(8-752)

[0339] Set the variable fRefWidth equal to PicOutputWidthL of the reference picture in luma samples. Set the variable fRefHeight equal to PicOutputHeightL of the reference picture in luma samples. Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0340] – If cIdx is equal to 0, the following operations are applied:

[0341] –The scaling factor and its fixed-point representation are defined as

[0342] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0343] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0344] – Let (xIntL, yIntL) be the luma position given in full sample units, and (xFracL, yFracL) be the offset given in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0345] – Set the upper left coordinate of the boundary block of the reference sample fill (xSbIntL, ySbIntL) to be equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0346] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted brightness sample value predSamplesLX[x L ][y L ] is derived as follows:

[0347] –Let (refxSb L ,refySb L ) and (refx L ,refy L) is the luminance position (refMvLX[0], refMvLX[1]) pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in 1 / 16 sample units. The variable refxSb L ,refx L ,refySb L and refy L The derivation is as follows:

[0348] sbX L =((xSb<<4)+refMvLX[0])*hori_scale_fp (8-755)

[0349] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6 (8-756)

[0350] sb L =((ySb<<4)+refMvLX[1])*vert_scale_fp (8-757)

[0351] refyL=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[0352] – Variable xInt L ,yInt L ,xFrac L and yFrac L The derivation is as follows:

[0353] xInt L =refx L >>4 (8-759)

[0354] yInt L =refy L >>4 (8-760)

[0355] xFrac L =refx L &15 (8-761)

[0356] yFrac L =refy L &15 (8-762)

[0357] – If bdofFlag is equal to true (TRUE) or (sps_affine_prof_enabled_flag is equal to TRUE and inter_affine_flag[xSb][ySb] is equal to TRUE), and one or more of the following conditions are true, the predicted luma sample value predSamplesLX[xSb][ySb] is L ][y L ] is obtained by calling the luminance integer sample acquisition process specified in clause 8.5.6.3.3 with (xInt L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX are used as input to derive.

[0358] 1.x L is equal to 0.

[0359] 2.x L Equal to sbWidth+1.

[0360] 3.y L is equal to 0.

[0361] 4.y L Equal to sbHeight+1.

[0362] – Otherwise, the predicted brightness sample value predSamplesLX[x L ][y L ] by calling the 8-tap interpolation filter process for luma samples specified in Section 8.5.6.3.2 with (xIntL-(brdExtSize>0?1:0), yIntL-(brdExtSize>0?1:0)), (xFracL, yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are used as input to derive.

[0363] – Otherwise (cIdx is not equal to 0), apply the following operation:

[0364] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0365] – Set the upper left coordinate of the boundary block of the reference sample fill (xSbIntC, ySbIntC) to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0366] – For each chroma sample position (xC = 0..sbWidth-1, yC = 0..sbHeight-1) within the predicted chroma sample array predSamplesLX, the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0367] –Let (refxSb C ,refySb C ) and (refx C ,refy C ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in 1 / 32 sample units. The variable refxSb C ,efySb C , efx C and refy C The derivation is as follows:

[0368] sbX C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp(8-763)

[0369] refx C =((Sign(refxSb C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[0370] sb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp (8-765)

[0371] refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>>5 (8-766)

[0372] – Variable xInt C,yInt C 、xFrac C and Frac C The derivation is as follows:

[0373] xInt C =refx C >>5 (8-767)

[0374] yInt C =refy C >>5 (8-768)

[0375] xFrac C =refy C &31 (8-769)

[0376] yFrac C =refy C &31 (8-770)

[0377] – The predicted sample values ​​predSamplesLX[xC][yC] are derived by calling the procedure specified in clause 8.5.6.3.4 with (xIntC, yIntC), xFracC, yFracC), xSbIntC, ySbIntC), bWidth, bHeight and refP as input.

[0378] 8.5.6.3.2 Luma Sample Interpolation Filtering Process

[0379] The input of this program is

[0380] – Luminance position in full sample units (xInt L ,yInt L ),

[0381] – Luma position in fractional samples (xFrac L ,yFrac L ),

[0382] – Luminance position in full sample units (xSbInt L ,ySbInt L ), specifies the upper left sample point of the boundary block filled with reference samples relative to the upper left luma sample point of the reference picture,

[0383] – Luminance reference sample array refPicLXL,

[0384] – half-sample interpolation filter index hpelIfIdx,

[0385] – The variable sbWidth specifies the width of the current sub-block.

[0386] –Variable sbHeight specifies the height of the current sub-block.

[0387] – Luma position (xSb, ySb), specifies the top left corner sample of the current sub-block relative to the top left corner luma sample of the current picture;

[0388] The output of this process is the predicted luminance sample value predSampleLX L

[0389] The variables shift1, shift2, and shift3 are derived as follows:

[0390] – Set the variable shift1 to be equal to Min(4, BitDepth Y -8), sets the variable shift2 to 6, and sets the variable shift3 to Max(2,14-BitDepth Y ).

[0391] – Set the variable picW equal to pic_width_in_luma_samples and the variable picH equal to pic_height_in_luma_samples.

[0392] The luma interpolation filter coefficient f for each 1 / 16 fractional sample position p equal to xFracL or yFracL L [p] is derived as follows:

[0393] – If MotionModelIdc[xSb][ySb] is greater than 0, and sbWidth and sbHeight are both equal to 4, the luma interpolation filter coefficients f are specified in Table 8-12 L [p].

[0394] – Otherwise, specify the luma interpolation filter coefficients f in Table 8-11 according to hpelIfIdx L [p].

[0395] For i = 0..7, the luminance position in units of full samples (xInt i ,yInt i ) is derived as follows:

[0396] – If subpic_drawn_as_pic_flag[SubPicIdx] is equal to 1, the following applies:

[0397] xInt i=Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xInt L +i-3)(8-771)

[0398] yInt i =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yInt L +i-3) (8-772)

[0399] – Otherwise (subpic_aged_as_pic_flag[SubPicIdx] is equal to 0), the following operations apply:

[0400] xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[0401] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3): (8-773)

[0402] xInt L +i-3)

[0403] yInt i =Clip3(0,picH-1,yInt L +i-3)(8-774)

[0404] For i = 0..7, the brightness position in units of all samples is further modified as follows:

[0405] xInt i =Clip3(xSbInt L -3,xSbInt L +sbWidth+4,xInt i ) (8-775)

[0406] yInt i =Clip3(ySbInt L -3,ySbInt L +sbHeight+4,yInt i ) (8-776)

[0407] Predicted brightness sample value predSampleLX L The derivation is as follows:

[0408] – If xFrac L and yFrac L are all equal to 0, then predSampleLX L The value of is derived as follows:

[0409] predSampleLX L =refPicLX L [xInt3][yInt3]< <shift3 (8-777)

[0410] – Otherwise, if xFrac L is not equal to 0 and yFrac L If equal to 0, predSampleLX L The value of is derived as follows:

[0411]

[0412] – Otherwise, if xFrac L Equal to 0 and yFrac L If not equal to 0, predSampleLX L The value of is derived as follows:

[0413]

[0414] – Otherwise, if xFrac L is not equal to 0 and yFrac L If not equal to 0, predSampleLX L The value of is derived as follows:

[0415] – The derivation of the sample array temp[n] for n=0..7 is as follows:

[0416]

[0417] –Predicted brightness sample value predSampleLX L The derivation is as follows:

[0418]

[0419] Table 8-11 – Luma interpolation filter coefficients f for each 1 / 16 fractional sample position p L Specification of [p]

[0420]

[0421] Table 8-12 – Luma interpolation filter coefficients f for each 1 / 16 fractional sample position p in affine motion mode L[p] specification.

[0422]

[0423]

[0424] 8.5.6.3.3 Luminance integer sample acquisition process

[0425] The input to this process is:

[0426] – Luminance position in full sample units (xInt L ,yInt L ),

[0427] – Luminance reference sample array refPicLX L ,

[0428] The output of this process is the predicted luminance sample value predSampleLX L

[0429] Set the variable shift equal to Max(2,14-BitDepth Y ).

[0430] The variable picW is set equal to pic_width_in_luma_samples, and the variable picH is set equal to pic_height_in_luma_samples.

[0431] The luminance position (xInt, yInt) in units of all samples is derived as follows:

[0432] xInt=Clip3(0,picW-1,sps_ref_wraparound_enabled_flag? (8-782)

[0433] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):xInt L )

[0434] yInt=Clip3(0,picH-1,yInt L ) (8-783)

[0435] Predicted brightness sample value predSampleLX L The derivation is as follows:

[0436] predSampleLX L =refPicLXL [xInt][yInt]< <shift3 (8-784)

[0437] 8.5.6.3.4 Chroma sample interpolation process

[0438] The input to this process is:

[0439] – Chroma position in units of full samples (xInt C ,yInt C ),

[0440] – Chroma position in 1 / 32 fractional samples (xFrac C ,yFrac C ),

[0441] – Chroma position (xSbIntC, ySbIntC) in full sample units, specifies the top left sample of the border block filled with reference samples relative to the top left chroma sample of the reference picture,

[0442] – The variable sbWidth specifies the width of the current sub-block.

[0443] –Variable sbHeight specifies the height of the current sub-block.

[0444] – Chroma reference sample array refPicLX C .

[0445] The output of this process is the predicted chrominance sample value predSampleLX C

[0446] The variables shift1, shift2, and shift3 are derived as follows:

[0447] – Set the variable shift1 to be equal to Min Min(4, BitDepth C -8), set the variable shift2 to 6, and set the variable shift3 to Max(2,14-BitDepth C ).

[0448] – Set the variable picW C Set it equal to pic_width_in_luma_samples / SubWidthC, and set the variable picH C Set equal to pic_height_in_luma_samples / SubHeightC.

[0449] Table 8-13 specifies the value equal to xFrac C or yFracC The chroma interpolation filter coefficient f for each 1 / 32 fractional sample position p C [p].

[0450] Set the variable xOffset equal to (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0451] For i = 0..3, the chrominance position in units of all samples (xInt i ,yInt i ) is derived as follows:

[0452] – If subpic_tained_as_pic_flag[SubPicIdx] is equal to 1, the following applies:

[0453] xInt i =Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,xInt L +i) (8-785)

[0454] yInt i =Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yInt L +i) (8-786)

[0455] – Otherwise (subpic_aged_as_pic_flag[SubPicIdx] is equal to 0), the following operations apply:

[0456] xInt i =Clip3(0,picW C -1,sps_ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i-1): (8-787)

[0457] xInt C +i-1)

[0458] yInt i =Clip3(0,picH C -1,yInt C +i-1)(8-788)

[0459] For i = 0..3, the chrominance position (xInt i ,yInt i ) is further modified as follows:

[0460] xInt i =Clip3(xSbIntC-1,xSbIntC+sbWidth+2,xInt i ) (8-789)

[0461] yInt i =Clip3(ySbIntC-1,ySbIntC+sbHeight+2,yInt i ) (8-790)

[0462] Predicted chroma sample value predSampleLX C The derivation is as follows:

[0463] – If xFrac C and yFrac C are all equal to 0, then predSampleLX C The value of is derived as follows:

[0464] predSampleLX C =refPicLX C [xInt1][yInt1]< <shift3 (8-791)

[0465] – Otherwise, if xFrac C is not equal to 0 and yFrac C If equal to 0, predSampleLX C The value of is derived as follows:

[0466]

[0467] – Otherwise, if xFrac C is equal to 0 and yFrac C If not equal to 0, predSampleLX C The value of is derived as follows:

[0468]

[0469] – Otherwise, if xFrac C is not equal to 0 and yFrac C If not equal to 0, predSampleLX C The value of is derived as follows:

[0470] – The sample array temp[n] for n=0..3 is derived as follows:

[0471]

[0472] –Predicted chroma sample value predSampleLX C The derivation is as follows:

[0473] predSampleLX C =(f C [yFrac C ][0]*temp[0]+f C [yFrac C ][1]*temp[1]+f C [yFrac C ][2]*temp[2]+

[0474] (8-795)

[0475] f C [yFrac C ][3]*temp[3])>>shift2

[0476] Table 8-13 – Chroma interpolation filter coefficients f for each 1 / 32 fractional sample position p C Specification of [p]

[0477]

[0478]

[0479] 2.6 Embodiment using sub-pictures

[0480] Using the current syntax design of sub-pictures in the existing implementation, the position and size of the sub-pictures are derived as follows:

[0481]

[0482]

[0483] subpics_present_flag equal to 1 indicates that sub-picture parameters are currently present in the SPS RBSP syntax. subpics_present_flag equal to 0 indicates that sub-picture parameters are currently not present in the SPS RBSP syntax.

[0484] NOTE 2 – When the bitstream is the result of a sub-bitstream extraction process and contains only a subset of the sub-pictures of the input bitstream of the sub-bitstream extraction process, it may be necessary to set the value of subpics_present_flag equal to 1 in the RBSP of the SPSs.

[0485] max_subpics_minus1 plus 1 specifies the maximum number of subpictures that may be present in the CVS. max_subpics_minus1 must be in the range 0 to 254. The value 255 is reserved for future use by ITU-T|ISO / IEC.

[0486] subpic_grid_col_width_minus1 plus 1 specifies the width of each element of the sub-picture identifier grid in units of 4 samples. The length of the syntax element is Ceil(Log2(pic_width_max_in_luma_samples / 4)) bits.

[0487] The variable NumSubPicGridCols is derived as follows:

[0488] NumSubPicGridCols=

[0489] (pic_width_max_in_luma_samples+subpic_grid_col_width_minus1*4+3) /

[0490] (subpic_grid_col_width_minus1*4+4) (7-5)

[0491] subpic_grid_row_height_minus1 plus 1 specifies the height of each element in the sub-picture identifier grid in units of 4 samples. The length of the syntax element is

[0492] Ceil(Log2(pic_height_max_in_luma_samples / 4)) bits.

[0493] The variable NumSubPicGridRows is derived as follows:

[0494] NumSubPicGridRows=

[0495] (pic_height_max_in_luma_samples+subpic_grid_row_height_minus1*4+3) /

[0496] (subpic_grid_row_height_minus1*4+4) (7-6)

[0497] subpic_grid_idx[i][j] specifies the sub-picture index of the grid position (i, j). The length of the syntax element is Ceil(Log2(max_subpics_minus1+1)) bits.

[0498] The variables SubPicTop[subpic_grid_idx[i][j]], SubPicLeft[subpic_grid_idx[i][j]], SubPicWidth[subpic_grid_idx[i][j]], SubPicHeight[subpic_grid_idx[i][j]], and NumSubPics are derived as follows:

[0499]

[0500] subpic_treated_as_pic_flag[i] equal to 1 specifies that the i-th sub-picture of each coded picture in the CVS is to be treated as a picture in the decoding process that does not include loop filtering operations.

[0501] subpic_tained_as_pic_flag[i] equal to 0 specifies that the i-th subpicture of each coded picture in the CVS is not to be considered as a picture in a decoding process that does not include a loop filtering operation. If subpic_aged_as_pic_flag[i] is not present, the value of subpic_aged_as_pic_flag[i] is inferred to be equal to 0.

[0502] 2.7 Combined Intra Prediction (CIIP)

[0503] Combined Inter-Intra Prediction (CIIP) is used as a special Merge candidate in VVC. It can only be enabled for WxH blocks with W<=64 and H<=64.

[0504] 3. Disadvantages of existing implementation methods

[0505] In the current design of VVC, ATMVP has the following problems:

[0506] 1) Whether the application of ATMVP at the stripe level and CU level does not match;

[0507] 2) In the slice header, ATMVP can be enabled even if TMVP is disabled. At the same time, the ATMVP flag is signaled before the TMVP flag.

[0508] 3) Masking is always done regardless of whether the MV is compressed;

[0509] 4) The effective corresponding area may be too large;

[0510] 5) The derivation of TMV is too complicated;

[0511] 6) In some cases, ATMVP may not be available, so a better default MV is needed.

[0512] 7) The MV scaling method in ATMVP may not be efficient;

[0513] 8) ATMVP should take CPR into consideration;

[0514] 9) Even if affine prediction is disabled, the default zero affine merge candidate can be put into the list.

[0515] 10) The current picture is considered as a long-term reference picture, while other pictures are considered as short-term reference pictures. For ATMVP and TMVP candidates, the motion information from the temporal blocks in the collocated pictures is scaled to a reference picture with a fixed reference index (i.e., 0 for each reference picture list in the current design). However, when CPR mode is enabled, the current picture is also considered as a reference picture, and the current picture can be added to the reference picture list 0 (RefPicList0) with index equal to 0.

[0516] a. For TMVP, if the time domain block is encoded and decoded in CPR mode and the reference picture of RefPicList0 is a short reference picture, the TMVP candidate is set to unavailable.

[0517] b. If the reference picture of RefPicList0 with index equal to 0 is the current picture, and the current picture is an intra random access point (IRAP) picture, the ATMVP candidate is set to unavailable.

[0518] c. For the ATMVP sub-block within a block, when the motion information of the sub-block is derived from the time domain block, if the time domain block is encoded and decoded in CPR mode, the default ATMVP candidate (derived from the time domain block identified by the starting TMV and center position of the current block) is used to fill the motion information of the sub-block.

[0519] 11) Shift the MV right to integer precision, but do not follow the rounding rules in VVC.

[0520] 12) Since it points to collocated pictures, the MV (MVx, MVy) used in ATMVP to locate corresponding blocks in different pictures is used directly (e.g., TMV in 0). This is based on the assumption that all pictures have the same resolution. However, when RPR is enabled, different picture resolutions can be used. Similar problems exist for identifying corresponding blocks in collocated pictures to derive sub-block motion information.

[0521] 13) If the width or height of a block is greater than 32, and the maximum transform block size is 32, for CIIP coded blocks, the intra prediction signal is generated in CU size; and the inter prediction signal is generated in TU size (recursively divide the current block into multiple 32x32 blocks). Using CU to derive the intra prediction signal will result in lower efficiency.

[0522] There are some problems with the current design. First, if the reference picture of RefPicList0 with index equal to 0 is the current picture, and the current picture is not an IRAP picture, the ATMVP process will still be called, but since there is no temporal motion vector that can be scaled to the current picture, it cannot find any available ATMVP candidates.

[0523] 4. Examples of Embodiments and Technologies

[0524] The following itemized list of techniques and embodiments should be considered as examples to explain the general concept. These techniques should not be interpreted narrowly. In addition, these techniques can be combined in any way in encoder or decoder embodiments.

[0525] 1. Whether TMVP is allowed and / or whether CPR is used should be considered to decide / parse the maximum number of candidates in the sub-block Merge candidate list and / or decide whether ATMVP candidates should be added to the candidate list. The maximum number of candidates in the sub-block Merge candidate list is denoted as ML.

[0526] a) In one example, when determining or parsing the maximum number of candidates in the sub-block Merge candidate list, if the ATMVP usage flag is off (equal to 0) or TMVP is disabled, it is inferred that ATMVP is not applicable.

[0527] i. In one example, the ATMVP usage flag is on (equal to 1) and TMVP is disabled, and the ATMVP candidate is not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0528] ii. In one example, the ATMVP usage flag is on (equal to 1), and the TMVP is disabled, and the affine usage flag is off (equal to 0), then ML is set equal to 0, which means that the sub-block Merge is not applicable.

[0529] iii. In one example, the ATMVP usage flag is on (equal to 1), and TMVP is enabled, and the affine usage flag is off (equal to 0), then ML is set equal to 1.

[0530] b) In one example, when determining or parsing the maximum number of candidates in the sub-block Merge candidate list, if the ATMVP usage flag is off (equal to 0) or the collocated reference picture of the current picture is the current picture itself, it is inferred that ATMVP is not applicable.

[0531] i. In one example, the ATMVP usage flag is on (equal to 1) and the collocated reference picture of the current picture is the current picture itself, and the ATMVP candidate is not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0532] ii. In one example, the ATMVP usage flag is on (equal to 1), and the collocated reference picture of the current picture is the current picture itself, and the affine usage flag is off (equal to 0), then ML is set equal to 0, which means that sub-block Merge is not applicable.

[0533] iii. In one example, the ATMVP usage flag is on (equal to 1), and the collocated reference picture of the current picture is not the current picture itself, and the affine usage flag is off (equal to 0), then ML is set equal to 1.

[0534] c) In one example, when determining or parsing the maximum number of candidates in a sub-block Merge candidate list, if the ATMVP usage flag is off (equal to 0), or the reference picture with reference picture index 0 in reference list 0 is the current picture itself, it is inferred that ATMVP is not applicable.

[0535] i. In one example, the ATMVP usage flag is on (equal to 1), and the collocated reference picture with reference picture index 0 in reference list 0 is the current picture itself, and the ATMVP candidate is not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0536] ii. In one example, the ATMVP usage flag is on (equal to 1), and the reference picture with reference picture index 0 in reference list 0 is the current picture itself, and the affine usage flag is off (equal to 0), then ML is set to 0, which means that sub-block Merge is not applicable.

[0537] iii. In one example, the ATMVP usage flag is on (equal to 1), and the reference picture with reference picture index 0 in reference list 0 is not the current picture itself, and the affine usage flag is off (equal to 0), then ML is set equal to 1.

[0538] d) In one example, when determining or parsing the maximum number of candidates in the sub-block Merge candidate list, if the ATMVP usage flag is off (equal to 0), or the reference picture with reference picture index 0 in reference list 1 is the current picture itself, it is inferred that ATMVP is not applicable.

[0539] i. In one example, the ATMVP usage flag is on (equal to 1) and the collocated reference picture with reference picture index 0 in reference list 1 is the current picture itself, and the ATMVP candidate is not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0540] ii. In one example, the ATMVP usage flag is on (equal to 1), and the reference picture with reference picture index 0 in reference list 1 is the current picture itself, and the affine usage flag is off (equal to 0), then ML is set to equal to 0, which means that sub-block Merge is not applicable.

[0541] iii. In one example, the ATMVP usage flag is on (equal to 1), and the reference picture with reference picture index 0 in reference list 1 is not the current picture itself, and the affine usage flag is off (equal to 0), then ML is set equal to 1.

[0542] 2. It is proposed that if TMVP is disabled at the slice / slice / picture level, ATMVP is implicitly disabled and the ATMVP flag is not signaled.

[0543] a) In one example, the ATMVP flag is signaled after the TMVP flag in the slice header / slice header / PPS.

[0544] b) In one example, the ATMVP or / and TMVP flag may not be signaled in the slice header / slice header / PPS, but only in the SPS header.

[0545] 3. Whether and how to mask the corresponding position in ATMVP depends on whether and how to compress MV. Assume (xN, yN) is the corresponding position calculated using the coordinates of the current block / subblock and the starting motion vector (eg, TMV) in the collocated picture.

[0546] a) In one example, if compression of the MV is not required (e.g., sps_disable_motioncompression signaled in the SPS is 1), (xN, yN) is not masked; otherwise, (compression of the MV is required) (xN, yN) is masked as xN = xN & MASK, yN = yN & MASK, where MASK is equal to ~(2 M -1), and M may be an integer such as 3 or 4.

[0547] b) Assume that each 2 K x2 K The MV compression method used for MV storage results in the block share the same motion information, and the masking in the ATMVP process is defined as ~(2 M -1). It is suggested that K may not be equal to M, for example, M=K+1.

[0548] c) The MASK used in ATMVP and TMVP can be the same or different.

[0549] 4. In one example, the MV compression method may be flexible.

[0550] a) In one example, the MV compression method may be selected between no compression, 8x8 compression (M=3 in item 3.a), or 16x16 compression (M=4 in item 3.a).

[0551] b) In one example, the MV compression method may be signaled in the VPS / SPS / PPS / slice header / slice group header.

[0552] c) In one example, the MV compression method may be set differently in different standard profiles / levels / layers.

[0553] 5. The effective corresponding area in ATMVP can be adaptive;

[0554] a) For example, the valid corresponding area may depend on the width and height of the current block;

[0555] b) For example, the valid corresponding area may depend on the MV compression method;

[0556] i. In one example, if the MV compression method is not used, the effective corresponding area is smaller.

[0557] If the MV compression method is used, the effective corresponding area is larger.

[0558] 6. The valid corresponding area in ATMVP can be based on a basic area of ​​size M×N that is smaller than the CTU area. For example, the CTU size in VTM-3.0 is 128×128, while the basic area size can be 64×64. Assume that the width and height of the current block are W and H.

[0559] a) In one example, if W<=M and H<=N, meaning the current block is within the base region, then the valid corresponding regions in ATMVP are the extensions in the collocated base region and the collocated picture. Fig. 27 An example is shown.

[0560] i. For example, assuming that the upper left corner position of the concatenated basic area is (xBR, yBR), the corresponding position at (xN, yN) will be clipped to the valid area xBR <= xN <xBR+M+4;yBR<=yN<yBR+N。

[0561] Fig. 27 An example embodiment of the proposed valid region when the current block is within the base region (BR) is shown.

[0562] Fig.28 An example embodiment of a valid region when a current block is not within a base region is shown.

[0563] b) In one example, if W>M and H>N, which means that the current block is not in the basic region, the current block is divided into several parts. Each part has a separate valid corresponding region in ATMVP. For position A in the current block, its corresponding position B in the collocated block should be in the valid corresponding region of the part where position A is located.

[0564] i. For example, the current block is divided into non-overlapping basic regions. The valid corresponding region of a basic region is its collocated basic region and its extension in the collocated picture. Fig.28 An example is shown.

[0565] 1. For example, assume that position A in the current block is in a basic region R. Denote the collocated basic region of R in the collocated picture as CR. The corresponding position of A in the collocated block is position B. The upper left corner position of CR is (xCR, yCR), then position B at (xN, yN) will be cropped to the valid region xCR <= xN <xCR+M+4;yCR<=yN<yCR+N。

[0566] 7. It is proposed to derive the motion vector used in ATMVP to locate corresponding blocks in different pictures (for example, TMV in 2.3.5.1.2) as:

[0567] a) In one example, the TMV is always set equal to a default MV, such as (0, 0).

[0568] i. In one example, the default MV is signaled in VPS / SPS / PPS / slice header / slice group header / CTU / CU.

[0569] b) In one example, the TMV is set to an MV stored in the HMVP table by the following method;

[0570] i. If the HMVP list is empty, set TMV equal to the default MV, such as (0, 0)

[0571] ii. Otherwise (HMVP list is not empty),

[0572] 1. TMV can be set equal to the first element stored in the HMVP table;

[0573] 2. Alternatively, TMV may be set equal to the last element stored in the HMVP table;

[0574] 3. Alternatively, the TMV may simply be set equal to a specific MV stored in the HMVP table.

[0575] a. In one example, a particular MV references reference list 0.

[0576] b. In one example, a particular MV references Reference List 1.

[0577] c. In one example, a particular MV references a particular reference picture in reference list 0, such as a reference picture with index 0.

[0578] d. In one example, a particular MV references a particular reference picture in reference list 1, such as a reference picture with index 0.

[0579] e. In one example, a particular MV references a collocated picture.

[0580] 4. Alternatively, if a specific MV (e.g., mentioned in entry 3) stored in the HMVP table cannot be found, the TMV can be set equal to the default MV;

[0581] a. In one example, only the first element stored in the HMVP table is searched to find a particular MV.

[0582] b. In one example, only the last element stored in the HMVP table is searched to find a specific MV.

[0583] c. In one example, some or all elements stored in the HMVP table are searched to find a particular MV.

[0584] 5. Alternatively, furthermore, the TMV obtained from the HMVP cannot reference the current picture itself.

[0585] 6. Alternatively, if the TMV obtained from the HMVP table does not refer to a collocated picture, the TMV obtained from the HMVP table may be scaled to the collocated picture.

[0586] c) In one example, TMV is set to an MV of a specific neighboring block. Other neighboring blocks are not involved.

[0587] i. A specific neighboring block can be Fig. 22 Blocks A0, A1, B0, B1, B2 in.

[0588] ii. TMV can be set equal to the default MV if the following conditions are met

[0589] 1. The specific adjacent block does not exist;

[0590] 2. The specific adjacent block is not inter-frame coded;

[0591] iii. The TMV can only be set equal to a specific MV stored in a specific adjacent block.

[0592] 1. In one example, a particular MV references reference list 0.

[0593] 2. In one example, a particular MV references Reference List 1.

[0594] 3. In one example, a particular MV references a particular reference picture in reference list 0, such as a reference picture with index 0.

[0595] 4. In one example, a particular MV references a particular reference picture in reference list 1, such as a reference picture with index 0.

[0596] 5. In one example, a particular MV references a juxtaposed picture.

[0597] 6. If a specific MV stored in a specific adjacent block cannot be found, the TMV can be set equal to the default MV;

[0598] iv. If the TMV obtained from a specific neighboring block does not refer to a collocated picture, the TMV obtained from the specific neighboring block may be scaled to the collocated picture.

[0599] v. The TMV obtained from a specific neighboring block cannot reference the current picture itself.

[0600] 8. MVdefault0 and MVdefault1 used in ATMVP as disclosed in 2.3.5.1.2 can be derived as

[0601] a) In one example, MVdefault0 and MVdefault1 are set equal to (0, 0);

[0602] b) In one example, MVdefaultX (X=0 or 1) is derived from HMVP,

[0603] i. If the HMVP list is empty, set MVdefaultX equal to a predefined default MV, such as (0, 0).

[0604] 1. The predefined default MV is signaled in VPS / SPS / PPS / slice header / slice group header / CTU / CU.

[0605] ii. Otherwise (HMVP list is not empty),

[0606] 1. MVdefaultX can be set equal to the first element stored in the HMVP table.

[0607] 2. MVdefaultX may be set equal to the last element stored in the HMVP table.

[0608] 3. You can simply set MVdefaultX equal to a specific MV stored in the HMVP table.

[0609] a. In one example, a particular MV references a reference list X.

[0610] b. In one example, a particular MV references a particular reference picture in reference list X, such as a reference picture with index 0.

[0611] 4. If a specific MV stored in the HMVP table cannot be found, MVdefaultX can be set equal to the predefined default MV;

[0612] a. In one example, only the first element stored in the HMVP table is searched.

[0613] b. In one example, only the last element stored in the HMVP table is searched.

[0614] c. In one example, some or all elements stored in the HMVP table are searched.

[0615] 5. If the MVdefaultX obtained from the HMVP table does not refer to a collocated picture, the MVdefaultX obtained from the HMVP table may be scaled to the collocated picture.

[0616] 6. The MVdefaultX obtained from HMVP cannot reference the current picture itself.

[0617] c) In one example, MVdefaultX (X=0 or 1) is derived from neighboring blocks.

[0618] i. Adjacent blocks may include Fig. 22 Blocks A0, A1, B0, B1, B2 in.

[0619] 1. For example, only one of these blocks is used to derive MVdefaultX.

[0620] 2. Alternatively, some or all of these blocks are used to derive MVdefaultX.

[0621] a. Check these blocks in turn until a valid MVdefaultX is found.

[0622] 3. If no valid MVdefaultX is found from the selected neighboring block or blocks, it is set equal to a predefined default MV, such as (0, 0).

[0623] a. The predefined default MV is signaled in VPS / SPS / PPS / slice header / slice group header / CTU / CU.

[0624] ii.Under the following conditions, a valid MVdefaultX cannot be found from a specific neighboring block

[0625] 1. The specific adjacent block does not exist;

[0626] 2. The specific adjacent block is not inter-frame coded;

[0627] iii. You can only set MVdefaultX equal to a specific MV stored in a specific neighboring block;

[0628] 1. In one example, a particular MV references a reference list X.

[0629] 2. In one example, a particular MV references a particular reference picture in reference list X, such as a reference picture with index 0.

[0630] iv. The MVdefaultX obtained from a specific neighboring block can be scaled to a specific reference picture, such as the reference picture with index 0 in reference list X.

[0631] v. The MVdefaultX obtained from a specific neighboring block cannot reference the current picture itself.

[0632] 9. For sub-block or non-sub-block ATMVP candidates, if the temporal block of the sub-block / whole block in the collocated picture is coded and decoded with CPR mode, a default motion candidate may be utilized instead.

[0633] a) In one example, the default motion candidate may be defined as a motion candidate associated with the center position of the current block (eg, MVdefault0 and / or MVdefault1 used in ATMVP as disclosed in 2.3.5.1.2).

[0634] b) In one example, if available, the default motion candidate may be defined as a (0, 0) motion vector with both reference picture list reference picture indices equal to 0.

[0635] 10. It is proposed that the default motion information for the ATMVP process (e.g., MVdefault0 and MVdefault1 for ATMVP as disclosed in 2.3.5.1.2) can be derived based on the positioning of the position used in the sub-block motion information derivation process. With the proposed method, for the sub-block, since the default motion information is directly assigned, there is no need to further derive the motion information.

[0636] a) In one example, instead of using the center position of the current block, the center position of a sub-block (eg, the center sub-block) within the current block is utilized.

[0637] b) Examples of existing and proposed implementations are respectively Fig.29A and 29B Shown in.

[0638] 11. Proposing ATMVP candidates is always available for the following methods:

[0639] a) Assuming the center point of the current block is (x0, y0), locate the corresponding position of (x0, y0) in the collocated picture as M = (x0 + MV'x, y0 + MV'y). Find the block Z covering M. If Z is intra-coded, MVdefault0 and MVdefault1 are derived by some method proposed in Article 6.

[0640] b) Alternatively, without locating block Z to obtain motion information, some of the methods proposed in clause 8 are directly applied to obtain MVdefault0 and MVdefault1.

[0641] c) Alternatively, the default motion candidate used in the ATMVP process is always available. If it is set to be unavailable based on the current design (e.g., the temporal block is intra-coded), other motion vectors can be used instead of the default motion candidate.

[0642] i. In one example, the solution in international application PCT / CN2018 / 124639, which is incorporated herein by reference, may be applied.

[0643] d) Alternatively, furthermore, whether an ATMVP candidate is always available depends on other high-level grammatical information.

[0644] i. In one example, the ATMVP candidate may be always set as available only when the ATMVP enabled flag at the slice / slice / picture header or other video unit is set to true.

[0645] ii. In one example, the above method is applicable only when the ATMVP enabled flag in the slice header / picture header or other video unit is set to true and the current picture is not an IRAP picture and the current picture is not inserted into RefPicList0 with reference index equal to 0.

[0646] e) Assign a fixed index or a fixed index group to the ATMVP candidate. When the ATMVP candidate is always unavailable, the fixed index / group index can be inferred to other types of motion candidates (such as affine candidates).

[0647] 12. It is proposed that whether to put the zero-motion affine Merge candidate into the sub-block Merge candidate list should depend on whether affine prediction is enabled.

[0648] a) For example, if the affine usage flag is off (sps_affine_enabled_flag is equal to 0), the zero-motion affine Merge candidate is not put into the sub-block Merge candidate list.

[0649] b) Alternatively, in addition, a default motion vector candidate is added which is not an affine candidate.

[0650] 13. It is proposed that non-affine filling candidates can be put into the sub-block Merge candidate list.

[0651] a) If the sub-block Merge candidate list is not full, zero-motion non-affine filling candidates can be added.

[0652] b) When such a filling candidate is selected, the affine_flag of the current block should be set to 0.

[0653] c) Alternatively, if the Sub-block Merge candidate list is not full and the affine usage flag is off, then put the zero-motion non-affine filling candidate into the Sub-block Merge candidate list.

[0654] 14. Assume that MV0 and MV1 represent the MVs in reference list 0 and reference list 1 covering the blocks at the corresponding positions (for example, MV0 and MV1 may be MVZ_0 and MVZ_1 or MVZS_0 and MVZS_1 described in Section 2.3.5.1.2). MV0' and MV1' represent the MVs in reference list 0 and reference list 1 to be derived for the current block or subblock. Then MV0' and MV1' should be derived by scaling the following.

[0655] a) MV0, if the collocated picture is in reference list 1;

[0656] b) MV1, if the collocated picture is in reference list 0.

[0657] 15. When the current picture is considered as a reference picture whose index is set to M (e.g., 0) in a reference picture list X (PicRefListX, e.g., X=0), the ATMVP and / or TMVP enable / disable flag may be inferred to be false for a slice / slice or other type of video unit. Here, for PicRefListX during the ATMVP / TMVP process, M may be equal to the target reference picture index to which the motion information of the temporal block should be scaled.

[0658] a) Alternatively, furthermore, the above method is only applicable when the current picture is an intra random access point (IRAP) picture.

[0659] b) In one example, when the current picture is considered as a reference picture whose index in PicRefListX is set to M (e.g., 0) and / or the current picture is considered as a reference picture whose index in PicRefListY is set to N (e.g., 0), the ATMVP and / or TMVP enable / disable flag may be inferred to be false. The variables M and N represent the target reference picture index used in the TMVP or ATMVP process.

[0660] c) For the ATMVP process, the constraint confirmation bitstream shall follow the following rule: the collocated picture from which the motion information of the current block is derived shall not be the current picture.

[0661] d) Alternatively, when the above condition is true, the ATMVP or TMVP process is not called.

[0662] 16. It is proposed that if the reference picture whose index is set to M (eg, 0) in the reference picture list X (PicRefListX, eg, X=0) of the current block is the current picture, ATMVP may still be enabled for the block.

[0663] a) In one example, the motion information of all sub-blocks points to the current picture.

[0664] b) In one example, when the motion information of a sub-block is obtained from a time domain block, the time domain block should be encoded and decoded through at least one reference picture pointing to the current picture of the time domain block.

[0665] c) In one example, when the motion information of a sub-block is obtained from a temporal block, no scaling operation is applied.

[0666] 17. Regardless of whether ATMVP is used, the encoding and decoding method of aligning sub-block Merge indexes is used.

[0667] a) In one example, for the first L bits, they are context-coded. For the remaining bits, they are bypass-coded. In one example, L is set to 1.

[0668] b) Alternatively, for all bins, they are context encoded and decoded.

[0669] 18. The MV (MVx, MVy) used in ATMVP to locate corresponding blocks in different pictures (eg, TMV in 0) can be right-shifted to integer precision (expressed as (MVx', MVy')) using the same rounding method as in the MV scaling process.

[0670] a) Alternatively, the MV used in ATMVP to locate corresponding blocks in different pictures (eg, TMV in 0) can be right-shifted to integer precision using the same rounding method as in the MV averaging process.

[0671] b) Alternatively, the MV used in ATMVP to locate corresponding blocks in different pictures (eg, TMV in 0) may be right-shifted to integer precision using the same rounding method as in the Adaptive MV Resolution (AMVR) process.

[0672] 19. The MVs (MVx, MVy) used in ATMVP to locate corresponding blocks in different pictures (eg, TMV in 0) may be right-shifted to integer precision (denoted as (MVx', MVy')) by rounding towards zero.

[0673] a) For example, MVx'=(MVx+((1<<N)> >1)-(MVx>=0?1:0))>>N; N is an integer representing the MV resolution, for example N=4.

[0674] i. For example, MVx′=(MVx+(MVx>=0?7:8))>>4.

[0675] b) For example, MVy'=(MVy+((1<<N)> >1)-(MVy>=0?1:0))>>N; N is an integer representing the MV resolution, for example N=4.

[0676] i. For example, MVy′=(MVy+(MVy>=0?7:8))>>4.

[0677] 20. In one example, the MVs (MVx, MVy) in entries 18 and 19 are used to locate the corresponding blocks to derive the default motion information used in ATMVP, such as using the center position and shifted MV of the sub-block, or using the upper left corner position and shifted MV of the current block.

[0678] a) In one example, the MV (MVx, MVy) is used to locate the corresponding block to derive the motion information of the sub-block in the current block during the ATMVP process, such as using the center position of the sub-block and the shifted MV.

[0679] 21. The methods proposed in items 18, 19 and 20 can also be applied to other coding tools that need to locate reference blocks in different pictures or the current picture by motion vectors.

[0680] 22. The MV (MVx, MVy) used in ATMVP to locate corresponding blocks in different pictures (e.g., TMV in 0) can be scaled even if it points to a collocated picture.

[0681] a) In one example, if the width and / or height of the collocated picture (or the consistency window therein) is different from the width and / or height of the current picture (or the consistency window therein), the MV may be scaled.

[0682] b) Assume that the width and height of the collocated picture (of the consistency window) are denoted as W1 and H1 respectively. The width and height of the current picture (of the consistency window) are denoted as W2 and H2 respectively. Then MV(MVx, MVy) can be scaled as MVx'=MVx*W1 / W2, and MVy'=MVy*H1 / H2.

[0683] 23. The center point of the current block used to derive motion information during the ATMVP process (such as position (x0, y0) in 2.3.5.1.2) can be further modified by scaling and / or adding an offset.

[0684] a) In one example, if the width and / or height of the collocated picture (or the consistency window therein) is different from the width and / or height of the current picture (or the consistency window therein), the center point may be further modified.

[0685] b) Assume that the upper left position of the consistency window in the collocated picture is denoted as X1 and Y1. The upper left position of the consistency window defined in the current picture is denoted as X2 and Y2. The width and height (of the consistency window) of the collocated picture are denoted as W1 and H1, respectively. The width and height (of the consistency window) of the current picture are denoted as W2 and H2, respectively. Then (x0, y0) can be modified as x0' = (x0-X2)*W1 / W2+X1 and y0' = (y0-Y2)*H1 / H2+Y1.

[0686] i. Alternatively, x0'=x0*W1 / W2, y0'=y0*H1 / H2.

[0687] 24. The corresponding position used to derive motion information in the ATMVP process (e.g., position M in 2.3.5.1.2) can be further modified by scaling and / or adding an offset.

[0688] a) In one example, if the width and / or height of the collocated picture (or the consistency window therein) is different from the width and / or height of the current picture (or the consistency window therein), the corresponding position may be further modified.

[0689] b) Assume that the upper left position of the consistency window in the collocated picture is denoted as X1 and Y1. The upper left position of the consistency window defined in the current picture is denoted as X2 and Y2. The width and height (of the consistency window) of the collocated picture are denoted as W1 and H1, respectively. The width and height (of the consistency window) of the current picture are denoted as W2 and H2, respectively. Then M(x,y) can be modified as x'=(x-X2)*W1 / W2+X1 and y'=(y-Y2)*H1 / H2+Y1.

[0690] i. Alternatively, x'=x*W1 / W2, y'=y*H1 / H2.

[0691] Sub-image related

[0692] 25. In one example, if positions (i, j) and (i, j-1) belong to different sub-pictures, the width of sub-picture S ending at the (j-1) column can be set equal to j minus the leftmost column of sub-picture S.

[0693] a) Examples based on existing implementations are highlighted below.

[0694]

[0695] 26. In one example, the height of a sub-picture S that ends at (NumSubPicGridRows-1) rows may be set equal to (NumSubPicGridRows-1) minus the topmost row of the sub-picture S plus 1.

[0696] a) Examples based on existing implementations are highlighted below.

[0697]

[0698]

[0699] 27. In one example, the width of a sub-picture S that ends at (NumSubPicGridColumns-1) columns may be set equal to (NumSubPicGridColumns-1) minus the leftmost column of the sub-picture S plus 1.

[0700] a) Examples based on existing implementations are highlighted below.

[0701]

[0702]

[0703] 28. The sub-picture grid must be an integer multiple of the CTU size.

[0704] a) Examples based on existing implementations are highlighted below.

[0705] subpic_grid_col_width_minus1 plus 1 specifies the width of each element of the sub-picture identifier grid in units of CtbSizeY. The length of the syntax element is Ceil(Log2(pic_width_max_in_luma_samples / CtbSizeY)) bits.

[0706] The variable NumSubPicGridCols is derived as follows:

[0707] NumSubPicGridCols=(pic_width_max_in_luma_samples+subpic_grid_col_width_minus1*CtbSizeY+CtbSizeY-1) /

[0708] (subpic_grid_col_width_minus1*CtbSizeY+CtbSizeY) (7-5)

[0709] subpic_grid_row_height_minus1 plus 1 specifies the height of each element in the sub-picture identifier grid in units of 4 samples. The length of the syntax element is Ceil(Log2(pic_height_max_in_luma_samples / CtbSizeY)) bits.

[0710] The variable NumSubPicGridRows is derived as follows:

[0711] NumSubPicGridRows=(pic_height_max_in_luma_samples+subpic_grid_row_height_minus1*CtbSizeY+CtbSizeY-1) /

[0712] (subpic_grid_row_height_minus1*CtbSizeY+CtbSizeY)(7-6)

[0713] 29. Added consistency constraints to ensure that sub-images cannot overlap each other and all sub-images must cover the entire image.

[0714] a) Examples based on existing implementations are highlighted as follows

[0715] Any subpic_grid_idx[i][j] must be equal to idx if all of the following conditions are met:

[0716] i>=SubPicTop[idx]and i <SubPicTop[idx]+SubPicHeight[idx].

[0717] j>=SubPicLeft[idx]and j <SubPicLeft[idx]+SubPicWidth[idx].

[0718] Any subpic_grid_idx[i][j] must be different from idx if the following conditions are not met at the same time:

[0719] i>=SubPicTop[idx]and i <SubPicTop[idx]+SubPicHeight[idx].

[0720] j>=SubPicLeft[idx]and j <SubPicLeft[idx]+SubPicWidth[idx].

[0721] RPR related

[0722] 30. Signal a syntax element (eg, a flag) denoted as RPR_flag to indicate whether RPR can be used in a video unit (eg, a sequence). RPR_flag can be signaled in SPS, VPS, or DPS.

[0723] a) In one example, if it is signaled that RPR is not used (eg, RPR_flag is 0), then all widths / heights signaled in the PPS must be the same as the maximum width / maximum height signaled in the SPS.

[0724] b) In one example, if it is signaled that RPR is not used (e.g., RPR_flag is 0), all widths / heights in PPS are not signaled and are inferred to the maximum width / maximum height signaled in SPS.

[0725] c) In one example, if it is signaled that RPR is not to be used (eg, RPR_flag is 0), the consistency window information is not used during the decoding process. Otherwise (it is signaled that RPR is to be used), the consistency window information may be used during the decoding process.

[0726] 31. It is proposed that the interpolation filter used to derive the prediction block of the current block during motion compensation can be selected depending on whether the resolution of the reference picture is different from the current picture, or whether the width and / or height of the reference picture is larger than the width and / or height of the current picture.

[0727] a. In one example, when condition A is satisfied, an interpolation filter with fewer taps may be applied, where condition A depends on the dimensions of the current picture and / or the reference picture.

[0728] i. In one example, condition A is that the resolution of the reference picture is different from that of the current picture.

[0729] ii. In one example, condition A is that the width and / or height of the reference picture is greater than the width and / or height of the current picture.

[0730] iii. In one example, condition A is W1>a*W2 and / or H1>b*H2, where (W1, H1) represents the width and height of the reference picture, and (W2, H2) represents the width and height of the current picture, and a and b are two factors, for example a=b=1.5.

[0731] iv. In one example, condition A may also depend on whether bidirectional prediction is used.

[0732] 1) Condition A is satisfied only when bidirectional prediction is used for the current block.

[0733] v. In one example, condition A may depend on M and N, where M and N represent the width and height of the current block.

[0734] 1) For example, condition A is satisfied only when M*N<=T, where T is an integer such as 64.

[0735] 2) For example, condition A is satisfied only when M<=T1 or N<=T2, where T1 and T2 are integers, such as T1=T2=4.

[0736] 3) For example, condition A is satisfied only when M<=T1 and N<=T2, where T1 and T2 are integers, such as T1=T2=4.

[0737] 4) For example, condition A is satisfied only when M*N<=T or M<=T1 or N<=T2, where T, T1 and T2 are integers, such as T=64, T1=T2=4.

[0738] 5) In one example, the smaller condition in the above sub-items can be replaced by the larger condition. vi. In one example, a 1-tap filter is applied. In other words, unfiltered integer pixels are output as the interpolation result.

[0739] vii. In one example, when the resolution of the reference picture is different from the current picture, a bilinear filter is applied.

[0740] viii. In one example, when the resolution of the reference picture is different from that of the current picture or the width and / or height of the reference picture is greater than that of the current picture, a 4-tap filter or a 6-tap filter is applied.

[0741] 1) The 6-tap filter can also be used for affine motion compensation.

[0742] 2) The 4-tap filter can also be used for interpolation of chroma samples.

[0743] b. Whether and / or how to apply the method disclosed in item 31 may depend on the color component.

[0744] i. For example, the method is applied only to the luma component.

[0745] c. Whether and / or how to apply the method disclosed in item 31 may depend on the interpolation filtering direction.

[0746] i. For example, the method is only applied to horizontal filtering.

[0747] ii. For example, the method is only applied to vertical filtering.

[0748] CIIP related

[0749] 32. The intra prediction signal used in the CIIP process can be done at the TU level instead of the CU level (for example, using reference samples outside the TU instead of the CU).

[0750] a) In one example, if the CU width or height is larger than the maximum transform block size, the CU may be partitioned into multiple TUs, and intra / inter prediction may be generated for each TU, for example, using reference samples outside the TU.

[0751] b) In one example, if the maximum transform size K is less than 64 (such as K=32), the intra prediction used in CIIP will be performed recursively like in a regular intra coding block.

[0752] c) For example, a CIIP codec block of KM×KN is divided into MN K×K blocks, where M and N are integers, and intra prediction is performed on each K×K block. The intra prediction of the subsequently coded / decoded K×K block may depend on the reconstructed samples of the previously coded / decoded K×K block.

[0753] 5. Additional Example Embodiments

[0754] 5.1 Example #1: Syntax Design Example in SPS / PPS / Slice Header / Slice Group Header

[0755] Changes compared to the VTM3.0.1rc1 reference software are highlighted in large bold font as follows:

[0756]

[0757] 5.2 Example #2: Syntax Design Example in SPS / PPS / Slice Header / Slice Group Header

[0758] 7.3.2.1 Sequence Parameter Set RBSP Syntax

[0759]

[0760]

[0761] sps_sbtmvp_enabled_flag equal to 1 specifies that the subblock based temporal motion vector predictor can be used in the CVS for decoding pictures of all slices with slice_type not equal to 1. sps_sbtmvp_enabled_flag equal to 0 specifies that the subblock based temporal motion vector predictor is not used in the CVS. When sps_sbtmvp_enabled_flag is not present, it is inferred to be equal to 0.

[0762] five_minus_max_num_subblock_merge_cand specifies the maximum number of sub-block-based Merge motion vector prediction (MVP) candidates supported in a slice, subtracted from 5. When five_minus_max_num_subblock_merge_cand is not present, it is inferred to be equal to 5-sps_sbtmvp_enabled_flag. The maximum number of sub-block-based Merge MVP candidates MaxNumSubblockMergeCand is derived as follows:

[0763] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand(7-45)

[0764] The value of MaxNumSubblockMergeCand should be in the range of 0 to 5 (inclusive).

[0765] 8.3.4.2 Derivation of motion vectors and reference indices in sub-block merge mode

[0766] The inputs to this process are:

[0767] ..[No changes to the current VVC draft specification].

[0768] The output of this process is:

[0769] ...[no changes to the current VVC draft specification].

[0770] The variables numSbX, numSbY and subblock Merge candidate list subblockMergeCandList are derived through the following ordered steps:

[0771] When sps_sbtmvp_enabled_flag is equal to 1 and (the current picture is an IRAP and index 0 of reference picture list 0 is the current picture) is not true, the following steps apply:

[0772] The derivation process of the Merge candidate from the adjacent codec unit as specified in clause 8.3.2.3 is called with the luma codec block position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight and luma codec block width as input, and the output is the availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1 and available FlagB2, reference indexes refIdxLXA0, refIdxLXA1, refIdxLXB0, refIdxLXB1, and refIdxLXB2, prediction list utilization flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1, and predFlagLXB2, and motion vectors mvLXA0, mvLXA1, mvLXB0, mvLXB1, and mvLXB2, where X is 0 or 1.

[0773] Using luma position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight, availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1, reference indexes refIdxLXA0, refIdxLXA1, refIdxLXB0, refIdxLXB1, prediction list utilization flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1, and motion vectors mvLXA0, mvLXA1, mvLXB 0, mvLXB1 as input, calling the derivation process of sub-block based temporal Mergee candidates as specified in clause 8.3.4.3, and outputs the availability flag availableFlagSbCol, the number of luminance codec sub-blocks numSbX and numSbY in the horizontal and vertical directions, the reference index refIdxLXSbCol, the luminance motion vector mvLXSbCol[xSbIdx][ySbIdx] and the prediction list utilization flag predFlagLXSbCol[xSbIdx][ySbIdx], where xSbIdx = 0..numSbX–1, ySbIdx = 0..numSbY-1 and X is 0 or 1.

[0774] When sps_affine_enabled_flag is equal to 1, the sample positions (xNbA0, yNbA0), (xNbA1, yNbA1), (xNbA2, yNbA2), (xNbB0, yNbB0), (xNbB1, yNbB1), (xNbB2, yNbB2), (xNbB3, yNbB3), and the variables numSbX and numSbY are derived as follows:

[0775] [No changes to the current VVC draft specification].

[0776] 5.3 Example #3 of MV rounding

[0777] The syntax changes are based on existing implementations.

[0778] 8.5.5.3 Derivation of Sub-Block-Based Time-Domain Mergee Candidates

[0779] …

[0780] – The position of the collocated sub-block inside ColPic (xColSb, yColSb) is derived as follows.

[0781] 1. Apply the following steps:

[0782] yColSb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1< <CtbLog2SizeY)-1),

[0783] ySb+((tempMv[1]+8–(tempMv[1]>=0))>>4))

[0784] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[0785] xColSb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0786] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following steps apply:

[0787] xColSb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+(tempMv[0]+8+(tempMV[0]> =0))>>4))

[0788] …

[0789] 8.5.5.4 Derivation of Sub-Block-Based Temporal Merge Basic Motion Data

[0790] …

[0791] The position (xColCb, yColCb) of the concatenated block inside ColPic is derived as follows.

[0792] Apply the following steps:

[0793] yColCb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+8–(tempMv[1]> =0))>>4))

[0794] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[0795] xColCb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0796] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0) the following steps apply:

[0797] xColCb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0798] 5.4 Example #4: Second example of MV rounding

[0799] 8.5.5.3 Derivation of Sub-Block-Based Temporal Merge Candidates

[0800] The input to this process is:

[0801] – The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0802] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0803] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0804] – the availability flag availableFlagA1 of the adjacent codec unit,

[0805] – The reference index of the adjacent codec unit refIdxLXA1, where X is 0 or 1

[0806] – The prediction list of the adjacent codec unit uses the flag predFlagLXA1, where X is 0 or 1

[0807] – The motion vector mvLXA1 of the adjacent codec unit with 1 / 16 fractional sample accuracy, where X is 0 or 1. The output of this process is:

[0808] – Availability flag availableFlagSbCol,

[0809] – The number of luma codec sub-blocks numSbX and numSbY in the horizontal and vertical directions,

[0810] – reference indexes refIdxL0SbCol and refIdxL1SbCol,

[0811] – Luma motion vectors mvL0SbCol[xSbIdx][ySbIdx] and mvL1SbCol[xSbIdx][ySbIdx] with 1 / 16 fractional sample accuracy, where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1,

[0812] – The prediction list utilizes the flags predFlagL0SbCol[xSbIdx][ySbIdx] and predFlagL1SbCol[xSbIdx][ySbIdx], xSbIdx=0..numSbX-1, where ySbIdx=0..numSbY-1.

[0813] The availability flag availableFlagSbCol is derived as follows.

[0814] – If one or more of the following conditions are true, availableFlagSbCol is set equal to 0.

[0815] ––slice_temporal_mvp_enabled_flag is equal to 0.

[0816] ––sps_sbtmvp_enabled_flag is equal to 0.

[0817] ––cbWidth is less than 8.

[0818] ––cbHeight is less than 8.

[0819] – Otherwise, the following sequential steps apply:

[0820] 1. The position of the upper left sample point (xCtb, yCtb) of the luma codec tree block containing the current codec block and the position of the lower right center sample point (xCtr, yCtr) of the current luma codec block are derived as follows:

[0821] -xCtb=(xCb>>CtuLog2Size)< <CtuLog2Size (8-542)

[0822] -yCtb=(yCb>>CtuLog2Size)< <CtuLog2Size (8-543)

[0823] -xCtr=xCb+(cbWidth / 2) (8-544)

[0824] -yCtr=yCb+(cbHeight / 2) (8-545)

[0825] 2. Set the luma position (xColCtrCb, yColCtrCb) equal to the top left sample of the collocated luma codec block within ColPic that covers the position given by (xCtr, yCtr) relative to the top left luma sample of the collocated picture specified by ColPic.

[0826] 3. Invoke the derivation of the sub-block based temporal Merge base motion data as specified in clause 8.5.5.4 with position (xCtb, yCtb), position (xColCtrCb, yColCtrCb), availability flag availableFlagA1, prediction list utilization flag predFlagLXA1, reference index refIdxLXA1, and motion vector mvLXA1 as input (where X is 0 and 1), and with motion vector ctrMvLX, prediction list utilization flag ctrPredFlagLX for the collocated block, and temporal motion vector tempMv as output (where X is 0 and 1).

[0827] 4. The variable availableFlagSbCol is derived as follows:

[0828] – If both ctrPredFlagL0 and ctrPredFlagL1 are equal to 0, then set availableFlagSbCol equal to 0.

[0829] – Otherwise, set availableFlagSbCol equal to 1.

[0830] When availableFlagSbCol is equal to 1, the following steps apply:

[0831] – The variables numSbX, numSbY, sbWidth, sbHeight and refIdxLXSbCol are derived as follows:

[0832] -numSbX=cbWidth>>3(8-546)

[0833] -numSbY=cbHeight>>3(8-547)

[0834] -sbWidth=cbWidth / numSbX(8-548)

[0835] -sbHeight=cbHeight / numSbY(8-549)

[0836] -refIdxLXSbCol=0(8-550)

[0837] – For xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY–1, the motion vector mvLXSbCol[xSbIdx][ySbIdx] and the prediction list are derived using the flag predFlagLXSbCol[xSbIdx][ySbIdx] as follows:

[0838] –Specify the luminance position (xSb, ySb) of the upper left sample of the current codec sub-block relative to the upper left luminance sample of the current picture as follows:

[0839] -xSb=xCb+xSbIdx*sbWidth+sbWidth / 2(8-551)

[0840] -ySb=yCb+ySbIdx*sbHeight+sbHeight / 2(8-552)

[0841] – The position of the collocated sub-block in ColPic (xColSb, yColSb) is derived as follows.

[0842] 1. Apply the following steps:

[0843] -yColSb=Clip3(yCtb,

[0844] Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+((tempMv[1]+8–(tempMv[1]> =0? 1:0))>>4)) (8-553)

[0845] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[0846] -xColSb=Clip3(xCtb,

[0847] Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8–(tempMv[0]> =0? 1:0))>>4)) (8-554)

[0848] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following steps apply:

[0849] -xColSb=Clip3(xCtb,

[0850] Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8–(tempMv[0]> =0? 1:0))>>4)) (8-555)

[0851] –The variable currCb specifies the luma codec block in the current picture that covers the current codec sub-block.

[0852] – The variable colCb specifies the luma codec block within ColPic that covers the modified position given by ((xColSb>>3)<<3,(yColSb>>3)<<3).

[0853] – Set the luma position (xColCb, yColCb) equal to the top left sample of the collocated luma codec block specified by colCb relative to the top left luma sample of the collocated picture specified by ColPic.

[0854] – Call the derivation process of the collocated motion vector as specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL0 set equal to 0, and sbFlag set equal to 1, and assign the output to the motion vector mvL0SbCol[xSbIdx][ySbIdx] and availableFlagL0SbCol of the sub-block.

[0855] – Invoke the derivation process of the collocated motion vector as specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL1 set equal to 0, and sbFlag set equal to 1, and assign the output to the motion vector mvL1SbCol[xSbIdx][ySbIdx] and availableFlagL1SbCol of the sub-block.

[0856] – When availableFlagL0SbCol and availableFlagL1SbCol are both equal to 0, the following steps are applied for X being 0 and 1:

[0857] -mvLXSbCol[xSbIdx][ySbIdx]=ctrMvLX (8-556)

[0858] -predFlagLXSbCol[xSbIdx][ySbIdx]=ctrPredFlagLX (8-557)

[0859] 8.5.5.4 Derivation of Sub-Block-Based Temporal Merge Basic Motion Data

[0860] The input to this process is:

[0861] – The position of the top left sample point of the luma codec tree block containing the current codec block (xCtb, yCtb),

[0862] – The position of the upper left sample of the collocated luma codec block covering the lower right center sample (xColCtrCb, yColCtrCb).

[0863] – the availability flag availableFlagA1 of the adjacent codec unit,

[0864] – the reference index of the adjacent codec unit refIdxLXA1,

[0865] – The prediction list of the adjacent codec unit uses the flag predFlagLXA1,

[0866] – Motion vector mvLXA1 of the adjacent codec unit with 1 / 16 fractional sample accuracy.

[0867] The output of this process is:

[0868] – motion vectors ctrMvL0 and ctrMvL1,

[0869] – Prediction using flags ctrPredFlagL0 and ctrPredFlagL1,

[0870] – Temporal motion vector tempMv.

[0871] Set the variable tempMv as follows:

[0872] -tempMv[0]=0 (8-558)

[0873] -tempMv[1]=0 (8-559)

[0874] The variable currPic specifies the current picture.

[0875] When availableFlagA1 equals TRUE, the following steps apply:

[0876] – Set tempMv equal to mvL0A1 if all of the following are true:

[0877] –predFlagL0A1 is equal to 1,

[0878] –DiffPicOrderCnt(ColPic,RefPicList[0][refIdxL0A1]) is equal to 0,

[0879] – Otherwise, if all of the following are true, set tempMv equal to mvL1A1:

[0880] –slice_type is equal to B,

[0881] –predFlagL1A1 is equal to 1,

[0882] –DiffPicOrderCnt(ColPic,RefPicList[1][refIdxL1A1]) is equal to 0.

[0883] The position (xColCb, yColCb) of the concatenated block within ColPic is derived as follows.

[0884] – Apply the following steps:

[0885] -yColCb=Clip3(yCtb,

[0886] Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+8-(tempMv[1]> =0? 1:0))>>4)) (8-560)

[0887] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[0888] -xColCb=Clip3(xCtb,

[0889] Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8–(tempMv[0]> =0? 1:0))>>4)) (8-561)

[0890] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following steps apply:

[0891] -xColCb=Clip3(xCtb,

[0892] Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8–(tempMv[0]> =0? 1:0))>>4)) (8-562)

[0893] Set the colPredMode array equal to the prediction mode array CuPredMode[0] of the collocated picture specified by ColPic.

[0894] The motion vectors ctrMvL0 and ctrMvL1, and the prediction list are derived using the flags ctrPredFlagL0 and ctrPredFlagL1 as follows:

[0895] – If colPredMode[xColCb][yColCb] is equal to MODE_INTER, then the following steps apply:

[0896] –The variable currCb specifies the luma codec block covering (xCtrCb, yCtrCb) in the current picture.

[0897] – The variable colCb specifies the luma codec block that covers the modified position given by ((xColCb>>3)<<3,(yColCb>>3)<<3) within ColPic.

[0898] – Set the luma position (xColCb, yColCb) equal to the top left sample of the collocated luma codec block specified by colCb relative to the top left luma sample of the collocated picture specified by ColPic.

[0899] – Invoke the collocated motion vector derivation process specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL0 set equal to 0, and sbFlag set equal to 1, and assign the output to ctrMvL0 and ctrPredFlagL0.

[0900] – Invoke the collocated motion vector derivation process specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL1 set equal to 0, and sbFlag set equal to 1, and assign the output to ctrMvL1 and ctrPredFlagL1.

[0901] – Otherwise, apply the following steps:

[0902] -ctrPredFlagL0=0 (8-563)

[0903] -ctrPredFlagL1=0 (8-564)

[0904] 5.5 Example #5: Third example of MV rounding

[0905] 8.5.5.3 Derivation of Sub-block-based Temporal Merge Candidates

[0906] The input to this process is:

[0907] – The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0908] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0909] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0910] – the availability flag availableFlagA1 of the adjacent codec unit,

[0911] – The reference index of the adjacent codec unit refIdxLXA1, where X is 0 or 1,

[0912] – The prediction list of the adjacent codec unit uses the flag predFlagLXA1, where X is 0 or 1,

[0913] – The motion vector mvLXA1 of the adjacent codec unit with 1 / 16 fractional sample accuracy, where X is 0 or 1.

[0914] The output of this process is:

[0915] – Availability flag availableFlagSbCol,

[0916] – The number of luma codec sub-blocks numSbX and numSbY in the horizontal and vertical directions,

[0917] – reference indexes refIdxL0SbCol and refIdxL1SbCol,

[0918] – Luma motion vectors mvL0SbCol[xSbIdx][ySbIdx] and mvL1SbCol[xSbIdx][ySbIdx] with 1 / 16 fractional sample accuracy, where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1,

[0919] – The prediction list utilizes the flags predFlagL0SbCol[xSbIdx][ySbIdx] and predFlagL1SbCol[xSbIdx][ySbIdx], where xSbIdx=0..numSbX-1, ySbIdx=0..numSbY-1.

[0920] The availability flag availableFlagSbCol is derived as follows.

[0921] – If one or more of the following conditions are true, availableFlagSbCol is set equal to 0.

[0922] –slice_temporal_mvp_enabled_flag is equal to 0.

[0923] –sps_sbtmvp_enabled_flag is equal to 0.

[0924] –cbWidth is less than 8.

[0925] –cbHeight is less than 8.

[0926] – Otherwise, the following sequential steps apply:

[0927] 5. The position of the upper left sample point (xCtb, yCtb) of the luma codec tree block containing the current codec block and the position of the lower right center sample point (xCtr, yCtr) of the current luma codec block are derived as follows:

[0928] -xCtb=(xCb>>CtuLog2Size)< <CtuLog2Size (8-542)

[0929] -yCtb=(yCb>>CtuLog2Size)< <CtuLog2Size (8-543)

[0930] -xCtr=xCb+(cbWidth / 2) (8-544)

[0931] -yCtr=yCb+(cbHeight / 2) (8-545)

[0932] 6. Set the luma position (xColCtrCb, yColCtrCb) equal to the top left sample of the collocated luma codec block covering the position given by (xCtr, yCtr) within ColPic, relative to the top left luma sample of the collocated picture specified by ColPic.

[0933] 7. With position (xCtb, yCtb), position (xColCtrCb, yColCtrCb), availability flag availableFlagA1, prediction list utilization flag predFlagLXA1, reference index refIdxLXA1, and motion vector mvLXA1 as input, where X is 0 and 1, call the derivation process of sub-block based temporal Merge base motion data as specified in clause 8.5.5.4, and output the motion vector ctrMvLX, the prediction list utilization flag ctrPredFlagLX of the collocated block, and the temporal motion vector tempMv, where X is 0 and 1.

[0934] 8. The variable availableFlagSbCol is derived as follows:

[0935] – If both ctrPredFlagL0 and ctrPredFlagL1 are equal to 0, then set availableFlagSbCol equal to 0.

[0936] – Otherwise, set availableFlagSbCol equal to 1.

[0937] When availableFlagSbCol is equal to 1, the following steps apply:

[0938] – The variables numSbX, numSbY, sbWidth, sbHeight and refIdxLXSbCol are derived as follows:

[0939] -numSbX=cbWidth>>3(8-546)

[0940] -numSbY=cbHeight>>3(8-547)

[0941] -sbWidth=cbWidth / numSbX(8-548)

[0942] -sbHeight=cbHeight / numSbY(8-549)

[0943] -refIdxLXSbCol=0(8-550)

[0944] – For xSbIdx=0..numSbX-1 and ySbIdx=0..numSbY-1, the motion vector mvLXSbCol[xSbIdx][ySbIdx] and the prediction list are derived using the flag predFlagLXSbCol[xSbIdx][ySbIdx] as follows:

[0945] –Specify the luminance position (xSb, ySb) of the upper left sample of the current codec sub-block relative to the upper left luminance sample of the current picture as follows:

[0946] -xSb=xCb+xSbIdx*sbWidth+sbWidth / 2(8-551)

[0947] -ySb=yCb+ySbIdx*sbHeight+sbHeight / 2(8-552)

[0948] – The position of the collocated sub-block within ColPic (xColSb, yColSb) is derived as follows:

[0949] 1. Apply the following steps:

[0950] -yColSb=Clip3(yCtb,

[0951] Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+((tempMv[1]+(tempMv[1]> =0? 7:8))>>4)) (8-553)

[0952] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[0953] -xColSb=Clip3(xCtb,

[0954] Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4)) (8-554)

[0955] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following steps apply:

[0956] -xColSb=Clip3(xCtb,

[0957] Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4)) (8-555)

[0958] – The variable currCb specifies the luma codec block that covers the current codec sub-block in the current picture.

[0959] – The variable colCb specifies the luma codec block that covers the modified position given by ((xColSb>>3)<<3,(yColSb>>3)<<3) within ColPic.

[0960] – Set the luma position (xColCb, yColCb) equal to the top left sample of the collocated luma codec block specified by colCb relative to the top left luma sample of the collocated picture specified by ColPic.

[0961] – Invoke the collocated motion vector derivation process as specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL0 set equal to 0, and sbFlag set equal to 1, and assign the output to the motion vector mvL0SbCol[xSbIdx][ySbIdx] and availableFlagL0SbCol of the sub-block.

[0962] – Invoke the collocated motion vector derivation process as specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL1 set equal to 0, and sbFlag set equal to 1, and assign the output to the motion vector mvL1SbCol[xSbIdx][ySbIdx] and availableFlagL1SbCol of the sub-block.

[0963] – When availableFlagL0SbCol and availableFlagL1SbCol are both equal to 0, for X 0 and 1, apply the following steps:

[0964] -mvLXSbCol[xSbIdx][ySbIdx]=ctrMvLX (8-556)

[0965] -predFlagLXSbCol[xSbIdx][ySbIdx]=ctrPredFlagLX (8-557)

[0966] 8.5.5.4 Derivation of Sub-Block-Based Temporal Merge Basic Motion Data

[0967] The input to this process is:

[0968] – The position of the top left sample point of the luma codec tree block containing the current codec block (xCtb, yCtb),

[0969] – The position of the upper left sample of the collocated luma codec block covering the lower right center sample (xColCtrCb, yColCtrCb).

[0970] – the availability flag availableFlagA1 of the adjacent codec unit,

[0971] – the reference index of the adjacent codec unit refIdxLXA1,

[0972] – The prediction list of the adjacent codec unit uses the flag predFlagLXA1,

[0973] – Motion vector mvLXA1 of the adjacent codec unit with 1 / 16 fractional sample accuracy.

[0974] The output of this process is:

[0975] – motion vectors ctrMvL0 and ctrMvL1,

[0976] – The prediction list uses the flags ctrPredFlagL0 and ctrPredFlagL1,

[0977] – Temporal motion vector tempMv.

[0978] Set the variable tempMv as follows:

[0979] -tempMv[0]=0 (8-558)

[0980] -tempMv[1]=0 (8-559)

[0981] The variable currPic specifies the current picture.

[0982] When availableFlagA1 equals TRUE, the following steps apply:

[0983] – Set tempMv equal to mvL0A1 if all of the following are true:

[0984] –predFlagL0A1 is equal to 1

[0985] –DiffPicOrderCnt(ColPic,RefPicList[0][refIdxL0A1]) is equal to 0,

[0986] – Otherwise, if all of the following are true, set tempMv equal to mvL1A1:

[0987] –slice_type is equal to B,

[0988] –predFlagL1A1 is equal to 1,

[0989] –DiffPicOrderCnt(ColPic,RefPicList[1][refIdxL1A1]) is equal to 0.

[0990] The position (xColCb, yColCb) of the concatenated block within ColPic is derived as follows.

[0991] – Apply the following steps:

[0992] -yColCb=Clip3(yCtb,

[0993] Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+(tempMv[1]> =0?7:8))>>4)) (8-560)

[0994] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[0995] -xColCb=Clip3(xCtb,

[0996] Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4)) (8-561)

[0997] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), the following steps apply:

[0998] -xColCb=Clip3(xCtb,

[0999] Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4)) (8-562)

[1000] Set the colPredMode array equal to the prediction mode array CuPredMode[0] of the collocated picture specified by ColPic.

[1001] The motion vectors ctrMvL0 and ctrMvL1, and the prediction list are derived using the flags ctrPredFlagL0 and ctrPredFlagL1 as follows:

[1002] – If colPredMode[xColCb][yColCb] is equal to MODE_INTER, then the following steps apply:

[1003] –The variable currCb specifies the luma codec block covering (xCtrCb, yCtrCb) in the current picture.

[1004] – The variable colCb specifies the luma codec block that covers the modified position given by ((xColCb>>3)<<3,(yColCb>>3)<<3) within ColPic.

[1005] – Set the luma position (xColCb, yColCb) equal to the top left sample of the collocated luma codec block specified by colCb relative to the top left luma sample of the collocated picture specified by ColPic.

[1006] – Invoke the derivation process of the collocated motion vector specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL0 set equal to 0, and sbFlag set equal to 1, and assign the output to ctrMvL0 and ctrPredFlagL0.

[1007] – Invoke the derivation process of the collocated motion vector specified in clause 8.5.2.12 with as input currCb, colCb, (xColCb, yColCb), refIdxL1 set equal to 0, and sbFlag set equal to 1, and assign the output to ctrMvL1 and ctrPredFlagL1.

[1008] – Otherwise, apply the following steps:

[1009] -ctrPredFlagL0=0 (8-563)

[1010] -ctrPredFlagL1=0 (8-564)

[1011] 8.5.6.3 Fractional Sample Interpolation Process

[1012] 8.5.6.3.1 Overview

[1013] The input to this process is:

[1014] –Specify the luminance position (xSb, ySb) of the upper left sample of the current codec sub-block relative to the upper left luminance sample of the current picture;

[1015] –Variable sbWidth specifies the width of the current codec sub-block.

[1016] –Variable sbHeight specifies the height of the current codec sub-block.

[1017] – motion vector offset mvOffset,

[1018] – the refined motion vector refMvLX,

[1019] – The selected reference picture sample array refPicLX,

[1020] – Half-sample interpolation filter index hpelIfIdx,

[1021] – bidirectional optical flow flag bdofFlag,

[1022] –Variable cIdx specifies the color component index of the current block.

[1023] The output of this process is:

[1024] – The (sbWidth+brdExtSize)x(sbHeight+brdExtSize) array of predicted sample values, predSamplesLX.

[1025] The prediction block boundary extension size brdExtSize is derived as follows:

[1026] -brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))? 2:0 (8-752)

[1027] Set the variable fRefWidth equal to PicOutputWidthL of the reference picture in luma samples.

[1028] Set the variable fRefHeight equal to PicOutputHeightL of the reference picture in luma samples.

[1029] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[1030] – If cIdx is equal to 0, the following steps apply:

[1031] –The scaling factor and its fixed-point representation are defined as

[1032] -hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL (8-753)

[1033] -vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[1034] – Let (xIntL, yIntL) be the luma position given in full sample units, and (xFracL, yFracL) be the offset given in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[1035] – The upper left coordinate of the boundary block filled with reference samples (xSbInt L ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[1036] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted brightness sample value predSamplesLX[x L ][y L ] is derived as follows:

[1037] –Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position (refMvLX[0], refMvLX[1]) pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in 1 / 16 sample units. The variable refxSb L ,refx L 、refySb L and refy L The derivation is as follows:

[1038] -refxSb L=((xSb<<4)+refMvLX[0])*hori_scale_fp (8-755)

[1039] -refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6 (8-756)

[1040] -refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp (8-757)

[1041] -refyL=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[1042] – Variable xInt L ,yInt L 、xFrac L and yFrac L The derivation is as follows:

[1043] -xInt L =refx L >>4(8-759)

[1044] -yInt L =refy L >>4(8-760)

[1045] -xFrac L =refx L &15(8-761)

[1046] -yFrac L =refy L &15(8-762)

[1047] – Using6TapFlag is set to 1 if all of the following conditions are met:

[1048] –cbWidth[0][xSb][ySb]<=4||cbHeight[0][xSb][ySb]<=4||cbWidth[0][xSb][ySb]*cbHeight[0][xSb][ySb]<=64.

[1049] –PredFlagL0[xSb][ySb]==1&&PredFlagL1[xSb][ySb]==1.

[1050] – If bdofFlag is equal to TRUE or (sps_affine_prof_enabled_flag is equal to TRUE and inter_affine_flag[xSb][ySb] is equal to TRUE), and one or more of the following conditions are true, then the luma integer sample acquisition process is invoked as specified in clause 8.5.6.3.3 and inter_affine_flag[xSb][ySb] is equal to TRUE. L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX as input to derive the predicted brightness sample value predSamplesLX[x L ][y L ].

[1051] 1.x L is equal to 0.

[1052] 2.x L Equal to sbWidth+1.

[1053] 3.y L is equal to 0.

[1054] 4.y L Equal to sbHeight+1.

[1055] – Otherwise, the 8-tap interpolation filter process for luma samples is invoked as specified in clause 8.5.6.3.2 with (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) and using6TapFlag are used as input to derive the predicted luminance sample value predSamplesLX[xL][yL].

[1056] – Otherwise (cIdx is not equal to 0), apply the following steps:

[1057] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[1058] – Set the upper left coordinate (xSbIntC, ySbIntC) of the reference sample filled boundary block to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[1059] – For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[1060] –Let (refxSb C ,refySb C ) and (refx C ,refy C ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in 1 / 32 sample units. The variable refxSb C 、refySb C ,refx C and refy C The derivation is as follows:

[1061] -refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp (8-763)

[1062] -refx C =((Sign(refxSb C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[1063] -refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp (8-765)

[1064] -refy C =((Sign(refySbC )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>>5 (8-766)

[1065] – Variable xInt C ,yInt C 、xFrac C and yFrac C The derivation is as follows:

[1066] -xInt C =refx C >>5(8-767)

[1067] -yInt C =refy C >>5(8-768)

[1068] -xFrac C =refy C &31(8-769)

[1069] -yFrac C =refy C &31(8-770)

[1070] – Derives the predicted sample values ​​predSamplesLX[xC][yC] by calling the procedure specified in clause 8.5.6.3.4 with (xIntC,yIntC), (xFracC,yFracC), (xSbIntC,ySbIntC), sbWidth, sbHeight and refPicLX as input.

[1071] 8.5.6.3.2 Luma Sample Interpolation Filtering Process

[1072] The input to this process is:

[1073] – Luminance position in full sample units (xInt L ,yInt L ),

[1074] – Luma position in fractional samples (xFrac L ,yFrac L ),

[1075] – Luminance position in full sample units (xSbInt L ,ySbInt L ), the upper left sample of the boundary block used for reference sample filling relative to the upper left luminance sample of the reference picture,

[1076] – Luminance reference sample array refPicLX L ,

[1077] – Half-sample interpolation filter index hpelIfIdx,

[1078] – The variable sbWidth specifies the width of the current sub-block.

[1079] –Variable sbHeight specifies the height of the current sub-block.

[1080] –Specify the luminance position (xSb, ySb) of the upper left corner sample of the current sub-block relative to the luminance sample of the upper left corner of the current picture;

[1081] –using6TapFlag flag, specifies whether to use a 6-tap interpolation filter.

[1082] The output of this process is the predicted brightness sample value predSampleLX L

[1083] The variables shift1, shift2, and shift3 are derived as follows:

[1084] – Set the variable shift1 to be equal to Min(4, BitDepth Y -8), set the variable shift2 to 6, and set the variable shift3 to Max(2,14-BitDepth Y ).

[1085] – Set the variable picW equal to pic_width_in_luma_samples and the variable picH equal to pic_height_in_luma_samples.

[1086] Equal to xFrac L or yFrac L The luma interpolation filter coefficient f for each 1 / 16 fractional sample position p L [p]The derivation is as follows:

[1087] – If at least one of the following conditions is met, the luma interpolation filter coefficients f are specified in Table 8-12 L [p].

[1088] –MotionModelIdc[xSb][ySb] is greater than 0, and sbWidth and sbHeight are both equal to 4,

[1089] –using6TapFlag is equal to 1.

[1090] – Otherwise, the luma interpolation filter coefficients f specified in Table 8-11 depend on hpelIfIdx L [p].

[1091] For i = 0..7, the luminance position in units of full samples (xInt i ,yInt i ) is derived as follows:

[1092] – If subpic_tained_as_pic_flag[SubPicIdx] is equal to 1, the following steps apply:

[1093] -xInt i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xInt L +i-3)(8-771)

[1094] -yInt i =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yInt L +i-3)(8-772)

[1095] – Otherwise (subpic_aged_as_pic_flag[SubPicIdx] is equal to 0), the following steps apply:

[1096] -xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[1097] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3): (8-773)

[1098] xInt L +i-3)

[1099] -yInt i =Clip3(0,picH-1,yInt L +i-3)(8-774)

[1100] For i = 0..7, the brightness position in units of all samples is further modified as follows:

[1101] -xInti =Clip3(xSbInt L -3,xSbInt L +sbWidth+4,xInt i ) (8-775)

[1102] -yInt i =Clip3(ySbInt L -3,ySbInt L +sbHeight+4,yInt i ) (8-776)

[1103] Predicted brightness sample value predSampleLX L The derivation is as follows:

[1104] – If xFrac L and yFrac L are all equal to 0, then predSampleLX L The value of is derived as follows:

[1105] -predSampleLX L =refPicLX L [xInt3][yInt3]< <shift3 (8-777)

[1106] – Otherwise, if xFrac L is not equal to 0 and yFrac L If equal to 0, predSampleLX L The value of is derived as follows:

[1107]

[1108] – Otherwise, if xFrac L Equal to 0 and yFrac L If not equal to 0, predSampleLX L The value of is derived as follows:

[1109]

[1110] – Otherwise, if xFrac L is not equal to 0 and yFrac L If not equal to 0, predSampleLX L The value of is derived as follows:

[1111] – The sample array temp[n] for n=0..7 is derived as follows:

[1112]

[1113] –Predicted brightness sample value predSampleLX L The derivation is as follows:

[1114]

[1115] Table 8-11 – Luma interpolation filter coefficients f for each 1 / 16 fractional sample position p L Specification of [p]

[1116]

[1117] Table 8-12 – Luma interpolation filter coefficients f for each 1 / 16 fractional sample position p in affine motion mode L Specification of [p]

[1118]

[1119] Fig.30 3000 is a flow chart of a method 3000 for video processing. The method 3000 includes, at operation 3010, for a conversion between a current block of a video and a bitstream representation of the video, determining a maximum number of candidates (ML) in a sub-block based Merge candidate list and / or whether to add a sub-block based temporal motion vector prediction (SbTMVP) candidate to a sub-block based Merge candidate list based on whether a temporal motion vector prediction (TMVP) is enabled during the conversion or whether a current picture reference (CPR) codec mode is used for the conversion.

[1120] The method 3000 includes, at operation 3020, performing a conversion based on the determination.

[1121] Fig.31 31 is a flow chart of a method 3100 for video processing. The method 3100 includes, at operation 3110, for a conversion between a current block of a video and a bitstream representation of the video, determining a maximum number of candidates (ML) in a sub-block-based Merge candidate list based on whether temporal motion vector prediction (TMVP), sub-block-based temporal motion vector prediction (SbTMVP), and an affine coding mode are enabled during the conversion.

[1122] The method 3100 includes, at operation 3120, performing a conversion based on the determination.

[1123] Fig.32is a flow chart of a method 3200 for video processing. The method 3200 includes, at operation 3210, for a conversion between a current block of a first video segment of a video and a bitstream representation of the video, determining that a sub-block based motion vector prediction (SbTMVP) mode is disabled for the conversion because a temporal motion vector prediction (TMVP) mode is disabled at a first video segment level.

[1124] The method 3200 includes, at operation 3220, performing conversion based on determining that the bitstream representation conforms to a format that specifies whether an indication of the SbTMVP mode is included and / or a position of the indication of the SbTMVP mode relative to the indication of the TMVP mode in the Merge candidate list.

[1125] Fig.33 is a flow chart of a method 3300 for video processing. The method 3300 includes, at operation 3310, performing a conversion between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool or a temporal motion vector prediction (TMVP) tool and a bitstream representation of the video, selectively masking coordinates of corresponding positions of the current block or a sub-block of the current block using a mask based on compression of a motion vector associated with the SbTMVP tool or the TMVP tool, and the application of the mask includes calculating a bitwise AND operation between values ​​of the coordinates and values ​​of the mask.

[1126] Fig.34 34 is a flow chart of a method 3400 for video processing. The method 3400 includes, at operation 3410, applying a sub-block based motion vector prediction (SbTMVP) tool to the current block based on one or more characteristics of the current block of the video segment of the video to determine a valid corresponding area of ​​the current block.

[1127] The method 3400 includes, at operation 3420, performing a conversion between the current block and a bitstream representation of the video based on the determination.

[1128] Fig.35 is a flow chart of a method 3500 for video processing. The method 3500 includes, at operation 3510, determining a default motion vector for a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool.

[1129] The method 3500 includes, at operation 3520, based on the determination, performing conversion between the current block and a bitstream representation of the video, determining a default motion vector without obtaining a motion vector from a block covering a corresponding position in a collocated picture associated with a center position of the current block.

[1130] Fig.3636 is a flow chart of a method 3600 for video processing. The method 3600 includes, at operation 3610, for a current block of a video segment of a video, in a case where a current picture of the current block is a reference picture whose index is set to M in a reference picture list X, inferring that a sub-block based temporal motion vector prediction (SbTMVP) tool or a temporal motion vector prediction (TMVP) tool is disabled for the video segment, M and X are integers, and X=0 or X=1.

[1131] The method 3600 includes, at operation 3620, performing conversion between the current block and a bitstream representation of the video based on the inference.

[1132] Fig.37 37 is a flow chart of a method 3700 for video processing. The method 3700 includes, at operation 3710, for a current block of a video, determining to enable application of a sub-block based temporal motion vector prediction (SbTMVP) tool when a current picture of the current block is a reference picture whose index is set to M in a reference picture list X, where M and X are integers.

[1133] The method 3700 includes, at operation 3720, performing a conversion between the current block and a bitstream representation of the video based on the determination.

[1134] Fig.38 38 is a flow chart of a method 3800 for video processing. The method 3800 includes, at operation 3810, performing conversion between a current block of a video and a bitstream representation of the video, the current block being encoded and decoded using a sub-block based encoding and decoding tool, wherein performing the conversion includes encoding and decoding a sub-block Merge index using a plurality of binary bits (N) by a unified method when a sub-block based temporal motion vector prediction (SbTMVP) tool is enabled or disabled.

[1135] Fig.39 39 is a flow chart of a method 3900 for video processing. The method 3900 includes, at operation 3910, for a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool, determining a motion vector used by the SbTMVP tool to locate a corresponding block in a picture different from a current picture including the current block.

[1136] The method 3900 includes, at operation 3920, performing a conversion between the current block and a bitstream representation of the video based on the determination.

[1137] Fig.404000 is a flowchart of a method for video processing. The method 4000 includes, at operation 4010, for a conversion between a current block of a video and a bitstream representation of the video, determining whether to insert a zero-motion affine Merge candidate into a sub-block Merge candidate list based on whether affine prediction is enabled for conversion of the current block.

[1138] The method 4000 includes, at operation 4020, performing a conversion based on the determination.

[1139] Fig.41 41 is a flowchart of a method 4100 for video processing. The method 4100 includes, at operation 4110, for a conversion between a current block of a video and a bitstream representation of the video using a sub-block Merge candidate list, inserting a zero-motion non-affine filling candidate into the sub-block Merge candidate list if the sub-block Merge candidate list is not full.

[1140] The method 4100 includes, at operation 4120, performing a conversion after the inserting.

[1141] Fig.42 4200. The method 4200 includes, at operation 4210, determining a motion vector for a conversion between a current block of a video and a bitstream representation of the video using a rule that the motion vector is derived from one or more motion vectors of a block covering a corresponding position in a collocated picture.

[1142] The method 4200 includes, at operation 4220, performing conversion based on the motion vector.

[1143] Fig.43 43 is a flow chart of a method 4300 for video processing. The method 4300 includes, at operation 4310, for a conversion between a current block of a video and a bitstream representation of the video, determining to use a sub-block based temporal motion vector prediction (sbTMVP) tool with a default motion candidate for conversion when a temporal block associated with the current block or a sub-block of the current block in a collocated picture is coded using a codec mode in which a video unit is reconstructed based on previously coded samples in the same picture.

[1144] The method 4300 includes, at operation 4320, performing the converting based on the default motion candidate.

[1145] Fig.444400 is a flow chart of a method 4400 for video processing. The method 4400 includes, at operation 4410, for a sub-block based temporal motion vector prediction (sbTMVP) process as part of a conversion between a current block of video and a bitstream representation of the video, deriving default motion information for the sbTMVP process based on locations used in a sub-block motion information derivation process for a sub-block of the current block.

[1146] The method 4400 includes, at operation 4420, performing conversion based on default motion information.

[1147] Fig.45 4500 is a flow chart of a method 4500 for video processing. The method 4500 includes, for a conversion between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (sbTMVP) tool and a bitstream representation of the video, determining a modified motion vector for locating a corresponding block in a picture different from the current picture of the current block, wherein the modified motion vector is generated by right-shifting a motion vector used for prediction in the sbTMVP tool to integer precision.

[1148] The method 4500 includes, at operation 4520, performing the converting based on the determining.

[1149] Fig.46 4600. The apparatus 4600 may be used to implement one or more of the methods described herein. The apparatus 4600 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 4600 may include one or more processors 4602, one or more memories 4604, and video processing hardware 4606. The processor 4302 may be configured to implement one or more of the methods described herein. Although some embodiments may operate without a memory, (one or more) memories 4604 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 4606 may be used to implement some of the techniques described in this document in hardware circuits. In some embodiments, the video processing hardware 4606 may be at least partially internal to a processor 4602 (e.g., a graphics coprocessor).

[1150] In some embodiments, the video encoding and decoding method can use Fig.46 The described apparatus is implemented on a hardware platform.

[1151] Some embodiments of the disclosed technology include making a decision or determining to enable a video processing tool or mode. In an example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a video block, but will not necessarily modify the resulting bitstream based on the use of the tool or mode. In other words, the conversion from a video block to a bitstream representation of a video will use the video processing tool or mode when enabling the video processing tool or mode based on a decision or determination. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream based on the video processing tool or mode when it is known that the bitstream has been modified. In other words, the video processing tool or mode enabled based on a decision or determination will be used to perform the conversion from the bitstream representation of a video to a video block.

[1152] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In an example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion of video blocks to a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode enabled based on the decision or determination.

[1153] Fig.47 4700 is a block diagram illustrating an example video processing system 4700 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 4700. System 4700 may include an input 4702 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or may be received in a compressed or encoded format. Input 4702 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[1154] System 4700 may include a codec component 4704 that can implement various codecs or encoding methods described herein. Codec component 4704 can reduce the average bit rate of the video from the input 4702 of codec component 4704 to the output to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. As represented by component 4706, the output of codec component 4704 can be stored or sent via the connected communication. The storage or transmitted bitstream (or codec) representation of the video received at input 4702 can be used by component 4708 to generate pixel values ​​or displayable video sent to display interface 4710. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations of the reverse codec results will be performed by the codec.

[1155] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described herein may be embodied in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[1156] In some embodiments, the following technical solutions may be implemented:

[1157] A1. A video processing method, comprising: for conversion between a current block of a video and a bitstream representation of the video, based on whether temporal motion vector prediction (TMVP) is enabled during the conversion or whether a current picture reference (CPR) coding mode is used for the conversion, determining a maximum number of candidates (ML) in a sub-block based Merge candidate list and / or whether to add a sub-block based temporal motion vector prediction (SbTMVP) candidate to the sub-block based Merge candidate list; and performing the conversion based on the determination.

[1158] A2. The method according to Solution A1, wherein in case the TMVP tool is disabled or the SbTMVP tool is disabled, the use of the SbTMVP candidate is disabled.

[1159] A3. The method according to solution A2, wherein determining the ML comprises: excluding SbTMVP candidates from a sub-block based Merge candidate list based on whether the SbTMVP tool or the TMVP tool is disabled.

[1160] A4. A video processing method, comprising: for conversion between a current block of a video and a bitstream representation of the video, determining a maximum number of candidates (ML) in a sub-block-based Merge candidate list based on whether temporal motion vector prediction (TMVP), sub-block-based temporal motion vector prediction (SbTMVP) and an affine coding mode are enabled during the conversion; and performing the conversion based on the determination.

[1161] A5. The method according to solution A4, wherein, in case the affine codec mode is enabled, the ML is dynamically set and signaled in the bitstream representation.

[1162] A6. The method according to solution A4, wherein the ML is predefined with the affine codec mode disabled.

[1163] A7. The method according to solution A2 or A6, wherein determining the ML comprises: setting the ML to zero when the TMVP tool is disabled, the SbTMVP tool is enabled, and the affine codec mode of the current block is disabled.

[1164] A8. The method according to solution A2 or A6, wherein determining the ML comprises: setting the ML to one when the SbTMVP tool is enabled, the TMVP tool is enabled, and the affine codec mode of the current block is disabled.

[1165] A9. The method according to Solution A1, wherein the use of SbTMVP candidates is disabled if the SbTMVP tool is disabled or the collocated reference picture of the current picture of the current block is the current picture.

[1166] A10. The method according to solution A9, wherein determining the ML comprises: excluding SbTMVP candidates from the sub-block based Merge candidate list based on whether the SbTMVP tool is disabled or whether a collocated reference picture of the current picture is the current picture.

[1167] A11. The method according to solution A9, wherein determining the ML comprises: setting the ML to zero if the collocated reference picture of the current picture is the current picture and affine coding of the current block is disabled.

[1168] A12. The method according to solution A9, wherein determining the ML comprises: setting the ML to 1 when the SbTMVP tool is enabled, the collocated reference picture of the current picture is not the current picture, and affine coding of the current block is disabled.

[1169] A13. The method according to solution A1, wherein the use of SbTMVP candidates is disabled if the SbTMVP tool is disabled or the reference picture with reference picture index 0 in reference picture list 0 (L0) is the current picture of the current block.

[1170] A14. The method according to solution A13, wherein determining ML comprises: excluding SbTMVP candidates from the sub-block based Merge candidate list based on whether the SbTMVP tool is disabled or whether the reference picture with reference picture index 0 in L0 is the current picture.

[1171] A15. The method according to solution A13, wherein determining ML comprises: setting ML to zero when the SbTMVP tool is enabled, the reference picture with reference picture index 0 in L0 is the current picture, and affine coding of the current block is disabled.

[1172] A16. The method according to solution A13, wherein determining ML comprises: setting ML to one when the SbTMVP tool is enabled, the reference picture with reference picture index 0 in L0 is not the current picture, and affine coding of the current block is disabled.

[1173] A17. The method according to solution A1, wherein the use of SbTMVP candidates is disabled when the SbTMVP tool is disabled or the reference picture with reference picture index 0 in reference picture list 1 (L1) is the current picture of the current block.

[1174] A18. The method according to solution A17, wherein determining ML comprises: excluding SbTMVP candidates from the sub-block based Merge candidate list based on whether the SbTMVP tool is disabled or whether the reference picture with reference picture index 0 in L1 is the current picture.

[1175] A19. The method according to solution A17, wherein determining ML comprises setting ML to zero when the SbTMVP tool is enabled, the reference picture with reference picture index 0 in L1 is the current picture, and affine coding of the current block is disabled.

[1176] A20. The method of solution A17, wherein determining the ML comprises setting the ML to one when the SbTMVP tool is enabled, the reference picture with reference picture index 0 in L1 is not the current picture, and affine coding of the current block is disabled.

[1177] A21. A video processing method, comprising: for a conversion between a current block of a first video fragment of a video and a bitstream representation of the video, determining that a sub-block based motion vector prediction (SbTMVP) mode is disabled for the conversion because a temporal motion vector prediction (TMVP) mode is disabled at the first video segment level; and performing the conversion based on the determination, wherein the bitstream representation conforms to a format, the format specifying whether to include an indication of the SbTMVP mode and / or the position of the indication of the SbTMVP mode in a Merge candidate list relative to the indication of the TMVP mode.

[1178] A22. The method according to solution A21, wherein the first video segment is a sequence, a slice, a film or a picture.

[1179] A23. A method according to solution A21, wherein, due to including an indication of the TMVP mode at the first video segment level, the format specifies omitting an indication of the SbTMVP mode.

[1180] A24. The method of solution A21, wherein the format specifies that the indication of SbTMVP mode follows the indication of being in TMVP mode at the first video fragment level in decoding order.

[1181] A25. The method of any of solutions A21 to A24, wherein the format specifies omitting an indication of SbTMVP mode in case TMVP mode is indicated as disabled.

[1182] A26. The method of solution A21, wherein the format specifies including an indication of the SbTMVP mode at a sequence level of the video and omitting the indication of the SbTMVP mode at a second video fragment level.

[1183] A27. The method according to solution A26, wherein the second video segment at the second video segment level is a slice, a slice or a picture.

[1184] A28. According to the method of any one of solutions A1 to A27, the conversion generates the current block from the bitstream representation.

[1185] A29. A method according to any of solutions A1 to A27, wherein the conversion generates a bitstream representation from the current block.

[1186] A30. The method of any of solutions A1 to A27, wherein performing the conversion comprises parsing the bitstream representation based on one or more decoding rules.

[1187] A31. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement one or more of the methods of Solutions A1 to A30.

[1188] A32. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing one or more methods of solutions A1 to A30.

[1189] In some embodiments, the following technical solutions may be implemented:

[1190] B1. A video processing method, comprising: performing a conversion between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool or a temporal motion vector prediction (TMVP) tool and a bitstream representation of the video, wherein a mask is used to selectively mask the coordinates of corresponding positions of the current block or a sub-block of the current block based on compression of a motion vector associated with the SbTMVP tool or the TMVP tool, and wherein application of the mask includes calculating a bitwise AND operation between the value of the coordinate and the value of the mask.

[1191] B2. The method according to solution 1, wherein the coordinates are (xN, yN), and the mask (MASK) is equal to ~(2 M -1), where M is an integer, where application of the mask results in masked coordinates (xN', yN'), where xN'=xN&MASK and yN'=yN&MASK, and where "~" is a bitwise NOT operation and "&" is a bitwise AND operation.

[1192] B3. The method according to solution B2, wherein M=3 or M=4.

[1193] B4. A method according to solution B2 or B3, wherein the multiple dimensions are 2 K ×2 K The sub-blocks share the same motion information based on the compression of motion vectors, and wherein K is an integer not equal to M.

[1194] B5. The method according to solution B4, wherein M=K+1.

[1195] B6. The method of solution B1, wherein after determining that the motion vectors associated with the SbTMVP tool or the TMVP tool are not compressed, no mask is applied.

[1196] B7. The method according to any of solutions B1 to B6, wherein the mask of the SbTMVP tool is the same as the mask of the TMVP tool.

[1197] B8. The method of any of solutions B1 to B6, wherein the mask of the ATMVP tool is different from the mask of the TMVP tool.

[1198] B9. A method according to solution B1, wherein the type of compression is no compression, 8×8 compression or 16×16 compression.

[1199] B10. A method according to solution B9, wherein the type of compression is signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header or a slice group header.

[1200] B11. A method according to solution B9 or B10, wherein the type of compression is based on a standard profile, level or tier corresponding to the current block.

[1201] B12. A video processing method, comprising: based on one or more characteristics of a current block of a video fragment of a video, applying a sub-block based motion vector prediction (SbTMVP) tool to the current block to determine a valid corresponding area of ​​the current block; and based on the determination, performing conversion between the current block and a bitstream representation of the video.

[1202] B13. A method according to solution B12, wherein the one or more characteristics include the height or width of the current block.

[1203] B14. The method according to solution B12, wherein the one or more characteristics include a type of compression of a motion vector associated with the current block.

[1204] B15. A method according to solution B14, wherein, when the type of compression includes no compression, the valid corresponding area is a first size, and wherein, when the type of compression includes K×K compression, the valid corresponding area is a second size larger than the first size.

[1205] B16. A method according to solution B12, wherein the size of the valid corresponding area is based on a basic area of ​​size M×N having a size smaller than a codec tree unit (CTU) area, and wherein the size of the current block is W×H.

[1206] B17. A method according to solution B16, wherein the size of the CTU region is 128×128, and wherein M=64 and N=64.

[1207] B18. A method according to solution B16, wherein, when W≤M and H≤N, the valid corresponding region is an extension in the collocated base region and the collocated picture.

[1208] B19. A method according to solution B16, wherein, when it is determined that W>M and H>N, the current block is divided into several parts, and wherein each of the several parts includes a separate valid corresponding area for application of the SbTMVP tool.

[1209] B20. A video processing method, comprising: determining a default motion vector for a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool; and based on the determination, performing conversion between the current block and a bitstream representation of the video, wherein the default motion vector is determined without obtaining a motion vector from a block covering a corresponding position in a collocated picture associated with a center position of the current block.

[1210] B21. The method according to solution B20, wherein the default motion vector is set to (0,0).

[1211] B22. A method according to solution B20, wherein the default motion vector is derived from a history-based motion vector prediction (HMVP) table.

[1212] B23. The method according to solution B22, wherein, in case the HMVP table is empty, the default motion vector is set to (0,0).

[1213] B24. A method according to solution B22, wherein a default motion vector is predefined and signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a slice group header, a codec tree unit (CTU) or a codec unit (CU).

[1214] B25. The method according to Solution 22, wherein, in case the HMVP table is non-empty, the default motion vector is set to the first element stored in the HMVP table.

[1215] B26. The method according to solution B22, wherein, in case the HMVP table is non-empty, the default motion vector is set to the last element stored in the HMVP table.

[1216] B27. The method according to solution B22, wherein, in case the HMVP table is non-empty, the default motion vector is set to a specific motion vector stored in the HMVP table.

[1217] B28. A method according to solution B27, wherein the particular motion vector references reference list 0.

[1218] B29. A method according to solution B27, wherein the specific motion vector references reference list 1.

[1219] B30. A method according to solution B27, wherein the specific motion vector references a specific reference picture in reference list 0.

[1220] B31. A method according to solution B27, wherein the specific motion vector references a specific reference picture in reference list 1.

[1221] B32. A method according to solution B30 or B31, wherein the specific reference picture has index 0.

[1222] B33. A method according to solution B27, wherein the particular motion vector references a collocated picture.

[1223] B34. A method according to solution B22, wherein, in case the search process in the HMVP table cannot find a specific motion vector, the default motion vector is set to a predetermined default motion vector.

[1224] B35. A method according to solution B34, wherein the search process searches only the first element or only the last element of the HMVP table.

[1225] B36. A method according to solution B34, wherein the search process searches only a subset of the elements of the HMVP table.

[1226] B37. A method according to solution B22, wherein the default motion vector does not reference the current picture of the current block.

[1227] B38. The method according to solution B22, wherein in case the default motion vector does not refer to a collocated picture, the default motion vector is scaled to the collocated picture.

[1228] B39. A method according to solution B20, wherein a default motion vector is derived from a neighbouring block.

[1229] B40. A method according to solution B39, wherein the upper right corner of the adjacent block (A0) is directly adjacent to the lower left corner of the current block, or the lower right corner of the adjacent block (A1) is directly adjacent to the lower left corner of the current block, or the lower left corner of the adjacent block (B0) is directly adjacent to the upper right corner of the current block, or the lower right corner of the adjacent block (B1) is directly adjacent to the upper right corner of the current block, or the lower right corner of the adjacent block (B2) is directly adjacent to the upper left corner of the current block.

[1230] B41. A method according to solution B40, wherein the default motion vector is derived from only one of the neighboring blocks A0, A1, B0, B1 and B2.

[1231] B42. A method according to solution B40, wherein the default motion vector is derived from one or more of the neighboring blocks A0, A1, B0, B1 and B2.

[1232] B43. A method according to solution B40, wherein, in case a valid default motion vector cannot be found in any of the neighboring blocks A0, A1, B0, B1 and B2, the default motion vector is set to a predefined default motion vector.

[1233] B44. A method according to solution B43, wherein a predefined default motion vector is signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a slice group header, a codec tree unit (CTU) or a codec unit (CU).

[1234] B45. A method according to solutions B43 and B44, wherein the predefined default motion vector is (0,0).

[1235] B46. A method according to solution B39, wherein the default motion vector is set to a specific motion vector from a neighboring block.

[1236] B47. A method according to solution B46, wherein the particular motion vector references reference list 0.

[1237] B48. A method according to solution B46, wherein the specific motion vector references reference list 1.

[1238] B49. A method according to solution B46, wherein the specific motion vector references a specific reference picture in reference list 0.

[1239] B50. A method according to solution B46, wherein the specific motion vector references a specific reference picture in reference list 1.

[1240] B51. A method according to solution B49 or B50, wherein the particular reference picture has index 0.

[1241] B52. A method according to solution B46, wherein the particular motion vector references a collocated picture.

[1242] B53. A method according to solution B20, wherein a default motion vector is used in case the block covering the corresponding position in the collocated picture is intra-coded.

[1243] B54. Method according to solution B20, wherein the derivation method is modified in case a block covering the corresponding position in the collocated picture is not located.

[1244] B55. A method according to solution B20, wherein a default motion vector candidate is always available.

[1245] B56. The method according to solution B20, wherein, when it is determined that the default motion vector candidate is set to be unavailable, the default motion vector is derived in an alternative manner.

[1246] B57. A method according to solution B20, wherein the availability of the default motion vector is based on syntax information in the bitstream representation associated with the video segment.

[1247] B58. A method according to solution B57, wherein the syntax information includes an indication to enable the SbTMVP tool, and wherein the video segment is a slice, a tile, or a picture.

[1248] B59. A method according to solution B58, wherein the current picture of the current block is not an intra random access point (IRAP) picture, and the current picture is not inserted into a reference picture list 0 (L0) with a reference index of 0.

[1249] B60. A method according to solution B20, wherein when the SbTMVP tool is enabled, a fixed index or a fixed index group is assigned to a candidate associated with the SbTMVP tool, and wherein when the SbTMVP tool is disabled, a fixed index or a fixed index group is assigned to a candidate associated with a codec tool other than the SbTMVP tool.

[1250] B61. A video processing method, comprising: for a current block of a video fragment of a video, when the current picture of the current block is a reference picture whose index is set to M in a reference picture list X, inferring that a sub-block based temporal motion vector prediction (SbTMVP) tool or a temporal motion vector prediction (TMVP) tool is disabled for the video fragment, where M and X are integers and where X=0 or X=1; and based on the inference, performing conversion between the current block and a bitstream representation of the video.

[1251] B62. The method according to solution B61, wherein M corresponds to a target reference picture index, and for a reference picture list X used for the SbTMVP tool or the TMVP tool, the motion information of the temporal block is scaled to the target reference picture index.

[1252] B63. A method according to solution B61, wherein the current picture is an intra random access point (IRAP) picture.

[1253] B64. A video processing method, comprising: for a current block of a video, when a current picture of the current block is a reference picture whose index is set to M in a reference picture list X, determining to enable application of a sub-block based temporal motion vector prediction (SbTMVP) tool, where M and X are integers; and based on the determination, performing conversion between the current block and a bitstream representation of the video.

[1254] B65. A method according to solution B64, wherein the motion information corresponding to each sub-block of the current block references the current picture.

[1255] B66. A method according to solution B64, wherein motion information of a sub-block of the current block is derived from a time domain block, and wherein the time domain block is encoded and decoded with at least one reference picture that refers to the current picture of the time domain block.

[1256] B67. A method according to solution B66, wherein the transformation does not include a scaling operation.

[1257] B68. A video processing method, comprising: performing conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded and decoded using a sub-block based encoding and decoding tool, and wherein performing the conversion includes encoding and decoding a sub-block Merge index using a plurality of binary bits (N) through a unified method when a sub-block based temporal motion vector prediction (SbTMVP) tool is enabled or disabled.

[1258] B69. A method according to solution B68, wherein a first number of bins (L) of the plurality of bins are context-coded and wherein a second number of bins (NL) are bypass-coded.

[1259] B70. A method according to solution B69, wherein L=1.

[1260] B71. A method according to solution B68, wherein context encoding and decoding is performed on each of the multiple binary bits.

[1261] B72. According to the method of any one of solutions B1 to B71, the conversion generates the current block from the bitstream representation.

[1262] B73. A method according to any of solutions B1 to B71, wherein the conversion generates a bitstream representation from the current block.

[1263] B74. A method according to any of solutions B1 to B71, wherein performing the conversion comprises parsing the bitstream representation based on one or more decoding rules.

[1264] B75. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement one or more methods of solutions B1 to B74.

[1265] B76. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for performing a method according to one or more of solutions B1 to B74.

[1266] In some embodiments, the following technical solutions may be implemented:

[1267] C1. A video processing method, comprising: for a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (SbTMVP) tool, determining a motion vector used by the SbTMVP tool to locate a corresponding block in a picture different from a current picture including the current block; and based on the determination, performing conversion between the current block and a bitstream representation of the video.

[1268] C2. The method according to solution C1, wherein the motion vector is set to a default motion vector.

[1269] C3. A method according to solution C2, wherein the default motion vector is (0,0).

[1270] C4. A method according to solution C2, wherein the default motion vector is signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a slice group header, a codec tree unit (CTU) or a codec unit (CU).

[1271] C5. The method according to solution C1, wherein the motion vector is set to a motion vector stored in a History Based Motion Vector Prediction (HMVP) table.

[1272] C6. The method according to solution C5, wherein, in case the HMVP table is empty, the motion vector is set to a default motion vector.

[1273] C7. A method according to solution C6, wherein the default motion vector is (0,0).

[1274] C8. The method according to solution C5, wherein, in case the HMVP table is non-empty, the motion vector is set to the first motion vector stored in the HMVP table.

[1275] C9. The method according to solution 5, wherein, if the HMVP table is non-empty, the motion vector is set to the last motion vector stored in the HMVP table.

[1276] C10. The method according to solution C5, wherein in case the HMVP table is non-empty, the motion vector is set to a specific motion vector stored in the HMVP table.

[1277] C11. A method according to solution C10, wherein the particular motion vector references reference list 0.

[1278] C12. A method according to solution C10, wherein the specific motion vector references reference list 1.

[1279] C13. The method according to solution C10, wherein the specific motion vector refers to a specific reference picture in reference list 0.

[1280] C14. The method according to solution C10, wherein the specific motion vector refers to a specific reference picture in reference list 1.

[1281] C15. A method according to solution C13 or C14, wherein the specific reference picture has index 0.

[1282] C16. The method according to solution C10, wherein the specific motion vector refers to a collocated picture.

[1283] C17. A method according to solution C5, wherein in case the search process in the HMVP table cannot find a specific motion vector, the motion vector is set to a default motion vector.

[1284] C18. A method according to solution C17, wherein the search process searches only the first element or only the last element of the HMVP table.

[1285] C19. A method according to solution C17, wherein the search process searches only a subset of the elements of the HMVP table.

[1286] C20. The method according to solution C5, wherein the motion vectors stored in the HMVP table do not reference the current picture.

[1287] C21. The method according to solution C5, wherein, in case the motion vector stored in the HMVP table does not refer to a collocated picture, the motion vector stored in the HMVP table is scaled to the collocated picture.

[1288] C22. The method according to solution C1, wherein the motion vector is set to a specific motion vector of a specific neighboring block.

[1289] C23. A method according to solution C22, wherein the upper right corner of the neighboring block (A0) is directly adjacent to the lower left corner of the current block, or the lower right corner of the neighboring block (A1) is directly adjacent to the lower left corner of the current block, or the lower left corner of the neighboring block (B0) is directly adjacent to the upper right corner of the current block, or the lower right corner of the neighboring block (B1) is directly adjacent to the upper right corner of the current block, or the lower right corner of the neighboring block (B2) is directly adjacent to the upper left corner of the current block.

[1290] C24. The method according to solution C1, wherein, in case a specific neighboring block does not exist, the motion vector is set to a default motion vector.

[1291] C25. The method according to solution C1, wherein the motion vector is set to a default motion vector when no inter-coding is performed on a specific neighboring block.

[1292] C26. A method according to solution C22, wherein the particular motion vector references reference list 0.

[1293] C27. A method according to solution C22, wherein the specific motion vector references reference list 1.

[1294] C28. The method according to solution C22, wherein the specific motion vector references a specific reference picture in reference list 0.

[1295] C29. A method according to solution C22, wherein the specific motion vector references a specific reference picture in reference list 1.

[1296] C30. A method according to solution C28 or C29, wherein the particular reference picture has index 0.

[1297] C31. A method according to solution C22 or C23, wherein the particular motion vector refers to a collocated picture.

[1298] C32. A method according to solution C22 or C23, wherein the motion vector is set to a default motion vector if a particular neighboring block does not reference a collocated picture.

[1299] C33. A method according to any of solutions C24 to C32, wherein the default motion vector is (0,0).

[1300] C34. The method according to solution C1, wherein, in case a specific motion vector stored in a specific neighboring block cannot be found, the motion vector is set to a default motion vector.

[1301] C35. The method according to Solution 22, wherein if the specific motion vector does not refer to a collocated picture, the specific motion vector is scaled to the collocated picture.

[1302] C36. A method according to solution C22, wherein the specific motion vector does not reference the current picture.

[1303] C37. According to the method of any one of solutions C1 to C36, the conversion generates the current block from the bitstream representation.

[1304] C38. A method according to any of solutions C1 to C36, wherein the conversion generates a bitstream representation from the current block.

[1305] C39. A method according to any of solutions C1 to C36, wherein performing the conversion comprises parsing the bitstream representation based on one or more decoding rules.

[1306] C40. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement one or more methods of solutions C1 to C39.

[1307] C41. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of one or more of solutions C1 to C39.

[1308] In some embodiments, the following technical solutions may be implemented:

[1309] D1. A video processing method, comprising: for a conversion between a current block of a video and a bitstream representation of the video, determining whether to insert a zero-motion affine Merge candidate into a sub-block Merge candidate list based on whether affine prediction is enabled for the conversion of the current block; and performing the conversion based on the determination.

[1310] D2. The method according to solution D1, wherein, when the affine usage flag in the bitstream representation is off, the zero-motion affine Merge candidate is not inserted into the sub-block Merge candidate list.

[1311] D3. The method according to solution D2 further includes: when the affine use flag is off, inserting a default motion vector candidate that is a non-affine candidate into the sub-block Merge candidate list.

[1312] D4. A video processing method, comprising: for a conversion between a current block of a video and a bitstream representation of the video using a sub-block Merge candidate list, if the sub-block Merge candidate list is not full, inserting a zero-motion non-affine filling candidate into the sub-block Merge candidate list; and performing the conversion after the insertion.

[1313] D5. The method according to solution D4 further includes: setting the affine usage flag of the current block to zero.

[1314] D6. A method according to solution D4, wherein the insertion is also based on whether an affine usage flag in the bitstream representation is turned off.

[1315] D7. A video processing method, comprising: for a conversion between a current block of a video and a bitstream representation of the video, using a rule to determine a motion vector, wherein the rule determines that the motion vector is derived from one or more motion vectors of blocks covering corresponding positions in a juxtaposed picture; and performing the conversion based on the motion vector.

[1316] D8. A method according to solution D7, wherein the one or more motion vectors include MV0 and MV1 representing motion vectors in reference list 0 and reference list D1, respectively, and wherein the motion vector to be derived includes MV0' and MV1' representing motion vectors in reference list 0 and reference list 1.

[1317] D9. The method according to solution D8, wherein, in case the collocated picture is in reference list 0, MV0' and MV1' are derived based on MV0.

[1318] D10. The method according to solution D8, wherein, in case the collocated picture is in reference list 1, MV0' and MV1' are derived based on MV1.

[1319] D11. According to the method of any one of solutions D1 to D10, the conversion generates the current block from the bitstream representation.

[1320] D12. A method according to any of solutions D1 to D10, wherein the conversion generates a bitstream representation from the current block.

[1321] D13. The method of any of solutions D1 to D10, wherein performing the conversion comprises parsing the bitstream representation based on one or more decoding rules.

[1322] D14. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement one or more methods of solutions D1 to D13.

[1323] D15. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of one or more of solutions D1 to D13.

[1324] In some embodiments, the following technical solutions may be implemented:

[1325] E1. A method for video processing, comprising: for conversion between a current block of a video and a bitstream representation of the video, when a time domain block associated with the current block or a sub-block of the current block in a juxtaposed picture is encoded and decoded using a codec mode, and a video unit is reconstructed based on previously encoded and decoded samples in the same picture in the codec mode, determining to use a sub-block-based temporal motion vector prediction (sbTMVP) tool with a default motion candidate for conversion; and performing the conversion based on the default motion candidate.

[1326] E2. The method according to solution E1, wherein a default motion candidate is used instead of a sub-block or non-sub-block sbTMVP candidate.

[1327] E3. The method according to solution E1 or E2, wherein the default motion candidate comprises a motion candidate associated with a center position of the current block.

[1328] E4. The method according to solution E1 or E2, wherein the default motion candidate is (0, 0), and wherein the reference picture index of reference picture list 0 or reference picture list 1 is zero.

[1329] E5. The method according to any of solutions E1 to E4, wherein the codec mode is an intra block copy (IBC) mode.

[1330] E6. A method of video processing, comprising: for a sub-block based temporal motion vector prediction (sbTMVP) processing as part of a conversion between a current block of a video and a bitstream representation of the video, deriving default motion information for the sbTMVP processing based on positioning of a position used in a sub-block motion information derivation process for a sub-block of the current block; and performing the conversion based on the default motion information.

[1331] E7. A method according to solution E6, wherein the position is the center position of a specific sub-block of the current block, rather than the center position of the current block.

[1332] E8. A method according to solution E7, wherein the specific sub-block is a central sub-block.

[1333] E9. The method according to any of the solutions E1 to E8, the conversion generating the current block from the bitstream representation.

[1334] E10. The method according to any of the solutions E1 to E8, wherein the conversion generates a bitstream representation from the current block.

[1335] E11. The method according to any of the solutions E1 to E8, wherein performing the conversion comprises parsing the bitstream representation based on one or more decoding rules.

[1336] E12. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of solutions E1 to E11.

[1337] E13. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of solutions E1 to E11.

[1338] In some embodiments, the following technical solutions may be implemented:

[1339] F1. A video processing method, comprising: for a conversion between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction (sbTMVP) tool and a bitstream representation of the video, determining a modified motion vector for locating a corresponding block in a picture different from the current picture of the current block, wherein the modified motion vector is generated by right-shifting the predicted motion vector in the sbTMVP tool to integer precision; and performing the conversion based on the determination.

[1340] F2. A method according to solution F1, wherein the right shift uses the same rounding operation as used in the motion vector scaling process.

[1341] F3. A method according to solution F1, wherein the right shift uses the same rounding operation as used in the motion vector averaging process.

[1342] F4. A method according to solution F1, wherein the right shift uses the same rounding operation as used in adaptive motion vector resolution (AMVR) processing.

[1343] F5. A method according to solution F1, wherein the right shift uses a rounding-towards-zero operation, wherein the motion vector is represented as MV=(MVx, MVy), and wherein the right-shifted motion vector is represented as MV'=(MVx', MVy').

[1344] F6. The method according to solution F5, wherein the right-shifted motion vector is calculated as: MVx'=(MVx+((1<<N)> >1)-(MVx≥0?1:0))>>N, and MVy'=(MVy+((1<<N)> >1)-(MVy≥0?1:0))>>N,

[1345] Where N is an integer representing the motion vector resolution.

[1346] F7. A method according to solution F6, wherein N=4.

[1347] F8. A method according to solution F5, wherein the right-shifted motion vector is calculated as: MVx'=(MVx+(MVx≥0?7:8))>>4, and MVy'=(MVy+(MVy≥0?7:8))>>4.

[1348] F9. A method according to any one of solutions F1 to F8, wherein the modified motion vector is also used to locate another corresponding block to derive default motion information used in the sbTMVP tool.

[1349] F10. A method according to solution F9, wherein deriving the default motion information is also based on the center position of the current block and the modified motion vector.

[1350] F11. A method according to solution F9, wherein deriving the default motion information of the sub-block of the current block is also based on the center position of the sub-block and the modified motion vector.

[1351] F12. A method according to any one of solutions F1 to F11, wherein the conversion uses another codec tool different from the sbTMVP tool, and the other codec tool uses the modified motion vector to locate a reference block in the different picture or the current picture.

[1352] F13. A method according to any one of solutions F1 to F12, wherein said converting generates said current block from said bitstream representation.

[1353] F14. A method according to any of solutions F1 to F12, wherein said conversion generates said bitstream representation from said current block.

[1354] F15. The method according to any of solutions F1 to F12, wherein performing the conversion comprises parsing the bitstream representation based on one or more decoding rules.

[1355] F16. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement a method according to any one of solutions F1 to F15.

[1356] F17. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing a method according to any one of solutions F1 to F15.

[1357] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more thereof. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded and decoded on a tangible and non-volatile computer-readable medium for execution by a data processing device or to control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing unit" or "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode and decode information for transmission to a suitable receiver device.

[1358] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[1359] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[1360] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory may be supplemented by, or incorporated into, dedicated logic circuits.

[1361] Although this patent document contains many details, they should not be interpreted as limitations on any subject matter or the scope of the claims, but rather as descriptions of features of specific embodiments of specific technologies. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially required to be so, in some cases, one or more features in the claim combination can be removed from the combination, and the combination of claims can be directed to subcombinations or variations of subcombinations.

[1362] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1363] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: Determining a temporal motion vector for a transition between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction tool and a bitstream of the video; modifying the temporal motion vector by right-shifting the temporal motion vector instead of left-shifting it to integer precision, wherein the right-shift is a fixed value of 4; Locating at least one region of the current block in the co-located picture based on the modified temporal motion vector, wherein the at least one region includes a plurality of pixels of integer precision; and performing the converting based on the at least one region; Wherein performing the conversion comprises: Obtaining a temporal motion vector prediction based on the at least one region; Wherein the determining the temporal motion vector comprises: Initializing the temporal motion vector to a default motion vector; and The temporal motion vector is set to a specific motion vector associated with a specific spatial neighboring block of the current block, wherein the specific spatial neighboring block is located in a current frame including the current block. The method of claim 1 , wherein the right shift uses a round-toward-zero operation.

3. The method of claim 1 , wherein performing the conversion based on the at least one region comprises: deriving at least one sub-block motion information related to the current block based on the at least one region; constructing a sub-block motion candidate list based at least on the at least one sub-block motion information; as well as The conversion is performed based on the sub-block motion candidate list, wherein a sub-block Merge index is included in the bitstream.

4. The method according to claim 3, wherein N binary bits are used to represent the sub-block Merge index, wherein a first number L of the N binary bits are context-coded, and a second number NL of the N binary bits are bypass-coded, wherein L=1. The method according to claim 1 , wherein the motion vector associated with the default motion information is (0, 0).

6. The method of claim 1, wherein the right shift uses a rounding operation that is the same as a rounding operation used in a motion vector averaging process in which paired Merge candidates are derived.

7. The method of claim 1, wherein the right shift uses the same rounding operation as used in a motion vector scaling process or an adaptive motion vector resolution process. The method of claim 1 , wherein the converting comprises decoding the current block from the bitstream.

9. The method of claim 1, wherein the converting comprises encoding the current block into the bitstream.

10. The method according to claim 1, wherein: The right shift uses a round-towards-zero operation, wherein the motion vector is represented as MV=(MVx, MVy), and wherein the right-shifted motion vector is represented as MV'=(MVx', MVy').

11. The method according to claim 10, wherein: The right-shift motion vector is calculated as: MVx'=(MVx+((1<<N)> >1)-(MVx≥0?1:0))>>N, and MVy'=(MVy+((1<<N)> >1)-(MVy≥0?1:0))>>N, Where N is an integer representing the motion vector resolution.

12. The method according to claim 11, wherein: N＝4。 13. The method according to claim 10, wherein: The right-shift motion vector is calculated as: MVx'=(MVx+(MVx≥0?7:8))>>4, and MVy'=(MVy+(MVy≥0?7:8))>>4.

14. The method according to claim 1, wherein: The modified motion vector is also used to locate another corresponding block to derive default motion information used in the sbTMVP tool.

15. The method according to claim 14, wherein: Deriving the default motion information is also based on the center position of the current block and the modified motion vector.

16. The method according to claim 14, wherein: Deriving the default motion information of the sub-block of the current block is also based on the motion vector modified by the center position of the sub-block.

17. The method according to claim 1, wherein: The conversion uses another codec tool different from the sbTMVP tool, and the other codec tool uses the modified motion vector to locate a reference block in a different picture or a current picture.

18. The method according to claim 1, wherein: Performing the conversion includes parsing the bitstream based on one or more decoding rules.

19. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions, wherein when the processor executes the instructions, the processor is caused to: Determining a temporal motion vector for a transition between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction tool and a bitstream of the video; The temporal motion vector is modified by right-shifting the temporal motion vector instead of left-shifting it to integer precision, wherein, The right shift is a fixed value of 4. Locating at least one region of the current block in the co-located picture based on the modified temporal motion vector, wherein the at least one region includes a plurality of pixels of integer precision; and performing the converting based on the at least one region; Wherein performing the conversion comprises: Obtaining a temporal motion vector prediction based on the at least one region; Wherein the determining the temporal motion vector comprises: Initializing the temporal motion vector to a default motion vector; and The temporal motion vector is set to a specific motion vector associated with a specific spatial neighboring block of the current block, wherein the specific spatial neighboring block is located in a current frame including the current block.

20. The apparatus of claim 19, wherein the right shift uses a round-toward-zero operation.

21. A non-transitory computer-readable storage medium storing instructions for causing a processor to: Determining a temporal motion vector for a transition between a current block of a video encoded and decoded using a sub-block based temporal motion vector prediction tool and a bitstream of the video; The temporal motion vector is modified by right-shifting the temporal motion vector instead of left-shifting it to integer precision, wherein, The right shift is a fixed value of 4. Locating at least one region of the current block in the co-located picture based on the modified temporal motion vector, wherein the at least one region includes a plurality of pixels of integer precision; and performing the converting based on the at least one region; Wherein performing the conversion comprises: Obtaining a temporal motion vector prediction based on the at least one region; Wherein the determining the temporal motion vector comprises: Initializing the temporal motion vector to a default motion vector; and The temporal motion vector is set to a specific motion vector associated with a specific spatial neighboring block of the current block, wherein the specific spatial neighboring block is located in a current frame including the current block.

22. A method for storing a bit stream of a video, comprising: Determining a temporal motion vector for a current block of a video encoded using a sub-block based temporal motion vector prediction tool; modifying the temporal motion vector by right-shifting the temporal motion vector instead of left-shifting it to integer precision, wherein the right-shift is a fixed value of 4; Locating at least one region of the current block in the co-located picture based on the modified temporal motion vector, wherein the at least one region includes a plurality of pixels of integer precision; and generating the bitstream based on the at least one region; Wherein generating the bit stream comprises: Obtaining a temporal motion vector prediction based on the at least one region; and storing the bitstream in a non-transitory computer-readable storage medium; Wherein the determining the temporal motion vector comprises: Initializing the temporal motion vector to a default motion vector; and The temporal motion vector is set to a specific motion vector associated with a specific spatial neighboring block of the current block, wherein the specific spatial neighboring block is located in a current frame including the current block.

Images