Storage of motion information in history-based motion vector prediction table

Abstract

Video coding and decoding techniques including history-based motion vector prediction with default parameters are described. An example video processing method includes maintaining a history-based motion vector prediction (HMVP) table including one or more HMVP motion candidates derived from one or more previous video blocks of a video, at least one HMVP motion candidate from the HMVP table including interpolation filter information and / or side information of a switchable interpolation filter, the interpolation filter information including parameters indicative of an interpolation filter, and performing a conversion between a current video block of the video and a bitstream representation of the video based on the HMVP table.

Description

[0001] Cross-reference to related applications

[0002] This application is filed in accordance with applicable patent law and / or the Paris Convention to promptly claim priority and benefit from International Patent Application No. PCT / CN2019 / 094703, filed on July 4, 2019. For all legal purposes, the entire disclosure of the aforementioned application is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to video encoding and decoding technologies, devices, and systems. Background Technology

[0004] Despite advancements in video compression, digital video still accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This document describes various embodiments and techniques in which video encoding or decoding is performed using history-based motion vector prediction (HMVP) with default parameters.

[0006] In one example aspect, a method for video processing is disclosed. The method includes maintaining a history-based motion vector prediction (HMVP) table, which includes one or more HMVP motion candidates derived from one or more previous video blocks of the video, wherein at least one HMVP motion candidate from the HMVP table includes interpolation filter information and / or side information of a switchable interpolation filter, the interpolation filter information including parameters indicating the interpolation filter; and performing a conversion between the current video block of the video and a bitstream representation of the video based on the HMVP table.

[0007] In another example, a method for video processing is disclosed. The method includes: populating an HMVP table with a set of K default HMVP candidates, where N and K are positive integers and K ≤ N, when a history-based motion vector prediction (HMVP) table of size N is determined to be reset; and performing a conversion between the current block and its bitstream representation based on the HMVP table.

[0008] In yet another example, a different method for video processing is disclosed. The method includes: for a conversion between current blocks and their bitstream representations, determining a set of default HMVP candidates to be used during the conversion; and performing the conversion using the set of default HMVP candidates, wherein different HMVP candidates are used during the conversion of different video units.

[0009] In yet another example, a different method for video processing is disclosed. This method includes: determining a set of N default history-based motion vector prediction (HMVP) candidates already populated in an HMVP table before performing a transformation between the current block and its bitstream representation; adding the motion candidates to the HMVP table; and performing the transformation using the HMVP table including the motion candidates.

[0010] In another example aspect, a different method for video processing is disclosed. This method includes: storing information about the interpolation filter used for the current video block, or side information about a switchable interpolation filter used for the current video block, along with past motion information for the current video block, in a table before converting between the current video block and its bitstream representation; performing the conversion based on entries in the table; and updating the table after the conversion is performed.

[0011] In yet another example, a video processing apparatus is disclosed. The apparatus includes a processor configured to perform the methods disclosed above. The apparatus may also perform video encoding or video decoding.

[0012] In yet another example, a computer-readable medium is disclosed. The medium stores code for a processor implementation of the methods described above.

[0013] This document describes these and other aspects. Attached Figure Description

[0014] Figure 1 An example export process for constructing the Merge candidate list is described.

[0015] Figure 2 An example of the location of the spatial merge candidate is shown.

[0016] Figure 3 An example of candidate pairs considered for redundancy checks of spatial merge candidates is shown.

[0017] Figures 4A-4B Example locations of the second PU divided into N×2N and 2N×N segments are shown.

[0018] Figure 5 This is a diagram illustrating the scaling of motion vectors for time-domain Merge candidates.

[0019] Figure 6 Examples of candidate positions C0 and C1 for time-domain Merge candidates are shown.

[0020] Figure 7 An example of combined bidirectional prediction of Merge candidates is shown.

[0021] Figure 8 The process of deriving motion vector prediction candidates is summarized.

[0022] Figure 9 A diagram illustrating motion vector scaling for spatial motion vector candidates is shown.

[0023] Figure 10A and 10B An example of a simplified affine motion model is shown.

[0024] Figure 11 An example of an affine MVF for each sub-block is shown.

[0025] Figure 12 An example of candidate positions for the affine Merge pattern is shown.

[0026] Figure 13 An example of the modified Merge list construction process is shown.

[0027] Figure 14 An example of inter-frame prediction based on triangulation is shown.

[0028] Figure 15 An example of applying the first weighting factor group to CU is shown.

[0029] Figure 16 An example of motion vector storage implementation is shown.

[0030] Figure 17 An example of the final motion vector representation (UMVE) search process is shown.

[0031] Figure 18 An example of a UMVE search point is shown.

[0032] Figure 19 An example of MVD(0,1) mirrored between list 0 and list 1 in DMVR is shown.

[0033] Figure 20 An example of a motion vector (MV) that can be examined in a single iteration is shown.

[0034] Figure 21 An example of intra-frame block copying is shown.

[0035] Figure 22 This is a block diagram of an example hardware platform used to implement the methods described in this document.

[0036] Figure 23 This is a flowchart of an example method for video processing.

[0037] Figure 24 A table showing the extended signaling notifications for AMVR is provided.

[0038] Figure 25An example of an interpolation filter used in video encoding or decoding is shown.

[0039] Figure 26 This is a block diagram illustrating an example video codec system that can utilize the techniques disclosed herein.

[0040] Figure 27 This is a block diagram illustrating an example of a video encoder.

[0041] Figure 28 This is a block diagram illustrating an example of a video decoder.

[0042] Figure 29 This is a block diagram of an example video processing system that can implement the various techniques disclosed in this document.

[0043] Figure 30 This is a flowchart of an example method for video processing. Detailed Implementation

[0044] This document uses section headings for ease of understanding and does not limit the embodiments disclosed in a section to that section alone. Furthermore, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed techniques are applicable to other video codec techniques. Additionally, while some embodiments describe video codec steps in detail, it will be understood that the corresponding decoding steps to decode the codecs will also be implemented by a decoder. Moreover, the term video processing encompasses video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compressed format to another or at different compression bitrates.

[0045] 1. Overview

[0046] This patent document relates to video codec technology. Specifically, it relates to symmetrical motion vector difference codec. It can be applied to existing video codec standards like HEVC or standards yet to be finalized (universal video codecs). It can also be applied to future video standards or video codecs.

[0047] 2. Preliminary Discussion

[0048] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed the H.261 and H.263 standards, while ISO / IEC developed the MPEG-1 and MPEG-4 Visual standards. The two organizations jointly developed the H.262 / MPEG-2 video standard, the H.264 / MPEG-4 Advanced Video Coding (AVC) standard, and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was established to work on the VVC standard, with the goal of reducing the bit rate by 50% compared to HEVC.

[0049] 2.1 Inter-frame prediction in HEVC / H.265

[0050] For inter-frame encoding / decoding, the codec unit (CU) can be encoded / decoded with one prediction unit (PU) or two PUs, depending on the segmentation mode. Each inter-frame prediction PU has motion parameters for one or two lists of reference images. The motion parameters include motion vectors and reference image indices. The use of one of the two reference image lists can also be signaled using `inter_pred_idc`. The motion vectors can be explicitly encoded / decoded as increments relative to the predictors.

[0051] When encoding and decoding a CU using the skip mode, a PU is associated with that CU, and there are no significant residual coefficients, no encoding / decoding motion vector increments, or reference picture indices. A merge mode is specified, thereby obtaining the motion parameters of the current PU from neighboring PUs (including spatial and temporal candidates). The merge mode can be applied to any PU for inter-frame prediction, not just the skip mode. An alternative to the merge mode is explicit transmission of motion parameters, where each PU explicitly signals: the motion vector (more precisely, the motion vector difference (MVD) relative to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the reference picture list usage. Such a mode is named Advanced Motion Vector Prediction (AMVP) in this disclosure.

[0052] When signaling indicates that one of the two reference image lists should be used, the PU is generated from a block of samples. This is called "one-way prediction". One-way prediction can be used for both P-strips and B-strips.

[0053] When signaling indicates that both from the list of reference images should be used, the PU is generated from two blocks of samples. This is called "bidirectional prediction". Bidirectional prediction can only be used for B-strips.

[0054] The following text provides details of the inter-frame prediction modes specified in HEVC. The description will begin with the Merge mode.

[0055] 2.1.1 List of Reference Images

[0056] In HEVC, the term inter-frame prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference images other than the currently decoded image. As in H.264 / AVC, images can be predicted from multiple reference images. The reference images used for inter-frame prediction are organized into one or more reference image lists. A reference index identifies which reference images in the list should be used to create the predicted signal.

[0057] A single list of reference images (list 0) is used for the P-strip, and two lists of reference images (list 0 and list 1) are used for the B-strip. It should be noted that, in terms of capture / display order, the reference images contained in lists 0 and 1 can be images from the past and future.

[0058] 2.1.2 Merge Mode

[0059] 2.1.2.1 Derivation of Candidates for Merge Pattern

[0060] When predicting the PU using the Merge mode, indices pointing to entries in the Merge candidate list are parsed from the bitstream, and these indices are used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized according to the following steps:

[0061] Step 1: Initial Candidate Export

[0062] Step 1.1: Spatial Candidate Export

[0063] Step 1.2: Redundancy check of airspace candidates

[0064] Step 1.3: Temporal Candidate Derivation

[0065] Step 2: Adding candidate insertions

[0066] Step 2.1: Create bidirectional prediction candidates

[0067] Step 2.2: Insert zero-motion candidates

[0068] Figure 1 These steps are also schematically depicted. For spatial merge candidate derivation, up to four merge candidates are selected from candidates located at five different positions. For temporal merge candidate derivation, up to one merge candidate is selected from two candidates. Since it is assumed at the decoder that the number of candidates per PU is constant, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the stripe header. Because the number of candidates is constant, truncated univariate binarization (TU) is used to encode the index of the best merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single merge candidate list, which is the same as the merge candidate list of the 2N×2N prediction unit.

[0069] The operations associated with the foregoing steps are described in detail below.

[0070] Figure 1 An example export process for constructing the Merge candidate list is described.

[0071] 2.1.2.2 Spatial Domain Candidate Derivation

[0072] In the export of the spatial Merge candidate, located in Figure 2 At most four merged candidates are selected from the candidates at the positions depicted. The derived order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because the PU belongs to another stripe or slice) or during intra-frame encoding / decoding. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates. This redundancy check ensures that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only those with the same motion information are considered. Figure 3 The arrows in the list link pairs, and a candidate is added to the list only if the corresponding candidate used for redundancy checking has different motion information. Another source of duplicate motion information is a "second PU" associated with a segmentation other than 2N×2N. As an example, Figures 4A-4BThe second prediction unit (PU) is described for the N×2N and 2N×N cases, respectively. When the current PU is segmented into N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate would result in two prediction units with the same motion information, which is redundant for an encoding / decoding unit with only one PU. Similarly, when the current PU is segmented into 2N×N, position B1 is not considered.

[0073] 2.1.2.3 Time-domain candidate derivation

[0074] In this step, only one candidate is added to the list. Specifically, in the export of this temporal merge candidate, the scaled motion vector is exported based on the juxtaposed PU belonging to the image that has the minimum POC difference with the current image within a given list of reference images. The list of reference images to be used for the export of the juxtaposed PU is explicitly signaled in the strip header. Figure 5 As shown by the dashed lines, a scaled motion vector for the temporal merge candidate is obtained. This scaled motion vector is scaled from the motion vector of the juxtaposed PU using the POC distances tb and td, where tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the juxtaposed image and the juxtaposed image. The reference image index of the temporal merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For the B-strip, two motion vectors are obtained and combined to produce a bidirectional prediction merge candidate, one of which is used for reference image list 0 and the other for reference image list 1.

[0075] 2.1.2.4 Juxtaposed Images and Juxtaposed PUs

[0076] When TMVP is enabled (i.e., slice_temporal_mvp_enabled_flag equals 1), the variable ColPic representing the juxtaposed images is exported as follows:

[0077] - If the current stripe is a B stripe and the signaling notification collocated_from_l0_flag is equal to 0, then ColPic is set to equal to RefPicList1[collocated_ref_idx].

[0078] - Otherwise (slice_type equals B and collocated_from_l0_flag equals 1, or slice_type equals P), ColPic is set to equal RefPicList0[collocated_ref_idx].

[0079] In this paper, collocated_ref_idx and collocated_from_l0_flag are two syntax elements that can be signaled in the stripe header.

[0080] like Figure 6 As shown, in the juxtaposed PU(Y) belonging to the reference frame, a position is selected between candidate C0 and C1 for the temporal candidate. If the PU at position C0 is unavailable, intra-frame encoded, or outside the current codec tree unit (CTU, also known as LCU, maximum codec unit) row, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0081] The relevant grammatical elements are described below:

[0082] 7.3.6.1 General Strip Header Syntax

[0083]

[0084]

[0085] 2.1.2.5 Exporting MVs of TMVP Candidates

[0086] More specifically, perform the following steps to derive TMVP candidates:

[0087] 1) Set the reference image list X = 0, and the target reference image is set to the reference image with index 0 in list X (i.e., curr_ref). Call the juxtaposition motion vector export procedure to obtain the MV pointing to list X of curr_ref.

[0088] 2) If the current stripe is a B stripe, set the reference image list X = 1, and the target reference image is set to the reference image with index 0 in list X (i.e., curr_ref). Call the derivation procedure for the juxtaposed motion vectors to obtain the MV of list X pointing to curr_ref.

[0089] The process of deriving the juxtaposed motion vectors is described in the next subsection 2.1.2.5.1.

[0090] 2.1.2.5.1 The process of deriving the juxtaposed motion vector

[0091] For contiguous blocks, intra-frame or inter-frame encoding / decoding can be performed using unidirectional or bidirectional prediction. If intra-frame encoding / decoding is performed, the TMVP candidate is set to unavailable.

[0092] If it is a one-way prediction from list A, then the motion vector of list A is scaled to the target reference image list X.

[0093] If it is a bidirectional prediction and the target reference image list is X, then the motion vectors of list A are scaled to the target reference image list X, and A is determined according to the following rules:

[0094] - If none of the reference images have a larger POC value than the current image, then A is set to equal to X.

[0095] Otherwise, set A to equal collocated_from_l0_flag.

[0096] 2.1.2.6 Additional Candidate Insertion

[0097] In addition to spatial and temporal merge candidates, two additional types of merge candidates exist: combined bidirectional prediction merge candidates and zero merge candidates. Combined bidirectional prediction merge candidates are generated by utilizing spatial and temporal merge candidates. These combined bidirectional prediction merge candidates are only used for B-strips. Combined bidirectional prediction candidates are generated by combining the motion parameters of the first reference image list of the initial candidate with the motion parameters of the second reference image list of another candidate. If these two tuples provide different motion hypotheses, they form a new bidirectional prediction candidate. As an example, Figure 7 The following scenario is described, in which two candidates from the original list (on the left) with either mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create a combined bidirectional predictive merge candidate, which is then added to the final list (on the right). There are numerous rules regarding the combinations considered to generate these additional merge candidates.

[0098] Zero-motion candidates are inserted to populate the remaining entries in the Merge candidate list, thus reaching the MaxNumMergeCand capacity. These candidates have zero spatial displacements and a reference image index that starts at zero and increments whenever a new zero-motion candidate is added to the list. Finally, no redundancy checks are performed on these candidates.

[0099] 2.1.3 AMVP

[0100] AMVP utilizes the spatial-temporal correlation between motion vectors and neighboring PUs, which is used for the explicit transmission of motion parameters. For each list of reference images, a candidate list of motion vectors is constructed by first checking the availability of the temporally adjacent PU positions to the left and top, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the Merge index signaling notification, a truncated unary is used to encode the index of the best motion vector candidate. The maximum value to be encoded in this case is 2 (see [link to relevant documentation]). Figure 8 The following sections provide details on the process of deriving motion vector prediction candidates.

[0101] 2.1.3.1 Exporting AMVP Candidates

[0102] Figure 8 The process of deriving motion vector prediction candidates is summarized.

[0103] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the spatial motion vector candidate derivation, the final result is based on the position of... Figure 2 The motion vectors of each PU at five different locations are shown to derive two motion vector candidates.

[0104] For temporal motion vector candidate derivation, select one motion vector candidate from two candidates derived based on two different juxtaposition positions. After creating a first list of spatial-temporal candidates, remove duplicate motion vector candidates from the list. If the number of potential candidates is greater than 2, remove motion vector candidates from the list whose reference image index is greater than 1 in the associated reference image list. If the number of spatial-temporal motion vector candidates is less than 2, add additional zero motion vector candidates to the list.

[0105] 2.1.3.2 Candidate Spatial Motion Vectors

[0106] In deriving the spatial motion vector candidates, at most two candidates are considered from five potential candidates, which are located in the following order: Figure 2 The PU (Programmable Utility) derivations for the shown positions are identical to those for the motion merge. The derivation order to the left of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order to the upper side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four possible motion vector candidates, two of which do not require spatial scaling and two of which do. The four different cases are summarized below.

[0107] • No spatial scaling

[0108] –(1) Same list of reference images, and same index of reference images (same POC)

[0109] –(2) Different lists of reference images, but the same reference image (same POC)

[0110] • Spatial scaling

[0111] –(3) Same list of reference images, but different reference images (different POCs)

[0112] –(4) A list of different reference images, and different reference images (different POCs)

[0113] First, check for the case without spatial scaling, then check for spatial scaling. Regardless of the reference image list, spatial scaling is considered when the reference image of the POC differs from the reference image of the current PU. If all PUs in the left-hand candidate are unavailable or intra-frame encoded / decoded, scaling of the upper motion vector is allowed to facilitate parallel export of the left-hand and upper MV candidates. Otherwise, spatial scaling of the upper motion vector is not allowed.

[0114] During spatial scaling, the motion vectors of the neighboring PU are scaled in a manner similar to temporal scaling, such as... Figure 9 As shown. The main difference is that the current PU's reference image list and index are given as input; the actual scaling process is the same as the temporal scaling process.

[0115] 2.1.3.3 Candidate Motion Vectors in the Time Domain

[0116] Apart from the reference image index export, all procedures for exporting temporal merge candidates are the same as those for exporting spatial motion vector candidates (see...). Figure 6 (Same as above.) The reference image index signaling is sent to the decoder.

[0117] 2.2 Inter-frame prediction methods in VVC

[0118] Several new codec tools exist for improving inter-frame prediction, such as Adaptive Motion Vector Difference Resolution (AMVR) for Signaling Notification MVD, Merge with Motion Vector Difference (MMVD), Triangular Prediction Mode (TPM), Combined Intra-Inter-Frame Prediction (CIIP), Advanced TMVP (ATMVP, also known as SbTMVP), Affine Prediction Mode, Generalized Bidirectional Prediction (GBI), Decoder-Side Motion Vector Refinement (DMVR), and Bidirectional Optical Flow (BIO, also known as BDOF).

[0119] VVC supports three different Merge list construction processes:

[0120] 1) Sub-block Merge candidate list: It includes ATMVP and affine Merge candidates. Both affine and ATMVP modes share a single Merge list construction process. Here, ATMVP and affine Merge candidates can be added sequentially. The size of the sub-block Merge list is signaled in the strip header and has a maximum value of 5.

[0121] 2) Regular Merge List: For inter-frame codec blocks, a shared Merge list construction process is used. Here, spatial / temporal Merge candidates, HMVP, paired Merge candidates, and zero-motion candidates can be inserted sequentially. The signaling in the strip header informs the size of the regular Merge list, with a maximum value of 6. MMVD, TPM, and CIIP rely on the regular Merge list.

[0122] 3) IBC Merge List: It is done in a similar way to a regular Merge list.

[0123] Similarly, VVC supports three lists of AMVPs:

[0124] 1) Affine AMVP Candidate List

[0125] 2) Regular AMVP Candidate List

[0126] 3) IBC AMVP Candidate List: Due to the adoption of JVET-N0843, the construction process is the same as that of the IBC Merge List.

[0127] 2.2.1 Codec Block Structure in VVC

[0128] In VVC, a quadtree / binary tree / multi-branch tree (QT / BT / TT) structure is used to divide the image into square or rectangular blocks.

[0129] In addition to QT / BT / TT, VVC also employs separate trees (also known as dual codec trees) for I-frames. These separate trees provide signaling for the luma and chroma components, respectively, to inform the codec block structure.

[0130] In addition, CU is set to be equal to PU and TU, except for blocks encoded and decoded using several specific encoding and decoding methods (such as intra-frame sub-segmentation prediction, where PU is equal to TU but less than CU, and sub-block transformation for inter-frame encoded and decoded blocks, where PU is equal to CU but TU is less than PU).

[0131] 2.2.2 Affine Prediction Mode

[0132] In HEVC, only translational motion models are applied for motion compensation prediction (MCP). However, in the real world, many types of motion exist, such as scaling, rotation, perspective motion, and other irregular motions. In VVC, simplified affine transformation motion compensation prediction is applied to both 4-parameter and 6-parameter affine models. Figure 10A and 10B As shown, the affine motion field of this block is described by the two control point motion vectors (CPMV) of the 4-parameter affine model and the three CPMVs of the 6-parameter affine model.

[0133] The motion vector field (MVF) of the block is described by the following equations, respectively, using a 4-parameter affine model in equation (1) (where the 4-parameters are defined as variables a, b, e, and f) and a 6-parameter affine model in equation (2) (where the 4-parameters are defined as variables a, b, c, d, e, and f):

[0134]

[0135]

[0136] In this article, (mv h 0,mv h 0) is the motion vector of the top-left control point, and (mv h 1, MV h 1) is the motion vector of the upper right control point, and (mv h 2, MV h 2) is the motion vector of the lower left control point. All three motion vectors are called the control point motion vectors (CPMV). (x, y) represents the coordinates of the representative point relative to the upper left sample point within the current block, and (mv) h (x, y), mv v (x, y) is the motion vector derived for the sample located at (x, y). The CP motion vector can be signaled (as in affine AMVP mode) or derived on the fly (as in affine Merge mode). w and h are the width and height of the current block. In practice, this division is implemented by right-shifting the rounding operation. In VTM, the representative point is defined as the center position of the sub-block; for example, when the coordinates of the top-left corner of the sub-block relative to the top-left sample within the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2). For each sub-block (i.e., 4×4 in VTM), the representative point is used to derive the motion vector for the entire sub-block.

[0137] To further simplify motion compensation prediction, a sub-block-based affine transformation prediction is applied. To derive the motion vector for each M×N (in the current VVC, both M and N are set to 4) sub-block, as follows... Figure 11 As shown, the motion vector of the center sample point of each sub-block is calculated according to equations (1) and (2) and rounded to 1 / 16 of the fractional accuracy. Then, a 1 / 16 pixel motion-compensated interpolation filter can be applied to generate the prediction for each sub-block using the derived motion vector. The 1 / 16 pixel interpolation filter is introduced by the affine pattern.

[0138] After MCP, the high-accuracy motion vector of each sub-block is rounded and saved with the same accuracy as the normal motion vector.

[0139] 2.2.3 MERGE of the entire block

[0140] 2.2.3.1 Construction of the Merge List for the Regular Merge Pattern with Translation

[0141] 2.2.3.1.1 History-Based Motion Vector Prediction (HMVP)

[0142] Unlike the Merge list design, VVC uses a history-based motion vector prediction (HMVP) method.

[0143] The HMVP stores motion information from previous encoding / decoding operations. Motion information from previously encoded / decoded blocks is defined as HMVP candidates. Multiple HMVP candidates are stored in a table named the HMVP table, which is maintained during the real-time encoding / decoding process. The HMVP table is cleared when encoding / decoding of a new slice / LCU line / strip begins. Whenever an inter-frame encoded / decoded block or non-sub-block, non-TPM mode exists, the associated motion information is added to the last entry of the table as a new HMVP candidate. The entire encoding / decoding process is as follows: Figure 12 As shown.

[0144] 2.2.3.1.2 Standard Merge List Construction Process

[0145] The construction of a regular Merge list (for translational motion) can be summarized according to the following steps:

[0146] Step 1: Export spatial domain candidates

[0147] Step 2: Insert HMVP candidate

[0148] Step 3: Insert pairwise average candidates

[0149] Step 4: Default motion candidates

[0150] HMVP candidates can be used in the construction of both the AMVP and Merge candidate lists. Figure 13The modified Merge Candidate List construction process is depicted (shaded in gray). When the Merge Candidate List is not full after TMVP candidate insertion, HMVP candidates stored in the HMVP table can be used to populate the Merge Candidate List. Considering that a block is generally more relevant to its nearest neighboring blocks in terms of motion information, HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first, and the first entry is added last. Similarly, redundancy removal is applied to HMVP candidates. The Merge Candidate List construction process terminates once the total number of available Merge Candidates reaches the maximum number of Merge Candidates allowed to be signaled.

[0151] Note that all spatial / temporal / HMVP candidates should be encoded and decoded in non-IBC mode. Otherwise, they are not allowed to be added to the regular Merge candidate list.

[0152] The HMVP table contains up to 5 regular motion candidates, and each candidate is unique.

[0153] 2.2.3.1.2.1 Pruning process

[0154] A candidate is added to the list only if the corresponding candidate used for redundancy checking has different motion information. This comparison process is called the pruning process.

[0155] The pruning process in the airspace candidates depends on the TPM used in the current block.

[0156] When encoding or decoding the current block without using TPM mode (e.g., regular Merge, MMVD, CIIP), the HEVC pruning process (e.g., five prunings) used for spatial merge candidates is utilized.

[0157] 2.2.3.1.2.2 Update the HMVP table

[0158] After decoding the inter-frame codec block in non-TPM, non-sub-block mode, the HMVP table is updated. The motion information for the current block is first pruned to include all available HMVP candidates in the HMVP table. If a duplicate HMVP candidate is found, the redundant candidate is removed from the table, and all remaining HMVP candidates are moved forward by decrementing their index by one. The motion information for the current block is then added to the HMVP table.

[0159] If the count represented by K (the number of available HMVP candidates in its record table) is less than 5, then K is incremented by 1. Otherwise, K remains unchanged.

[0160] The relevant specifications are defined as follows:

[0161] 8.5.2.16 Update process of the historical motion vector predictor candidate list

[0162] The input for this process is:

[0163] Luminance motion vectors mvL0 and mvL1 with a fractional sample accuracy of -1 / 16.

[0164] -Refer to indices refIdxL0 and refIdxL1,

[0165] - The prediction list uses the flags predFlagL0 and predFlagL1.

[0166] - Bidirectional prediction weight index bcwIdx.

[0167] The MVP candidate hMvpCand includes the brightness motion vectors mvL0 and mvL1, the reference indices refIdxL0 and refIdxL1, the prediction list using the flags predFlagL0 and predFlagL1, and the bidirectional prediction weight index bcwIdx.

[0168] The candidate list HmvpCandList is modified using the following ordered steps:

[0169] 1. Set the variable identicalCandExist to false and the variable removeIdx to 0.

[0170] 2. When NumHmvpCand is greater than 0, for each index hMvpIdx, where hMvpIdx = 0..NumHmvpCand-1, apply the following steps until identicalCandExist equals TRUE:

[0171] – When hMvpCand equals HmvpCandList[hMvpIdx], identicalCandExist is set to true, and removeIdx is set to equal hMvpIdx.

[0172] 3. The candidate list HmvpCandList is updated as follows:

[0173] – If identicalCandExist equals true or NumHmvpCand equals MaxNumMergeCand-1, then the following applies:

[0174] – For each index, where i = (removeIdx+1)...(NumHmvpCand-1), HmvpCandList[i-1] is set to equal HmvpCandList[i].

[0175] –HmvpCandList[NumHmvpCand-1] is set to equal to mvCand.

[0176] – Otherwise (identicalCandExist equals false or NumHmvpCand is less than MaxNumMergeCand-1), then the following applies:

[0177] –HmvpCandList[NumHmvpCand++] is set to equal to mvCand.

[0178] Here, MaxNumMergeCand is set to 5.

[0179] 2.2.4 Triangular Prediction Model (TPM)

[0180] In VCC, triangular segmentation mode is supported for inter-frame prediction. Triangular segmentation mode is only applied to CUs of 8×8 size or larger that are encoded / decoded in Merge mode and not in MMVD or CIIP mode. For CUs meeting these conditions, signaling informs CU-level flags to indicate whether triangular segmentation mode has been applied.

[0181] When using this mode, such as Figure 14 As shown, the CU is uniformly divided into two triangular segments using either diagonal or anti-diagonal partitioning. Each triangular segment in the CU performs inter-frame prediction using its own motion; each segment allows only unidirectional prediction, meaning each segment has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, as with regular bidirectional prediction, each CU requires only two motion-compensated predictions.

[0182] If the CU level flag indicates that the current CU is encoded and decoded using the triangular segmentation mode, further signaling indicates a flag indicating the direction of the triangular segmentation (diagonal or anti-diagonal) and two merge indices (one for each segment). After predicting each triangular segment, a blending process with adaptive weights is used to adjust the sample values ​​along the diagonal or anti-diagonal edges. This is the predicted signal for the entire CU, and as in other prediction modes, the transform and quantization processes are applied to the entire CU. Finally, the motion field of the CU predicted using the triangular segmentation mode is stored in 4×4 cells.

[0183] The regular Merge candidate list is reused for triangulation Merge prediction without additional motion vector pruning. For each Merge candidate in the regular Merge candidate list, exactly one of its L0 or L1 motion vectors is used for triangulation prediction. Furthermore, the order of selection of the L0 and L1 motion vectors is based on their Merge index parity. With this scheme, the regular Merge list can be used directly.

[0184] 2.2.4.1 TPM Merge List Construction Process

[0185] Basically, it follows the standard Merge list construction process as proposed in JVET-N0340. However, some modifications have been added.

[0186] Specifically, the following applies:

[0187] 1) How the pruning process is performed depends on the TPM used for the current block.

[0188] - If TPM is not used to encode or decode the current block, then invoke the 5 pruning steps applied to the HEVC of the spatial merge candidate.

[0189] Otherwise (if the current block is encoded using TPM encoding / decoding), full pruning is applied when adding new spatial merge candidates. In other words, B1 is compared with A1; B0 is compared with A1 and B1; A0 is compared with A1, B1, and B0; and B2 is compared with A1, B1, A0, and B0.

[0190] 2) Whether or not to check motion information from B2 depends on the current block's TPM usage.

[0191] - If the current block is not encoded or decoded using TPM, B2 is accessed and checked only if there are fewer than 4 space merge candidates before checking B2.

[0192] Otherwise (if the current block is encoded or decoded using TPM), B2 is always accessed and checked before adding B2, regardless of how many Merge candidates are available.

[0193] 2.2.4.2 Adaptive Weighting Process

[0194] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangular prediction units to derive the final prediction for the entire CU. The two sets of weighting factors are defined as follows:

[0195] The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luminance and chrominance samples, respectively;

[0196] The second weighting factor group: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luminance and chrominance samples, respectively.

[0197] The weighting factor set is selected based on a comparison of the motion vectors of the two triangular prediction units. The second weighting factor set is used if any of the following conditions are true:

[0198] - The reference images for the two triangular prediction units are different from each other.

[0199] - The absolute value of the difference between the horizontal values ​​of two motion vectors is greater than 16 pixels.

[0200] - The absolute value of the difference between the perpendicular values ​​of two motion vectors is greater than 16 pixels.

[0201] Otherwise, use the first weighted factor group. Figure 15 An example is shown in the figure.

[0202] 2.2.4.3 Motion Vector Storage

[0203] The motion vector of the triangular prediction unit ( Figure 16 Mv1 and Mv2 in the CU are stored in a 4×4 grid. For each 4×4 grid, depending on the location of the 4×4 grid in the CU, either unidirectional or bidirectional predicted motion vectors are stored. Figure 16 As shown, for a 4×4 grid located in an unweighted region (i.e., not on the diagonal edge), a unidirectional predicted motion vector—Mv1 or Mv2—is stored. Conversely, for a 4×4 grid located in a weighted region, a bidirectional predicted motion vector is stored. The bidirectional predicted motion vector is derived from Mv1 and Mv2 according to the following rules:

[0204] 1) When Mv1 and Mv2 have motion vectors from different directions (L0 or L1), Mv1 and Mv2 can be simply combined to form a bidirectional predicted motion vector.

[0205] 2) When both Mv1 and Mv2 originate from the same L0 (or L1) direction,

[0206] - If the reference image for Mv2 is the same as an image in the L1 (or L0) reference image list, then Mv2 is scaled to that image. Mv1 and the scaled Mv2 are combined to form a bidirectional predicted motion vector.

[0207] - If the reference image for Mv1 is the same as an image in the L1 (or L0) reference image list, then scale Mv1 to that image. Combine the scaled Mv1 and Mv2 to form a bidirectional predicted motion vector.

[0208] Otherwise, only Mv1 is stored for the weighted region.

[0209] 2.2.4.4 The syntax, semantics, and decoding process of the Merge pattern

[0210] 7.3.5.1 General Strip Header Syntax

[0211]

[0212]

[0213] 7.3.7.5 Encoding / Decoding Unit Syntax

[0214]

[0215]

[0216]

[0217] 7.3.7.7 Merge Data Syntax

[0218]

[0219]

[0220]

[0221] 7.4.6.1 General Strip Header Semantics

[0222] `six_minus_max_num_merge_cand` specifies the maximum number of Merge Motion Vector Prediction (MVP) candidates supported from the strips subtracted from 6. The maximum number of Merge MVP candidates, `MaxNumMergeCand`, is derived as follows:

[0223] MaxNumMergeCand=6-six_minus_max_num_merge_cand(7-57)

[0224] The value of MaxNumMergeCand should be in the range of 1 to 6 (inclusive).

[0225] `five_minus_max_num_subblock_merge_cand` specifies the maximum number of subblock-based Merge Motion Vector Prediction (MVP) candidates supported from the strips subtracted from 5. If `five_minus_max_num_subblock_merge_cand` does not exist, it is inferred to be equal to 5 - `sps_sbtmvp_enabled_flag`. The maximum number of subblock-based Merge MVP candidates, `MaxNumSubblockMergeCand`, is derived as follows:

[0226] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand(758)

[0227] The value of MaxNumSubblockMergeCand should be in the range of 0 to 5 (inclusive).

[0228] 7.4.8.5 Semantics of Encoding / Decoding Units

[0229] A pred_mode_flag value of 0 indicates that the current codec unit is encoded in inter-prediction mode. A pred_mode_flag value of 1 indicates that the current codec unit is encoded in intra-prediction mode.

[0230] When pred_mode_flag does not exist, the following inference can be made:

[0231] – If cbWidth equals 4 and cbHeight equals 4, then pred_mode_flag is inferred to be equal to 1.

[0232] Otherwise, when decoding I stripes, pred_mode_flag is inferred to be equal to 1, while when decoding P or B stripes respectively, pred_mode_flag is inferred to be equal to 0.

[0233] For x = x0..x0 + cbWidth-1 and y = y0..y0 + cbHeight-1, the variable CuPredMode[x][y] is derived as follows:

[0234] – If pred_mode_flag equals 0, then CuPredMode will be set to 0. [y] is set to equal MODE_INTER.

[0235] Otherwise (pred_mode_flag equals 1), set CuPredMode[x][y] to equal MODE_INTRA.

[0236] A pred_mode_ibc_flag value of 1 indicates that the current codec unit is encoded / decoded in IBC prediction mode. A pred_mode_ibc_flag value of 0 indicates that the current codec unit is not encoded / decoded in IBC prediction mode.

[0237] When pred_mode_ibc_flag is not present, the following inference can be made:

[0238] – If cu_skip_flag[x0][y0] equals 1, and cbWidth equals 4, and cbHeight equals 4, then pred_mode_ibc_flag is inferred to be equal to 1.

[0239] Otherwise, if both cbWidth and cbHeight are equal to 128, then pred_mode_ibc_flag is inferred to be equal to 0.

[0240] Otherwise, when decoding an I stripe, pred_mode_ibc_flag is inferred to be equal to the value of sps_ibc_enabled_flag; and correspondingly, when decoding a P or B stripe, the value of pred_mode_ibc_flag is inferred to be equal to 0.

[0241] When pred_mode_ibc_flag equals 1, for x = x0..x0 + cbWidth-1 and y = y0..y0 + cbHeight-1, the variable CuPredMode[x][y] is set to equal MODE_IBC.

[0242] `general_merge_flag[x0][y0]` specifies whether to infer the inter-prediction parameters of the current codec unit from the inter-prediction segments of the neighborhood. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0243] When general_merge_flag[x0][y0] does not exist, the following inference can be made:

[0244] – If cu_skip_flag[x0][y0] equals 1, then it is inferred that general_merge_flag[x0][y0] equals 1.

[0245] Otherwise, infer that general_merge_flag[x0][y0] equals 0.

[0246] mvp_l0_flag[x0][y0] specifies the motion vector predictor index of list 0, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance image sample of the image.

[0247] If mvp_l0_flag[x0][y0] does not exist, it is inferred to be equal to 0.

[0248] mvp_l1_flag[x0][y0] has the same semantics as mvp_l0_flag, where l0 and list 0 are replaced by l1 and list 1, respectively.

[0249] inter_pred_idc[x0][y0] specifies whether list 0, list 1, or bidirectional prediction is used for the current codec unit according to Table 7-10. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0250] Table 7-10 – Names Associated with Inter-Frame Prediction Modes

[0251]

[0252] If inter_pred_idc[x0][y0] does not exist, it is inferred to be equal to PRED_L0.

[0253] 7.4.8.7 Merge Data Semantics

[0254] The regular_merge_flag[x0][y0] value being 1 indicates that the regular Merge mode is used to generate the inter-frame prediction parameters for the current codec unit. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0255] When regular_merge_flag[x0][y0] does not exist, the following inference can be made:

[0256] – If all of the following conditions are true, then regular_merge_flag[x0][y0] is inferred to be equal to 1:

[0257] –Sps_mmvd_enabled_flag equals 0.

[0258] –general_merge_flag[x0][y0] equals 1.

[0259] –cbWidth*cbHeight equals 32.

[0260] Otherwise, infer that regular_merge_flag[x0][y0] equals 0.

[0261] The value of mmvd_merge_flag[x0][y0] equal to 1 indicates that the Merge mode with motion vector difference is used to generate the inter-frame prediction parameters for the current codec unit. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0262] When mmvd_merge_flag[x0][y0] does not exist, the following inference can be made:

[0263] – If all of the following conditions are true, then mmvd_merge_flag[x0][y0] is inferred to be equal to 1:

[0264] –sps_mmvd_enabled_flag equals 0.

[0265] –general_merge_flag[x0][y0] equals 1.

[0266] –cbWidth*cbHeight equals 32.

[0267] –regular_merge_flag[x0][y0] equals 0.

[0268] Otherwise, infer that mmvd_merge_flag[x0][y0] equals 0.

[0269] mmvd_cand_flag[x0][y0] specifies whether to use the first (0) or second (1) candidate in the Merge candidate list with the motion vector difference derived from mmvd_distance_idx[x0][y0] and mmvd_direction_idx[x0][y0]. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0270] If mmvd_cand_flag[x0][y0] does not exist, it is inferred to be equal to 0.

[0271] mmvd_distance_idx[x0][y0] specifies the index used to derive MmvdDistance[x0][y0], as shown in Table 7-12. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0272] Table 7-12 – Specifications of MmvdDistance[x0][y0] based on mmvd_distance_idx[x0][y0]

[0273]

[0274] mmvd_direction_idx[x0][y0] specifies the index used to derive MmvdSign[x0][y0], as specified in Table 7-13. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0275] Table 7-13 – Specifications of MmvdSign[x0][y0] based on mmvd_direction_idx[x0][y0]

[0276]

[0277] The two components MmvdOffset[x0][y0] of Merge plus MVD offset are exported as follows:

[0278] MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]<<2)*MmvdSi gn[x0][y0][0](7-124)

[0279] MmvdOffset[x0][y0][1]=(MmvdDistance[x0][y0]<<2)*MmvdSi gn[x0][y0][1](7-125)

[0280] `merge_subblock_flag[x0][y0]` specifies whether to infer the sub-block-based inter-frame prediction parameters of the current codec unit from neighboring blocks. Array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the considered codec block relative to the top-left luminance sample of the image. If `merge_subblock_flag[x0][y0]` does not exist, it is inferred to be equal to 0.

[0281] merge_subblock_idx[x0][y0] specifies the merge candidate index of the merge candidate list based on subblocks, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0282] If merge_subblock_idx[x0][y0] does not exist, it is inferred to be equal to 0.

[0283] ciip_flag[x0][y0] specifies whether to apply the combined inter-frame merge and intra-frame prediction to the current codec unit. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0284] If ciip_flag[x0][y0] does not exist, it is inferred to be equal to 0.

[0285] When ciip_flag[x0][y0] equals 1, the variable IntraPredModeY[x][y] with x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1 is set to equal INTRA_PLANAR.

[0286] The variable MergeTriangleFlag[x0][y0] specifies whether to use triangle-based motion compensation to generate prediction samples for the current codec unit when decoding B-strips. The variable MergeTriangleFlag[x0][y0] is derived as follows:

[0287] – If all of the following conditions are true, then MergeTriangleFlag[x0][y0] will be set to 1:

[0288] –sps_triangle_enabled_flag equals 1.

[0289] –slice_type equals B.

[0290] –general_merge_flag[x0][y0] equals 1.

[0291] –MaxNumTriangleMergeCand is greater than or equal to 2.

[0292] –cbWidth*cbHeight is greater than or equal to 64.

[0293] –regular_merge_flag[x0][y0] equals 0.

[0294] –mmvd_merge_flag[x0][y0] equals 0.

[0295] –merge_subblock_flag[x0][y0] equals 0.

[0296] –ciip_flag[x0][y0] equals 0.

[0297] Otherwise, set MergeTriangleFlag[x0][y0] to 0.

[0298] `merge_triangle_split_dir[x0][y0]` specifies the direction of the Merge triangle pattern. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0299] If merge_triangle_split_dir[x0][y0] does not exist, it is inferred to be equal to 0.

[0300] merge_triangle_idx0[x0][y0] specifies the first merge candidate index of the triangle-based motion compensation candidate list, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0301] If merge_triangle_idx0[x0][y0] does not exist, it is inferred to be equal to 0.

[0302] merge_triangle_idx1[x0][y0] specifies the second Merge candidate index of the triangle-based motion compensation candidate list, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0303] If merge_triangle_idx1[x0][y0] does not exist, it is inferred to be equal to 0.

[0304] merge_idx[x0][y0] specifies the Merge candidate index in the Merge candidate list, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0305] When merge_idx[x0][y0] does not exist, the following inference can be made:

[0306] – If mmvd_merge_flag[x0][y0] equals 1, then merge_idx[x0][y0] is inferred to be equal to mmvd_cand_flag[x0][y0].

[0307] Otherwise (mmvd_merge_flag[x0][y0] equals 0), then merge_idx[x0][y0] is inferred to be equal to 0.

[0308] 2.2.4.4.1 Decoding process

[0309] The decoding process provided in JVET-N0340 is defined as follows:

[0310] 8.5.2.2 Derivation process of brightness motion vector in Merge mode

[0311] This procedure is called only when general_merge_flag[XCB][YCB] equals 1, where (xCb, yCb) specifies the top-left sample of the current luminance codec block relative to the top-left luminance sample of the current image.

[0312] The input for this process is:

[0313] - The brightness position (xCb, yCb) of the top-left sample of the current luminance block relative to the top-left sample of the current image.

[0314] - The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0315] - The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0316] The output of this process is:

[0317] The brightness motion vectors in the -1 / 16 fractional sample accuracy mvL0[0][0] and mvL1[0][0]

[0318] -Refer to indices refIdxL0 and refIdxL1,

[0319] - The prediction list uses the flags predFlagL0[0][0] and predFlagL1[0][0].

[0320] - Bidirectional prediction weight index bcwIdx.

[0321] -Merge candidate list mergeCandList.

[0322] The bidirectional prediction weight index bcwIdx is set to equal to 0.

[0323] Motion vectors mvL0[0][0] and mvL1[0][0], reference indices refIdxL0 and refIdxL1, and prediction utilization flags predFlagL0[0][0] and predFlagL1[0][0] are derived through the following ordered steps:

[0324] 1. Call the derivation procedure for spatial merge candidates from neighboring codec units as specified in Section 8.5.2.4 with the following conditions: luma codec block position (xCb, yCb), luma codec block width cbWidth, and luma codec block height cbHeight as inputs; and the outputs are availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1, and availableFlagB2, with reference indices refIdxLXA0, refIdxL XA1, refIdxLXB0, refIdxLXB1 and refIdxLXB2, the prediction list uses the flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1 and predFlagLXB2, and motion vectors mvLXA0, mvLXA1, mvLXB0, mvLXB1 and mvLXB2, where X is 0 or 1, and bidirectional prediction weight indices bcwIdxA0, bcwIdxA1, bcwIdxB0, bcwIdxB1, bcwIdxB2.

[0325] 2. The bidirectional prediction weight index bcwIdxCol of the reference index refIdxLXCol (where X is 0 or 1) and the time-domain Merge candidate Col is set to equal to 0.

[0326] 3. Call the derivation procedure for temporal luminance motion vector prediction as specified in Section 8.5.2.11 using the following conditions: luminance position (xCb, yCb), luminance codec block width cbWidth, luminance codec block height cbHeight, and variable refIdxL0Col as input; and the output is the availability flag availableFlagL0Col and the temporal motion vector mvL0Col. The available variables FlagCol, predFlagL0Col, and predFlagL1Col can be derived as follows:

[0327] availableFlagCol=availableFlagL0Col(8-263)

[0328] predFlagL0Col=availableFlagL0Col(8-264)

[0329] predFlagL1Col=0(8-265)

[0330] 4. When slice_type equals B, the derivation process for the temporal lumen motion vector prediction specified in Section 8.5.2.11 is invoked with the following conditions: lumen position (xCb, yCb), lumen codec block width cbWidth, lumen codec block height cbHeight, and variable refIdxL1Col as input; and the output is the availability flag availableFlagL1Col and the temporal motion vector mvL1Col. The variables availableFlagCol and predFlagL1Col are derived as follows: availableFlagCol = availableFlagL0Col || availableFlagL1Col (8-266) predFlagL1Col = availableFlagL1Col (8-267)

[0331] 5. The Merge Candidate List (mergeCandList) is constructed as follows:

[0332] i=0

[0333] if(availableFlagA1)

[0334] mergeCandList[i++] = A1

[0335] if(availableFlagB1)

[0336] mergeCandList[i++] = B1

[0337] if(availableFlagB0)

[0338] mergeCandList[i++]=B0(8-268)

[0339] if(availableFlagA0)

[0340] mergeCandList[i++] = A0

[0341] if(availableFlagB2)

[0342] mergeCandList[i++] = B2

[0343] if(availableFlagCol)

[0344] mergeCandList[i++] = Col

[0345] 6. Set the variables numCurrMergeCand and numOrigMergeCand to be equal to the number of Merge candidates in mergeCandList.

[0346] 7. When numCurrMergeCand is less than (MaxNumMergeCand-1) and NumHmvpCand is greater than 0, the following applies:

[0347] - Call the history-based Merge candidate derivation procedure as specified in 8.5.2.6 with the following conditions: mergeCandList and numCurrMergeCand as inputs, and modified mergeCandList and numCurrMergeCand as outputs.

[0348] -numOrigMergeCand is set to be equal to numCurrMergeCand.

[0349] 8. When numCurrMergeCand is less than MaxNumMergeCand but greater than 1, the following applies:

[0350] The derivation procedure for pairwise average Merge candidates, as specified in Section 8.5.2.4, is invoked under the following conditions: mergeCandList, reference indices refIdxL0N and refIdxL1N, the prediction list taking flags predFlagL0N and predFlagL1N, the motion vectors mvL0N and mvL1N for each candidate N in mergeCandList, and numCurrMergeCand as input; and the output is assigned to mergeCandList, numCurrMergeCand, reference indices refIdxL0avgCand and refIdxL1avgCand, the prediction list taking flags predFlagL0avgCand and predFlagL1avgCand, and the motion vectors mvL0avgCand and mvL1avgCand of the candidate avgCands added to mergeCandList. The bidirectional prediction weight index bcwIdx of the candidate avgCands added to mergeCandList is set to equal to 0.

[0351] -numOrigMergeCand is set to be equal to numCurrMergeCand.

[0352] 9. Call the zero motion vector Merge candidate derivation procedure as specified in Section 8.5.2.5 with the following conditions: mergeCandList, reference indices refIdxL0N and refIdxL1N, the prediction list using flags predFlagL0N and predFlagL1N, the motion vectors mvL0N and mvL1N of each candidate N in mergeCandList, and numCurrMergeCand as input; and assign the output to mergeCandList, numCurrMergeCand, reference indices refIdxL0zeroCandm and refIdxL1zeroCandm, the prediction list using flags predFlagL0zeroCandm and predFlagL1zeroCandm, and the motion vectors mvL0zeroCandm and mvL1zeroCandm of each new candidate zeroCandm added to mergeCandList. The bidirectional prediction weight index bcwIdx of each new candidate zeroCandm added to mergeCandList is set to equal to 0. The number of candidates added, numSeroMergeCand, is set to equal to (numCurrMergeCand - numOrigMergeCand). When numZeroMergeCand is greater than 0, m ranges from 0 to numZeroMergeCand-1 (inclusive).

[0353] 10. Perform the following allocation, where N is the candidate at position merge_idx[xCb][yCb] in the merge candidate list mergeCandList(N=mergeCandList[merge_idx[xCb][yCb]]), and X is replaced by 0 or 1:

[0354] refIdxLX=refIdxLXN(8-269)

[0355] predFlagLX[0][0]=predFlagLXN(8-270)

[0356] mvLX[0][0][0]=mvLXN[0](8-271)

[0357] mvLX[0][0][1]=mvLXN[1](8-272)

[0358] bcwIdx=bcwIdxN(8-273)

[0359] 11. When mmvd_merge_flag[xCb][yCb] equals 1, the following applies:

[0360] - Call the Merge motion vector difference derivation procedure as specified in 8.5.2.7 with the following conditions: brightness position (xCb, yCb), reference index refIdxL0, refIdxL1 and prediction list using flags predFlagL0[0][0] and predFlagL1[0][0] as input, and motion vector difference mMvdL0 and mMvdL1 as output.

[0361] - The motion vector difference mMvdLX is added to the merged motion vector mvLX, where X is 0 and 1, as follows:

[0362] mvLX[0][0][0]+=mMvdLX[0](8-274)

[0363] mvLX[0][0][1]+=mMvdLX[1](8-275)

[0364] mvLX[0][0][0]=Clip3(-2 17 ,2 17 -1,mvLX[0][0][0])(8-276)

[0365] mvLX[0][0][1]=Clip3(-2 17 ,2 17 -1,mvLX[0][0][1])

[0366] (8-277)

[0367] 2.2.5 MMVD

[0368] In JVET-L0054, the Ultimate Motion Vector Expression (UMVE, also known as MMVD) was proposed. UMVE uses the motion vector expression method for skip mode or merge mode.

[0369] UMVE reuses the same Merge candidates included in the regular Merge candidate list in VVC. From among the Merge candidates, a basic candidate can be selected and further extended using the proposed motion vector representation method.

[0370] UMVE provides a novel representation of motion vector difference (MVD) using a starting point, motion amplitude, and motion direction.

[0371] Figure 17 An example of the UMVE search process is shown.

[0372] Figure 18 An example of a UMVE search point is shown.

[0373] The proposed technique uses the Merge candidate list as is. However, only candidates of the default Merge type (MRG_TYPE_DEFAULT_N) will be considered for UMVE extensions.

[0374] The basic candidate index defines the starting point. The basic candidate index indicates the best candidate among the candidates in the list, as follows.

[0375] Table 1: Basic Candidate IDX

[0376]

[0377] If the number of basic candidates is equal to 1, then no signaling is sent to the basic candidate IDX.

[0378] The distance index is motion amplitude information. The distance index indicates a predefined distance from the starting point. The predefined distances are as follows:

[0379] Table 2: Distance from IDX

[0380]

[0381] The direction index represents the direction of MVD relative to the starting point. The direction index can represent four directions, as shown below.

[0382] Table 3: Directional IDX

[0383] Directional IDX 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0384] Immediately after sending the skip and merge flags, signal the UMVE flag. If the skip and merge flags are true, the UMVE flag is resolved. If the UMVE flag is equal to 1, the UMVE syntax is resolved. However, if it is not 1, the affine flag is resolved. If the affine flag is equal to 1, it is an affine mode; however, if it is not equal to 1, the skip / merge index will be resolved for the VTM's skip / merge mode.

[0385] No additional line buffer is needed due to UMVE candidates, as the software's skip / merge candidates are used directly as the base candidates. Using the input UMVE index, MV supplementation can be determined before motion compensation. There is no need to reserve a long line buffer for this.

[0386] Under the current general testing conditions, the first or second Merge candidate in the Merge candidate list can be selected as the basic candidate.

[0387] UMVE is also known as Merge with MV difference (MMVD).

[0388] 2.2.6 Combined Intra-Inter-Frame Prediction (CIIP)

[0389] In JVET-L0100, multi-hypothesis prediction is proposed, in which combined intra-frame prediction and inter-frame prediction is a method for generating multiple hypotheses.

[0390] When multi-hypothesis prediction is applied to improve intra-mode, it combines an intra-mode prediction with a merge index prediction. In the merge CU, when a flag is true, signaling informs a flag for the merge mode to select an intra-mode from the intra-candidate list. For the luma component, the intra-candidate list is derived from only one intra-prediction mode (i.e., planar mode). The weights applied to the prediction block from intra- and inter-prediction are determined by the encoding / decoding modes (intra- or non-intra-) of two neighboring blocks (A1 and B1).

[0391] 2.2.7 Sub-block-based Merge technology

[0392] It is recommended that all motion candidates related to sub-blocks be placed not only in the regular Merge list of non-sub-block Merge candidates, but also in a separate Merge list.

[0393] Place motion candidates related to sub-blocks in a separate Merge list, which is named "Sub-block Merge Candidate List".

[0394] In one example, the list of sub-block merge candidates includes ATMVP candidates and affine merge candidates.

[0395] Fill the sub-block Merge candidate list with candidates in the following order:

[0396] a) ATMVP candidates (may be available or not);

[0397] b) Affine Merge list (including inherited affine candidates; and constructed affine candidates)

[0398] c) Zero-filled MV 4-parameter affine model

[0399] 2.2.7.1 ATMVP (also known as Sub-Block Temporal Motion Vector Predictor, SbTMVP)

[0400] The basic idea of ​​ATMVP is to derive multiple sets of temporal motion vector predictors for a block. Each sub-block is assigned a set of motion information. When generating ATMVP Merge candidates, full motion compensation is performed at the 8×8 level rather than the entire block level.

[0401] In the current design, ATMVP predicts the motion vectors of sub-CUs within a CU in two steps, which are described in the following two subsections.

[0402] 2.2.7.1.1 Derivation of Initialization Motion Vectors

[0403] The initial motion vector is represented by tempMv. When block A1 is available and not intra-frame encoded (i.e., using inter-frame or IBC mode encoding), the following is used to derive the initial motion vector.

[0404] - If all of the following conditions are true, then tempMv is set to be equal to the motion vector of block A1 from list 1, denoted by mvL1A1:

[0405] - The reference image index for list 1 is available (not equal to -1) and has the same POC value as the juxtaposed image (i.e., DiffPicOrderCnt(ColPic, RefPicList[1][refIdxL1A1]) equals 0).

[0406] - Compared to the current image, none of the reference images have a larger Proof of Concept (POC) (e.g., for each image aPic in the reference image list for the current strip, DiffPicOrderCnt(aPic, currPic) is less than or equal to 0).

[0407] -The current stripe is equal to stripe B.

[0408] -collocated_from_l0_flag equals 0.

[0409] Otherwise, if all of the following conditions are true, then tempMv is set to be equal to the motion vector of block A1 from list 0, denoted by mvL0A1: - If all of the following conditions are true, then tempMv is set to be equal to the motion vector of block A1 from list 0, denoted by mvL1A1:

[0410] - The reference image index for list 0 is available (not equal to -1).

[0411] - Has the same POC value as the juxtaposed image (e.g., DiffPicOrderCnt(ColPic,RefPicList[0][refIdxL0A1]) equals 0).

[0412] Otherwise, the zero motion vector is used to initialize the MV.

[0413] In the juxtaposed image of the signaling notification at the strip header with initialized motion vector, identify the corresponding block (the center position of the current block plus the rounded MV, which is pruned within a specific range as necessary).

[0414] If the block is inter-frame encoded, proceed to step two. Otherwise, the ATMVP candidate is set to unavailable.

[0415] 2.2.7.1.2 Sub-CU Motion Derivation

[0416] The second step is to divide the current CU into sub-CUs and obtain the motion information of each sub-CU from the block corresponding to each sub-CU in the juxtaposed image.

[0417] If the corresponding block of a sub-CU is encoded and decoded in inter-frame mode, the final motion information of the current sub-CU is derived using motion information by calling the derivation procedure for the concatenated MV. This derivation procedure differs from that used in the traditional TMVP procedure. Basically, if the corresponding block is predicted from the target list X for unidirectional or bidirectional prediction, the motion vector is used; otherwise, if the corresponding block is predicted from list Y (Y = 1-X) for unidirectional or bidirectional prediction, and NoBackwardPredFlag equals 1, the MV of list Y is used. Otherwise, motion candidates may not be found.

[0418] If the blocks in the juxtaposed image identified by the initial MV and the position of the current sub-CU are intra-frame or IBC encoded, or if motion candidates may not be found as described above, then the following further applies:

[0419] The motion vectors used to extract the motion field from the juxtaposed image Rcol are denoted as MVcol. To minimize the impact of MV scaling, the MVs in the spatial candidate list for MVcol export can be selected as follows: if the reference image of a candidate MV is a juxtaposed image, then that MV is selected and used as MVcol without any scaling. Otherwise, the MV with the reference image closest to the juxtaposed image is selected to export MVcol with scaling.

[0420] 2.2.8 Refinement of motion information

[0421] 2.2.8.1 Decoder-Side Motion Vector Refinement (DMVR)

[0422] In bidirectional prediction, for the prediction of a block region, two prediction blocks formed by the motion vectors (MV) from list 0 and list 1 are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined.

[0423] For DMVR in VVC, assuming the MVD mirror between list 0 and list 1 is as follows: Figure 19As shown, bilateral matching is performed to refine the MV, i.e., finding the optimal MVD among several MVD candidates. Let MVL0(L0X, L0Y) and MVL1(L1X, L1Y) represent the MVs of two lists of reference images. The optimal MVD is defined as the MVD represented by (MvdX, MvdY) of list 0 that minimizes the cost function (e.g., SAD). The SAD function is defined as the SAD between the reference block of list 0 derived from the motion vectors (L0X+MvdX, L0Y+MvdY) in the list 0 reference image and the reference block of list 1 derived from the motion vectors (L1X-MvdX, L1Y-MvdY) in the list 1 reference image.

[0424] The motion vector thinning process can be iterated twice. In each iteration, up to six MVDs (with integer pixel precision) can be checked in two steps, such as... Figure 20 As shown. In the first step, MVD(0,0), (-1,0), (1,0), (0,-1), and (0,1) are checked. In the second step, one of MVD(-1,-1), (-1,1), (1,-1), or (1,1) can be selected and further checked. Assume the function Sad(x,y) returns the SAD value of MVD(x,y). The MVD represented by (MvdX, MvdY) checked in the second step is determined as follows:

[0425] MvdX = -1;

[0426] MvdY = -1;

[0427] If (Sad(1,0)) <Sad(-1,0))

[0428] MvdX = 1;

[0429] If (Sad(0,1) <Sad(0,-1))

[0430] MvdY = 1;

[0431] In the first iteration, the starting point is the MV of the signaling notification, and in the second iteration, the starting point is the MV of the signaling notification plus the best MVD selected in the first iteration. DMVR is applied only when one reference image is the preceding image and the other reference image is the following image, and the image order count distances between the two reference images and the current image are the same.

[0432] To further simplify the DMVR process, JVET-M0147 proposes several changes to the JEM design. More specifically, the DMVR design adopted for VTM-4.0 (to be released) has the following key features:

[0433] - Terminate early when the SAD at position (0,0) between list 0 and list 1 is less than the threshold.

[0434] - Terminate early when the SAD between list 0 and list 1 is zero for some position.

[0435] - DMVR block size: W*H>=64&&H>=8, where W and H are the width and height of the block.

[0436] For DMVRs with CU dimensions > 16x16, divide the CU into multiple 16x16 sub-blocks. If only the width or height of the CU is greater than 16, divide it only in the vertical or horizontal direction.

[0437] ο Reference block size (W+7)*(H+7) (for brightness).

[0438] ο25-point SAD-based integer pixel search (i.e., (+-)2 refines the search range, single stage)

[0439] ο DMVR based on bilinear interpolation.

[0440] Subpixel thinning based on the "parameter error surface equation". This process is only performed if the minimum SAD cost is not equal to zero and the optimal MVD was (0, 0) in the previous MV thinning iteration.

[0441] o Brightness / Chromatography MC w / Reference block fill (if needed).

[0442] This is only for refining the MV for MC and TMVP.

[0443] 2.2.8.1.1 Use of DMVR

[0444] DMVR can be enabled when all of the following conditions are true:

[0445] - The DMVR enable flag in SPS (i.e., sps_dmvr_enabled_flag) is equal to 1.

[0446] - The TPM flag, inter-frame affine flag, and sub-block merge flag (ATMVP or affine merge), and the MMVD flag are all equal to 0.

[0447] -Merge flag equals 1

[0448] - The current block is predicted bidirectionally, and the POC distance between the current image and the reference image in List 1 is equal to the POC distance between the reference image and the current image in List 0.

[0449] - The current CU height is greater than or equal to 8

[0450] - The number of luminance samples (CU width * height) is greater than or equal to 64

[0451] 2.2.8.1.2 Sub-pixel thinning based on the “parameter error surface equation”.

[0452] The method is summarized as follows:

[0453] 1) Parameter error surface fitting is calculated only if the center position is the optimal cost position in a given iteration.

[0454] 2) The cost at the center location and the costs at locations (-1, 0), (0, -1), (1, 0), and (0, 1) from the center are used to fit the following two-dimensional parabolic error surface equation.

[0455] E(x,y)=A(x-x0) 2 +B(y-y0) 2 +C

[0456] Where x0, y0) corresponds to the position with minimum cost, and C corresponds to the minimum cost value. By solving the five equations involving the five unknowns, x0, y0) can be calculated as follows:

[0457] x0=(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0)))

[0458] y0=(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0)))

[0459] The precision of the division (i.e., how many bits of the quotient are calculated) can be adjusted to compute x0, y0) to any desired subpixel precision. For 1 / 16 pixel accuracy, only 4 bits of the absolute value of the quotient need to be computed, which facilitates the implementation of fast shift-based subtraction with only 2 divisions per CU.

[0460] 3) Add the calculated x0, y0) to the integer distance thinning MV to obtain the sub-pixel accurate thinning increment MV.

[0461] 2.3 Intra-Block Copying (IBC)

[0462] HEVC Screen Content Codec Extension (HEVC-SCC) and the current VVC test model (VTM-4.0) have adopted Intra-Block Copy (IBC), also known as Current Picture Reference. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. For example... Figure 21As shown, when IBC is applied, the current block is predicted from a reference block in the same image. Samples in the reference block must be reconstructed before the current block can be encoded or decoded. Although IBC is inefficient for most camera-captured sequences, it demonstrates significant encoding / decoding gains for screen content. This is because screen content images contain many repeating patterns, such as icons and text characters. IBC effectively removes redundancy between these repeating patterns. In HEVC-SCC, the codec unit (CU) of inter-frame encoding / decoding can apply IBC if the current image is chosen as its reference image. In this case, the MV is renamed to a block vector (BV), and the BV always has integer pixel precision. For compatibility with the main configuration document HEVC, the current image is marked as the "long-term" reference image in the Decoded Picture Buffer (DPB). It should be noted that, similarly, in multi-view / 3D video codec standards, inter-view reference images are also marked as "long-term" reference images.

[0463] After BV finds its reference block, predictions can be generated by copying the reference block. The residual can be obtained by subtracting the reference pixel from the original signal. Then, transforms and quantization can be applied as in other encoding / decoding modes.

[0464] However, some or all pixel values ​​are undefined when the reference block is outside the image, overlaps with the current block, is outside the reconstructed region, or is outside the valid region subject to certain constraints. Basically, there are two solutions to this problem. One is to prevent this situation, such as in bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0465] 2.3.1 IBC (VTM4.0) in the VVC test model

[0466] In the current VVC test model (i.e., the VTM-4.0 design), the entire reference block should be together with the current codec tree unit (CTU) and should not overlap with the current block. Therefore, there is no need to fill the reference or prediction block. The IBC flag is encoded and decoded into the prediction mode of the current CU. Therefore, there are a total of three prediction modes per CU: MODE_INTRA, MODE_INTER, and MODE_IBC.

[0467] 2.3.1.1 IBC Merge Mode

[0468] In IBC Merge mode, indices pointing to entries in the IBC Merge candidate list are parsed from the bitstream. The construction of the IBC Merge list can be summarized according to the following sequence of steps:

[0469] Step 1: Export spatial domain candidates

[0470] Step 2: Insert HMVP candidate

[0471] Step 3: Insert pairwise average candidates

[0472] In the export of the spatial Merge candidate, located as shown in the appendix Figure 2 Up to four merged candidates are selected from the candidates shown at positions A1, B1, B0, A0, and B2. The derived order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another strip or slice) or if encoding / decoding is not performed using IBC mode. After adding the candidate at position A1, a redundancy check is performed on the insertion of the remaining candidates, ensuring that candidates with the same motion information are excluded from the list, thus improving encoding / decoding efficiency.

[0473] After inserting an empty domain candidate, if the IBC Merge list size is still smaller than the maximum IBC Merge list size, an IBC candidate from the HMVP table can be inserted. Redundancy checks are performed when inserting an HMVP candidate.

[0474] Finally, the paired average candidates are inserted into the IBC Merge list.

[0475] A Merge candidate is considered an invalid Merge candidate when the reference block identified by the Merge candidate is outside the image, overlaps with the current block, is outside the reconstructed region, or is outside the valid region subject to certain constraints.

[0476] Note that invalid Merge candidates can be inserted into the IBC Merge list.

[0477] 2.3.1.2 IBC AMVP Mode

[0478] In IBC AMVP mode, the AMVP index pointing to an entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized according to the following steps:

[0479] Step 1: Export spatial domain candidates

[0480] • Check A0 and A1 until a usable candidate is found.

[0481] • Check B0, B1, and B2 until a usable candidate is found.

[0482] Step 2: Insert HMVP candidate

[0483] Step 3: Insert zero candidate

[0484] After inserting a spatial candidate, if the size of the IBC AMVP list is still smaller than the maximum IBC AMVP list size, then an IBC candidate from the HMVP table can be inserted.

[0485] Finally, the zero candidate is inserted into the IBC AMVP list.

[0486] 2.3.1.3 Chromaticity IBC Mode

[0487] In the current VVC, motion compensation in the chroma IBC mode is performed at the sub-block level. The chroma block is divided into several sub-blocks. Each sub-block determines whether its corresponding luma block has a block vector and its validity (if present). Encoder constraints exist in the current VTM, where the chroma IBC mode is tested if all sub-blocks in the current chroma CU have valid luma block vectors. For example, on YUV 420 video, the chroma block is NxM, and the luma region is then juxtaposed as 2Nx2M. The sub-block size of the chroma block is 2x2. Several steps are involved in performing chroma mv export, followed by a block copying process.

[0488] 1) The chroma block will first be divided into (N>>1)*(M>>1) sub-blocks.

[0489] 2) For each sub-block with a top-left sample point at coordinates (x, y), retrieve the corresponding luminance block, which covers the same top-left sample point at coordinates (2x, 2y).

[0490] 3) The encoder checks the block vector (bv) of the retrieved luminance block. If one of the following conditions is met, the bv is considered invalid.

[0491] a. The bv corresponding to the luminance block does not exist.

[0492] b. The predicted block identified by bv has not yet been reconstructed.

[0493] c. The predicted block identified by bv partially or completely overlaps with the current block.

[0494] 4) Set the chromaticity motion vector of the sub-block to the motion vector of the corresponding luminance sub-block.

[0495] When all sub-blocks find a valid bv, enable IBC mode at the encoder.

[0496] 2.3.2 Latest Developments in IBC (in VTM5.0)

[0497] 2.3.2.1 Single BV List

[0498] JVET-N0843 is used for VCC. In JVET-N0843, BV predictors used in IBC for Merge mode and AMVP mode will share a common predictor list, which contains the following elements:

[0499] ο2 spatial neighborhood locations (e.g.) Figure 2 (A1, B1)

[0500] ο5 HMVP entries

[0501] ο Default zero vector

[0502] The number of candidates in the list is controlled by a variable derived from the stripe header. For Merge mode, a maximum of the first 6 entries of this list will be used; for AMVP mode, the first 2 entries will be used. Furthermore, this list must conform to the shared Merge list region requirement (the same list is shared within SMR).

[0503] In addition to the aforementioned list of BV predictor candidates, JVET-N0843 proposes to simplify the pruning operation between HMVP candidates and existing Merge candidates (A1, B1). In this simplification, a maximum of two pruning operations will be performed, as it only compares the first HMVP candidate with one or more spatial Merge candidates.

[0504] 2.3.2.2 Update the IBC HMVP table

[0505] Similar to the design of the HMVP table update process used for inter-frame mode encoding and decoding, pruning is applied and a count is maintained to record the number of IBC candidates available in the IBC HMVP table.

[0506] The relevant regulations are as follows:

[0507] 8.6.2.6 Update process of the historical motion vector predictor candidate list

[0508] The input for this process is:

[0509] The brightness motion vector mvL with an accuracy of -1 / 16 fractional sample points.

[0510] The candidate list HmvpIbcCandList is modified by the following ordered steps:

[0511] 1. Set the variable identicalCandExist to false and the variable removeIdx to 0.

[0512] 2. When NumHmvpIbcCand is greater than 0, for each index hMvpIdx, where hMvpIdx = 0..NumHmvpIbcCand-1, apply the following steps until identicalCandExist equals true:

[0513] – When hMvpCand equals HmvpIbcCandList[hMvpIdx], identicalCandExist is set to TRUE and removeIdx is set to hMvpIdx.

[0514] 3. The candidate list HmvpIbcCandList is updated as follows:

[0515] If identicalCandExist equals true or NumHmvpIbcCand equals MaxNumMergeCand-1, then the following applies:

[0516] – For each index, where i = (removeIdx+1)...(NumHmvpIbcCand-1), HmvpIbcCandList[i-1] is set to equal HmvpIbcCandList[i].

[0517] –HmvpIbcCandList[NumHmvpIbcCand-1] is set to equal to mvCand.

[0518] Otherwise (identicalCandExist equals false or NumHmvpIbcCand is less than MaxNumMergeCand-1), then the following applies:

[0519] –HmvpIbcCandList[NumHmvpIbcCand++] is set to equal to mvCand.

[0520] Here, MaxNumMergeCand is set to 6.

[0521] 2.4 Switchable Interpolation Filter

[0522] 2.4.1 Half-pixel AMVR mode

[0523] An additional AMVR mode for the CU in non-affine, non-merge inter-frame encoding / decoding is proposed, which allows motion vector differences to be notified with half-pixel accuracy signaling. The existing AMVR scheme in the current VVC draft is extended directly as follows: immediately following the syntax element `amvr_flag`, if `amvr_flag == 1`, there exists a new context-modeled binary syntax element `hpel_amvr_flag`, which indicates that if `hpel_amvr_flag == 1`, the new half-pixel AMVR mode is used. Otherwise, i.e., if `hpel_amvr_flag == 0`, the choice between full-pixel and 4-pixel AMVR modes is indicated by the syntax element `amvr_precision_flag`, as in the current VVC draft.

[0524] like Figure 24 As shown in the table, extended AMVR signaling notifications.

[0525] In JVET-N0309, an alternative half-pixel luminance interpolation filter is proposed, wherein two 6-tap filters with smoothing characteristics represented as FlatTop and Gauss are tested using CE4-1. See also Figure 25 .

[0526] 2.4.2 Explicit Signaling Notification

[0527] For non-affine and non-Merge inter-frame codecs using half-pixel motion vector accuracy (i.e., half-pixel AMVR mode), an alternative luma half-pixel interpolation filter is used. For the test described in 2.4.1, a Gauss luma half-pixel interpolation filter is used. For this test, switching between the two alternative half-pixel interpolation filters is performed based on the value of the new syntax element hpel_if_idx. In the case of half-pixel AMVR mode, only the signaling of the syntax element hpel_if_idx is communicated, as follows:

[0528]

[0529] 2.4.3 Implicit Signaling Notification

[0530] When using the skip / merge mode of the spatial merge candidate, information about the position of the interpolation filter applied to half a pixel is inherited from the neighborhood block.

[0531] 3. Examples of problems solved by the disclosed embodiments

[0532] The current HMVP design must record the number of available HMVP candidates in the HMVP table. Two different counts are maintained for the HMVP table used for inter-frame codec blocks and the IBC HMVP table used for IBC codec blocks. Regardless of whether the HMVP table needs updating, the counts must be checked and modified accordingly.

[0533] 4. Example Implementation

[0534] The examples listed below should be considered as examples to illustrate the overall concept. These embodiments should not be interpreted in a narrow sense. Furthermore, these inventions can be combined in any manner.

[0535] LUT-based motion vector prediction (which can improve both existing and future video codec standards) based on publicly available techniques is illustrated below as examples of various implementations described. In LUT-based motion vector prediction methods, one or more tables containing motion information from previous codec blocks are maintained during the encoding / decoding process. During the encoding / decoding of a block, the associated motion information in the LUT can be added to a motion candidate list, and the LUT can be updated after the encoding / decoding of a block. LUT-based methods are also known as history-based motion vector prediction (HMVP) methods. Motion information includes at least one of the following: prediction direction, reference picture index, motion vector value, intensity compensation flag, affine flag, motion vector difference precision, or motion vector difference. Motion candidates may also include other types of codec information, such as intra-prediction modes for intra-mode codecs; luminance compensation parameters for IC parameter codecs; and filter parameters.

[0536] Before encoding and decoding video units (e.g., strips / pieces / blocks / pictures / CTU lines / CTUs), the HMVP table can be reset by populating it with one or more default motion candidates, rather than resetting the count of available HMVP candidates in the table to 0. The HMVP table size is represented as N, meaning it can store up to N HMVP candidates.

[0537] In the following discussion, the term "strip" may also be referred to as "slice group" or "slice / block / CTU line" or other video data units.

[0538] 1. Regardless of whether the HMVP table is reset, K (K<=N) default HMVP candidates can be used to populate the HMVP table.

[0539] a. In one example, the K default HMVP candidates are the same.

[0540] i. Alternatively, at least two of the K default HMVP candidates are different.

[0541] ii. Alternatively, a default HMVP candidate can be added to the HMVP table multiple times.

[0542] 1) Alternatively, the first m (m < K) default HMVP candidates can be different from each other, while the remaining (K - m) candidates are the same as one of the first m candidates.

[0543] 2) Alternatively, the last m (m < K) default HMVP candidates can be different from each other, while the remaining first (K - m) candidates are the same as one of the first m candidates.

[0544] iii. Alternatively, the first m (m < K) default HMVP candidates can be different from each other, while the remaining (K - m) candidates are filled by repeatedly copying the first m candidates.

[0545] b. In one example, the default HMVP candidate can be set to a uni - directional prediction motion candidate.

[0546] i. In one example, for P - slices / pictures, the prediction direction can be set to list 0.

[0547] ii. In one example, for B - slices / pictures, the prediction direction can be set to list 0 or list 1

[0548] iii. Alternatively, in addition, the motion vector (MV) of the default HMVP candidate can be set to (0, 0).

[0549] iv. Alternatively, in addition, the MV of the default HMVP candidate can be set to (-m * W, 0), where m and W are two integers. For example, W is one of the allowed widths of the coding / decoding unit / prediction unit / transform unit.

[0550] v. Alternatively, in addition, the MV of the default HMVP candidate can be set to (0, -q * H), where q and H are two integers. For example, H is one of the allowed heights of the coding / decoding unit / prediction unit / transform unit.

[0551] vi. Alternatively, in addition, the MV of the default HMVP candidate can be set to (-m * W, -q * H), where m, W, q, H are integers. For example, W and H are respectively one of the allowed widths and heights of the coding / decoding unit / prediction unit / transform unit.

[0552] vii. Alternatively, in addition, the reference index of the default HMVP candidate can be set to any value less than the number of reference pictures in the prediction direction (i.e., list 0 or 1).

[0553] viii. Alternatively, in addition, the reference picture can be set to the current picture.

[0554] ix. Alternatively, the IBC flag can be set to 1.

[0555] c. In one example, the default HMVP candidate can be set as a bidirectional prediction motion candidate. i. In one example, the bullet number under 1.ii can be used to set the motion information associated with each prediction direction.

[0556] ii. Alternatively, for two MVs, the weighting factors of the two predictions from the two directions can be set to be equal.

[0557] 1) In one example, the Generalized Bidirectional Prediction (GBI) index is set to 0.

[0558] d. Default HMVP candidates may include motion information from another video unit (such as another CTU line / another picture).

[0559] e. In one example, for P stripes / images, the default HMVP candidate can include (Mv(0,0),i), where i is the index of the reference image, and i is in the range [0, the number of reference images in list 0 minus 1].

[0560] Alternatively, candidates can be added to the HMVP table in ascending / descending order of i.

[0561] ii. Alternatively, if the number of reference images in list 0 is less than K, one of the default motion candidates can be added repeatedly until K candidates are available.

[0562] f. Alternatively, if the number of reference images in list 0 is less than K, the available default HMVP candidates can be repeatedly added until K candidates are available. In one example, for B strips / images, the default HMVP candidates may include K0 one-way predicted HMVP candidates from list X, K1 one-way predicted HMVP candidates from list Y (Y equals (1-X)), and K2 two-way predicted HMVP candidates, where (K0+K1+K2=K, and K0 / K1 / K2>=0).

[0563] i. In one example, for B-band / image, the default HMVP candidate may only include bidirectional predicted HMVP candidates.

[0564] ii. In one example, for B strips / images, the default HMVP candidates can include staggered one-way predicted HMVP candidates and two-way predicted HMVP candidates.

[0565] iii. In one example, for a B strip / image, the default HMVP candidate may include a one-way predicted HMVP candidate and a subsequent two-way predicted HMVP candidate.

[0566] g. Alternatively, when K equals N, the count maintained in the record table of available HMVP candidates is removed.

[0567] h. In one example, the default HMVP candidate may refer only to a reference image with a reference index equal to a specific value (e.g., 0).

[0568] 2. The default HMVP candidate can be adaptively changed from one video unit to another.

[0569] a. The default HMVP candidate can be adaptively changed from one CTU line to another.

[0570] b. The default HMVP candidate can be adapted from one slice to another.

[0571] 3. The default HMVP candidates can be different for different HMVP tables.

[0572] a. In one example, the selection of the default HMVP candidate can depend on the HMVP table used for encoding / decoding modes.

[0573] i. In one example, the default HMVP candidates used to populate the HMVP table can be different for inter-frame codec blocks and IBC codec blocks.

[0574] 4. When N default HMVP candidates are populated in the HMVP table before encoding / decoding video units, the following can also be applied when adding new motion candidates.

[0575] a. In one example, a new motion candidate is compared with available HMVP candidates based on the ascending or descending order of the index of the HMVP candidates in the HMVP table.

[0576] i. In one example, the comparison process terminates when identical HMVP candidates are found. Alternatively, identical HMVP candidates are removed from the table, and the new motion candidate is added to the last entry of the table.

[0577] ii. In one example, the comparison process terminates after all HMVP candidates in the table have been checked. Alternatively, all identical M HMVP candidates are removed from the table, and new motion candidates are added repeatedly until the table is full.

[0578] 5. The default HMVP candidate can be notified via signaling in VPS / DPS / SPS / APS / PPS / image header / strip header / group header.

[0579] a. Alternatively, multiple sets of default HMVP candidates can be predefined, and one or more indices can be signaled to indicate which set of HMVP candidates to use for the picture / strip / piece group / piece / CTU line.

[0580] b. Alternatively, multiple sets of default HMVP candidates can be predefined, and an index of a codec mode (e.g., IBC or non-IBC) can be signaled to indicate which set of HMVP candidates is used by the picture / strip / piece group / piece / CTU line.

[0581] 6. When updating the HMVP table / HMVP list, a new motion candidate to be added can be added before all existing motion candidates in the HMVP table / HMVP list. In this case, the entry index of the table with the minimum value can indicate the latest motion candidate in the decoding order.

[0582] a. In one example, if a new motion candidate is the same as or similar to an existing candidate in the table, then no new motion candidate needs to be added.

[0583] i. Alternatively, if a new motion candidate is the same as or similar to an existing candidate in the table (e.g., the MV difference is less than a threshold), then a new motion candidate may not be added, and the same or similar existing candidate may be removed from the table.

[0584] 7. The methods mentioned above can be applied to all HMVP tables (e.g., those HMVP tables to be used by inter-frame codec blocks or IBC codec blocks).

[0585] a. Alternatively, the method mentioned above should be applied to which HMVP table, depending on the table’s usage, such as whether it is used for IBC codec blocks.

[0586] 8. The proposed method can be applied to other encoding and decoding information with historical information, such as history-based intra-mode encoding and decoding methods with LUTs filled by default intra-prediction modes.

[0587] Storage of motion information in the HMVP table / HMVP list

[0588] 9. Indication of interpolation filters for blocks and / or side information of switchable interpolation filters can be stored in motion candidates stored in the HMVP table / HMVP list.

[0589] a. In one example, motion candidates include indications of interpolation filters for blocks stored along with other motion information such as MV, reference image index, GBi (Generalized Bidirectional Prediction) / BCW (Also known as Bidirectional Prediction with CU Weights) index, etc.

[0590] b. In one example, two motion candidates with different interpolation filters can be considered as different motion candidates.

[0591] c. In one example, two motion candidates with different interpolation filters can be considered as the same motion candidate. For example, for motion candidate X in the HMVP table, if motion candidate Y has the same information as X except for the interpolation filter, X can be removed from the HMVP table and Y can be inserted into the HMVP table.

[0592] d. In one example, whether the indication of the interpolation filter for the motion candidate is stored in the HMVP table may depend on the MV resolution of the motion candidate.

[0593] i. For example, if a candidate MV has a resolution of half a pixel, the indication of the interpolation filter for the motion candidate can be stored in the HMVP table.

[0594] ii. In one example, an interpolation filter can be stored for each prediction direction.

[0595] Use of motion information in the HMVP table / HMVP list

[0596] 10. During the pruning process of adding new candidates to the candidate list of existing candidates (e.g., comparing and / or suppressing the addition of new candidates), the interpolation filter information should also be compared.

[0597] e. In one example, suppose there are two candidates that have the same motion information except for the interpolation filters associated with them; in this case, they are considered different. New candidates can also be added to the candidate list.

[0598] 11. When a motion (HMVP) candidate from the HMVP table / HMVP list is added to a list (e.g., the AMVP or Merge list) as a new (e.g., AMVP or Merge) candidate, the interpolation filter associated with the motion candidate can also be inherited by the new candidate.

[0599] f. In one example, interpolation filter information can also be considered during the pruning process.

[0600] 12. Based on a motion (HMVP) candidate in an HMVP table / list, multiple new candidates can be derived into a list (e.g., an AMVP or Merge list), and each new candidate can be associated with a different interpolation filter.

[0601] Figure 22This is a block diagram of video processing device 2200. Device 2200 can be used to implement one or more of the methods described herein. Device 2200 can be implemented as a smartphone, tablet computer, Internet of Things (IoT) receiver, etc. Device 2200 may include one or more processors 2202, one or more memories 2204, and video processing hardware 2206. The processors 2202 can be configured to implement one or more methods described in this document. The memories 2204 can be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 2206 can be used to implement some of the techniques described in this document in hardware circuitry.

[0602] Figure 23 This is a flowchart of an example method 2300 for video processing. Method 2300 includes: (2302) populating the HMVP table with a set of K default HMVP candidates when a history-based motion vector prediction (HMVP) table of size N is determined to be reset, where N and K are positive integers and K≤N; and (2304) performing a conversion between the current block and the bitstream representation of the current block based on the HMVP table.

[0603] Some embodiments may preferably include the following examples of technology.

[0604] 1. A video processing method comprising: populating an HMVP table with a set of K default HMVP candidates when a history-based motion vector prediction (HMVP) table of size N is determined to be reset, wherein N and K are positive integers and K ≤ N; and performing a conversion between the current block and the bitstream representation of the current block based on the HMVP table.

[0605] 2. As described in Example 1, wherein each default HMVP candidate in the set of K default HMVP candidates is identical.

[0606] 3. The method as described in Example 1, wherein at least two of the K default HMVP candidates in the set of default HMVP candidates are different.

[0607] 4. As described in Example 1, wherein at least one default HMVP candidate from the set of K default HMVP candidates is added multiple times to the HMVP table.

[0608] 5. The method as described in Example 4, wherein each of the first m candidates in the HMVP table is different, and each of the last (Km) candidates in the HMVP table is the same as one of the first m candidates, and wherein m is a positive integer.

[0609] 6. The method as described in Example 4, wherein each of the last (Km) candidates in the HMVP table is different, and wherein each of the first m candidates in the HMVP table is the same as one of the last (Km) candidates, and wherein m is a positive integer.

[0610] 7. The method as described in Example 4, wherein each of the first m candidates in the HMVP table is different, and wherein the last (Km) candidates in the HMVP table include one or more copies of the first m candidates.

[0611] 8. The method as described in Example 1, wherein at least one of the set of K default HMVP candidates is a unidirectional predictive motion candidate.

[0612] 9. The method as described in Example 8, wherein the strip or image including the current block is a P strip or image, and wherein the prediction direction of the unidirectional predicted motion candidate is set to list 0 (L0).

[0613] 10. The method as described in Example 8, wherein the strip or image including the current block is a B strip or image, and wherein the prediction direction of the unidirectional predicted motion candidate is set to list 0 (L0) or list 1 (L1).

[0614] 11. The method as described in Example 8, wherein the motion vector of the unidirectional predicted motion candidate is set to (0,0).

[0615] 12. The method as described in Example 8, wherein the motion vector of the unidirectional predicted motion candidate is set to (-m×W,0), where m and W are integers.

[0616] 13. The method as described in Example 8, wherein the motion vector of the unidirectional predicted motion candidate is set to (0, -q×H), where q and H are integers.

[0617] 14. The method as described in Example 8, wherein the motion vector of the unidirectional predicted motion candidate is set to (-m×W, -q×H), where m, q, H and W are integers.

[0618] 15. The method as described in Example 12 or 14, wherein W is the allowed width of the prediction unit (PU), encoding / decoding unit (CU), or transform unit (TU) associated with the current block.

[0619] 16. The method as described in Example 13 or 14, wherein H is the allowed height of the prediction unit (PU), encoding / decoding unit (CU), or transform unit (TU) associated with the current block.

[0620] 17. The method as described in Example 1, wherein at least one of the set of K default HMVP candidates is a bidirectional predictive motion candidate.

[0621] 18. The method as described in Example 1, wherein at least one of the set of K default HMVP candidates includes motion information from a video unit different from the video unit that includes the current block.

[0622] 19. The method as described in Example 18, wherein the video unit comprises a codec tree unit (CTU) row or picture.

[0623] 20. A video processing method comprising: for a conversion between current blocks and bitstream representations of the current blocks, determining a set of default history-based motion vector prediction (HMVP) candidates to be used during the conversion; and performing the conversion using the set of default HMVP candidates, wherein different HMVP candidates are used during the conversion of different video units.

[0624] 21. The method as described in Example 20, wherein different video units include codec tree unit (CTU) rows or different slices.

[0625] 22. A video processing method comprising: determining a set of N default history-based motion vector prediction (HMVP) candidates that are already populated in an HMVP table before a conversion between the current block and the bitstream representation of the current block; adding the motion candidates to the HMVP table; and performing the conversion using the HMVP table including the motion candidates.

[0626] 23. The method as shown in Example 22 further includes: comparing the motion candidate with each candidate in the HMVP table based on ascending or descending order of the index of the set of N default HMVP candidates.

[0627] 24. The method as described in Example 23, wherein the comparison is terminated when a motion candidate matches a specific candidate in the HMVP table.

[0628] 25. The method as described in Example 24, wherein a specific candidate is removed from the HMVP table, and wherein a motion candidate is added to the last entry of the HMVP table.

[0629] 26. The method as described in any one of Examples 1 to 25, wherein the signaling notifies the set of default HMVP candidates in the following: Video Parameter Set (VPS), Decoder Parameter Set (DPS), Sequence Parameter Set (SPS), Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Picture Header, Strip Header, or Slice Header.

[0630] 27. The method as described in any one of Examples 1 to 25, wherein a plurality of sets of default HMVP candidates are predefined, and wherein signaling notifies the index of the set of default HMVP candidates within the plurality of sets of default HMVP candidates.

[0631] 28. The method as described in any one of Examples 1 to 27, wherein the current block is encoded and decoded using an inter-frame encoding / decoding mode or an intra-block copy (IBC) mode.

[0632] 29. A video processing method comprising: storing information of an interpolation filter for the current video block or side information of the current video block with respect to a switchable interpolation filter, together with past motion information of the current video block, in a table prior to a conversion between a current video block and a bitstream representation of the current video block; performing a conversion based on entries in the table; and updating the table after performing the conversion.

[0633] 30. The method as described in Example 29, wherein the past motion information includes motion candidates, which include interpolation filter information or side information and at least one of the following: motion vector, reference image index, generalized bidirectional prediction information, and bidirectional prediction weight index.

[0634] 31. The method as described in any one of Examples 29 to 30, wherein the update includes updating based on information from the interpolation filter.

[0635] 32. The method as described in any one of Examples 29 to 31, wherein the update includes updating based on edge information.

[0636] 33. The method as described in any one of Examples 31 to 32, wherein the information of the interpolation filter or the edge information is used to determine the uniqueness of the corresponding motion candidate.

[0637] 34. The method as described in any one of Examples 29 to 33, wherein the update is based on a pruning process in which a new motion candidate is compared with information stored in a table and the new motion candidate is selectively added based on the unique information contained in the new motion candidate.

[0638] 35. The method described in Example 34, wherein the pruning process determines uniqueness based on information from the interpolation filter.

[0639] 36. The method as described in any one of Examples 29 to 35, wherein the update includes updating the table using motion candidates or merge candidates from the high-level motion vector resolution list along with corresponding interpolation filter information or edge information.

[0640] 37. The method as described in any one of Examples 29 to 35, wherein the update includes generating new candidates by deriving from other candidates that have different interpolation information or different edge information.

[0641] 38. The method as described in any one of Examples 29 to 37, wherein the conversion includes an encoding process for generating a bitstream representation from the current video block.

[0642] 39. The method as described in any one of Examples 29 to 37, wherein the conversion includes a decoding process for generating the current video block from the bitstream representation.

[0643] Items 9-12 in the preceding chapters provide additional aspects and variations related to Examples 29-39.

[0644] 40. A device in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of Examples 1 to 39.

[0645] 41. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for carrying out the method of any one of Examples 1 to 39.

[0646] In the examples listed in this document, the term "conversion" can refer to generating a bitstream representation for the current video block or generating the current video block from a bitstream representation. The bitstream representation does not need to represent consecutive groups of bits; instead, it can be divided into bits included in header fields or codewords that represent encoding / decoding pixel value information.

[0647] It will be understood that the disclosed techniques can be implemented in video encoders or decoders to improve compression efficiency using techniques including the use of various implementation rules regarding the use of history-based motion vector prediction (HMVP) with default parameters, as described in this document.

[0648] Some embodiments of the technology disclosed herein include determining or enabling a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode during the processing of video blocks, but it is not necessary to modify the resulting bitstream based on the use of the tool or mode. In other words, when a video processing tool or mode is determined or enabled, the conversion from video blocks to a video bitstream will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream based on the video processing tool or mode, knowing that the bitstream has been modified. In other words, the conversion from a bitstream representation of video to video blocks will be performed using a video processing tool or mode determined or enabled.

[0649] Some embodiments of the technology disclosed herein include deciding or determining to disable video processing tools or modes. In one example, when video processing tools or modes are disabled, the encoder will not use the tools or modes in the conversion of video blocks to a bitstream representation of video. In another example, when video processing tools or modes are disabled, the decoder will process the bitstream knowing that no modifications have been made to the bitstream based on the decision or determination to disable video processing tools or modes.

[0650] Figure 26 This is a block diagram illustrating an example video encoding / decoding system 100 that can utilize the techniques disclosed herein. Figure 26 As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data; this source device may be referred to as a video encoding device. The destination device 120 can decode the encoded video data generated by the source device 110; this destination device may be referred to as a video decoding device. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0651] Video source 112 may include sources such as video capture devices, interfaces for receiving video data from video content providers, and / or computer graphics systems that generate video data, or combinations of these sources. Video data may include one or more pictures. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming a codec representation of the video data. The bitstream may include codec pictures and associated data. A codec picture is a codec representation of a picture. Associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. Encoded video data may be transmitted directly to destination device 120 via network 130a through I / O interface 116. Encoded video data may also be stored on storage medium / server 130b for access by destination device 120.

[0652] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[0653] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be externally mounted on destination device 120 configured to connect to an external display device.

[0654] The video encoder 114 and video decoder 124 can operate according to the following video compression standards, such as the High Efficiency Video Codec (HEVC) standard, the Universal Video Codec (VVM) standard, and other current and / or other standards.

[0655] Figure 27 This is a block diagram illustrating an example of a video encoder 200, which can be... Figure 26 The system 100 shown in the diagram contains video encoder 114.

[0656] The video encoder 200 can be configured to perform any or all of the techniques disclosed herein. Figure 27 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques of this disclosure.

[0657] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra-frame prediction unit 206), a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0658] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0659] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, can be highly integrated, but for interpretive purposes... Figure 27 The examples are shown separately.

[0660] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0661] The mode selection unit 203 can, for example, select one of the intra-frame or inter-frame encoding / decoding modes based on the error result, and provide the resulting intra-frame or inter-frame encoded / decoded block to the residual generation unit 207 to generate residual block data and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 can select a combination of intra-frame and inter-frame prediction (CIIP) modes, where prediction is based on inter-frame prediction signals and intra-frame prediction signals. The mode selection unit 203 can also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision) in the case of inter-frame prediction.

[0662] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on the motion information of the image from buffer 213 (rather than the image associated with the current video block) and decoded samples.

[0663] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations for the current video block. For example, the different operations performed depend on whether the current video block is in an I-strip, a P-strip, or a B-strip.

[0664] In some examples, motion estimation unit 204 can perform unidirectional prediction of the current video block, and can search for a reference video block for the current video block in the reference images of list 0 or list 1. Motion estimation unit 204 can then generate a reference index indicating the reference image containing the reference video block in list 0 or list 1, and generate a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0665] In other examples, motion estimation unit 204 can perform bidirectional prediction of the current video block. Motion estimation unit 204 can search for a reference video block for the current video block in the reference images of list 0 and also search for another reference video block for the current video block in the reference images of list 1. Motion estimation unit 204 can then generate a reference index indicating the reference image containing the reference video block in list 0 or list 1, and generate a motion vector indicating the spatial displacement between the reference video block and the current video block. Motion estimation unit 204 can output the reference index and the motion vector of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0666] In some examples, the motion estimation unit 204 can output the complete set of motion information for use in the decoder's decoding process.

[0667] In some examples, motion estimation unit 204 may not output the complete set of motion information for the current video. Instead, motion estimation unit 204 may signal the motion information of the current video block by referencing the motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0668] In one example, the motion estimation unit 204 may indicate in the syntax structure associated with the current video block that the current video block has the same motion information value as another video block.

[0669] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicating video block. Video decoder 300 can use the motion vector of the indicating video block and the motion vector difference to determine the motion vector of the current video block.

[0670] As discussed above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling notification techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling Notification.

[0671] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[0672] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) one or more predicted video blocks from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of the samples in the current video block.

[0673] In other examples, such as in skip mode, residual data for the current video block may not exist, and the residual generation unit 207 may not perform the subtraction operation.

[0674] The transform processing unit 208 can generate one or more transform coefficient video blocks of the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0675] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0676] The inverse quantization unit 210 and the inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block respectively to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples from one or more predicted video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block, which is then stored in the buffer 213.

[0677] After the video block is reconstructed by the reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts.

[0678] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream including the entropy encoded data.

[0679] Figure 28 This is a block diagram illustrating an example of a video decoder 300, which can be... Figure 26 The video decoder 114 in the system 100 shown in the figure.

[0680] The video decoder 300 can be configured to perform any or all of the techniques disclosed herein. Figure 28In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques of this disclosure.

[0681] exist Figure 28 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform operations related to the video encoder 200 ( Figure 27 The decoding process is the overall inversion of the encoding process described.

[0682] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy-encoded video data, and based on the entropy-decoded video, the motion compensation unit 302 can determine motion information including motion vectors, motion vector precision, reference image list index, and other motion information. The motion compensation unit 302 can determine such information, for example, by executing AMVP and Merge modes.

[0683] The motion compensation unit 302 can generate motion compensation blocks, possibly performing interpolation based on an interpolation filter. The identifier of the interpolation filter to be used at sub-pixel precision can be included in the syntax element.

[0684] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during the encoding of the video block to calculate the interpolation values ​​of a sub-integer number of pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information and use the interpolation filter to generate the prediction block.

[0685] The motion compensation unit 302 can use some syntax information to determine: the size of the blocks used to encode (one or more) frames and / or (one or more) stripes of the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a mode indicating how each segment is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence.

[0686] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 303 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[0687] The reconstruction unit 306 can sum the residual block and the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to form a decoded block. If necessary, a deblocking filter can also be applied to the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and generates the encoded / decoded video for presentation on a display device.

[0688] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from the pixel representation of a video to the corresponding bitstream representation, and vice versa. The bitstream representation or codec representation of the current video block can, for example, correspond to juxtaposed or extended bits at different locations within the bitstream. For example, the video block can be encoded based on the transformed and encoded / decoded error residuals and using bits in the header and other fields in the bitstream. Furthermore, as described in the above scheme, during the conversion, the decoder can parse the bitstream based on determining whether some fields may or may not be present. Similarly, the encoder can determine whether certain syntax fields are included or excluded and generate the codec representation accordingly by including or excluding syntax fields from the codec representation.

[0689] Figure 29 This is a block diagram illustrating an example video processing system 2900 that can implement the various techniques disclosed herein. Various implementations may include some or all of the components in system 2900. System 2900 may include an input 2902 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or in a compressed or encoded format. Input 2902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, Passive Optical Networking (PON), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0690] System 2900 may include an encoding / decoding component 2904 capable of implementing the various encoding / decoding or coding methods described in this document. Encoding / decoding component 2904 can reduce the average bit rate of the video from input 2902 to the output of encoding / decoding component 2904 to produce an encoded / decoded representation of the video. Therefore, encoding / decoding techniques are sometimes referred to as video compression or video transcoding techniques. The output of encoding / decoding component 2904 can be stored or transmitted via connected communication, as represented by component 2906. The stored or communicated bitstream (or encoded / decoded) representation of the video received at input 2902 can be used by component 2908 to generate pixel values ​​or displayable video that is sent to display interface 2910. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “encoding / decoding” operations or tools, it should be understood that encoding / decoding tools or operations are used at the encoder, and the corresponding decoding tools or operations that will be performed by the decoder to reverse the encoding / decoding results are also used.

[0691] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described in this document can be implemented in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0692] Figure 30 This is a flowchart of an example method 3000 for video processing. Method 3000 includes: 3002, maintaining a history-based motion vector prediction (HMVP) table, which includes one or more HMVP motion candidates derived from one or more previous video blocks of the video, wherein at least one HMVP motion candidate from the HMVP table includes interpolation filter information and / or side information of a switchable interpolation filter, the interpolation filter information including parameters indicating the interpolation filter; and 3004, performing a conversion between the current video block of the video and the bitstream representation of the video based on the HMVP table.

[0693] In some embodiments of method 3000, the edge information indicates the use of a switchable interpolation filter at a specific resolution. In some embodiments of method 3000, the specific resolution is half a pixel resolution. In some embodiments of method 3000, at least one HMVP motion candidate further includes a motion vector, a reference image index, and / or a generalized bidirectional prediction information (GBI) index. In some embodiments of method 3000, maintenance includes: for a second HMVP motion candidate, determining that motion information excluding at least the interpolation filter information is the same as the motion information of a first HMVP motion candidate in the HMVP table; and based on this determination, removing the first HMVP motion candidate from the HMVP table and adding the second HMVP motion candidate to the HMVP table. In some embodiments of method 3000, performing a transformation based on the HMVP table includes: adding an HMVP motion candidate from at least one HMVP motion candidate to a list as a new motion candidate, wherein the interpolation filter information of the at least one HMVP motion candidate is inherited by the new motion candidate; and performing a transformation based on the list.

[0694] In some embodiments of method 3000, performing a transformation based on a list includes: selecting a motion candidate from the list; and performing a transformation using the selected motion candidate, wherein interpolation filter information of the selected motion candidate is used during the transformation. In some embodiments of method 3000, the list is an Advanced Motion Vector Prediction (AMVP) list and the new candidate is an AMVP candidate, or the list is a Merge list and the new candidate is a Merge candidate. In some embodiments, method 3000 further includes updating an HMVP table using motion information, which includes interpolation filter information derived for the current video block. In some embodiments, method 3000 further includes determining that at least one HMVP motion candidate is different from the second HMVP motion candidate in response to the interpolation filter information of at least one HMVP motion candidate being different from the interpolation filter information of a second HMVP motion candidate.

[0695] In some embodiments of method 3000, the HMVP table stores interpolation filter information by first determining whether interpolation filter information is included in the HMVP table based on the resolution of at least one HMVP motion candidate. In some embodiments of method 3000, the interpolation filter information is stored in the HMVP table in response to the resolution of at least one HMVP motion candidate being half a pixel. In some embodiments of method 3000, the interpolation filter information is stored in the HMVP table for each prediction direction associated with at least one HMVP motion candidate. In some embodiments of method 3000, adding includes: comparing first motion information including first interpolation filter information of the HMVP motion candidate to be added with second motion information including second interpolation filter information of at least one motion candidate already added to the list; and adding the HMVP motion candidate to be added to the list when it is determined that the first motion information of the HMVP motion candidate to be added is different from the second motion information of at least one motion candidate.

[0696] In some embodiments, the method 3000 further includes performing a first determination, namely, that the second HMVP motion candidate is the same as at least one HMVP motion candidate; wherein the first determination is performed by determining that the second interpolation filter or the second switchable interpolation filter is the same as the interpolation filter or the switchable interpolation filter of at least one HMVP motion candidate; and based on the first determination, performing a second determination, namely, not adding the second HMVP motion candidate to the HMVP table. In some embodiments, the method 3000 further includes performing a first determination, namely, that the second HMVP motion candidate includes a second interpolation filter or the second switchable interpolation filter that is different from the interpolation filter or the switchable interpolation filter of at least one HMVP motion candidate; and based on the first determination, performing a second determination, namely, adding the second HMVP motion candidate to the HMVP table.

[0697] In some embodiments, method 3000 further includes adding a plurality of motion candidates obtained based on at least one HMVP motion candidate to a list, wherein each motion candidate from the plurality of motion candidates is associated with an interpolation filter, the interpolation filter being different from the interpolation filter of another motion candidate from the plurality of motion candidates. In some embodiments of method 3000, the list is an Advanced Motion Vector Prediction (AMVP) list or a Merge list. In some embodiments of method 3000, performing the transformation includes an encoding operation in which a bitstream representation is generated from the current video block. In some embodiments of method 3000, performing the transformation includes a decoding operation in which the current video block is generated from the bitstream representation.

[0698] Based on the foregoing, it will be understood that specific embodiments of the technology disclosed herein have been described for illustrative purposes, but various modifications may be made without departing from the scope of the invention. Therefore, the technology currently disclosed is not limited except for the appended claims.

[0699] Other schemes, examples, embodiments, modules, and functional operations disclosed in this document may be implemented in digital electronic circuits or in computer software, firmware, or hardware, including the structures disclosed in this document and their equivalents, or combinations thereof. The disclosed and other embodiments may be implemented as one or more computer program products, such as one or more computer program instruction modules encoded on a computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a complex influencing machine-readable propagating signals, or a combination thereof. The term "data processing apparatus" encompasses all devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagating signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[0700] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file storing one or more modules, subroutines, or code sections). A computer program can be deployed to execute on one computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communication network.

[0701] The processes and logic flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by special-purpose logic circuitry (e.g., FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits)), and the device can be implemented as special-purpose logic circuitry (e.g., FPGAs or ASICs).

[0702] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magneto-optical, magneto-optical, or optical disc) for storing data, or operatively coupled to receive data from or transfer data to a mass storage device (e.g., magneto-optical, magneto-optical, or optical disc), or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by dedicated logic circuitry or integrated into dedicated logic circuitry.

[0703] While this patent document contains numerous details, these details should not be construed as limiting any subject matter or the scope of the claims, but rather as descriptions of features specific to particular embodiments of a particular technology. In this patent document, certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in various suitable sub-combinations. Furthermore, although features may be described above as operating in certain combinations and even initially claimed in this way, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may be for sub-combinations or variations thereof.

[0704] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order or sequence shown, or to perform all of the shown operations to achieve the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0705] Only a few implementations and examples are described, and other implementations, enhancements and variations can be made based on what is described and shown in this patent document.

Claims

1. A method of video processing, comprising: maintaining a history-based motion vector prediction (HMVP) table that includes one or more HMVP motion candidates derived from one or more previous video blocks of a video, wherein at least one HMVP motion candidate from the HMVP table includes interpolation filter information and / or side information of a switchable interpolation filter, the interpolation filter information including parameters indicative of an interpolation filter; and performing a conversion between a current video block of the video and a bitstream of the video based on the HMVP table, wherein the side information indicates to use the switchable interpolation filter at a particular resolution.

2. The method of claim 1, wherein, the particular resolution is a half-pel resolution.

3. The method of claim 1, wherein, the at least one HMVP motion candidate further includes a motion vector, a reference picture index, and / or a generalized bi-prediction information (GBI) index.

4. The method of claim 1, wherein, the maintaining includes: for a second HMVP motion candidate, determining that motion information excluding at least the interpolation filter information is identical to motion information of a first HMVP motion candidate in the HMVP table; and based on the determining, removing the first HMVP motion candidate from the HMVP table and adding the second HMVP motion candidate to the HMVP table.

5. The method of claim 1, wherein, the performing the conversion based on the HMVP table includes: adding a HMVP motion candidate from the at least one HMVP motion candidate to a list as a new motion candidate, wherein interpolation filter information of the at least one HMVP motion candidate is inherited by the new motion candidate; and performing the conversion based on the list.

6. The method of claim 5, wherein, the performing the conversion based on the list includes: selecting a motion candidate from the list; and performing the conversion using the selected motion candidate, wherein during the conversion, interpolation filter information of the selected motion candidate is used.

7. The method of claim 5, wherein the list is an advanced motion vector prediction (AMVP) list and the new motion candidate is an AMVP candidate, or wherein the list is a Merge list and the new motion candidate is a Merge candidate.

8. The method of claim 1, further comprising: updating the HMVP table using motion information including interpolation filter information derived for the current video block.

9. The method of claim 1, further comprising: in response to interpolation filter information of the at least one HMVP motion candidate being different from interpolation filter information of a second HMVP motion candidate, determining that the at least one HMVP motion candidate is different from the second HMVP motion candidate.

10. The method of claim 1, wherein, the HMVP table stores the interpolation filter information by first determining whether to include the interpolation filter information in the HMVP table based on a resolution of the at least one HMVP motion candidate.

11. The method of claim 10, wherein, in response to the resolution of the at least one HMVP motion candidate being a half-pel resolution, storing the interpolation filter information in the HMVP table.

12. The method of claim 1, wherein, For each prediction direction associated with the at least one HMVP motion candidate, the interpolation filter information is stored in the HMVP table.

13. The method of claim 5, wherein, The adding includes: comparing first motion information including first interpolation filter information of a HMVP motion candidate to be added with second motion information including second interpolation filter information of at least one motion candidate already added to the list; and based on determining that the first motion information of the HMVP motion candidate to be added is different from the second motion information of the at least one motion candidate, adding the HMVP motion candidate to be added to the list.

14. The method of claim 1, further comprising: performing a first determination that a second HMVP motion candidate is identical to the at least one HMVP motion candidate; wherein the first determination is performed by determining that a second interpolation filter or a second switchable interpolation filter is identical to an interpolation filter or a switchable interpolation filter of the at least one HMVP motion candidate; and based on the first determination, performing a second determination that the second HMVP motion candidate is not added to the HMVP table.

15. The method of claim 1, further comprising: performing a first determination that a second HMVP motion candidate includes a second interpolation filter or a second switchable interpolation filter that is different from an interpolation filter or a switchable interpolation filter of the at least one HMVP motion candidate; and based on the first determination, performing a second determination that the second HMVP motion candidate is added to the HMVP table.

16. The method of claim 1, further comprising: adding a plurality of motion candidates obtained based on the at least one HMVP motion candidate to a list, wherein each motion candidate from the plurality of motion candidates is associated with an interpolation filter that is different from an interpolation filter of another motion candidate from the plurality of motion candidates.

17. The method of claim 16, wherein, The list is an advanced motion vector prediction (AMVP) list or a Merge list.

18. The method of any one of claims 1 to 17, wherein, The performing the conversion includes an encoding operation that generates the bitstream from the current video block.

19. The method of any one of claims 1 to 17, wherein, The performing the conversion includes a decoding operation that generates the current video block from the bitstream.

20. A device in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein, The instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1-19.

21. A non-transitory computer-readable medium storing a computer program product, the computer program product comprising program code for implementing the method of any one of claims 1-19.

22. A method of storing a bitstream of a video, comprising: performing the video processing method of claim 1 to generate a bitstream; and storing the bitstream.

Images