Which lookup tables need to be updated or not

Abstract

A video decoding method for determining which lookup table needs to be updated or not updated, comprising: maintaining a plurality of tables, wherein each table includes a set of motion candidates, and each motion candidate is associated with corresponding motion information derived from a previously encoded video block; performing conversion between a current video block in a video region and a bitstream representation of the current video block; and updating one or more tables based on an update rule.

Description

[0001] This application is a divisional application of the invention patent application with application date of July 1, 2019, application number 201910586798.5, and invention name “Which lookup table needs to be updated or not updated”. Technical Field

[0002] This application document relates to video coding technology, equipment and systems. Background Art

[0003] Despite advances in video compression, digital video still uses the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video are expected to continue to grow. Summary of the Invention

[0004] This document discloses methods, systems, and apparatus for encoding and decoding digital video using a Merge list of motion vectors.

[0005] In one example aspect, a video decoding method is disclosed that includes maintaining several tables, wherein each table includes a set of motion candidates and each motion candidate is associated with corresponding motion information derived from a previously encoded video block; performing conversion between a current video block in a video region and a bitstream representation of the current video block; and updating one or more tables based on an update rule.

[0006] In another example aspect, another video decoding method is disclosed. The method includes: checking a set of tables, each table including one or more motion candidates, each motion candidate associated with motion information of the motion candidate; processing motion information of a video block based on the one or more tables; and updating the one or more tables based on the video block generated from the processing.

[0007] In yet another example aspect, another video decoding method is disclosed. The method includes: checking a set of tables, each table including one or more motion candidates, each motion candidate associated with motion information for the motion candidate; selecting one or more tables based on a position of a video block in a picture; processing the motion information of the video block based on the selected one or more tables; and updating the selected tables based on the video block generated from the processing.

[0008] In yet another example aspect, another video decoding method is disclosed. The method includes: examining a set of tables, each table including one or more motion candidates, each motion candidate associated with motion information for the motion candidate; selecting one or more tables based on a distance between a video block and a motion candidate in the one or more tables; processing the motion information of the video block based on the selected one or more tables; and updating the selected tables based on a video block generated from the processing.

[0009] In yet another example aspect, a video encoding device implementing the video encoding method described herein is disclosed.

[0010] In another exemplary aspect, the various techniques described herein can be implemented as a computer program product stored on a non-transitory computer-readable medium. The computer program product includes program code for executing the methods described herein.

[0011] In yet another exemplary aspect, a video decoding device implementing the video decoding method described herein is disclosed.

[0012] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram illustrating an example of a video encoder implementation.

[0014] Figure 2 The diagram illustrates macroblock partitioning in the H.264 video coding standard.

[0015] Figure 3 An example of dividing a coding block (CB) into prediction blocks (PB) is illustrated.

[0016] Figure 4 Illustrated is an example implementation of subdividing a coding tree block (CTB) into CBs and transform blocks (TBs). Solid lines represent CB boundaries and dashed lines represent TB boundaries, including an example CTB with partitioning and the corresponding quadtree.

[0017] Figure 5 An example of a quad-tree binary tree (QTBT) structure for partitioning video data is shown.

[0018] Figure 6 An example of video block segmentation is shown.

[0019] Figure 7 An example of quadtree partitioning is shown.

[0020] Figure 8 An example of tree signaling is shown.

[0021] Figure 9 An example of the derivation process of Merge candidate list construction is shown.

[0022] Figure 10 Example locations of spatial merge candidates are shown.

[0023] Figure 11 An example of candidate pairs considering redundancy checking of spatial Merge candidates is shown.

[0024] Figure 12 Examples of the location of the second PU for Nx2N and 2NxN partitions are shown.

[0025] Figure 13 Illustrated is an example motion vector scaling of a temporal merge candidate.

[0026] Figure 14 The candidate positions of the time-domain Merge candidates and their collocated pictures are shown.

[0027] Figure 15 An example of combining bi-predictive Merge candidates is shown.

[0028] Figure 16 An example of a process of deriving motion vector prediction candidates is shown.

[0029] Figure 17 An example of motion vector scaling of spatial motion vector candidates is shown.

[0030] Figure 18 An example alternative temporal motion vector prediction (ATMVP) for motion prediction of a coding unit (CU) is shown.

[0031] Figure 19 An example of identification of source blocks and source pictures is diagrammatically depicted.

[0032] Figure 20 An example of one CU having four subblocks and neighboring blocks is shown.

[0033] Figure 21 An example of bilateral matching is illustrated.

[0034] Figure 22 An example of template matching is illustrated.

[0035] Figure 23 An example of unilateral motion estimation (ME) in frame rate up-conversion (FRUC) is depicted.

[0036] Figure 24 An example of decoder-side motion vector refinement (DMVR) based on bilateral template matching is shown.

[0037] Figure 25 An example of spatial neighboring blocks used to derive spatial Merge candidates is shown.

[0038] Figure 26 Depicts an example of how a selection of typical locations for a lookup table is updated.

[0039] Figure 27 An example of updating a lookup table with a new set of motion information is illustrated.

[0040] Figure 28 is a block diagram of an example hardware platform for implementing the visual media decoding or visual media encoding techniques described herein.

[0041] Figure 29 is a flow chart of another example method of video bitstream processing.

[0042] Figure 30 is a flow chart of another example method of video bitstream processing. DETAILED DESCRIPTION

[0043] In order to improve the compression ratio of videos, researchers are constantly looking for new technologies to encode videos.

[0044] 1. Introduction

[0045] This document relates to video coding techniques. Specifically, it relates to motion information coding (such as Merge mode and AMVP mode) in video coding. It is applicable to existing video coding standards or to standard multifunctional video coding (SMVCC) to be finalized. It may also be applicable to future video coding standards or video codecs.

[0046] Brief Discussion

[0047] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Vision, and the two organizations jointly developed the H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding architecture that employs temporal prediction plus transform coding. An example of a typical HEVC encoder framework is shown in Figure 1. Figure 1 shown.

[0048] 2.1 Segmentation Structure

[0049] 2.1.1 Segmentation Tree Structure in H.264 / AVC

[0050] The core of the coding layer in previous standards is the macroblock, which contains a 16×16 block of luminance samples and, in the case of conventional 4:2:0 color sampling, two corresponding 8×8 blocks of chrominance samples.

[0051] Intra-coded blocks use spatial prediction to explore the spatial correlation between pixels. Two splits are defined: 16x16 and 4x4.

[0052] Inter-coded blocks use temporal prediction instead of spatial prediction by estimating the motion between pictures. Motion can be estimated for 16x16 macroblocks alone or for any of their sub-macroblock partitions: 16x8, 8x16, 8x8, 8x4, 4x8, 4x4 (see Figure 2 ). Only one motion vector (MV) is allowed per sub-macroblock partition.

[0053] 2.1.2 Segmentation Tree Structure in HEVC

[0054] In HEVC, various local characteristics are adapted by dividing the CTU into CUs using a quadtree structure (expressed as a coding tree). Whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to encode the picture area is decided at the CU level. Depending on the partition type of the PU, each CU can be further divided into one, two or four PUs. In a PU, the same prediction process is applied, and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU partition type, the CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree of the CU. An important feature of the HEVC structure is that it has multiple partitioning concepts, including CU, PU and TU.

[0055] Below, various features involved in hybrid video coding using HEVC are highlighted as follows.

[0056] 1) Coding Tree Unit and Coding Tree Block (CTB) Structure: A similar structure in HEVC is the Coding Tree Unit (CTU), which has a size selected by the encoder and can be larger than a traditional macroblock. A CTU consists of a luma CTB and corresponding chroma CTBs, along with syntax elements. The size L×L of the luma CTB can be selected to be L=16, 32, or 64 samples, with larger sizes generally achieving better compression. HEVC then supports splitting the CTB into smaller blocks using a tree structure and quadtree signaling.

[0057] 2) Coding Unit (CU) and Coding Block (CB): The quadtree syntax of a CTU specifies the size and position of its luma and chroma CBs. The root of the quadtree is associated with the CTU. Therefore, the size of the luma CTB is the maximum size supported by the luma CB. The partitioning of the luma and chroma CBs of a CTU is signaled jointly. One luma CB and usually two chroma CBs together with the associated syntax form a coding unit (CU). A CTB can contain only one CU or be partitioned to form multiple CUs, and each CU has an associated partitioning into prediction units (PUs) and transform unit trees (TUs).

[0058] 3) Prediction Unit and Prediction Block (PB): The decision on whether to use inter-prediction or intra-prediction for encoding a picture region is made at the CU level. The root of the PU partitioning structure is at the CU level. Depending on the basic prediction type decision, the luma and chroma CBs can be further partitioned in size and predicted from luma and chroma prediction blocks (PBs). HEVC supports variable PB sizes from 64×64 to 4×4 samples. Figure 3 An example of allowed PBs for an MxM CU is shown.

[0059] 4) TU and transform block: The prediction residual is encoded using block transform. The root of the TU tree structure is at the CU level. The luma CB residual may be the same as the luma transform block (TB), or it may be further divided into smaller luma TBs. The same applies to chroma TBs. Integer basis functions similar to discrete cosine transform (DCT) are defined for square TBs of 4×4, 8×8, 16×16, and 32×32. For the 4×4 transform of the luma intra prediction residual, an integer transform derived from the discrete sine transform (DST) form can also be specified.

[0060] Figure 4 An example of subdividing a CTB into CBs (and transform blocks (TBs)) is shown. Solid lines indicate CB boundaries, and dashed lines indicate TB boundaries. (a) CTB with partitioning. (b) Corresponding quadtree.

[0061] 2.1.2.1 Tree Structure Division into Transform Blocks and Units

[0062] For residual coding, CBs can be recursively partitioned into transform blocks (TBs). The partitioning is signaled by the residual quadtree. Only square CB and TB partitioning is specified, where a block can be recursively partitioned into four quadrants, such as Figure 4 As shown. For a given luma CB of size M×M, a flag indicates whether it is partitioned into four blocks of size M / 2×M / 2. If further partitioning is possible, as indicated by the maximum depth of the residual quadtree indicated in the SPS, each quadrant is assigned a flag indicating whether it is partitioned into four quadrants. The leaf node blocks generated by the residual quadtree are transform blocks that are further processed by transform coding. The encoder indicates the maximum and minimum luma TB sizes that it will use. When the CB size is larger than the maximum TB size, a partition is implied. When a partition would result in a luma TB size smaller than the indicated minimum, no partition is implied. The chroma TB size is half the luma TB size in each dimension, except when the luma TB size is 4×4, in which case the area covered by four 4×4 luma TBs uses a single 4×4 chroma TB. In the case of an intra-predicted CU, the decoded samples of the nearest neighboring TB (inside or outside the CB) are used as reference data for intra prediction.

[0063] Different from previous standards, for inter-predicted CUs, the HEVC design allows the TB to span multiple PBs to maximize the potential coding efficiency benefiting from the TB partitioning of the quadtree structure.

[0064] 2.1.2.2 Parent Node and Child Node

[0065] The CTB is divided according to a quadtree structure, wherein the nodes are coding units. The multiple nodes in the quadtree structure include leaf nodes and non-leaf nodes. A leaf node has no child nodes in the tree structure (e.g., a leaf node is not further divided). The non-leaf nodes include the root node of the tree structure. The root node corresponds to an initial video block of video data (e.g., a CTB). For each respective non-root node of the multiple nodes, the respective non-root node corresponds to a video block that is a child block of the video block corresponding to the parent node in the tree structure of the respective non-root node. Each respective non-leaf node of the multiple non-leaf nodes has one or more child nodes in the tree structure.

[0066] 2.1.3 Quadtree plus Binarytree Block Structure with Large CTUs in JEM

[0067] To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and applied them to reference software called the Joint Exploration Model (JEM).

[0068] 2.1.3.1 QTBT block segmentation structure

[0069] Unlike HEVC, the QTBT structure eliminates the concept of multiple partition types, that is, the QTBT structure eliminates the separation of CU, PU and TU concepts and supports more flexibility in CU partition shape. In the QTBT block structure, CU can be square or rectangular. Figure 5 As shown, the coding tree unit (CTU) is first partitioned using a quadtree structure. The quadtree leaf nodes are further partitioned by a binary tree structure. There are two types of partitioning in the binary tree partitioning: symmetrical horizontal partitioning and symmetrical vertical partitioning. The binary tree leaf nodes are called coding units (CUs), which are used for prediction and conversion processing without further partitioning. This means that in the QTBT coding block structure, CU, PU and TU have the same block size. In JEM, a CU sometimes consists of coding blocks (CBs) of different color components, for example, in the P slice and B slice of the 4:2:0 chroma format, a CU contains one luminance CB and two chroma CBs, and a CU sometimes consists of CBs of a single component, for example, in the case of an I slice, a CU contains only one luminance CB or only two chroma CBs.

[0070] The following parameters are defined for the QTBT segmentation scheme.

[0071] –CTU size: The root node size of the quadtree, which is the same as the concept in HEVC.

[0072] –MiNQTSize: Minimum allowed quadtree leaf node size

[0073] –MaxBTSize: Maximum allowed binary tree root node size

[0074] –MaxBTDePTh: Maximum allowed binary tree depth

[0075] –MiNBTSize: Minimum allowed binary tree leaf node size

[0076] In one example of a QTBT partitioning structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 chroma sample blocks, MiNQTSize is set to 16×16, MaxBTSize is set to 64×64, MiNBTSize (width and height) is set to 4×4, and MaxBTSize is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. The size of the quadtree leaf node can have a size from 16×16 (i.e., MiNQTSize) to 128×128 (i.e., CTU size). If the leaf quadtree node is 128×128, it will not be further partitioned by the binary tree because its size exceeds MaxBTSize (e.g., 64×64). Otherwise, the leaf quadtree node can be further partitioned by the binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, and its binary tree depth is 0. When the binary tree depth reaches MaxBTDePTh (i.e., 4), further partitioning is not considered. When the width of a binary tree node is equal to MiNBTSize (i.e., 4), no further horizontal splitting is considered. Similarly, when the height of a binary tree node is equal to MiNBTSize, no further vertical splitting is considered. Leaf nodes of the binary tree are further processed by prediction and transform processing without further splitting. In JEM, the maximum CTU size is 256×256 luma samples.

[0077] Figure 5 (Left) illustrates an example of block segmentation using QTBT. Figure 5(Right) The corresponding tree representation is shown. Solid lines represent quadtree partitioning, and dashed lines represent binary tree partitioning. In each partitioned (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which partitioning type (i.e., horizontal or vertical) is used, where 0 indicates horizontal partitioning and 1 indicates vertical partitioning. For quadtree partitioning, there is no need to specify the partitioning type, as quadtree partitioning always partitions a block horizontally and vertically to generate four sub-blocks of the same size.

[0078] In addition, the QTBT scheme supports the ability to have separate QTBT structures for luma and chroma. Currently, for P and B slices, the luma and chroma CTBs in a CTU share the same QTBT structure. However, for I slices, the luma CTBs are split into CUs using the QTBT structure, and the chroma CTBs are split into chroma CUs using another QTBT structure. This means that a CU in an I slice consists of coding blocks for the luma component or coding blocks for two chroma components, and a CU in a P or B slice consists of coding blocks for all three color components.

[0079] In HEVC, to reduce memory access for motion compensation, inter-frame prediction for small blocks is restricted, so that 4×8 and 8×4 blocks do not support bidirectional prediction, and 4×4 blocks do not support inter-frame prediction. In JEM's QTBT, these restrictions are removed.

[0080] 2.1.4 Versatile Video Coding (VVC) Ternary Tree

[0081] In some embodiments, tree types other than quadtree and binary tree are supported. In the implementation, two other ternary tree (TT) partitions are introduced, namely horizontal and vertical center-side ternary trees, such as Figure 6 (d) and Figure 6 (e) shown.

[0082] Figure 6 Shown are: (a) quadtree partition (b) vertical binary tree partition (c) horizontal binary tree partition (d) vertical center-side ternary tree partition (e) horizontal center-side ternary tree partition.

[0083] In some implementations, there are two levels of trees: a region tree (quadtree) and a prediction tree (binary or ternary tree). The CTU is first partitioned using the region tree (RT). The RT leaves can be further partitioned using the prediction tree (PT). The PT leaves can also be further partitioned using the PT until the maximum PT depth is reached. The PT leaf is the basic coding unit. For convenience, it is still referred to as a CU. The CU cannot be further partitioned. Prediction and transform are applied to the CU in the same way as JEM. The entire partitioning structure is called a "multi-type tree."

[0084] 2.1.5 Split Structure

[0085] The tree structure used in this response, called Multi-Tree (MTT), is a generalization of QTBT. In QTBT, Figure 5 As shown in FIG, the coding tree unit (CTU) is first segmented using a quadtree structure, and then the quadtree leaf nodes are further segmented using a binary tree structure.

[0086] The basic structure of MTT consists of two types of tree nodes: region tree (RT) and prediction tree (PT), which supports nine types of partitioning, such as Figure 7 shown.

[0087] Figure 7 The following examples are illustrated: (a) quadtree split, (b) vertical binary tree split, (c) horizontal binary tree split, (d) vertical ternary tree split, (e) horizontal ternary tree split, (f) horizontal upward asymmetric binary tree split, (g) horizontal downward asymmetric binary tree split, (h) vertical left asymmetric binary tree split, and (i) vertical right asymmetric binary tree split.

[0088] The region tree can recursively divide the CTU into square blocks until the region tree leaf node of size 4x4. At each node of the region tree, a prediction tree can be formed from one of three tree types: binary tree (BT), ternary tree (TT) and / or asymmetric binary tree (ABT). In PT partitioning, quadtree partitioning in the branches of the prediction tree may be prohibited. As with JEM, the luminance tree and chrominance tree are separated in the I slice. The signaling method of RT and PT is as follows Figure 8 shown.

[0089] 2.2 Inter-frame prediction in HEVC / H.265

[0090] Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include motion vectors and reference picture indices. The use of one of the two reference picture lists can also be signaled using inter_pred_idc. Motion vectors can be explicitly coded as deltas relative to the predicted value, a coding mode called Advanced Motion Vector Prediction (AMVP) mode.

[0091] When a CU is coded in skip mode, a PU is associated with the CU and has no significant residual coefficients, no coded motion vector increments or reference picture indices. A Merge mode is specified whereby the motion parameters of the current PU are obtained from neighboring PUs (including spatial and temporal candidates). Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to Merge mode is explicit transmission of motion parameters, where the motion vectors, the reference picture indices corresponding to each reference picture list, and the use of reference picture lists are explicitly signaled in each PU.

[0092] When signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of samples. This is called "unidirectional prediction." Unidirectional prediction is available for both P and B slices.

[0093] When signaling indicates that two reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bidirectional prediction." Bidirectional prediction is only available for B slices.

[0094] The following text provides details of the inter prediction modes specified in HEVC. The description will start with the Merge mode.

[0095] 2.2.1 Merge Mode

[0096] 2.2.1.1 Derivation of Merge Mode Candidates

[0097] When using Merge mode to predict a PU, the index pointing to the entry in the Merge candidate list is analyzed from the bitstream and used to retrieve motion information. The structure of this list is specified in the HEVC standard and can be summarized in the following order:

[0098] Step 1: Initial candidate derivation

[0099] Step 1.1: Spatial Candidate Derivation

[0100] Step 1.2: Redundancy check of spatial candidates

[0101] Step 1.3: Time Domain Candidate Derivation

[0102] Step 2: Additional candidate insertions

[0103] Step 2.1: Creation of bidirectional prediction candidates

[0104] Step 2.2: Insertion of zero motion candidates

[0105] exist Figure 9These steps are also schematically described in . For spatial Merge candidate derivation, at most four Merge candidates are selected from candidates located at five different positions. For time domain Merge candidate derivation, at most one Merge candidate is selected from two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share a Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

[0106] The operations related to the above steps are described in detail below.

[0107] 2.2.1.2 Spatial Candidate Derivation

[0108] In the derivation of spatial Merge candidates, Figure 10 A maximum of four Merge candidates are selected from the candidates at the positions shown. The derivation order is A1, B1, B0, A0 and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is unavailable (for example, because it belongs to another slice or piece) or is intra-coded. After adding the candidate at position A1, a redundancy check is performed on the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce the complexity of the calculation, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only the pairs with the same motion information are considered. Figure 11 Only pairs linked by arrows in are considered, and candidates are added to the list only if the corresponding candidates for redundancy checking do not have the same motion information. Another source of duplicate motion information is the "second PU" associated with a different 2N×2N partition. For example, Figure 12 The second PU is described for the N×2N and 2N×N cases, respectively. When the current PU is split into N×2N, the candidate at position A1 is not considered for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant for having only one PU in the coding unit. Similarly, position B1 is not considered when the current PU is split into 2N×N.

[0109] 2.2.1.3 Time Domain Candidate Derivation

[0110] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the collocated PU with the minimum picture order count (POC) difference from the current picture in a given reference picture list. The reference picture list used to derive the collocated PU is explicitly signaled in the slice header. Figure 13 The dotted line in the figure shows the derivation of the scaled motion vector of the temporal merge candidate, which is scaled from the motion vector of the collocated PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For B slices, two motion vectors are obtained (one for reference picture list 0 and the other for reference picture list 1) and combined to make a bidirectional prediction merge candidate. Illustration of motion vector scaling for temporal merge candidates.

[0111] In the collocated PU(Y) belonging to the reference frame, the position of the temporal candidate is selected between candidates C0 and C1, such as Figure 14 If the PU at position C0 is unavailable, intra-coded, or outside the current CTU, position C1 is used. Otherwise, position C0 is used for the derivation of the time-domain Merge candidate.

[0112] 2.2.1.4 Additional Candidate Insertion

[0113] In addition to spatiotemporal merge candidates, there are two additional types of merge candidates: combined bi-predictive merge candidates and zero merge candidates. Combined bi-predictive merge candidates are generated using spatiotemporal merge candidates. Combined bi-predictive merge candidates are used only for B slices. A combined bi-predictive candidate is generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they form a new bi-predictive candidate. Figure 15 Shown is a case where two candidates in the original list (on the left) are used to create a combined bi-predictive Merge candidate that is added to the final list (on the right), which has two candidates of MvL0 and refIdxL0 or MvL1 and refIdxL1. There are many rules about combining that need to be considered to generate these additional Merge candidates.

[0114] Zero-motion candidates are inserted to fill the remaining entries in the Merge candidate list, reaching the capacity of MaxNumMergeCand. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero-motion candidate is added to the list. The number of reference frames used by these candidates is 1 frame for unidirectional prediction and 2 frames for bidirectional prediction. Finally, no redundancy check is performed on these candidates.

[0115] 2.2.1.5 Parallel Motion Estimation Regions

[0116] To speed up the coding process, motion estimation can be performed in parallel, thereby deriving motion vectors for all prediction units in a given region at the same time. Deriving Merge candidates from spatial neighbors may interfere with parallel processing because one prediction unit cannot derive motion parameters from neighboring PUs before completing the relevant motion estimation. To ease the balance between coding efficiency and processing delay, HEVC defines a motion estimation region (MER). The size of the MER can be signaled in the picture parameter set using the syntax element "log2_parallel_merge_level_minus2" as described below. When the MER is defined, Merge candidates that fall into the same region are marked as unavailable and are therefore not considered in the list construction.

[0117] 7.3.2.3 Picture Parameter Set Raw Byte Sequence Payload (RBSP) Syntax

[0118] 7.3.2.3.1 General Picture Parameter Set RBSP Syntax

[0119] pic_paraMeter_set_rbsp(){ Descriptor pps_pic_paraMeter_set_id ue(v) pps_seq_paraMeter_set_id ue(v) depeNdeNt_slice_segMeNts_eNabled_flag u(1) … pps_scaliNg_list_data_preseNt_flag u(1) if(pps_scaliNg_list_data_preseNt_flag) scaliNg_list_data() lists_ModificatioN_preseNt_flag u(1) log2_parallel_Merge_level_MiNus2 ue(v) slice_segMeNt_header_exteNsioN_preseNt_flag u(1) pps_exteNsioN_preseNt_flag u(1) … rbsp_trailiNg_bits() }

[0120] log2_parallel_Merge_level_MiNus2 plus 2 specifies the value of the variable Log2ParMrgLevel, which is used in the derivation of the Merge mode luma motion vector as specified in clause 8.5.3.2.2 and the derivation of the spatial Merge candidates as specified in clause 8.5.3.2.3. The value of log2_parallel_Merge_level_MiNus2 shall be in the range of 0 to CtbLog2SizeY-2, inclusive.

[0121] The variable Log2ParMrgLevel is derived as follows:

[0122] Log2ParMrgLevel=log2_parallel_Merge_level_MiNus2+2

[0123] Note 3 – The value of Log2ParMrgLevel indicates the built-in function of parallel derivation of Merge candidate lists. For example, when Log2ParMrgLevel is equal to 6, the Merge candidate lists of all prediction units (PUs) and coding units (CUs) contained in a 64x64 block can be derived in parallel.

[0124] 2.2.2 Motion Vector Prediction in AMVP Mode

[0125] Motion vector prediction exploits the spatiotemporal correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. A motion vector candidate list is first constructed by checking the availability of the time-domain neighboring PU positions to the upper left, removing redundant candidate positions, and adding zero vectors to make the candidate list length constant. The encoder can then select the best prediction value from the candidate list and send the corresponding index indicating the selected candidate. Similar to Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary. The maximum value to be encoded in this case is 2 ( Figures 2 to 8 ). In the following sections, the derivation process of motion vector prediction candidates will be introduced in detail.

[0126] 2.2.2.1 Derivation of Motion Vector Prediction Candidates

[0127] Figure 16 The derivation process of motion vector prediction candidates is summarized.

[0128] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, based on the Figure 11 The motion vectors of each PU at the five different positions shown ultimately lead to two motion vector candidates.

[0129] For temporal motion vector candidate derivation, a motion vector candidate is selected from two candidates derived based on two different collocated positions. After the first list of spatiotemporal candidates is made, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates with reference picture indices greater than 1 in the associated reference picture list are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0130] 2.2.2.2 Spatial Motion Vector Candidates

[0131] When deriving spatial motion vector candidates, at most two candidates are considered among five potential candidates, which are from Figure 11The PUs at the depicted positions are the same as the positions of the motion merge. The derivation order on the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order above the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, there are four cases on each side that can be used as motion vector candidates, two of which do not require spatial scaling, and two use spatial scaling. The four different cases are summarized as follows:

[0132] No spatial scaling

[0133] (1) Same reference picture list and same reference picture index (same POC)

[0134] (2) Different reference picture lists, but same reference picture index (same POC)

[0135] --Space Zoom

[0136] (3) Same reference picture list, but different reference picture index (different POC)

[0137] (4) Different reference picture lists and different reference picture indexes (different POCs)

[0138] First, the case without spatial scaling is checked, followed by the case with spatial scaling. Whenever the POC differs between the reference pictures of the neighboring PU and the reference picture of the current PU, spatial scaling is considered, regardless of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, scaling of the above motion vectors is allowed to facilitate the parallel derivation of the left and above MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

[0139] In the spatial scaling process, the motion vectors of neighboring PUs are scaled in a similar way to temporal scaling, e.g. Figure 17 The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as the time domain scaling process.

[0140] 2.2.2.3 Temporal Motion Vector Candidates

[0141] Except for the derivation of the reference picture index, all derivation processes of the temporal Merge candidate are the same as those of the spatial motion vector candidate ( Figures 2 to 6 ). Signaling of reference picture index to the decoder.

[0142] 2.2.2.4 Signaling of AMVP Information

[0143] For the AMVP mode, four parts may be signaled in the bitstream, including prediction direction, reference index, MVD, and MV prediction candidate index.

[0144] Syntax table:

[0145]

[0146] 7.3.8.9 Motion Vector Difference Syntax

[0147] Mvd_codiNg(x0,y0,refList){ Descriptor abs_Mvd_greater0_flag[0] ae(v) abs_Mvd_greater0_flag[1] ae(v) if(abs_Mvd_greater0_flag[0]) abs_Mvd_greater1_flag[0] ae(v) if(abs_Mvd_greater0_flag[1]) abs_Mvd_greater1_flag[1] ae(v) if(abs_Mvd_greater0_flag[0]){ if(abs_Mvd_greater1_flag[0]) abs_Mvd_MiNus2[0] ae(v) Mvd_sigN_flag[0] ae(v) } if(abs_Mvd_greater0_flag[1]){ if(abs_Mvd_greater1_flag[1]) abs_Mvd_MiNus2[1] ae(v) Mvd_sigN_flag[1] ae(v) } }

[0148] 2.3 New Inter-frame Prediction Method in Joint Exploration Model (JEM)

[0149] 2.3.1 Motion Vector Prediction Based on Sub-CU

[0150] In JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. By splitting a large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU, two sub-CU-level motion vector prediction methods are considered in the encoder. The optional temporal motion vector prediction (ATMVP) method allows each CU to obtain multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the spatio-temporal motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by utilizing the temporal motion vector predictor and the spatial neighboring motion vectors.

[0151] In order to maintain a more accurate motion field for sub-CU motion prediction, motion compression of reference frames is currently disabled.

[0152] 2.3.1.1 Optional Temporal Motion Vector Prediction

[0153] In the optional temporal motion vector prediction (ATMVP) method, the motion vector temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. Figure 18 As shown, the sub-CU is a square N×N block (N is set to 4 by default).

[0154] ATMVP predicts the motion vectors of sub-CUs within a CU in two steps. The first step is to use the so-called time domain vector to identify the corresponding block in the reference picture. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs and obtain the motion vector and reference index of each sub-CU from the block corresponding to each sub-CU, such as Figure 18 shown.

[0155] In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. In order to avoid repeated scanning processing of neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set to the time domain vector and the index of the motion source picture. In this way, in ATMVP, the corresponding block can be identified more accurately than in TMVP, where the corresponding block (sometimes called the collocated block) is always located in the lower right corner or center position relative to the current CU. In one example, if the first Merge candidate comes from the left neighboring block (for example, Figure 19 A1 in , the relevant MV and reference picture are used to identify the source block and source picture.

[0156] Figure 19 An example of identification of source blocks and source pictures is shown.

[0157] In the second step, the corresponding block of the sub-CU is identified by the time domain vector in the motion source picture by adding the time domain vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (the minimum motion grid covering the center sample point) is used to derive the motion information of the sub-CU. After the motion information of the corresponding N×N block is identified, it is converted into the motion vector and reference index of the current sub-CU, which is the same as the TMVP method of HEVC, where motion scaling and other processing are applied. For example, the decoder checks whether the low latency condition is met (for example, the POC of all reference pictures of the current picture is less than the POC of the current picture), and may use the motion vector MVx (the motion vector corresponding to the reference picture list X) to predict the motion vector MVy for each sub-CU (X is equal to 0 or 1 and Y is equal to 1-X).

[0158] 2.3.1.2 Space-time motion vector prediction

[0159] In this method, the motion vector of the sub-CU is recursively derived in raster scan order. Figure 20 The concept is illustrated. Let us consider an 8×8 CU, which contains four 4×4 sub-CUs A, B, C, and D. The adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

[0160] The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU A (block c). If block c is not available or is internally coded, check the other N×N blocks above sub-CU A (from left to right, starting from block c). The second neighbor is a block to the left of sub-CU A (block b). If block b is not available or is internally coded, check the other blocks to the left of sub-CU A (from top to bottom, starting from block b). The motion information obtained from the adjacent blocks in each list is scaled to the first reference frame of the given list. Next, the temporal motion vector prediction (TMVP) of sub-block A is derived according to the same procedure as TMVP specified in HEVC. The motion information of the collocated block at position D is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

[0161] Figure 20 An example of one CU having four sub-blocks (AD) and a neighboring block (ad) is shown.

[0162] 2.3.1.3 Sub-CU Motion Prediction Mode Signaling

[0163] Sub-CU mode is enabled as an additional Merge candidate mode, and no additional syntax elements are required to signal the mode. Two additional Merge candidates are added to the Merge candidate list of each CU to represent ATMVP mode and STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, a maximum of seven Merge candidates are used. The encoding logic of the additional Merge candidates is the same as that of the Merge candidates in HM, which means that for each CU in a P slice or B slice, two additional RD checks are required for the two additional Merge candidates.

[0164] In JEM, all bin files of the Merge index are context coded by CABAC. However, in HEVC, only the first bin file is context coded, and the remaining bin files are context bypass coded.

[0165] 2.3.2 Adaptive Motion Vector Difference Resolution

[0166] In HEVC, when use_integer_mv_flag is equal to 0 in the slice header, the motion vector difference (MVD) (between the PU's motion vector and the predicted motion vector) is signaled in units of quarter luma samples. In JEM, local adaptive motion vector resolution (LAMVR) is introduced. In JEM, MVD can be coded in units of quarter luma samples, integer luma samples, or four luma samples. MVD resolution is controlled at the coding unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU that has at least one non-zero MVD component.

[0167] For a CU with at least one non-zero MVD component, the first flag will signal whether quarter luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, the other flag signals whether integer luma sample MV precision or four luma sample MV precision is used.

[0168] When the first MVD resolution flag of a CU is zero or not encoded for the CU (meaning all MVDs in the CU are zero), the CU uses a quarter-luminance sample MV resolution. When a CU uses integer luminance sample MV precision or four-luminance sample MV precision, the MVP in the AMVP candidate list of the CU will be rounded to the corresponding precision.

[0169] In the encoder, CU-level RD check is used to determine which MVD resolution will be used for the CU. That is, three CU-level RD checks are performed for each MVD resolution. To speed up the encoder, the following encoding scheme is applied in JEM.

[0170] During RD check of a CU with normal quarter luma sample MVD resolution, the motion information of the current CU (integer luma sample precision) is stored. When performing RD check on the same CU with integer luma sample and 4 luma sample MVD resolution, the stored motion information (after rounding) is used as a starting point for further small-scale motion vector refinement, so that the time-consuming motion estimation process is not repeated three times.

[0171] Conditionally call RD check for CUs with 4 luma sample MVD resolution. For a CU, when the RD check cost of integer luma sample MVD resolution is much greater than the RD check cost of quarter luma sample MVD resolution, the RD check for the CU's 4 luma sample MVD resolution will be skipped.

[0172] 2.3.3 Pattern Matching Motion Vector Derivation

[0173] Pattern Matching Motion Vector Derivation (PMMVD) mode is a special Merge mode based on the Frame Rate Up Conversion (FRUC) technique. In this mode, the motion information of the block is not signaled but derived at the decoder side.

[0174] For a CU, when its Merge flag is true, the FRUC flag is signaled. When the FRUC flag is false, the Merge index is signaled and the normal Merge mode is used. When the FRUC flag is true, another FRUC mode flag is signaled to indicate which mode (bilateral matching or template matching) will be used to derive the motion information of the block.

[0175] On the encoder side, the decision on whether to use FRUCMerge mode for a CU is based on the RD cost selection made for the normal merge candidate. That is, the two matching modes (bilateral matching and template matching) of the CU are checked by using RD cost selection. The mode that results in the lowest cost is further compared with other CU modes. If the FRUC matching mode is the most effective mode, then for the CU, the FRUC flag is set to true and the relevant matching mode is used.

[0176] The motion inference process in FRUC Merge mode consists of two steps: first, a CU-level motion search is performed, followed by sub-CU-level motion optimization. At the CU level, an initial motion vector for the entire CU is derived based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate resulting in the lowest matching cost is selected as the starting point for further CU-level optimization. A local search based on bilateral matching or template matching is then performed near the starting point, and the resulting MV with the lowest matching cost is used as the MV value for the entire CU. Next, using the derived CU motion vector as a starting point, motion information is further refined at the sub-CU level.

[0177] For example, the following derivation process is performed for the W×H CU motion information derivation. In the first stage, the MV of the entire W×H CU is derived. In the second stage, the CU is further divided into M×M sub-CUs. The value of M is calculated according to equation (1), and D is the predefined partition depth, which is set to 3 by default in JEM. The MV value of each sub-CU is then derived.

[0178]

[0179] like Figure 21As shown in Figure 1, bilateral matching is used to derive the motion information of the current CU by finding the closest match between two blocks in two different reference pictures along the motion trajectory of the current CU. Under the assumption of continuous motion trajectory, the motion vectors MV0 and MV1 pointing to the two reference blocks are proportional to the temporal distance (e.g., TD0 and TD1) between the current picture and the two reference pictures. As a special case, when the current picture is temporally located between the two reference pictures and the temporal distance from the current picture to the two reference pictures is the same, bilateral matching becomes mirror-based bidirectional MV.

[0180] like Figure 22 As shown, template matching is used to derive the motion information of the current CU by finding the closest match between the template in the current picture (the top and / or left adjacent blocks of the current CU) and the block in the reference picture (the same size as the template). In addition to the above-mentioned FRUC Merge mode, template matching is also applied to AMVP mode. In JEM, as in HEVC, there are two candidates for AMVP. Using the template matching method, a new candidate is derived. If the candidate newly derived by template matching is different from the first existing AMVP candidate, it is inserted at the beginning of the AMVP candidate list, and then the list size is set to 2 (ie, the second existing AMVP candidate is removed). When applied to AMVP mode, only CU-level search is applied.

[0181] 2.3.3.1 CU-level MV candidate set

[0182] CU-level MV candidate sets include:

[0183] (i) Original AMVP candidate, if the current CU is in AMVP mode,

[0184] (ii) All Merge candidates,

[0185] (iii) Several MVs in the interpolated MV field introduced in Section 2.3.3.2.

[0186] (iv) Top and left neighboring motion vectors

[0187] When using bilateral matching, each valid MV of the Merge candidate is used as input to generate a pair of MVs that are assumed to be bilaterally matched. For example, a valid MV of the Merge candidate at reference list A is (MVa, ref a ). Then find the reference image ref of its paired bilateral MV in another reference list B b , so that ref a and ref b Located on different sides of the current picture in time. If the reference ref in reference list B b If not available, the referenceb Determined to be the same as reference ref a A different reference, and its temporal distance to the current picture is the minimum distance in list B. Determine reference ref b After that, based on the current image and reference ref a , reference b The temporal distance between MVa is scaled to derive MVb.

[0188] The four MVs from the interpolated MV field are also added to the CU-level candidate list. More specifically, the interpolated MVs at positions (0, 0), (W / 2, 0), (0, H / 2), and (W / 2, H / 2) of the current CU are added.

[0189] When FRUC is applied in AMVP mode, the original AMVP candidates are also added to the MV candidate set at the CU level.

[0190] At the CU level, up to 15 MVs of an AMVP CU and up to 13 MVs of a Merge CU can be added to the candidate list.

[0191] 2.3.3.2 Sub-CU Level MV Candidate Set

[0192] The MV candidates set at the sub-CU level include:

[0193] (i) MV determined from CU-level search,

[0194] (ii) the top, left, upper-left, and upper-right adjacent MVs,

[0195] (iii) a scaled version of the collocated MV from the reference picture,

[0196] (iv) Up to 4 ATMVP candidates,

[0197] (v) A maximum of 4 STMVP candidates.

[0198] The scaled MV from the reference picture is derived as follows: All reference pictures in both lists are traversed. The MV at the collocated position of the sub-CU in the reference picture is scaled to the reference of the starting CU level MV.

[0199] ATMVP and STMVP candidates are limited to the first four. At the sub-CU level, up to 17 MVs are added to the candidate list.

[0200] 2.3.3.3 Generation of interpolated MV fields

[0201] Before encoding a frame, an interpolated motion field of the entire picture is generated based on unidirectional ME. This motion field can then be used as an MV candidate at the CU level or sub-CU level.

[0202] First, the motion fields of each reference picture in both reference lists are traversed at the 4×4 block level. For each 4×4 block, if the motion associated with the block passes through a 4×4 block in the current picture (e.g. Figure 23 As shown), and the block is not assigned any interpolated motion, the motion of the reference block is scaled to the current picture according to the temporal distances TD0 and TD1 (the same as the MV scaling of TMVP in HEVC), and the scaled motion is assigned to the block in the current frame. If no scaled MV is assigned to the 4×4 block, the motion of the block is marked as unavailable in the interpolated motion field.

[0203] 2.3.3.4 Imputation Matching Cost

[0204] When motion vectors point to fractional sample locations, motion compensated interpolation is required. To reduce complexity, bilinear interpolation is used for both bilateral matching and template matching instead of the conventional 8-tap HEVC interpolation.

[0205] The calculation of the matching cost is slightly different at different steps. When selecting a candidate from the candidate set at the CU level, the matching cost is the sum of absolute differences (SAD) of bilateral matching or template matching. After determining the starting MV, the matching cost C of bilateral matching at the sub-CU level is calculated as follows:

[0206]

[0207] Here, w is the weight coefficient, which is empirically set to 4. MV and MV s Indicates the current MV and the starting MV respectively. SAD is still used as the matching cost for pattern matching at the sub-CU level.

[0208] In FRUC mode, MVs are derived using only luma samples. The derived motion is used for inter-frame MC prediction of luma and chroma. After determining the MVs, final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0209] 2.3.3.5 MV Refinement

[0210] MV refinement is a pattern-based MV search using either bilateral cost or template matching cost as the criterion. In JEM, two search modes are supported—unrestricted center biased diamond search (UCBDS) and adaptive cross search—for MV refinement at the CU level and sub-CU level, respectively. For both CU-level and sub-CU-level MV refinement, the MV is directly searched at quarter-luminance sample accuracy, followed by eighth-luminance sample MV refinement. The search range for MV refinement at both CU and sub-CU steps is set to 8 luma samples.

[0211] 2.3.3.6 Prediction Direction Selection in Template Matching FRUC Merge Mode

[0212] In bilateral merge mode, bidirectional prediction is always applied because the motion information of the CU is derived from the closest match between two blocks on the current CU's motion trajectory in two different reference pictures. Template matching merge mode does not have this restriction. In template matching merge mode, the encoder can choose from unidirectional prediction in list 0, unidirectional prediction in list 1, or bidirectional prediction for the CU.

[0213] The selection is based on the template matching cost as follows:

[0214] If costBi<=factor*min(cost0,cost1)

[0215] Bidirectional prediction is used;

[0216] Otherwise, if cost0 <= cost1

[0217] Then use the one-way prediction in list 0;

[0218] otherwise,

[0219] Use the one-way prediction in Listing 1;

[0220] Where cost0 is the SAD of list 0 template matching, cost1 is the SAD of list 2 template matching, and costBi is the SAD of bi-prediction template matching. The value of factor equal to 1.25 means that the selection process is biased towards bi-prediction. Inter prediction direction selection can only be applied to CU-level template matching process.

[0221] 2.3.4 Decoder-side Motion Vector Refinement

[0222] In bidirectional prediction, two prediction blocks, formed by motion vectors (MVs) from list 0 and list 1, are combined to form a single prediction signal for a block region. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined through a bilateral template matching process. Bilateral template matching, applied at the decoder, performs a distortion-based search between the bilateral template and the reconstructed samples in the reference picture to obtain the refined MVs without transmitting additional motion information.

[0223] In DMVR, a bilateral template is generated as a weighted combination (i.e., average) of two prediction blocks, where the two prediction blocks are from the initial MV0 of list 0 and MV1 of list 1, respectively. The template matching operation involves calculating a cost metric between the generated template and the sample area in the reference picture (around the initial prediction block). For each of the two reference pictures, the MV that produces the minimum template cost is considered as the updated MV for that list to replace the original MV. In JEM, nine MV candidates are searched for each list. The nine MV candidates include the original MV and 8 surrounding MVs that have a luminance sample offset from the original MV in the horizontal or vertical direction or both. Finally, the MV is selected using Figure 24 The two new MVs shown (ie, MV0' and MV1') generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as the cost metric.

[0224] Without transmitting additional syntax elements, DMVR is applied to the Merge mode of bi-prediction, where one MV comes from the past reference picture and the other MV comes from the future reference picture. In JEM, DMVR is not applied when LIC, affine motion, FRUC or sub-CU Merge candidate is enabled for the CU.

[0225] 2.3.5 Merge / Skip Mode with Bilateral Matching Refinement

[0226] First, the Merge candidate list is constructed by inserting the motion vectors and reference indices of spatial neighboring blocks and temporal neighboring blocks into the candidate list and performing redundancy checks until the number of available candidates reaches the maximum number of candidates, which is 19. The Merge candidate list for Merge / Skip mode is constructed by inserting spatial candidates ( Figure 11 ), time domain candidates, affine candidates, advanced time domain MVP (ATMVP) candidates, space-time MVP (STMVP) candidates and additional candidates used in HEVC (combined candidates and zero candidates):

[0227] - Spatial candidates for blocks 1-4.

[0228] - Extrapolated affine candidates for blocks 1-4.

[0229] -ATMVP.

[0230] -STMVP.

[0231] -Virtual affine candidate.

[0232] - Spatial candidates (block 5) (used only when the number of available candidates is less than 6).

[0233] - Extrapolate affine candidates (block 5).

[0234] - Temporal candidates (as derived in HEVC).

[0235] - Non-adjacent spatial candidates followed by extrapolated affine candidates (blocks 6 to 49, e.g. Figure 25 shown).

[0236] -Combination candidates.

[0237] -Zero Candidates

[0238] It is worth noting that, in addition to STMVP and Affine, the IC flag is also inherited from the Merge candidate. In addition, for the first four spatial candidates, the bidirectional prediction candidate is inserted before the unidirectional prediction candidate.

[0239] In some implementations, blocks that are not connected to the current block can be accessed. If non-adjacent blocks are encoded using non-intra mode, the related motion information can be added as additional merge candidates.

[0240] 3. Examples of Problems Solved by the Embodiments Disclosed herein

[0241] The current HEVC design can use the correlation between the current block and its neighboring blocks (adjacent to the current block) to better encode motion information. However, the neighboring blocks may correspond to different objects with different motion trajectories. In this case, prediction from neighboring blocks is not effective.

[0242] Predicting from motion information of non-neighboring blocks can bring additional coding benefits at the cost of storing all motion information (usually at a 4x4 level) in the cache, which significantly increases the complexity of hardware implementation.

[0243] 4. Some Examples

[0244] To overcome the shortcomings of existing implementations, LUT-based motion vector prediction techniques can be implemented in various embodiments to provide video encoding with higher coding efficiency. LUT-based motion vector prediction techniques use one or more lookup tables that store at least one motion candidate to predict motion information of a block. Each LUT may include one or more motion candidates, each associated with corresponding motion information. The motion information of a motion candidate may include part or all of the prediction direction, reference index / picture, motion vector, LIC flag, affine flag, motion vector derivation (MVD) accuracy, and / or MVD value. The motion information may also include block position information to indicate where the motion information comes from.

[0245] The examples described below for various implementations illustrate LUT-based motion vector prediction based on the disclosed technology, which can enhance existing and future video coding standards. Since the LUT allows the encoding / decoding process to be performed based on historical data (e.g., processed blocks), the LUT-based motion vector prediction can also be referred to as a history-based motion vector prediction (HMVP) method. In the LUT-based motion vector prediction method, one or more tables are maintained during the encoding / decoding process, whose motion information comes from previously encoded blocks. During the encoding / decoding of a block, the relevant motion information in the LUT can be added to the motion candidate list, and the LUT can be updated after encoding / decoding the block. The following examples should be regarded as examples to explain the general concept. These examples should not be understood narrowly. In addition, these examples can be combined in any way.

[0246] Some embodiments may use one or more lookup tables storing at least one motion candidate to predict motion information for a block. Embodiments may use a motion candidate to indicate a set of motion information stored in a lookup table. For traditional AMVP or Merge modes, embodiments may use AMVP or Merge candidates to store motion information.

[0247] The following example illustrates the general concept.

[0248] Example of a lookup table

[0249] Example A1: Each lookup table may contain one or more motion candidates, where each candidate is associated with its motion information.

[0250] a. Here, the motion information of the candidate motion may include part or all of the prediction direction, reference index / picture, motion vector, LIC flag, affine flag, MVD accuracy, and MVD value.

[0251] b. Motion information may also include block location information to indicate where the motion information comes from.

[0252] c. A counter may be further assigned to each lookup table.

[0253] i. At the beginning of encoding / decoding a picture / slice / LCU (CTU) row / slice, the counter may be initialized to zero.

[0254] ii. In one example, the counter may be updated after encoding / decoding a CTU / CTB / CU / CB / PU / specific region size (eg, 8x8 or 16x16).

[0255] iii. In one example, each time a candidate is added to the lookup table, a counter is incremented by one.

[0256] iv. In one example, the counter should not be larger than the size of the table (the number of allowed motion candidates).

[0257] v. Alternatively, a counter may be used to indicate how many motion candidates have been attempted to be added to the lookup table (some of them are in the lookup table, but may be removed from the table later). In this case, the counter may be larger than the size of the table.

[0258] d. The size of the table (the number of motion candidates allowed) and / or the number of tables can be fixed or adaptive. The table size can be the same for all tables, or different tables can have different table sizes.

[0259] i. Alternatively, different lookup tables may use different sizes (eg, 1 or 2).

[0260] ii. In one example, the size of the table and / or the number of tables may be predefined.

[0261] iii. In one example, the size of the table and / or the number of tables may be signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a slice header, a coding tree unit (CTU), a coding tree block (CTB), a coding unit (CU) or a prediction unit (PU), or an area covering multiple CTUs / CTBs / CUs / PUs.

[0262] iv. The size of the table and / or the number of tables may further depend on the slice type, the temporal layer index of the picture, and the picture order count (POC) distance between a slice and the nearest inner slice.

[0263] e. Assuming there are N tables for encoding threads, N*P tables may be needed to encode a stripe, where P represents the number of LCU rows or slices.

[0264] i. Alternatively, only P tables are needed for coding a slice, where P represents the number of LCU rows, where only one lookup table is used per LCU row, even though N may be greater than 1 when slices are disabled.

[0265] LUT Selection

[0266] Example B1: For a coding block, some or all motion candidates from a lookup table may be checked sequentially. When a motion candidate is checked during a coding block, it may be added to a motion candidate list (eg, AMVP, Merge candidate list).

[0267] a. Alternatively, motion candidates from multiple lookup tables can be checked sequentially.

[0268] b. The lookup table index may be signaled in a CTU, CTB, CU or PU, or in a region covering multiple CTUs / CTBs / CUs / PUs.

[0269] Example B2: The choice of lookup table may depend on the location of the block.

[0270] a. It may depend on the CTU address covering the block. Here, we take two lookup tables (dual lookup table, DLUT) as an example to illustrate this concept:

[0271] i. If the block is located in one of the first M CTUs in a CTU row, the first lookup table can be used to encode the block, while for blocks located in the remaining CTUs in the CTU row, the second lookup table can be used.

[0272] ii. If the block is located in one of the first M CTUs in a CTU row, the motion candidates in the first lookup table may be checked first to encode the block. If there are not enough candidates in the first table, the second lookup table may be used. For blocks located in the remaining CTUs in the CTU row, the second lookup table may be used.

[0273] iii. Alternatively, for blocks located in the remaining CTUs in the CTU row, the motion candidates in the second lookup table may be checked first to encode the block, and if there are not enough candidates in the second table, the first lookup table may be further used.

[0274] b. May depend on the distance between the location of the block and a location associated with a motion candidate in one or more lookup tables.

[0275] iv. In one example, if one motion candidate is associated with a smaller distance to the block to be encoded, it may be checked earlier than another motion candidate.

[0276] Use of lookup tables

[0277] Example C1: The total number of motion candidates in the lookup table to be checked can be predefined.

[0278] a. It may further depend on coding information, block size, block shape, etc. For example, for AMVP mode, only m motion candidates may be checked, while for Merge mode, n motion candidates may be checked (e.g., m=2, n=44).

[0279] b. In one example, the total number of motion candidates to be checked may be signaled in a video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, slice header, coding tree unit (CTU), coding tree block (CTB), coding unit (CU) or prediction unit (PU), or in an area covering multiple CTUs / CTBs / CUs / PUs.

[0280] Example C2: The motion candidates contained in the lookup table can be directly inherited by the block.

[0281] a. They can be used for Merge mode coding, i.e. motion candidates can be checked during Merge candidate list derivation.

[0282] b. They can be used for affine Merge mode encoding.

[0283] i. If the affine flag of the motion candidate in the lookup table is 1, it can be added as an affine Merge candidate.

[0284] c. Checking for motion candidates in the lookup table can be enabled when:

[0285] i. After inserting the TMVP candidate, the Merge candidate list is not full;

[0286] ii. After checking a certain spatial neighboring block for spatial merge candidate derivation, the merge candidate list is not full;

[0287] iii. After all spaces are merged as candidates, the merge candidate list is not full;

[0288] iv. After combining the bidirectional prediction merge candidates, the merge candidate list is not full;

[0289] v. Pruning can be applied before adding motion candidates to the Merge candidate list.

[0290] Example C3: The motion candidates contained in the lookup table may be used as predictors of the motion information of the coding block.

[0291] a. They can be used for AMVP mode coding, i.e., motion candidates can be checked during the AMVP candidate list derivation process.

[0292] b. Checking for motion candidates in the lookup table may be enabled when:

[0293] i. After the TMVP candidate is inserted, the AMVP candidate list is not full;

[0294] ii. Before inserting TMVP candidates, the AMVP candidate list is not full after selecting and pruning from spatial neighbors;

[0295] iii. When there is no AMVP candidate without scaling from the upper neighboring block and / or when there is no AMVP candidate without scaling from the left neighboring block.

[0296] iv. Before adding a motion candidate to the AMVP candidate list, pruning may be applied.

[0297] c. Check motion candidates that have the same reference picture as the current reference picture.

[0298] i. Optionally, in addition, motion candidates (MV with scaling) with different reference pictures than the current reference picture are also checked.

[0299] ii. Alternatively, first check all motion candidates that have the same reference picture as the current reference picture, and then check motion candidates that have different reference pictures from the current reference picture.

[0300] iii. Alternatively, check motion candidates in the same way as in Merge.

[0301] Example C4: The order of checking motion candidates in the lookup table is defined as follows (assuming that K (K>=1) motion candidates are allowed to be checked):

[0302] a. Look up the last K motion candidates in the table, eg, in descending order of the entry index of the LUT.

[0303] b. Top K%L candidates, where L is the lookup table size when K>=L, eg, in descending order of entry index of the LUT.

[0304] c. When K>=L, search all candidates in the table (L candidates).

[0305] d. Alternatively, or in addition, based on descending order of motion candidate indices.

[0306] e. Alternatively, K motion candidates are selected based on candidate information (eg, the position relative to the motion candidate and the distance from the current block).

[0307] f. The order in which different lookup tables are checked is defined in the next section, Use of Lookup Tables.

[0308] g. Once the Merge / AMVP candidate list reaches the maximum allowed number of candidates, the checking process will terminate.

[0309] h. Alternatively, once the number of added motion candidates reaches the maximum allowed number of motion candidates, the checking process will terminate.

[0310] i. One syntax element indicating the size of the allowed table and the number of motion candidates allowed to be checked (ie, K=L) may be signaled in the SPS, PPS, slice header, or slice header.

[0311] Example C5: The use of a lookup table to enable / disable motion information encoding of a block may be signaled in an SPS, PPS, slice header, slice header, CTU, CTB, CU or PU, or in an area covering multiple CTUs / CTBs / CUs / PUs.

[0312] Example C6: Whether to apply prediction from a lookup table may further depend on coding information. When inference is not applied to a block, additional signaling indicating prediction is skipped. Alternatively, when inference is not applied to a block, motion candidates in the lookup table do not need to be accessed, and checking for relevant motion candidates is omitted.

[0313] a. Whether to apply prediction from the lookup table may depend on the block size / block shape. In one example, for smaller blocks (such as 4x4, 8x4 or 4x8 blocks), prediction from the lookup table is not allowed.

[0314] b. Whether to apply prediction from a lookup table may depend on whether the block is coded in AMVP or Merge mode. In one example, for AMVP mode, prediction from a lookup table is not allowed.

[0315] c. Whether to apply prediction from a lookup table may depend on whether the block is coded with affine motion or other motion (such as translational motion). In one example, for affine mode, prediction from a lookup table is not allowed.

[0316] Example C7: Motion candidates from a lookup table in a previously coded frame / slice / slice may be used to predict motion information for a block in a different frame / slice / slice.

[0317] a. In one example, only the lookup table associated with the reference picture of the current block can be used to encode the current block.

[0318] b. In one example, only lookup tables associated with pictures having the same slice type and / or the same quantization parameter as the current block may be used for encoding the current block.

[0319] Update of the lookup table

[0320] Example D1: After a block is encoded using motion information (ie, intra block copy mode, inter coding mode), one or more lookup tables may be updated.

[0321] a. In one example, whether to update the lookup table can reuse the previous rule for selecting the lookup table.

[0322] b. The lookup table may be updated based on coding information and / or the location of the block / LCU.

[0323] c. If the block is encoded with directly signaled motion information (such as AMVP mode), the motion information of the block can be added to the lookup table.

[0324] i. Alternatively, if the block is encoded with motion information inherited directly from spatially neighboring blocks without any refinement (eg, a spatial merge candidate without refinement), the motion information of the block should not be added to the lookup table.

[0325] ii. Alternatively, if the block is encoded with refined motion information inherited directly from spatially neighboring blocks (e.g., DMVR, FRUC), the motion information of the block should not be added to any lookup table.

[0326] iii. Alternatively, if the block is encoded with motion information inherited directly from motion candidates stored in a lookup table, the motion information of the block should not be added to any lookup table.

[0327] d. Select M (M>=1) typical positions within the block and update the lookup table using the motion information associated with the typical positions.

[0328] i. In one example, the typical positions are defined as the four corner positions within the block (e.g. Figure 26 One of C0-C3).

[0329] ii. In one example, the typical location is defined as the center location within the block (e.g. Figure 26 Ca-Cd in the

[0330] iii. When sub-block prediction is not allowed for the block, M is set to 1.

[0331] iv. When the block allows sub-block prediction, M can be set to 1, or the total number of sub-blocks, or any value between 1 and the total number of sub-blocks, including 1 and the total number of sub-blocks.

[0332] v. Alternatively, when the block allows sub-block prediction, M can be set to 1, and the selection of representative sub-blocks is based on

[0333] 1. Frequency of use of motion information,

[0334] 2. Whether it is a bidirectional prediction block,

[0335] 3. Based on reference image index / reference image,

[0336] 4. The difference of the motion vector compared to other motion vectors (e.g., choose the largest MV difference)

[0337] 5. Other coding information.

[0338] e. When M (M>=1) groups of representative positions are selected to update the lookup table, further conditions may be checked before adding them as additional motion candidates to the lookup table.

[0339] i. The new motion information set can be applied to prune existing motion candidates in the lookup table.

[0340] ii. In one example, the new set of motion information should not be identical to any or part of the existing motion candidates in the lookup table.

[0341] iii. Alternatively, for the same reference picture from the new motion information set and an existing motion candidate, the MV difference should be no less than one / multiple thresholds. For example, the horizontal and / or vertical components of the MV difference should be greater than 1 pixel distance.

[0342] iv. Alternatively, when K>L, the new motion information set is pruned only to the last K candidates or the first K%L existing motion candidates to allow old motion candidates to be reactivated.

[0343] v. Alternatively, no pruning should be applied.

[0344] f. If the lookup table is updated with M sets of motion information, the corresponding counter should be increased by M.

[0345] g. Assume that before encoding the current block, the counter of the lookup table to be updated is denoted by K. After encoding the block, for a set of selected motion information (using the above method), it is added as an additional motion candidate with an index equal to K%L (where L is the lookup table size). For example Figure 27 shown.

[0346] i. Alternatively, add it as an additional motion candidate with index equal to min(K+1, L-1). Otherwise, if K>=L, remove the first motion candidate (index equal to 0) from the lookup table and decrement the indexes of the subsequent K candidates by 1.

[0347] h. After encoding an internal constraint block, the lookup table may be cleared.

[0348] i. If a motion information entry is added to the lookup table, more motion information entries may be added to the table by inferring from the motion information. In this case, the counter associated with the lookup table may be increased by more than 1.

[0349] i. In one example, the MV of the motion information entry is scaled and placed in a table;

[0350] ii. In one example, the MV of the motion information entry is added by (dx, dy) and placed in the table;

[0351] iii. In one example, the average of the MVs of two or more motion information items is calculated and put into a table.

[0352] Example D2: If a block is located at a picture / slice / slice boundary, then updating the lookup table may never be allowed.

[0353] Example D3: The motion information of the above LCU row may be disabled to encode the current LCU row.

[0354] a. In this case, the number of available motion candidates may be reset to 0 at the beginning of a new slice / slice / LCU row.

[0355] Example D4: At the beginning of encoding a slice / slice with a new temporal layer index, the number of available motion candidates may be reset to 0.

[0356] Example D5: The lookup table may be continuously updated using a stripe / slice / LCU row / stripes with the same temporal layer index.

[0357] a. Alternatively, the lookup table can only be updated after encoding / decoding every S (S>=1) CTUs / CTBs / CUs / CBs or encoding / decoding a certain region (e.g., size equal to 8x8 or 16x16).

[0358] b. Alternatively, a lookup table may stop updating once it reaches the maximum allowed counter.

[0359] c. In one example, the counter may be predefined or signaled in a video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, slice header, coding tree unit (CTU), coding tree block (CTB), coding unit (CU) or prediction unit (PU), or in an area covering multiple CTUs / CTBs / CUs / PUs.

[0360] Figure 28 is a block diagram of a video processing device 2800. Device 2800 can be used to implement one or more methods described herein. Device 2800 can be implemented in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Device 2800 may include one or more processors 2802, one or more memories 2804, and video processing hardware 2806. Processor 2802 can be configured to implement one or more methods described herein. Memory(s) 2804 can be used to store data and code used to implement the methods and techniques described herein. Video processing hardware 2806 can be used to implement some of the techniques described herein in hardware circuitry.

[0361] Figure 29is a flow chart of an example of a video decoding method 2900. The method 2900 includes maintaining (2902) several tables (e.g., lookup tables; LUTs), where each table includes a set of motion candidates and each motion candidate is associated with corresponding motion information derived from a previously encoded video block, performing (2904) conversion between a current video block and a bitstream of the current video block in a video region, and updating (2906) one or more tables based on an update rule.

[0362] Figure 30 is a flow chart of an example video decoding method 3000. The method 3000 includes checking (3002) a number of tables (e.g., lookup tables; LUTs), each table including one or more motion candidates, each motion candidate associated with motion information of the motion candidate, processing motion information of a video block based on a selected one or more tables, and updating (3006) the one or more tables based on a video block generated from the processing.

[0363] From the foregoing, it should be understood that for ease of illustration, specific embodiments of the technology disclosed herein have been described herein, but various modifications may be made without departing from the scope of the invention. Therefore, the technology disclosed herein is not limited to the limitations of the claims except as otherwise provided.

[0364] The disclosed and other described embodiments, modules, and functional operations herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or any combination thereof. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing apparatus or to control the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a composition of matter that effects a machine-readable propagated signal, or any combination thereof. The term "data processing apparatus" includes all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or a multiprocessor or group of computers. In addition to hardware, the apparatus may also include code that creates an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or any combination thereof. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.

[0365] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers, located at one site or distributed across multiple sites and interconnected by a communications network.

[0366] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special-purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0367] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more of any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CDROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated into, special-purpose logic circuitry.

[0368] While this patent document contains many specifics, they should not be construed as limitations on the scope of any invention or the claims, but rather as descriptions of features of particular embodiments of particular inventions. Some features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable subcombination. Furthermore, while the features described above may be described as functioning in some combinations, or even initially claimed to be so, in some cases one or more features in a claim combination may be removed from the combination, and a claim combination may be directed to a subcombination or variations of a subcombination.

[0369] Likewise, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, in order to achieve desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0370] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: maintaining one or more tables, wherein each of the one or more tables includes one or more motion candidates derived from one or more coded video blocks, and an arrangement of the motion candidates in the tables is based on an order in which the motion candidates were added to the tables; Constructing a motion candidate list for the current video block, wherein during the constructing process, at least one motion candidate of a table in the one or more tables may be selectively checked in sequence; Determining motion information of the current video block using the motion candidate list; and encoding and decoding the current video block based on the determined motion information; wherein the table is reset at the beginning of encoding and decoding a new region comprising a plurality of video blocks, wherein the resetting comprises resetting the number of motion candidates in the table to zero, The one or more tables include N tables, where N is equal to K*P, where K is an integer representing the number of tables corresponding to slices of a slice of video data or CTU rows for each codec thread, and P is an integer representing the number of CTU rows or the number of slices of a slice.

2. The method according to claim 1, wherein The table of the one or more tables has a size that indicates the number of motion candidates allowed in the table.

3. The method according to claim 2, wherein: Before new motion candidates are added to the table, when the number of motion candidates in the table reaches the size of the table, the candidates in the table are deleted due to the addition of the new motion candidate in the table.

4. The method according to claim 2, wherein: A value of the size of the table is a predefined value, and the sizes of the plurality of tables are the same.

5. The method of claim 1 , wherein the motion candidates in the table are associated with motion information, the motion information comprising at least one of a prediction direction, a reference picture index, a motion vector value, an intensity compensation flag, an affine flag, a motion vector difference precision, or a motion vector difference value. The method of claim 1 , wherein the motion candidates in the table are associated with intra prediction modes for intra-coded blocks. The method of claim 1 , wherein the motion candidates in the table are associated with a plurality of IC parameters for an illumination-compensated IC coding block. The method of claim 1 , wherein the motion candidates in the table are associated with filter parameters used in a filtering process.

9. The method according to claim 2, further comprising: A counter is maintained for the table, wherein the counter indicates a number of candidate motions in the table and the counter is no larger than a size of the table.

10. The method according to claim 2, wherein: The size of the table is based on at least one of a slice type, a temporal layer index of a picture, and a picture order count (POC) distance between a slice and a nearest intra slice.

11. The method according to claim 1, wherein The new area is a new codec tree unit, a new slice or a new slice.

12. The method according to claim 1, wherein The one or more tables include N tables, where N is equal to P when slices of a slice or CTU rows of the video data use a single table, and P is an integer representing the number of CTU rows or the number of slices of a slice.

13. The method according to claim 1, wherein The number of the one or more tables is based on at least one of a slice type, a temporal layer index of a picture, and a picture order count (POC) distance between a slice and a nearest intra slice.

14. The method according to claim 1, wherein The motion candidate list is one of a motion vector prediction list and a merge candidate list. The method according to claim 1 , further comprising adding one or more examined candidates to the motion candidate list based on an examination result.

16. The method according to claim 1, wherein The encoding and decoding includes encoding the current video block into a bitstream.

17. The method according to claim 1, wherein The encoding and decoding includes decoding the current video block from a bitstream.

18. An apparatus for encoding and decoding video data, comprising a processor and non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: maintaining one or more tables, wherein each of the one or more tables includes one or more motion candidates derived from one or more coded video blocks, and an arrangement of the motion candidates in the tables is based on an order in which the motion candidates were added to the tables; Build a motion candidate list for the current video block, where During the construction process, at least one motion candidate of a table in the one or more tables may be selectively checked in sequence; Determining motion information of the current video block using the motion candidate list; as well as encoding and decoding the current video block based on the determined motion information; wherein the table is reset at the beginning of encoding and decoding a new region comprising a plurality of video blocks, wherein the resetting comprises resetting the number of motion candidates in the table to zero, The one or more tables include N tables, where N is equal to K*P, where K is an integer representing the number of tables corresponding to slices of a slice of video data or CTU rows for each codec thread, and P is an integer representing the number of CTU rows or the number of slices of a slice.

19. The device according to claim 18, wherein The table of the one or more tables has a size that indicates the number of motion candidates allowed in the table.

20. The device according to claim 19, wherein Before new motion candidates are added to the table, when the number of motion candidates in the table reaches the size of the table, the candidates in the table are deleted due to the addition of the new motion candidate in the table.

21. The device according to claim 20, wherein A value of the size of the table is a predefined value, and the sizes of the plurality of tables are the same.

22. A non-transitory computer-readable storage medium storing instructions for causing a processor to: maintaining one or more tables, wherein each of the one or more tables includes one or more motion candidates derived from one or more coded video blocks, and an arrangement of the motion candidates in the tables is based on an order in which the motion candidates were added to the tables; Build a motion candidate list for the current video block, where During the construction process, at least one motion candidate of a table in the one or more tables may be selectively checked in sequence; Determining motion information of the current video block using the motion candidate list; as well as encoding and decoding the current video block based on the determined motion information; wherein the table is reset at the beginning of encoding and decoding a new region comprising a plurality of video blocks, wherein the resetting comprises resetting the number of motion candidates in the table to zero, The one or more tables include N tables, where N is equal to K*P, where K is an integer representing the number of tables corresponding to slices of a slice of video data or CTU rows for each codec thread, and P is an integer representing the number of CTU rows or the number of slices of a slice.

23. The non-transitory computer-readable storage medium of claim 22, wherein: The table of the one or more tables has a size that indicates the number of motion candidates allowed in the table.

24. The non-transitory computer-readable storage medium of claim 23, wherein: Before new motion candidates are added to the table, when the number of motion candidates in the table reaches the size of the table, the candidates in the table are deleted due to the addition of the new motion candidate in the table.

25. The non-transitory computer-readable storage medium of claim 23, wherein: A value of the size of the table is a predefined value, and the sizes of the plurality of tables are the same.

26. A method for storing a bitstream of a video, the method comprising: maintaining one or more tables, wherein each of the one or more tables includes one or more motion candidates derived from one or more coded video blocks, and an arrangement of the motion candidates in the tables is based on an order in which the motion candidates were added to the tables; Constructing a motion candidate list for the current video block, wherein during the constructing process, at least one motion candidate of a table in the one or more tables may be selectively checked in sequence; Determining motion information of the current video block using the motion candidate list; generating the bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium; wherein the table is reset at the beginning of encoding and decoding a new region comprising a plurality of video blocks, wherein the resetting comprises resetting the number of motion candidates in the table to zero, The one or more tables include N tables, where N is equal to K*P, where K is an integer representing the number of tables corresponding to slices of a slice of video data or CTU rows for each codec thread, and P is an integer representing the number of CTU rows or the number of slices of a slice.

27. The method for storing a video bitstream according to claim 26, wherein: The table of the one or more tables has a size that indicates the number of motion candidates allowed in the table.

28. The method for storing a video bitstream according to claim 27, wherein: Before new motion candidates are added to the table, when the number of motion candidates in the table reaches the size of the table, the candidates in the table are deleted due to the addition of the new motion candidate in the table.

29. The method for storing a video bitstream according to claim 27, wherein: A value of the size of the table is a predefined value, and the sizes of the plurality of tables are the same.

30. An apparatus for encoding and decoding video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 5-17.

31. A non-transitory computer-readable storage medium storing instructions for causing a processor to execute the method according to any one of claims 5 to 17.

Images