Interaction between intra block copy modes and inter predictors

Abstract

This disclosure relates to the interaction between intra block copy modes and inter prediction tools. A method for video processing is provided, comprising: determining that an Intra Block Copy (IBC) mode is applied to a current video block of a video, where in the IBC mode at least one reference picture used by the current video block is a current picture in which the current video block is located; making a decision regarding disabling a particular coding mode for the current block; and performing, based on the decision, a conversion between the current video block and a bitstream representation, the particular coding mode using motion vectors and non-current pictures to derive a prediction for the video block.

Description

[Technical field]

[0001] This patent document relates to video coding techniques, devices and systems.

[0002] This application claims priority to and the benefit of International Patent Application No. PCT / CN2018 / 118167, filed November 29, 2018. The entire disclosure of International Patent Application No. PCT / CN2018 / 118167 is incorporated by reference as part of the disclosure of this application. [Background technology]

[0003] Despite advances in video compression, digital video still accounts for the largest percentage of bandwidth in the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demands for digital video usage are expected to continue to increase. Summary of the Invention [Problem to be solved by the invention]

[0004] Signaling for motion vector derivation and Intra-Block-Copy (IBC) in digital video coding, and in particular in video and picture coding, is described. The described methods may be applied to both existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards or video codecs. [Means for solving the problem]

[0005] In one representative aspect, the disclosed techniques may be used to provide a method for video processing, the method including: determining that an intra block copy (IBC) mode is applied to a current video block of a video, where in the IBC mode at least one reference picture used by the current video block is a current picture in which the current video block is located; making a decision regarding disabling a particular coding mode for the current video block; and performing a conversion between the current video block and a bitstream representation based on the decision, where the particular coding mode uses motion vectors and non-current pictures to derive a prediction of the video block.

[0006] In another representative aspect, the disclosed techniques may be used to provide a method for video processing that includes determining that a current video block is encoded using a particular coding mode, making a decision regarding disabling an intra block copy (IBC) mode for the current block based on the determination, where in the IBC mode at least one reference picture used by the current video block is a current picture in which the current block is located, and performing a conversion between the current block and a bitstream representation based on the determination, where the particular coding mode uses motion vectors and non-current pictures to derive a prediction of the current block.

[0007] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing that includes determining an affine mode for a current video block, where whether the current video block inherits an affine mode from a neighboring block depends on a reference list, and performing a conversion between the current video block of a video and a consistent bitstream representation of the video based on the determination.

[0008] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, comprising: performing a conversion between a current video block of a video and a bitstream representation of the current video block, wherein during the conversion, an Intra Block Copy (IBC) mode and a Merge with Motion Vector Difference (MMVD) mode are used, and in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is disposed, and in the MMVD mode, a motion vector of the video block is derived based on a merged motion candidate list and further refined by at least one motion vector difference.

[0009] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, comprising: performing a conversion between a current video block of a video and a bitstream representation of the video, wherein during the conversion, an intra block copy (IBC) mode and an inter-intra prediction mode are used, and in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is located, and in the inter-intra prediction mode, a prediction signal of the current video block is generated based on at least an intra prediction signal and an inter prediction signal.

[0010] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, the method including: determining, during a conversion between a current video block and a bitstream representation of the current video block, decoded information of at least one coding method different from an IBC mode, in which at least one reference picture used by the current video block is a current picture in which the current video block is located, based on the decoded information, determining whether to skip signaling of an IBC flag for the current video block in a bitstream representation of the current video block, the IBC flag being associated with an IB mode, and performing the conversion based on the determination.

[0011] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, wherein during conversion between a current video block and a bitstream representation of the current video block, the current video block includes a luma component that is coded using a first coding structure tree and a chroma component that is coded using a second coding structure tree that is different from the first coding structure tree, the method including: deriving first motion vector information for chroma blocks from second motion vector information for one or more corresponding blocks of the luma component of the current video block; and performing the conversion based on the first motion vector information and the second motion vector information.

[0012] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, the method including: determining whether to apply an intra block copy mode (IBC) to a current video block, where at least one reference picture used by the current video block is the same as a picture in which the current video block is located; signaling whether the IBC mode applies to the current video block; withholding a signaling flag for a group of prediction modes based on determining that the IBC mode applies to the current video block; and performing a transformation of the current video block based on the application of the IBC mode.

[0013] In yet another representative aspect, the disclosed technology may be used to provide a method for video processing, which includes analyzing a bitstream to determine whether at least one reference picture used by a current video block is the same as a picture in which the current video block is located, and performing a transformation on the current video block, where a flag for a group of prediction modes is excluded from the bitstream based on determining that CPR is applied to the current video block.

[0014] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, the method including: in a process of obtaining motion information of a current video block, during a conversion between the current video block and a bitstream representation of the current video block, the motion information of the current video block is based on at least one affine model of at least one neighboring block of the current video block, determining whether the neighboring block of the current video block uses an intra block copy mode (IBC) in which at least one reference picture used by the neighboring block is the same as a picture in which the neighboring block is located, disabling, for the current video block, deriving affine candidates from the neighboring block based on determining that the neighboring video block uses the IBC mode, and performing the conversion based on the motion information of the current video block.

[0015] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, the method including: dividing a current video block into sub-blocks during conversion to / from a bitstream representation of the current video block based on a sub-block based temporal motion vector prediction (SbTMVP) mode, identifying a corresponding block in a motion source picture having a temporal motion vector based on motion information of spatial neighboring blocks of the current video block, generating motion vectors and reference indices for the sub-blocks from the corresponding blocks of the sub-blocks based on coding mode information of the corresponding blocks of the sub-blocks, and performing the conversion based on the generated motion vectors and reference indices.

[0016] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing, the method including: dividing a current video block into sub-blocks during conversion between a current video block and a bitstream representation of the current video block based on a sub-block-based temporal motion vector prediction (SbTMVP) mode, identifying corresponding blocks in a motion source picture having temporal motion vectors based on coding mode information of spatially neighboring blocks of the current video block, generating motion vectors and reference indexes for the sub-blocks from corresponding blocks of the sub-blocks, and performing the conversion based on the generated motion vectors and reference indexes.

[0017] In yet another exemplary embodiment, the above-described methods are embodied in the form of processor-executable code and stored on a computer-readable program medium.

[0018] In yet another exemplary embodiment, a device configured or operable to perform the above-described method is disclosed. The apparatus may include a processor programmed to implement the method.

[0019] In yet another representative aspect, a video decoding device may implement the methods described herein.

[0020] The above aspects and features of the disclosed technology are explained in more detail in the drawings, specification, and claims. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 illustrates the construction of an exemplary merge candidate list. [Diagram 2] FIG. 2 shows one exemplary spatial candidate location. [Diagram 3]FIG. 3 shows one exemplary spatial merge candidate pair undergoing redundancy checking. [Figure 4A] FIG. 4A illustrates a location of a second prediction unit (PU) based on an example current block size and shape. [Figure 4B] FIG. 4B illustrates a location of a second prediction unit (PU) based on an example current block size and shape. [Diagram 5] FIG. 5 illustrates motion vector scaling for one exemplary temporal merging candidate. [Figure 6] FIG. 6 shows candidate positions for one exemplary temporal merge candidate. [Figure 7] FIG. 7 illustrates the generation of one exemplary combined bi-predictive merge candidate. [Figure 8] FIG. 8 illustrates the construction of one exemplary motion vector prediction candidate. [Figure 9] FIG. 9 illustrates motion vector scaling for one exemplary spatial motion vector candidate. [Figure 10] FIG. 10 illustrates an exemplary motion prediction using an alternative temporal motion vector prediction (ATMVP) algorithm for a coding unit (CU). [Figure 11] FIG. 11 shows a coding unit with one exemplary sub-block and neighboring blocks used by the spatial-temporal motion vector prediction (STMVP) algorithm. [Figure 12] FIG. 12 shows an exemplary flowchart for encoding with different MV precisions. [Figure 13A] FIG. 13A shows an exemplary simplified four-parameter affine model and a simplified six-parameter affine model. [Figure 13B] FIG. 13B shows an exemplary simplified four-parameter affine model and a simplified six-parameter affine model. [Figure 14] FIG. 14 shows one exemplary affine motion vector field (MVF) per sub-block. [Figure 15A] FIG. 15A shows exemplary four-parameter and six-parameter affine models. [Figure 15B] FIG. 15B shows exemplary four-parameter and six-parameter affine models. [Figure 16] FIG. 16 shows motion vector prediction for AF_INTER for one example inherited affine candidate. [Figure 17] FIG. 17 shows motion vector prediction for AF_INTER for one exemplary constructed affine candidate. [Figure 18A] FIG. 18A shows candidates for an exemplary AF_MERGE affine motion mode. [Figure 18B] FIG. 18B shows candidates for an exemplary AF_MERGE affine motion mode. [Figure 19] Figure 19 shows candidate positions for one exemplary affine merge mode. [Figure 20] FIG. 20 shows an example intra-picture block copy. [Figure 21] FIG. 21 shows the valid correspondence regions in one example collocated picture. [Figure 22] FIG. 22 shows the encoding flow for one exemplary history-based motion vector prediction (HMVP) candidate. [Diagram 23] FIG. 23 illustrates one exemplary modified merge list building process. [Figure 24] FIG. 24 shows a flow chart of an exemplary method for video processing. [Diagram 25] FIG. 25 shows a flow chart of another exemplary method for video processing. [Figure 26] FIG. 26 illustrates a flow chart of yet another exemplary method for video processing. [Figure 27] FIG. 27 illustrates a flow chart of yet another exemplary method for video processing. [Figure 28] FIG. 28 is an exemplary block diagram of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document. [Figure 29] FIG. 29 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 30] FIG. 30 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 31] FIG. 31 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 32] FIG. 32 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 33] FIG. 33 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 34] FIG. 34 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 35] FIG. 35 illustrates a flow chart of yet another exemplary method for video processing. [Diagram 36] FIG. 36 illustrates a flow chart of yet another exemplary method for video processing. [Figure 37] FIG. 37 illustrates a flow chart of yet another exemplary method for video processing. [Figure 38] FIG. 38 illustrates a flow chart of yet another exemplary method for video processing. [Figure 39] FIG. 39 shows a flow chart of yet another exemplary method for video processing. [Diagram 40] FIG. 40 shows a flow chart of yet another exemplary method for video processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Due to the increasing demand for higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress or decompress digital video and are continually being improved to provide higher coding efficiency. Video codecs convert uncompressed video into compressed formats and vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). Compression formats often comply with standard video compression specifications, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the finalized Versatile Video Coding standard, or other current and / or future video coding standards.

[0023] Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve readability of the description, and in no way limit the description or embodiments (and / or implementations) to only the respective section.

[0024] In this document, the term "video processing" may refer to video coding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion of a pixel representation of a video to a corresponding bitstream representation, or vice versa.

[0025] 1 Example of inter prediction in HEVC / H.265 Video coding standards have improved significantly over the years and now, in part, provide high coding efficiency and support higher resolution. Recent standards, such as HEVC and H.265, are based on hybrid video coding structures, in which temporal prediction plus transform coding is utilized.

[0026] 1.1 Prediction mode examples Each inter-predicted PU (prediction unit) has motion parameters for one or two reference picture lists. In some embodiments, the motion parameters include a motion vector and a reference picture index. In other embodiments, the use of one of the two reference picture lists may also be signaled using inter_pred_idc. In yet other embodiments, the motion vector may be explicitly coded as a delta to the predictor.

[0027] When a CU is coded in skip mode, one PU is associated with the CU, and there are no significant residual coefficients, coded motion vector deltas, or reference picture indexes. A merge mode is specified, whereby motion parameters for the current PU, including spatial and temporal candidates, are obtained from neighboring PUs. The merge mode can be applied not only to skip mode, but also to any inter-predicted PU. An alternative to the merge mode is explicit transmission of motion parameters, where the motion vectors, the corresponding reference picture index for each reference picture list, and the use of the reference picture list are explicitly signaled for each PU.

[0028] If the signaling indicates that one of the two reference picture lists should be used, the PU is generated from one block of samples. This is referred to as "uni-prediction." Uni-prediction is available for both P slices and B-slices.

[0029] If the signaling indicates that both reference picture lists should be used, the PU is generated from two blocks of samples. This is referred to as "bi-prediction." Bi-prediction is available only for B-slices.

[0030] Reference Picture List

[0031] In HEVC, the term inter prediction refers to a prediction derived from reference picture data elements (e.g., sample values ​​or motion vectors) other than the currently decoded picture. Similar to H.264 / AVC, a picture may be predicted from multiple reference pictures. The reference pictures used for inter prediction are organized in one or more reference picture lists. A reference picture index identifies which of the reference pictures in the list should be used to generate the prediction signal.

[0032] A single reference picture list, List0, is used for P slices, and two reference picture lists, List0 and List1, are used for B slices. The reference pictures included in List0 / List1 can be from past and future pictures with respect to capturing / display.

[0033] 1.1.1 Implementation of Building Candidates for Merge Modes When a PU is predicted using the merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to retrieve the motion information. The construction of this list can be summarized according to the following sequence of steps:

[0034] Step 1: Derive initial candidates Step 1.1: Derive spatial candidates Step 1.2: Redundancy check for spatial candidates Step 1.3: Deriving Temporal Candidates Step 2: Insert additional candidates Step 2.1: Creating bi-predictive candidates Step 2.2: Inserting zero motion candidates

[0035] Figure 1 shows an example of constructing a merge candidate list based on the sequence of steps summarized above. For spatial merge candidate derivation, up to four merge candidates are selected from among candidates located at five different positions. For temporal merge candidate derivation, up to one merge candidate is selected from among two candidates. Since a constant number of candidates is assumed for each PU in the decoder, additional candidates are generated if the number of candidates does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, the index of the best merge candidate is coded using truncated unary binarization (TU). If the size of a CU is equal to 8, all PUs of the current CU share a single merge candidate list, which is identical to the merge candidate list of a 2Nx2N prediction unit.

[0036] 1.1.2 Constructing spatial merging candidates In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates located at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any PU in positions A1, B1, B0, A0 is unavailable (e.g., because it belongs to another slice or tile) or is intra-coded. After the candidate in position A1 is added, the addition of the remaining candidates is subject to a redundancy check that ensures that candidates with identical motion information are removed from the list so that the coding efficiency is improved.

[0037] In order to reduce computational complexity, not all possible candidate pairs are considered in the above redundancy check. Instead, only pairs linked with arrows in FIG. 3 are considered, and candidates are added to the list only if the corresponding candidates used in the redundancy check do not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a partition different from 2N×2N. As an example, FIG. 4A and FIG. 4B show the second PU for the N×2N and 2N×N cases, respectively. If the PU is currently partitioned as N×2N, the candidate A1 is not considered in the list construction. In some embodiments, adding this candidate may lead to two prediction units with identical motion information, which is redundant to having only one PU in the coding unit. Similarly, if the PU is currently partitioned as 2N×, the position B1 is not considered.

[0038] 1.1.3 Building Temporal Merge Candidates In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the co-located PU belonging to the picture with the smallest POC difference with the current picture in a given reference picture list. The reference picture list used for the derivation of the co-located PU is explicitly signaled in the slice header.

[0039] FIG. 5 shows the derivation of a scaled motion vector for one exemplary temporal merge candidate (as a dotted line), which is scaled from the motion vector of the co-located PU using POC distance, tb, and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. For a B-slice, two motion vectors, one for reference picture list 0 and the other for reference picture list 1, are obtained and combined to create a bi-predictive merge candidate.

[0040] For a co-located PU(Y) belonging to the reference frame, the location of the temporal candidate is selected between candidates C0 and C1, as shown in Figure 6. If the PU at location C0 is unavailable, intra-coded, or outside the current CTU, location C1 is used. Otherwise, location C0 is used to derive the temporal merging candidate.

[0041] 1.1.4 Building Additional Types of Merge Candidates Besides the spatial-temporal merge candidates, there are two additional types of merge candidates: combined bi-predictive merge candidates and zero merge candidates. The combined bi-predictive merge candidates are generated by utilizing the spatial-temporal merge candidates. The combined bi-predictive merge candidates are used only for B-slices. The combined bi-predictive candidate is generated by combining the first reference picture list motion parameters of an initial candidate with another second reference picture list motion parameters. If these two tuples provide different motion hypotheses, they form a new bi-predictive candidate.

[0042] Figure 7 shows an example of this process, where two candidates in the original list (710 on the left), with mvL0 and refIdxL0 or mvL1 and refIdxL1, are used to create a combined bi-predictive merge candidate that is added to the final list (720 on the right).

[0043] Zero motion candidates are inserted to fill the remaining entries in the merge candidate list, and thus reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional and bidirectional prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

[0044] In some embodiments, the following steps are performed in order until the merge list is full. (1) Set the variable numRef to either the number of reference pictures associated with list 0 for a P slice, or the minimum number of reference pictures in the two lists for a B slice. (2) Adding non-iterative zero motion candidates For a variable i, 0...numRef-1, add to list 0 (for P slices) or both lists (for B-slices) a default motion candidate with MV set to (0,0) and reference picture index set to i. (3) Add a repeated zero motion candidate with MV set to (0,0), reference picture index of list 0 set to 0 (for P slices), and reference picture index of both lists set to 0 (for B slices).

[0045] 1.2 Advanced Motion Vector Prediction (AMVP) Implementation AMVP exploits the spatial-temporal correlation of motion vectors with neighboring PUs, which is used for the explicit transmission of motion parameters. It first builds a motion vector candidate list by checking the availability of the temporally neighboring PU positions on the left, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similarly, in merge index signaling, the index of the best motion vector candidate is coded using a truncated unary. The maximum value to be coded in this case is 2 (see Figure 8). In the following sections, details on the derivation process of motion vector prediction candidates are provided.

[0046] 1.2.1 Example of AMVP candidate derivation FIG. 8 summarizes the derivation process for motion vector prediction candidates, and can be implemented for each reference picture list using refidx as input.

[0047] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For spatial motion vector candidate derivation, two motion vector candidates are finally derived based on the motion vectors of each PU located at five different positions as previously shown in FIG. 2.

[0048] For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates derived based on two different positions of the same location. After the first list of spatial-temporal candidates is created, duplicate motion vector candidates in the list are removed. When the number of possible candidates is greater than two, motion vector candidates with reference picture indexes greater than 1 in the associated reference picture list are removed from the list. When the number of spatial-temporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0049] 1.2.2 Construction of spatial motion vector candidates In the derivation of spatial motion vector candidates, a maximum of two candidates out of five possible candidates are considered, which are derived from PUs located at positions as shown earlier in FIG. 2, and their positions are the same as those of motion merging. The derivation order for the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order for the top side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Thus, for each side, there are four cases that can be used as motion vector candidates. Two cases do not need to use spatial scaling, and two cases use spatial scaling. The four different cases are summarized as follows: -No spatial scaling (1) Same reference picture list and same reference picture index (same POC) (2) Different reference picture lists, but the same reference picture (same POC) -Spatial Scaling (3) Same reference picture list, but different reference pictures (different POC) (4) Different reference picture lists and different reference pictures (different POC)

[0050] The cases without spatial scaling are checked first, followed by the cases that allow spatial scaling. Spatial scaling is considered when the POC differs between the reference pictures of the neighboring PUs and the reference picture of the current PU, regardless of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, scaling is allowed for the above motion vector to aid in the parallel derivation of the left and above MV candidates. Otherwise, spatial scaling is not allowed for the above motion vector.

[0051] As shown in the example of Figure 9, for spatial scaling, the motion vectors of neighboring PUs are scaled in a similar manner to that of temporal scaling. One difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as that of temporal scaling.

[0052] 1.2.3 Construction of Temporal Motion Vector Candidates Apart from the derivation of the reference picture index, the entire process for the derivation of the temporal merge candidate is the same as that for the derivation of the spatial motion vector candidate (as shown in the example of Figure 6). In some embodiments, the reference picture index is signaled to the decoder.

[0053] 2. Example of Inter-Prediction Method in Joint Exploration Model (JEM) In some embodiments, future video coding techniques are explored using reference software known as the Joint Exploration Model (JEM), which employs subblock-based prediction in several coding tools, including affine prediction, alternative temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), bidirectional optical flow (BIO), frame rate up-conversion (FRUC), locally adaptive motion vector decomposition (LAMVR), overlapped block motion compensation (OBMC), local illumination compensation (LIC), and decoder-side motion vector refinement (DMVR).

[0054] 2.1 Example of sub-CU-based motion vector prediction In JEM with quadtree plus binary tree (QTBT), each CU can have at most one set of motion parameters for each prediction direction. In some embodiments, two sub-CU level motion vector prediction methods are considered in the encoder: by splitting a large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU. With the alternative temporal motion vector prediction (ATMVP) method, each CU can fetch multiple sets of motion information from multiple blocks smaller than the current CU in the co-located reference picture. In the spatial-temporal motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using the temporal motion vector predictor and the spatial neighboring motion vector. In some embodiments, then, motion compression on the reference frame can be disabled to preserve a more accurate motion field for sub-CU motion prediction.

[0055] 2.1.1 Alternative Temporal Motion Vector Prediction (ATMVP) Implementation Example In the ATMVP method, the temporal motion vector prediction (TMVP) method is modified by fetching multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU.

[0056] 10 shows an example ATMVP motion prediction process for a CU 1000. The ATMVP method predicts a motion vector of a sub-CU 1001 in a CU 1000 in two steps. The first step is to identify a corresponding block 1051 in a reference picture 1050 using a temporal vector. The reference picture 1050 is also referred to as a motion source picture. The second step is to split the current CU 1000 into sub-CUs 1001 and obtain the motion vector and reference index of each sub-CU from the block corresponding to each sub-CU.

[0057] In the first step, the reference picture 1050 and corresponding blocks are determined by the motion information of the spatial neighboring blocks of the current CU 1000. To avoid repeated scanning process of the neighboring blocks, the first merging candidate in the merging candidate list of the current CU 1000 is used. The first available motion vector and the associated reference picture index are set to be the temporal vector and the index to the motion source picture. In this way, the corresponding blocks can be more accurately identified and compared with the TMVP. Here, the corresponding blocks (sometimes called co-located blocks) are always in the bottom-right or center position with respect to the current CU.

[0058] In the second step, the corresponding block of the sub-CU 1051 is identified by the temporal vector in the motion source picture 1050 by adding the temporal vector to the coordinates of the current CU. For each sub-CU, the motion information of the corresponding block (e.g., the smallest motion grid covering the center sample) is used to derive the motion information for the sub-CU. After the motion information of the corresponding N×N block is identified, it is converted to the motion vector and reference index of the current sub-CU in the same manner as TMVP in HEVC. Here, motion scaling and other procedures are applied. For example, the decoder checks whether a low-delay condition (e.g., the POC of all reference pictures of the current picture is smaller than the POC of the current picture) is met, and then possibly uses the motion vector MVx (e.g., the motion vector corresponding to the reference picture list X) to predict the motion vector MVy for each sub-CU (e.g., X is equal to 0 or 1, and Y is equal to 1-X).

[0059] 2.1.2 Example of Spatial-Temporal Motion Vector Prediction (STMVP) In the STMVP method, the motion vectors of sub-CUs are derived recursively according to the raster scan order. Figure 11 shows an example of one CU with four sub-blocks and neighboring blocks. Assume an 8x8 CU 1100 that contains four 4x4 sub-CUs A (1101), B (1102), C (1103), and D (1104). The neighboring 4x4 blocks in the current frame are labeled as a (1111), b (1112), c (1113), and d (1114).

[0060] For sub-CU A, motion derivation starts by identifying its two spatial neighbors. The first neighbor is an N×N block (block c 1113) above sub-CU A 1101. If this block c (1113) is not available or is intra-coded, other N×N blocks above sub-CU A (1101) are checked (starting from block c 1113, left to right). The second neighbor is a block (block b 1112) to the left of sub-CU A 1101. If block B (1112) is not available or is intra-coded, other blocks to the left of sub-CU A 1101 are checked (starting from block b 1112, top to bottom). The motion information obtained from the neighboring blocks for each list is scaled with respect to the first reference frame for a given list. Then, the temporal motion vector predictor (TMVP) of sub-block A 1101 is derived by following the same TMVP derivation procedure as specified in HEVC. The motion information of the co-located block in block D1104 is fetched and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors are averaged for each reference list separately. The averaged motion vector is assigned as the motion vector of the current sub-CU.

[0061] 2.1.3 Example of Sub-CU Motion Prediction Mode Signaling In some embodiments, the sub-CU modes are enabled as additional merge candidates, and no additional syntax elements are required to signal the modes. Two additional merge candidates are added to the merge candidate list of each CU to represent ATMVP and STMVP modes. In other embodiments, up to seven merge candidates may be used when the sequence parameter set indicates that ATMVP and STMVP are enabled. The coding logic of the additional merge candidates is the same as that of the merge candidates in HM. This means that for each CU in a P slice or B slice, two more RD checks may be required for the two additional merge candidates. In some embodiments, such as JEM, all bins of the merge index are context coded by Context-based Adaptive Binary Arithmetic Coding (CABAC). In other embodiments, such as HEVC, only the first bin is context coded, and the remaining bins are context bypass coded.

[0062] 2.2 Example of Inter Prediction Method in VVC There are several new coding tools for improving inter prediction: Adaptive Motion Vector Differential Decomposition (AMVR) for signaling MVD, Affine Prediction Mode, Triangular Prediction Mode (TPM), ATMVP, Generalized Bi-Prediction (GBI), Bidirectional Optical Flow (BIO).

[0063] 2.2.1 Example of coding block structure in VVC In VVC, a QuadTree / BinaryTree / MultipleTree (QT / BT / TT) structure is adopted to divide a picture into square or rectangular blocks. Besides QT / BT / TT, a separate tree (aka Dual coding tree) is also adopted in VVC for I-frames. In the separate trees, the coding block structures are signaled separately for the luma and chroma components.

[0064] 2.2.2 Example of Adaptive Motion Vector Difference Decomposition In some embodiments, when use_integer_mv_flag is equal to 0 in the slice header, the motion vector difference (MVD) (between the motion vector and the predicted motion vector of the PU) is signaled in units of 1 / 4 luma sample. Locally Adaptive Motion Vector Decomposition (LAMVR) is introduced in JEM. In JEM, MVD can be coded in units of 1 / 4 luma sample, integer luma sample, or 4 luma sample. MVD decomposition is controlled at the coding unit (CU) level, and the MVD decomposition flag is conditionally signaled for each CU that has at least one non-zero MVD component.

[0065] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether 1 / 4 luma sample MV precision is used in the CU. If the first flag (equal to 1) indicates that 1 / 4 luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or 4 luma sample MV precision is used.

[0066] If the first MVD decomposition flag of a CU is zero or not coded for the CU (meaning all MVDs in the CU are zero), then quarter luma sample MV decomposition is used for the CU. If the CU uses integer luma sample MV precision or 4 luma sample MV precision, the MVP in the AMVP candidate list for the CU is rounded to the corresponding precision.

[0067] In the encoder, a CU-level RD check is used to determine which MVD decomposition should be used for a CU. That is, the CU-level RD check is performed three times for each MVD decomposition. To accelerate the encoder speed, the following encoder scheme is applied in JEM.

[0068] During RD check of a CU with normal 1 / 4 luma sample MVD decomposition, the motion information of the current CU (with integer luma sample precision) is stored. The stored motion information (after rounding) is used as a starting point to refine the motion vectors in a smaller range during RD check for the same CU with integer luma sample and 4 luma sample MVD decomposition, so that the time-consuming motion estimation process is not duplicated three times.

[0069] The RD check for a CU with -4 luma sample MVD decomposition is conditionally invoked. If the RD cost of integer luma sample MVD decomposition for a CU is much larger than that of 1 / 4 luma sample MVD decomposition, the RD check of 4 luma sample MVD decomposition for the CU is skipped.

[0070] The symbolization process is shown in FIG. 12. First, the 1 / 4 pel MV is tested, and the RD cost is calculated and shown as RDCost0. Then, the integer MV is tested, and the RD cost is shown as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive threshold value), the 4-pel MV is tested; otherwise, the 4-pel MV is skipped. Basically, when checking the integer or 4-pel MV, the motion information and RD cost, etc. for the 1 / 4 pel MV are already known, and it can be reused to speed up the encoding process of the integer or 4 pel MV.

[0071] 2.2.3 Examples of Affine Motion Compensation Prediction In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, cameras and objects can have many types of motions. For example, zoom in / out, rotation, perspective motions, and / or other irregular motions. In VVC, simplified affine transform motion compensation prediction is applied using the 4-parameter affine model and the 6-parameter affine model. As shown in FIGS. 13A and 13B, the affine motion field of a block is described by two control point motion vectors (in the 4-parameter affine model using variables a, b, e, and f), or three control point motion vectors (in the 6-parameter affine model using variables a, b, c, d, e, and f), respectively.

[0072] The motion vector field (MVF) of a block is described by the following equations using the 4-parameter affine model and the 6-parameter affine model, respectively.

[0073]

Equation

[0074]

number

[0075] Here, (mv h 0,mv h 0) is the motion vector of the control point (CP) in the upper left corner, and (mv h 1,mv h 1) is the motion vector of the control point in the upper right corner, and (mv h 2,mv h 2) is the motion vector of the control point of the lower left corner. (x,y) represents the coordinates of the representative point for the top-left sample in the current block. The CP motion vector is signaled (as in affine AMVP mode) or derived on-the-fly (as in affine merge mode). w and h are the width and height of the current block. In practice, the division is implemented by a right shift with a rounding operation. In one embodiment, the representative point is defined to be the center position of the subblock. For example, if the coordinates of the top-left corner of the subblock for the top-left sample in the current block are (xs,ys), then the coordinates of the representative point are defined to be (xs+2,ys+2). For each subblock (e.g., 4x4), the representative point is used to derive a motion vector for the entire subblock.

[0076] FIG. 14 shows an example of affine MVF per subblock for block 1400, where subblock-based affine transformation prediction is applied to further simplify motion compensation prediction. To derive the motion vector of each M×N subblock, the motion vector of the center sample of each subblock is calculated according to equations (1) and (2), and may be rounded to a fraction of the motion vector precision (e.g., 1 / 16 in JEM). Then, a motion compensation interpolation filter may be applied to generate a prediction of each subblock using the derived motion vector. A 1 / 16-pel interpolation filter is introduced by the affine mode. After MCP, the high-precision motion vector of each subblock is rounded and stored with the same precision as the normal motion vector.

[0077] 2.2.4 Example of signaling for affine prediction Similar to the translational motion model, there are also two modes for signaling side information with affine prediction: AFFINE_INTER and AFFINE_MERGE modes.

[0078] 2.2.5 Example of AF_INTER mode The AF_INTER mode may be applied for CUs whose width and height are both greater than 8. An affine flag at the CU level is signaled in the bitstream to indicate whether the AF_INTER mode is used or not.

[0079] In this mode, for each reference picture list (List0 or List1), an affine AMVP candidate list is built with three types of affine motion predictors in the following order, where each candidate contains the estimated CPMV of the current block. The difference between the best CPMV found at the encoder side (such as mv0mv1mv2 in Figure 17) and the estimated CPMV is signaled. In addition, the index of the affine AMVP candidate from which the estimated CPMV is derived is further signaled. 1) Inherited Affine Motion Predictor The check order is similar to that of spatial MVP in HEVC AMVP list construction. First, the left inherited affine motion predictor is derived from the first block in {A1, A0} that is affine coded and has the same reference picture as the current block. Second, the above inherited affine motion predictor is derived from the first block in {B1, B0, B2} that is affine coded and has the same reference picture as the current block. Five blocks A1, A0, B1, B0, B2 are shown in Figure 16. Once it is known that the neighboring blocks are coded in affine mode, the CPMV of the coding unit covering the neighboring blocks is used to derive the predictor of the CPMV of the current block. For example, if A1 is coded in non-affine mode and A0 is coded in 4-parameter affine mode, the left inherited affine MV predictor is derived from A0. In this case, the CPMV of the CU covering A0 is MV0 for the top-left CPMV in Figure 18B. N , and MV1 for the CPMV in the upper right N The estimated CPMV of the current block is derived as shown by the MV0 for the top-left (having coordinates (x0, y0)), top-right (having coordinates (x1, y1)), and bottom-right (having coordinates (x2, y2)) locations of the current block. C , MV1 C , MV2 C , as shown by: 2) The constructed affine motion predictor The constructed affine motion predictor is composed of control point motion vectors (CPMVs) derived from adjacent inter coded blocks with the same reference picture, as shown in FIG. 17. If the current affine motion model is 4-parameter affine, the number of CPMVs is 2, otherwise if the current affine motion model is 6-parameter affine, the number of CPMVs is 3. The top left CPMV mv0 is inter coded and derived by the MV at the first block in group {A,B,C} with the same reference picture as the current block. The top right CPMV mv1 is inter coded and derived by the MV at the first block in group {D,E} with the same reference picture as the current block. The bottom left CPMV mv2 is inter coded and derived by the MV at the first block in group {F,G} with the same reference picture as the current block.

[0080] - If the current affine motion model is 4-parameter affine, the constructed affine motion predictor is inserted into the candidate list only if both mv0 and mv1 are established, i.e., mv0 and mv1 are used as estimated CPMVs for the top-left (with coordinates (x0, y0)), top-right (with coordinates (x1, y1)) positions of the current block.

[0081] - If the current affine motion model is 6-parameter affine, the constructed affine motion predictor is inserted into the candidate list only if all of mv0, mv1, and mv2 are established, i.e., mv0, mv1, and mv2 are used as estimated CPMVs for the top-left (with coordinates (x0,y0)), top-right (with coordinates (x1,y1)), and bottom-right (with coordinates (x2,y2)) positions of the current block. No pruning process is applied when inserting the constructed affine motion predictor into the candidate list. 3) Normal AMVP motion predictor The following is applied until the number of affine motion predictors reaches a maximum:

[0082] 1) If possible, derive an affine motion predictor by setting all CPMVs equal to mv̂2.

[0083] 2) If possible, derive affine motion predictors by setting all CPMVs equal to mv1.

[0084] 3) If possible, derive affine motion predictors by setting all CPMVs equal to mv0.

[0085] 4) If possible, derive affine motion predictors by setting all CPMVs equal to HEVC TMVP.

[0086] 5) Derive the affine motion predictor by setting all CPMVs to zero MV. mv i Note that {overscore (x)} is already derived in the constructed affine motion predictor.

[0087] In AF_INTER mode, when the 4 / 6 parameter affine mode is used, 2 / 3 control points are needed, and therefore 2 / 3 MVDs need to be coded for these control points, as shown in Figures 15A and 15B. In the existing implementation, the MVs can be derived as follows: for example, predict mvd1 and mvd2 from mvd0.

[0088] mv0=mv0￣+mvd0 mv1=mv1￣+mvd1+mvd0 mv2=mv2￣+mvd2+mvd0

[0089] Here, mv i ￣、mvd i , and mv imvA(xA,yA) and mvB(xB,yB) are the predicted motion vector, motion vector difference, and motion vector of the top-left pixel (i=0), top-right pixel (i=1), or bottom-left pixel (i=2), respectively, as shown in Figure 15B. In some embodiments, the addition of two motion vectors (e.g., mvA(xA,yA) and mvB(xB,yB)) is equal to the separate sum of the two components. For example, newMV=mvA+mvB means that the two components of newMV are set to (xA+xB) and (yA+yB), respectively.

[0090] 2.2.6 Example of AF_MERGE Mode When a CU is applied in AF_MERGE mode, obtain the first block coded in affine mode from the valid neighboring reconstructed blocks. And the selection order of the candidate blocks is from left, above, above right, left bottom to above left, as shown in FIG. 18A (shown in the order of A, B, C, D, E). For example, if the neighboring bottom-left block is coded in affine mode, as shown by A0 in FIG. 18B, the control point (CP) motion vector mv0 for the top-left corner, top-right corner, and bottom-left corner of the neighboring CU / PU containing block A is selected. N , mv1 N , mv2 N Then, the upper left corner / upper right / lower left motion vector mv0 in the current CU / PU is fetched. C , mv1 C , mv2 C (Only used for 6-parameter affine models) but mv0 N , mv1 N , mv2 NIt should be noted that the sub-block located in the upper left corner (e.g. 4x4 block) stores mv0 and the sub-block located in the upper right corner stores mv1 if the current block is affine coded. If the current block is coded using a 6-parameter affine model, the sub-block located in the lower left corner stores mv2, otherwise (with a 4-parameter affine model) the LB stores mv2'. The other sub-blocks store the MV used for MC.

[0091] The CPMVs of current CUs v0 and v1 are calculated according to the affine motion model in Equation (1) and (2), and the MVF of the current CU may be generated according to Equation (1). To identify whether the current CU is coded using the AF_MERGE mode, an affine flag may be signaled in the bitstream if at least one neighboring block is coded in an affine mode.

[0092] In some embodiments, the affine merge candidate list may be constructed using the following steps.

[0093] 1) Insertion of inherited affine candidates

[0094] An inherited affine candidate means that the candidate is derived from the affine motion models of its valid neighboring affine coded blocks. As shown in Figure 19, in the common base, the scan order for the candidate positions is A1, B1, B0, A0, and B2.

[0095] After a candidate is derived, a full pruning process is performed to check if the same candidate is inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0096] 2) Insertion of constructed affine candidates

[0097] If the number of candidates in the affine merge candidate list is less than MaxNumAffineCanD (which is set to 5 in this contribution), a constructed affine candidate is inserted into the candidate list. This means the constructed affine candidate and the candidate constructed by combining the neighboring motion information of each control point.

[0098] The motion information for a control point is first derived from the particular spatial and temporal neighborhood shown in Figure 19. CPk (k=1,2,3,4) represents the kth control point. A0, A1, A2, B0, B1, B2, and B3 are the spatial positions for predicting CPk (k=1,2,3), and T is the temporal position for predicting CP4.

[0099] The coordinates of CP1, CP2, CP3, and CP4 are (0,0), (W,0), (H,0), and (W,H), respectively, where W and H are the width and height of the current block.

[0100] The motion information of each control point is obtained according to the following priorities:

[0101] For CP1, the check priority is B2→B3→A2. If available, B2 is used. Otherwise, if B2 is unavailable, B3 is used. If both B2 and B3 are unavailable, A2 is used. If all three candidates are unavailable, the motion information of CP1 cannot be obtained.

[0102] For CP2, the check priority is B1→B0.

[0103] For CP3, the check priority is A1 → A0.

[0104] For CP4, T is used.

[0105] Second, a combination of the control points is used to construct an affine merge candidate.

[0106] To construct a 6-parameter affine candidate, we need motion information of three control points. The three control points can be selected from one of the following four combinations: {CP1,CP2,CP4}, {CP1,CP2,CP3}, {CP2,CP3,CP4}, {CP1,CP3,CP4}. The combinations {CP1,CP2,CP3}, {CP2,CP3,CP4}, {CP1,CP3,CP4} are transformed into a 6-parameter motion model represented by the top-left, top-right, and bottom-left control points.

[0107] To construct a four-parameter affine candidate, motion information of two control points is required. The two control points can be selected from one of the following six combinations: {CP1,CP4}, {CP2,CP3}, {CP1,CP2}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4}. The combinations {CP1, CP4}, {CP2,CP3}, {CP2,CP4}, {CP1,CP3}, {CP3 CP4} is converted into a four-parameter motion model represented by the top-left and top-right control points.

[0108] The combinations of constructed affine candidates are inserted into the candidate list in the following order:

[0109] {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}}

[0110] For a combination of reference list X (X is 0 or 1), the reference index with the highest usage in the control points is selected as the reference index of list X, and then the motion vector points for different reference pictures are scaled.

[0111] After a candidate is derived, a full pruning process is performed to check if the same candidate is inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0112] 3) Motion vector zero padding

[0113] If the number of candidates in the affine merge candidate list is less than five, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.

[0114] In some embodiments, for the subblock merging candidate list, the four-parameter merging candidate has the MV set to (0,0) and the prediction direction set to uni-prediction (for P slices) and bi-prediction (for B slices) from list 0.

[0115] 2.3 Example of Intra Block Copy In the HEVC Screen Content Coding Extension (SCC), Intra Block Copy (IBC, or Intra Picture Block Compensation), also called Current Picture Referencing (CPR), was adopted. This tool is very efficient for coding screen content video in that repeated patterns of text and graphics-rich content occur frequently in the same picture. Having a previously reconstructed block with the same or similar pattern as a predictor can effectively reduce the prediction error and thus improve the coding efficiency. One example of intra block compensation is shown in Figure 20.

[0116] Similar to the design of the CRP in the HEVC SCC, in VVC, the use of IBC mode is signaled at both the sequence and picture levels. When IBC mode is enabled in the sequence parameter set (SPS), it can be enabled at the picture level. When IBC mode is enabled at the picture level, the current reconstructed picture is treated as a reference picture. Therefore, no block-level syntax change is required on top of the existing VVC inter mode to signal the use of IBC mode.

[0117] Some of the features of CPR include:

[0118] ·It is treated as normal inter mode. Therefore, merge and skip modes are also available for IBC mode. The merge candidate list construction is unified, including merge candidates from neighboring positions coded in either IBC mode or HEVC inter mode. Depending on the selected merge index, the current block under merge or skip mode can be merged either with a neighbor coded in IBC mode or with one coded in normal inter mode using a different picture as a reference picture otherwise.

[0119] The block vector prediction and coding scheme for IBC mode reuses the scheme used for motion vector prediction and coding in HEVC Inter mode.

[0120] · Motion vectors in IBC mode, also referred to as block vectors, are coded with integer-pel precision, but are stored in memory with 1 / 16-pel precision after decoding, since the interpolation and deblocking stages require 1 / 4-pel precision. When used in motion vector prediction for IBC mode, the stored vector predictor is right-shifted by 4.

[0121] · Search Range: It is currently limited to being within the CTU.

[0122] · CPR is not allowed when affine mode / triangular mode / GBI weighted prediction is enabled.

[0123] 2.3.1 Harmonization with CPR In PCT / CN2018 / 089920, several methods are proposed to harmonize the CPR (aka IBC) with some of the new coding tools.

[0124] 2.4 Example of Merge List Design in VVC There are three different merge list building processes supported in VVC:

[0125] (1) Sub-block merge candidate list: It contains ATMVP and affine merge candidates. One merge list construction process is shared for both affine and ATMVP modes, where ATMVP and affine merge candidates can be added in order. The size of the sub-block merge list is signaled in the slice header, and the maximum value is 5.

[0126] (2) Uni-Prediction TPM Merge List: For triangular prediction mode, one merge list construction process for two partitions is shared even though the two partitions can select their own merge candidate indexes. When constructing this merge list, the spatial neighboring blocks and two temporal blocks of the block are checked. The motion information derived from the spatial neighboring and temporal blocks is called regular motion candidates in IDF. These regular motion candidates are further utilized to derive multiple TPM candidates. Note that the transformation is performed at the whole block level even though the two partitions can use different motion vectors to generate their own prediction blocks. In some embodiments, the size of the uni-prediction TPM merge list is fixed at 5.

[0127] (3) Regular merge list: For the remaining coding blocks, one merge list construction process is shared, where spatial / temporal / HMVP, pairwise combined bi-predictive merge candidates and zero motion candidates can be inserted in order. The regular merge list size is signaled in the slice header, and the maximum value is 6.

[0128] Subblock Merge Candidate List

[0129] It is suggested that all sub-block related motion candidates are placed in a separate merge list in addition to the regular merge list for non-sub-block merge candidates.

[0130] The sub-block related motion candidates are placed in a separate merge list named as the "sub-block merge candidate list."

[0131] In one example, the sub-block merging candidate list includes affine merging candidates, and ATMVP candidates and / or sub-block based STMVP candidates.

[0132] In another example, the ATMVP merge candidate in the regular merge list is moved to the first position of the affine merge list such that all merge candidates in the new list (i.e., the sub-block based merge candidate list) are based on sub-block coding tools.

[0133] In yet another example, a special merge candidate list known as the sub-block merge candidate list (aka affine merge candidate list) is added in addition to the regular merge candidate list. The sub-block merge candidate list is populated with candidates in the following order:

[0134] (a) ATMVP candidates (which may or may not be available)

[0135] (b) Inheritance affine candidates

[0136] (c) Constructing affine candidates

[0137] (d) Padded as a zero MV 4-parameter affine model

[0138] The maximum number of candidates in the sub-block merging candidate list (denoted as ML) is derived as follows.

[0139] (1) If the ATMVP usage flag (e.g., a flag that may be named as "sps_stmvp_enabled_flag") is on (equal to 1) but the affine usage flag (e.g., a flag that may be named as "sps_affine_enabled_flag") is off (equal to 0), then ML is set equal to 1.

[0140] (2) If the ATMVP use flag is off (equal to 0) and the affine use flag is off (equal to 0), then ML is set equal to 0. In this case, the sub-block merge candidate list is not used.

[0141] (3) Otherwise (the use affine flag is on (equal to 1) and the use ATMVP flag is on or off), ML is signaled from the encoder to the decoder. Valid ML is 0≦ML≦5.

[0142] When creating the sub-block merging candidate list, the ATMVP candidate is checked first. If any one of the following conditions is true, the ATMVP candidate is skipped and not placed into the sub-block merging candidate list:

[0143] (1) The ATMVP usage flag is off.

[0144] (2) Any TMVP usage flag (e.g., a flag that may be named "slice_temporal_mvp_enabled_flag" when signaled at the slice level) is off.

[0145] (3) The reference picture with reference index 0 in reference list 0 is the same as the current picture (it is CPR).

[0146] The ATMVP in the above example is much simpler than the JEM. When an ATMVP merge candidate is generated, the following process is applied:

[0147] (a) Check adjacent blocks A1, B1, B0, A0 in the order shown in FIG. 2 to find a first Inter-coded block, denoted as block X, which is not a CPR-coded block.

[0148] (b) Initialize TMV = (0,0). If an MV for block X (denoted as MVn) exists, then TMV is set equal to MVn, with reference to the located reference picture (as signaled in the slice header).

[0149] (c) Assuming the center point of the current block is (x0,y0), locate the corresponding position of (x0,y0) as M=(x0+MV*x,y0+MV*y) in the collocated picture. Find a block Z that covers M.

[0150] (i) If Z is intra-coded, then ATMVP is not available.

[0151] (ii) If Z is inter-coded, then MVZ_0 and MVZ_1 for the two lists of block Z are scaled to (Reflist0 index0) and (Reflist1 index1) as MVdefault0, MVdefault1, and then stored.

[0152] (d) For each 8x8 sub-block, assume its center point is (x0S,y0S), locate the corresponding position of (x0S,y0S) as MS=(x0S+MV*x,y0S+MV*y) in the aligned picture. Find the block ZS that covers MS.

[0153] (i) If ZS is intra-coded, MVdefault0 and MVdefault1 are assigned to the sub-block.

[0154] (ii) If ZS is inter-coded, MVZS_0 and MVZS_1 for the two lists of block ZS are scaled to (Reflist0 index0) and (Reflist1 index0) and then assigned to the sub-blocks.

[0155] MV Clipping and Masking in ATMVP: When locating the corresponding positions such as M or MS within the configured picture, it is clipped to be inside the pre-defined area. The CTU size is S×S, and in this embodiment, S = 128. Assuming that the upper-left position of the placed CTU is (xCTU, yCTU), the corresponding position M or MS at (xN, yN) is clipped to the valid area xCTU ≦ xN < xCTU + S + 4; yCTU ≦ yN < yCTU + S. Figure 21 shows an example of the valid area.

[0156] In addition to clipping, (xN, yN) is also masked as xN = xN & MASK, yN = yN & MASK. Here, MASK is an integer equal to ~(2 N - 1), and N = 3, with the lower 3 bits set to 0. Thus, xN and yN need to be multiples of 8. (「~」 represents the bitwise complement operator.)

[0157] Regular merge list

[0158] Different from the merge list design, in VVC, the history-based motion vector prediction (HMVP) method is adopted. In some embodiments, the combined bi-prediction merge candidates are replaced by bi-prediction merge candidates in pairs.

[0159] HMVP

[0160] In HMVP, previously coded motion information is stored. The motion information of previously coded blocks is defined as HMVP candidates. Multiple HMVP candidates are stored in a table, named as HMVP table, and this table is maintained on-the-fly during the encoding / decoding process. The HMVP table is emptied when starting a new slice of encoding / decoding. Whenever an inter-coded block exists, the associated motion information is added to the last entry of the table as a new HMVP candidate. The overall coding flow is shown in Figure 22.

[0161] HMVP candidates can be used in both AMVP and merge candidate list construction processes. Figure 23 shows the modified merge candidate list construction process (highlighted in gray). If the merge candidate list is not full after the insertion of the TMVP candidate, the HMVP candidates stored in the HMVP table can be used to fill the merge candidate list. Considering that a block usually has a high correlation with its nearest neighbor block in terms of motion information, the HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first, while the first entry is added to the end. Similarly, redundancy elimination is applied for the HMVP candidates. Once the total number of available merge candidates reaches the maximum number of merge candidates allowed to be signaled, the merge candidate list construction process is terminated.

[0162] 2.5 Example of Generalized Bi-Prediction (GBi) Improvements In one embodiment, a proposed Generalized Bi-prediction improvement (GBi) is provided.

[0163] In one example, GBi is proposed. An improved embodiment with improved gain-complexity tradeoff for GBi is provided, which applies unequal weighting from L0 and L1 to the predictor in bi-prediction mode. In inter-prediction mode, multiple weighting pairs including equal weighted pair (1 / 2,1 / 2) are evaluated based on rate-distortion optimization (RDO), and the GBi index of the selected weighting pair is signaled to the decoder. In merge mode, the GBi index is inherited from the neighboring CU. In this embodiment, the predictor generation in bi-prediction mode is shown in the following equation:

[0164] PGBi=(w0*PL0+w1*PL1+RoundingOffsetGBi)>>ShitNumGBi

[0165] Here, PGBi is the final predictor of GBi. w0 and w1 are the selected GBi weighting pairs and are applied to the predictors in list0(L0) and list(L1), respectively. RoundingOffsetGBi and shiftNumGBi are used to normalize the final predictor in GBi. The supported w1 weighting set is {-1 / 4, 3 / 8, 1 / 2, 5 / 8, 5 / 4}, with five weights corresponding to one equal weighting pair and four unequal weighting pairs. The blending gain, i.e., the sum of w1 and w0, is fixed to 1.0. Thus, the corresponding w0 weighting set is {5 / 4, 5 / 8, 1 / 2, 3 / 8, -1 / 4}. The weighting pair selection is at the CU level.

[0166] For non-low delay pictures, the weighting set size is reduced from 5 to 3, where the w1 weighting set is {3 / 8, 1 / 2, 5 / 8} and the w0 weighting set is {5 / 8, 1 / 2, 3 / 8}. The weighting set size reduction for non-low delay pictures is applied to GBi and all GBi tests in this contribution.

[0167] In one embodiment, a combined solution is proposed to further improve the GBi performance. Specifically, the following modifications are applied on top of the existing GBi design in the embodiment:

[0168] GBi encoder bug fix

[0169] To reduce the GBi encoding time, in the current encoder design, the encoder stores the uni-predictive motion vectors estimated from the GBi weighting equal to 4 / 8 and reuses them for the uni-predictive search of other GBi weightings. This fast encoding method is applied to both translational and affine motion models. In one example, a 6-parameter affine model is adopted together with a 4-parameter affine model. The encoder does not distinguish between the 4-parameter affine model and the 6-parameter affine model when storing the uni-predictive affine MV when the GBi weighting is equal to 4 / 8. As a result, the 4-parameter affine MV may be overwritten by the 6-parameter affine MV after encoding with the GBi weighting 4 / 8. The stored 6-parameter affine MV may be used for the 4-parameter affine ME for other GBi weightings, or the stored 4-parameter affine MV may be used for the 6-parameter affine ME. The proposed bug fix for the GBi encoder is to separate the storage of the 4-parameter and 6-parameter affine MVs. The encoder stores those affine MVs based on the affine model type if the GBi weighting is equal to 4 / 8, and reuses the corresponding affine MVs based on the affine model type for other GBi weightings.

[0170] CU size constraints for GBi

[0171] In this method, GBi is disabled for small CUs. In inter prediction mode, when bi-prediction is used and the CU region is smaller than 128 luma samples, GBi is disabled without signaling.

[0172] Merge mode with GBi

[0173] In merge mode, the GBi index is not signaled. Instead, it is inherited from the neighboring block to which it is merged. When a TMVP candidate is selected, GBi is turned off in this block.

[0174] Affine prediction using GBi

[0175] GBi may be used if the current block is coded using affine prediction. For affine inter mode, the GBi index is signaled. For affine merge mode, the GBi index is inherited from the neighboring block to merge into. When the constructed affine model is selected, GBi is turned off in this block.

[0176] 2.6 Example of Inter-Intra Prediction (IIP) Mode In inter-intra prediction mode, multi-hypothesis prediction combines one intra prediction and one merge index prediction. Such blocks are treated as special inter-coded blocks. In merge CU, one flag is signaled for merge mode to select an intra mode from the intra candidate list when the flag is true. For luma component, the intra candidate list is derived from four intra prediction modes including DC, planar, horizontal, and vertical modes, and the size of the intra candidate list can be 3 or 4 depending on the shape of the block. If the CU width is greater than twice the CU height, the horizontal mode is removed from the list of intra modes, and if the CU height is greater than twice the CU width, the vertical mode is removed from the list of intra modes. One intra prediction mode selected by the intra mode index and one merge index prediction selected by the merge index are combined using a weighted average. For the chroma components, the DM can Always applied.

[0177] The weighting for combining predictions is described as follows: If DC or planar mode is selected, or if the CB width or height is smaller than 4, equal weighting is applied. For those CBs whose CB width and height are equal to or greater than 4, if horizontal / vertical mode is selected, one CB is first divided vertically / horizontally into four equal-area regions. Each set of weightings, denoted as (w_intra,w_inter), is applied to the corresponding region, where i runs from 1 to 4, and (w_intrA1,w_inter1)=(6,2), (w_intra2,w_inter2)=(5,3), (w_intra3,w_inter3)=(3,5), and (w_intra4,w_inter4)=(2,6). (w_intra1,w_inter1) are for the region closest to the reference sample, and (w_intra4,w_inter4) are for the region farthest from the reference sample. A joint prediction may then be calculated by summing the two weighted predictions and right-shifting 3 bits. Furthermore, the intra-prediction mode for the intra-hypothesis of the predictor may be stored for subsequent reference of neighboring CUs.

[0178] Detailed design of one embodiment 1. inter_intra_flag (one new context) is signaled if all conditions are satisfied.

[0179] a. Merge but don't skip.

[0180] b. It is not affine.

[0181] c. Not MMVD.

[0182] dW*H<64 and W!=128 and H!=128 (meaning we can have 2*8 chroma intra prediction here) 2. Inter-intra prediction can be applied in triangular prediction.

[0183] 3. Inter-intra prediction is applied to Y, Cb, and Cr.

[0184] 4. If inter_intra_flag is on, the intra prediction mode (IPM) is signaled. IPM must be one of {DC, Planar, Ver, Hor}.

[0185] a. Three MPMs are included.

[0186] If bW>2*H or H>2*W then IPM must be one of the three MPMs and MPM-flag is not signaled, else MPM-flag is signaled (one new context, why not share the same context as Intra?).

[0187] i.MPM_flag, when on, signals the selected MPM. 0,10,11 represent the 3 MPMs (bypass).

[0188] ii. Otherwise, find one in {DC, Planar, Ver, Hor} that is not in the MPM list as the IPM.

[0189] c. Configuration of MPM list (different from intra-MPM list) Set i.LeftIPM=AboveIPM=DC.

[0190] ii. If the left neighboring block (A2) is intra-coded or inter-intra-coded, then set leftIPM to the IPM of the left neighboring block.

[0191] iii. If the above neighboring block (B3) is intra-coded or inter-intra-coded, then set aboveIPM to the IPM of the above neighboring block.

[0192] iv.leftIPM=(leftIPM>DC_IDX)?((leftIPM≦DIA_IDX)?HOR_IDX:VER_IDX):leftIPM; v.IPM=(IPM>DC_IDX)?((aboveIPM≦DIA_IDX)?HOR_IDX:VER_IDX):aboveIPM; (What about wide angle mode?) vi.leftIPM==aboveIPM 1. If the Left IPM is not DC or planar a.MPM[0]=leftIPM, MPM[1]=Planar, MPM[2]=DC 2.Otherwise a.MPM[0]=DC, MPM[1]=Planar, MPM[2]=Ver vii.Otherwise 1.MPM[0]=leftIPM, MPM[1]=aboveIPM 2.If leftIPM and aboveIPM are not planar, MPM[2]=Planar 3. Otherwise, MPM[2]=(leftIPM+aboveIPM)<2?Ver:DC viii.When W>2×H 1. If MPM[x]==Hor, find ModeX in {DC,Planar,Ver,Hor} instead of the MPM list, and set MPM[x]=ModeX.

[0193] ix. If H>2*W 1. If MPM[x]==Ver, find ModeX in {DC,Planar,Ver,Hor} instead of the MPM list, and set MPM[x]=ModeX.

[0194] 5. Note that the IPM used by inter-intra prediction cannot be used to predict the IPM of an intra-coded block.

[0195] a. An internal encoding block that processes an internal encoding block as an internal encoding block 6. Filtering of adjacent samples: For intra prediction: a. Do not filter for DC / VER / HOR.

[0196] b. If W*H > 32, filter for planar.

[0197] c. PDPC is retained.

[0198] 7. Weighted sum method a. P(x,y) = Clip((W0(x,y)*P_Intra(x,y)+W1(x,y)*P_Inter(x,y))>>3) 8. Weight values (which can be improved or simplified) a. If mode == DC / Planar, ||W < 4||H < 4: W0 = W1 = 4; b. If mode == HOR: i. If x < W / 4, W0 = 6, W1 = 2; ii. If W / 4 ≤ x < W / 2, W0 = 5, W1 = 3; iii. If W / 2 ≤ x < 3*W / 4, W0 = 3, W1 = 5; iv. If x ≥ 3×W / 4, W0 = 2, W1 = 6; c. If mode == VER: i. If y < H / 4, W0 = 6, W1 = 2; ii. If H / 4 ≤ y < H / 2, W0 = 5, W1 = 3; iii. If H / 2 ≤ y < 3×H / 4, W0 = 3, W1 = 5; iv. If y ≥ 3×H / 4, W0 = 2, W1 = 6

[0199] 2.7 Examples of the ultimate motion vector representation (UMVE) In some embodiments, an ultimate motion vector representation (UMVE) is presented. The UMVE is used for either skip mode or merge mode using the proposed motion vector representation method.

[0200] UMVE reuses the same merge candidates contained in the regular merge candidate list in VVC, among which a base candidate can be selected, and is further extended by the proposed motion vector representation method.

[0201] UMVE provides a new motion vector difference (MVD) representation method, in which the starting point, the motion magnitude, and the motion direction are used to represent the MVD.

[0202] The proposed technique uses the merge candidate list as is. However, only candidates with the default merge type (MRG_TYPE_DEFAULT_N) are considered for extending the UMVE.

[0203] The base candidate index defines a starting point. The base candidate index indicates the best candidate among the candidates in the list as follows:

[0204] [Table 1] Table 1: Base candidate IDX

[0205] If the number of base candidates is equal to 1, the base candidate IDX is not signaled.

[0206] The distance index is the magnitude information of the movement. The distance index indicates a predefined distance from the starting point information. The predefined distances are as follows:

[0207] [Table 2] Table 2: Distance IDX

[0208] The direction index represents the direction of the MVD relative to the starting point. The direction index can represent four directions as shown below.

[0209] [Table 3] Table 3: Direction IDX

[0210] The UMVE flag is signaled immediately after sending the skip or merge flag. If the skip or merge flag is true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. But if it is not 1, the AFFINE flag is parsed. If the AFFINE flag is equal to 1, then it is AFFINE mode, but if it is not 1, the skip / merge index is parsed for skip / merge mode.

[0211] No additional line buffers with UMVE candidates are required because the software skip / merge candidates are directly used as base candidates. The input UMVE index is used to determine the supplement correction of MV just before motion compensation. Therefore, there is no need to maintain long line buffers.

[0212] In the current general test conditions, either the first or second merge candidate in the merge candidate list can be selected as the base candidate.

[0213] UMVE is known as Merge with MVD (MMVD).

[0214] 3 Shortcomings of existing implementations In existing implementations (eg, the current VVC design), CPR suffers from these problems.

[0215] (1) The interaction between CPR and affine is still unclear (e.g., for a CPR coded block, a merge candidate can be inserted into the sub-block merge list or the regular merge list).

[0216] (2) The interaction between CPR and general bi-prediction is still unclear (e.g., for CPR coded blocks, we always use a fixed half-half weighting, so there is no need to signal a weighting index).

[0217] (3) The interaction between CPR and MMVD (aka UMVE) remains unclear.

[0218] (4) The interaction between CPR and inter-intra prediction remains unclear.

[0219] (5) Interaction between ATMVP and CPR

[0220] (6) It remains unclear how to apply CPR with two (separate) coding trees.

[0221] 4. Example of a Motion Vector Derivation and Signaling Method for CPR The embodiments of the presently disclosed technology overcome the shortcomings of existing implementations, thereby providing video coding with higher coding efficiency. The use of motion vector derivation and signaling for a current picture referring to video and image coding based on the disclosed technology can improve both existing and future video coding standards, as elucidated in the following examples described for various implementations. The examples of the disclosed technology provided below are illustrative of the general concept and are not intended to be construed as limiting. In the examples, various features described in these examples may be combined, unless explicitly indicated to the contrary. It should be noted that "a block or a candidate uses CPR" may mean the following:

[0222] (i) all reference pictures used by the block or candidate are CPR; or

[0223] (ii) At least one reference picture used by the block or candidate is CPR.

[0224] Example 1. When affine motion compensation is used, it is not permitted to use CPR.

[0225] (a) In one example, a conformance bitstream cannot simultaneously use CPR and affine motion compensation on a single block.

[0226] (b) In one example, in the parsing process, a block-level affine enable / disable flag can be signaled first. If a block is coded in affine mode, there is no need to signal whether the reference picture index of the current block is equal to the one the current picture is associated with.

[0227] (c) Alternatively, the reference picture index may be signaled before the block-level affine enable / disable flag. If the block uses CPR, the signaling of the affine flag is always skipped.

[0228] (d) Alternatively, the reference picture index may be signaled before the block-level affine enable / disable flag. If the block is coded using bi-prediction and only one reference picture is the current reference picture, the affine flag may still be signaled.

[0229] (i) In one example, it is further required that integer motion vectors be used for such blocks.

[0230] Example 2. A block cannot inherit an affine model from a neighboring block that uses CPR.

[0231] (a) In one example, neighboring blocks that use CPR cannot be used to derive affine merging candidates in the sub-block merging candidate list.

[0232] (i) In one example, neighboring blocks that use CPR are treated as unavailable in the sub-block merging candidate list construction process.

[0233] (b) In one example, neighboring blocks that use CPR cannot be used to derive affine AMVP candidates in the affine AMVP list.

[0234] (i) In one example, neighboring blocks that use CPR are treated as unavailable in the affine AMVP candidate list building process.

[0235] Example 3. Whether a block can inherit an affine model from a neighboring block that uses CPR depends on the reference list.

[0236] (a) As an example, assume that the neighboring blocks apply bi-prediction, and a reference picture in reference list X (X can be 0 or 1) is the current picture, and a reference picture in reference list (1-X) is not the current picture.

[0237] (i) The motion information of neighboring blocks that reference reference list X cannot be used to derive affine merge candidates in the sub-block merging candidate list.

[0238] (1) In one example, motion information of neighboring blocks referenced in the reference list (1-X) may be used to derive affine merging candidates in the sub-block merging candidate list.

[0239] (ii) The motion information of neighboring blocks that reference reference list X cannot be used to derive affine AMVP candidates in the affine AMVP list.

[0240] (1) For example, motion information of neighboring blocks that reference reference list X cannot be used to derive affine AMVP candidates in the affine AMVP list for reference list X.

[0241] (iii) Motion information of neighboring blocks referenced in the reference list (1-X) may be used to derive affine AMVP candidates in the affine AMVP list.

[0242] (1) For example, motion information of neighboring blocks that reference reference list (1-X) may be used to derive affine AMVP candidates in an affine AMVP list for reference list (1-X).

[0243] Example 4. If GBi is applied within a block (GBi weighting values ​​are not equal in the two reference pictures), it is not allowed to use CPR.

[0244] (a) In one example, a conforming bitstream cannot use CPR and GBi simultaneously in a single block.

[0245] (b) In one example, if CPR is used in a block, the GBi weighting index is not signaled and is assumed to be 0 (weighting values ​​are equal in the two reference pictures).

[0246] (c) In one example, if unequal weighting is applied within GBi in a block (GBi weighting values ​​are not equal in the two reference pictures), the CPR indication is not signaled and is assumed to be 0 (CPR is not used).

[0247] Example 5. If MMVD is applied in block, the use of CPR is not permitted.

[0248] (a) In one example, a conforming bitstream cannot use CPR and MMVD simultaneously in a single block.

[0249] (b) In one example, if CPR is used in a block, the MMVD flag is not signaled and is assumed to be 0 (weighting values ​​are equal in the two reference pictures).

[0250] (c) In one example, if MMVD is applied within a block, the CPR indication is not signaled and is presumed to be 0 (CPR is not used).

[0251] Example 6. If MMVD is applied within the block, CPR is permitted to be used.

[0252] (a) In one example, the base MV candidate has at least one reference picture that references the current picture.

[0253] (i) In one example, the MMVD is based on MVs that reference the current picture and omits MVs (if any) that do not reference the current picture.

[0254] (ii) In one example, the distances signaled by the MMVD can only be integer numbers.

[0255] (iii) In one embodiment, the MMVD is unipredictive.

[0256] (b) In one example, the base MV candidate has at least one reference picture that does not reference the current picture.

[0257] (i) In one example, the MMVD is based on MVs that do not reference the current picture, and omits MVs that do reference the current picture (if any).

[0258] Example 7. When inter-intra prediction is applied within a block, it is not allowed to use CPR.

[0259] (a) In one example, a conforming bitstream cannot use CPR and intra prediction simultaneously on a single block.

[0260] (b) In one example, if CPR is used within a block, the inter-intra prediction flag is not signaled and is assumed to be 0.

[0261] (c) As one example, if inter-intra prediction is applied within a block, the CPR indication is not signaled and is assumed to be 0 (no CPR is used).

[0262] Example 8. When inter-intra prediction is applied within a block, it is allowed to use CPR.

[0263] (a) In one example, a merge candidate for performing inter-intra prediction has at least one reference picture that references the current picture.

[0264] (i) In one example, inter-intra prediction is based on MVs that reference the current picture and omits MVs (if any) that do not reference the current picture.

[0265] (ii) In one example, the intra prediction portion of the inter-intra prediction may be modified as follows:

[0266] (1) No boundary filtering is performed.

[0267] (2) Position Dependent Prediction Combination (PDPC) is not performed.

[0268] (3) Only these specific modes are permitted.

[0269] ·MPM

[0270] DC

[0271] ·vertical

[0272] ·Horizontal

[0273] (iii) In one example, the inter prediction portion of the inter-intra prediction is uni-predicted.

[0274] (b) In one example, a merge candidate for performing inter-intra prediction has at least one reference picture that does not reference the current picture.

[0275] (i) In one example, inter-intra prediction is based on MVs that do not reference the current picture, omitting MVs that reference the current picture (if any).

[0276] Example 9. In ATMVP process

[0277] (a) Adjacent blocks are treated as unavailable when using CPR.

[0278] (b) When a block in a collocated picture uses CPR.

[0279] (i) It may be treated as unavailable in the ATMVP process.

[0280] (ii) may be treated as available, and MV' may be derived from that block for the current block or a sub-block.

[0281] (1) In one example, MV' refers to the current picture and is copied from a block in the aligned picture.

[0282] Example 10. When CPR is coded as a new mode in addition to intra and inter modes (instead of being indicated as a specific reference picture in the reference picture list), whether to skip the CPR mode signal depends on the decoded information of other coding methods.

[0283] (a) In one example, other coding methods may include, for example, an affine flag, an inter-intra prediction flag, a GBI index representing unequal weighting, MMVD, and ATMVP.

[0284] (b) If any of the other coding methods is applied to a block, the CPR mode signaling is skipped for this block.

[0285] (c) Alternatively, whether to skip the signaling of the use of other coding methods may depend on the decoded information of the CPR mode.

[0286] Example 11. Luma and chroma components, coded using separate coding structure trees

[0287] (a) In one example, when a chroma component is coded, the MV information of the corresponding block of the luma component (wherein the MV may refer to the current picture) may be used as an MV prediction or merging candidate.

[0288] (i) The corresponding block of the luma component may be in any position within the current chroma block (such as top left, top right, bottom left, bottom right, or center).

[0289] (1) If the color format is not 4:4:4, the current chroma block should be scaled to find the corresponding luma block.

[0290] (ii) In one example, there may be MV information from several corresponding blocks of the luma component that serve as MV prediction or merging candidates for the current chroma block, and which one of them is selected may be signaled to the decoder or derived at the decoder.

[0291] (b) The width / height of a chroma block using CPR may have constraints.

[0292] (i) In one example, the width / height of chroma blocks using CPR may be constrained differently than intra-coded chroma blocks.

[0293] (ii) In one example, for intra-coded blocks, width>2 && height>2, but for CPR-coded blocks, width≧2 and height≧2.

[0294] The above examples may be incorporated in the context of the methods described below, such as methods 2400, 2500, 2600, and 2700, which may be implemented in a video decoder or video encoder.

[0295] 24 shows a flow chart of an example method for video processing. The method 2400 includes, at step 2410, performing a conversion between a current video block and a bitstream representation of the current video block that is consistent with a first coding rule that specifies applicability of Current Picture Referencing (CPR) during coding or decoding of the current video block, where the first coding rule specifies at least one of: (a) refraining from simultaneously using CPR and affine motion compensation during the conversion, or (b) disabling, for the conversion, inheriting an affine model of a neighboring block that uses CPR for coding, or (c) for the conversion, specifying whether a coding condition is met. or (d) refraining from using CPR and generalized bi-prediction (GBi) modes simultaneously, where GBi uses unevenly weighted reference pictures, or (e) refraining from using CPR and merge with motion vector difference (MMVD) modes simultaneously, or (f) using CPR and MMVD modes simultaneously, or (g) refraining from using CPR and inter-intra prediction simultaneously, or (h) using CPR and inter-intra prediction simultaneously, or (i) using alternative temporal motion vector predictors simultaneously.

[0296] In some embodiments, and in the context of Example 1, integer precision is used for coding rule (a).

[0297] In some embodiments, and in the context of Example 6, the MMVD mode includes uni-prediction for coding rules (e) or (f).

[0298] In some embodiments, and in the context of Example 8, inter-intra prediction includes an intra prediction portion and an inter prediction portion for coding rules (g) and (h). In one example, the intra prediction portion excludes boundary filtering. In another example, the intra prediction portion excludes a combination of position-dependent predictions. In yet another example, the intra prediction portion includes a most probable mode selection, a DC prediction mode, a horizontal prediction mode, or a vertical prediction mode. In yet another example, the inter prediction portion includes a single prediction.

[0299] 25 shows a flow chart of another example method for video processing. The method 2500 includes, at step 2510, determining that neighboring blocks of a current video block are coded using the current picture reference mode.

[0300] Method 2500 includes, at step 2520, deriving merge candidates based on the determination. In some embodiments, and in the context of Example 2, the merge candidates include a first affine model that is different from a second affine model of the neighboring block. In one example, the merge candidates include affine merge candidates, and where the merge candidate list includes a sub-block merge candidate list. In another example, the merge candidates include affine AMVP candidates, and where the merge candidate list includes an affine AMVP candidate list.

[0301] In some embodiments, and in the context of Example 9, the merge candidates may include alternative temporal motion vector prediction (ATMVP) candidates.

[0302] The method 2500 includes, at step 2530, inserting the merge candidate into a merge candidate list.

[0303] The method 2500 includes, at step 2540, reconstructing the current video block based on the merging candidate list.

[0304] In some embodiments, and in the context of Example 3, the neighboring blocks apply bi-prediction, and method 2500 further includes determining an affine model of the merging candidates based on the first reference picture list or the second reference picture list. In one example, the first reference picture list includes a current picture of the current video block, and the affine model is determined based on at least one picture in the second reference picture list. In another example, the merging candidates include AMVP candidates, and the merging candidate list includes an affine AMVP candidate list.

[0305] 26 shows a flow chart of yet another example method for video processing. The method 2600 includes generating decoded information by decoding a first coding mode of a current video block that is coded using a first coding mode and a second coding mode different from the first coding mode, at step 2610.

[0306] The method 2600 includes, at step 2620, determining, based on the decoded information, whether the first coding mode or the second coding mode is signaled in the bitstream representation of the current video block.

[0307] The method 2600 includes, at step 2630, reconstructing the current video block based on the determination.

[0308] In some embodiments, and in the context of Example 10, the first coding mode includes an affine coding mode, an inter-intra prediction mode, a generalized bi-predictive prediction mode with unequal weighting, a motion vector differential mode or a merge with alternative temporal motion vector prediction mode, the second coding mode includes a CPR mode, and the bitstream representation excludes a flag for the CPR mode.

[0309] In some embodiments, and in the context of Example 10, the first coding mode includes a current picture reference (CPR) mode, the second coding mode includes an affine coding mode, an inter-intra prediction mode, a generalized bi-predictive prediction mode with unequal weighting, a motion vector difference mode or a merge with alternative temporal motion vector prediction mode, and the bitstream representation excludes a flag for the second coding mode.

[0310] 27 shows a flow chart of yet another example method for video processing. The method 2700 includes determining, at step 2710, that a current video block includes a luma component that is coded using a first coding structure tree and a chroma component that is coded using a second coding structure tree that is different from the first coding structure tree.

[0311] Method 2700 includes, at step 2720, reconstructing the current video block based on the determination. In some embodiments, and in the context of Example 11, the reconstructing includes decoding the chroma components based on the motion vector information derived from the one or more corresponding blocks of the luma component. In one example, the one or more corresponding blocks of the luma component are in a top-left position, a top-right position, a bottom-left position, a bottom-right position, or a center position of the current video block. In another example, method 2700 further includes scaling the current video block to determine the one or more corresponding blocks of the luma component upon determining that the color format of the chroma components is different from a 4:4:4 color format.

[0312] In some embodiments, and in the context of Example 11, the current video block is coded using a current picture reference (CPR) mode. In one example, a first set of constraints on the height and width of the current video block are different from a second set of constraints on the height and width of an intra-coded video block that is different from the current video block.

[0313] 5. Examples of the Disclosed Technology FIG. 28 is a block diagram of a video processing device 2800. The device 2800 may be used to implement one or more of the methods described herein. The device 2800 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 2800 may include one or more processors 2802, one or more memories 2804, and video processing hardware 2806. The processor 2802 may be configured to implement one or more of the methods described herein (including, but not limited to, methods 2400, 2500, 2600, and 2700). The memory(s) 2804 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 2806 may be used to implement some of the techniques described herein in hardware circuitry.

[0314] 29 shows a flow chart of yet another example method for video processing. The method 2900 includes, at step 2910, determining that an Intra-Block-Copy (IBC) mode is applied to a current video block of a video, where in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is located. At step 2920, the method includes making a decision regarding disabling a particular coding mode for the current block. At step 2930, the method includes performing a conversion between the current video block and a bitstream representation based on the decision, where the particular coding mode uses a motion vector and a non-current picture to derive a prediction of the video block.

[0315] 30 shows a flow chart of yet another example method for video processing. The method 3000 includes, at step 3010, determining that a current video block is coded using a particular coding mode. At step 3020, based on the determination, making a decision regarding disabling an intra block copy (IBC) mode for the current block, where in the IBC mode, at least one reference picture used by the current video block is the current picture in which the current video block is located. At step 3030, based on the determination, performing a conversion between the current block and a bitstream representation, where the particular coding mode uses motion vectors and non-current pictures to derive a prediction for the current block.

[0316] 31 shows a flow chart of yet another example method for video processing. The method 3100 includes, at step 3110, determining an affine mode for a current video block, where whether the current video block inherits an affine mode from a neighboring block depends on the reference list. At step 3120, performing a conversion between the current video block of the video and a consistent bitstream representation of the video based on the determination.

[0317] 32 shows a flowchart of yet another example method for video processing. The method 3200 includes, at step 3210, performing a conversion between a current video block of a video and a consistent bitstream representation of the video, where during the conversion, an Intra Block Copy (IBC) mode and a Merge with Motion Vector Difference (MMVD) mode are used, where in the IBC mode, at least one reference picture used by the current video block is the current picture in which the current video block is located, and where in the MMVD mode, a motion vector of the video block is derived based on the merged motion candidate list and further refined by at least one motion vector difference.

[0318] 33 shows a flowchart of yet another example method for video processing. The method 3300 includes, at step 3310, performing a conversion between a current video block of a video and a bitstream representation of the video, where during the conversion, an intra block copy (IBC) mode and an inter-intra prediction mode are used, and where in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is located, and where in the inter-intra prediction mode, a prediction signal for the current video block is generated based at least on the intra prediction signal and the inter prediction signal.

[0319] 34 shows a flow chart of yet another example method for video processing. The method 3400 includes, at step 3410, determining decoded information of at least one coding method different from an IBC mode during conversion between the current video block and a bitstream representation of the current video block, where in the IBC mode, at least one reference picture used by the current video block is the current picture in which the current video block is located. At step 3420, the method includes determining, based on the decoded information, whether to skip signaling of an IBC flag for the current video block in the bitstream representation of the current video block, where the IBG flag is associated with the IB mode. And, at step 3430, the method includes performing the conversion based on the determination.

[0320] 35 shows a flow chart of yet another example method for video processing. The method 3500 includes, at step 3510, during conversion between a current video block and a bitstream representation of the current video block, where the current video block includes a luma component coded using a first coding structure tree and a chroma component coded using a second coding structure tree different from the first coding structure tree, deriving first motion vector information for the chroma blocks from second motion vector information of one or more corresponding blocks of the luma components of the current video block. At step 3520, the method includes performing the conversion based on the first motion vector information and the second motion vector information.

[0321] FIG. 36 shows a flow chart of yet another example method for video processing. The method 3600 includes, at step 3610, determining whether to apply an intra block copy (IBC) mode to a current video block, where at least one reference picture used by the current video block is the same as the picture in which the current video block is located. At step 3620, the method includes signaling whether the IBC mode is applied to the current video block, at step 3630, based on determining that the IBC mode is applied to the current video block, includes refraining from signaling a flag for a group of prediction modes. And at step 3640, the method includes performing a transform on the current video block based on the application of the IBC mode.

[0322] 37 shows a flowchart of yet another example method for video processing. The method 3700 includes, at step 3710, analyzing a bitstream to determine whether an intra block copy mode is applied to the current video block, where at least one reference picture used by the current video block is the same as the picture in which the current video block is located. At step 3720, the method includes performing a transformation on the current video block based on application of the IBC mode, where a flag for the group of prediction modes is excluded from the bitstream based on determining that CPR is applied to the current video block.

[0323] FIG. 38 shows a flow chart of yet another example method for video processing. The method 3800 includes, at step 3810, a process of acquiring motion information of a current video block during conversion between the current video block and a bitstream representation of the current video block, where the motion information of the current video block is based on at least one affine model of at least one neighboring block of the current video block. At step 3820, the method includes determining whether a neighboring block of the current video block uses an intra block copy (IBC) mode, where at least one reference picture used by the neighboring block is the same as the picture in which the neighboring block is located. At step 3830, the method includes disabling, for the current video block, deriving affine candidates from the neighboring blocks based on determining that the neighboring blocks use the IBC mode. And at step 3830, the method includes performing a conversion based on the motion information of the current video block.

[0324] 39 shows a flowchart of yet another example method for video processing. The method 3900 includes, at step 3910, dividing a current video block into sub-blocks during conversion between the current video block and a bitstream representation of the current video block based on a sub-block-based temporal motion vector prediction (SbTMVP) mode. At step 3920, the method includes identifying a corresponding block in a motion source picture with a temporal vector for the sub-block based on motion information of spatial neighboring blocks of the current video block, at step 3930, generating a motion vector and a reference index of the reference index from a corresponding block of the sub-block based on coding information of the corresponding block, and at step 3940, performing the conversion based on the generated motion vector and reference index.

[0325] 40 shows a flowchart of yet another example method for video processing. The method 4000 includes, in step 4010, dividing a current video block into sub-blocks during conversion between the current video block and a bitstream representation of the current video block based on a sub-block-based temporal motion vector prediction (SbTMVP) mode. In step 4020, the method includes identifying a corresponding block in a motion source picture with a temporal vector for the sub-block based on coding information of spatial neighboring blocks of the current video block, in step 4030, generating a motion vector and a reference index for the sub-block from the corresponding block of the sub-block, and in step 4040, performing the conversion based on the generated motion vector and reference index.

[0326] In some embodiments, the video coding method may be performed using an apparatus implemented on a hardware platform, such as that described with respect to FIG.

[0327] Some embodiments can be illustrated using the following examples.

[0328] 1.1. A method for video processing, comprising: determining that an intra block copy (IBC) mode is applied to a current video block of a video, where in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is located; making a decision regarding disabling a particular coding mode for the current video block; and performing a conversion between the current video block and a bitstream representation based on the decision, where the particular coding mode uses motion vectors and non-current pictures to derive a prediction for the video block.

[0329] 1.2. The method of Example 1.1, wherein the converting includes encoding the current video block into the bitstream representation.

[0330] 1.3. The method of Example 1.1, wherein the transforming includes decoding the bitstream representation of the current video block to generate pixel values ​​of the current video block.

[0331] 1.4. Any of the methods of Examples 1.1-1.3, wherein the particular coding mode includes an affine motion compensation mode, the affine motion compensation mode using at least one control point motion vector.

[0332] 1.5. The method of Example 1.4, wherein a reference picture index is signaled and the signaling of an affine flag is skipped in response to the current video block using the IBC mode.

[0333] 1.6. The method of Example 1.4, further comprising the steps of signaling a reference picture index before the affine flag is signaled, and determining whether a bi-predictive mode is used for the current video block, wherein if a bi-predictive mode is used for the current video block and only one reference picture is the current reference picture, the affine flag is signaled, and if a bi-predictive mode is used for the current video block and both reference pictures are current reference pictures, the affine flag is not signaled.

[0334] 1.7. The method of example 1.6, wherein integer precision is used for the motion vector of the current video block.

[0335] 1.8. Any of the methods of Examples 1.1-1.7, wherein the particular coding mode includes a GBi mode, in which different weighting values ​​are associated with different reference pictures.

[0336] 1.9. The method of Example 1.8, wherein a GBi weighting index is not signaled in response to the IBC mode being used in the current video block.

[0337] 1.10. The method of Example 1.8, wherein the GBi weighting index is assumed to be zero.

[0338] 1.11. Any of the methods of Examples 1.1-1.10, wherein the particular coding mode includes a merge with a motion vector difference (MMVD) mode, in which a motion vector for a video block is derived based on a merged motion candidate list and further refined by at least one motion vector difference.

[0339] 1.12. The method of Example 1.11, wherein an MMVD flag is not signaled in response to the IBC mode being used in the current video block.

[0340] 1.13. The method of example 1.12, wherein the MMVD flag is presumed to be zero.

[0341] 1.14. Any of Examples 1.1-1.13, wherein the particular coding mode includes a combined inter-intra prediction (CIIP) mode, in which a prediction signal for the current video block is generated based on at least an intra prediction signal and an inter prediction signal.

[0342] 1.15. The method of Example 1.14, wherein a CIIP flag for the current video block is not signaled in response to the IBC mode being used in the current video block.

[0343] 1.16. The method of Example 1.15, wherein the CIIP flag is presumed to be zero.

[0344] 1.17. A method for video processing, comprising: determining that a current video block is encoded using a particular coding mode; and making a decision regarding disabling an intra block copy (IBC) mode for the current block based on the determination, wherein in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current block is located; and performing a conversion between the current block and a bitstream representation based on the determination, wherein the particular coding mode uses motion vectors and non-current pictures to derive a prediction of the current block.

[0345] 1.18. The method of Example 1.17, wherein the converting includes encoding the current video block into the bitstream representation.

[0346] 1.19. The method of Example 1.17, wherein the transforming includes decoding the bitstream representation of the current video block to generate pixel values ​​of the current video block.

[0347] 1.20. Any of the methods of Examples 1.1-1.3, wherein the particular coding mode includes an affine motion compensation mode, the affine motion compensation mode using at least one control point motion vector.

[0348] 1.21. The method of Example 1.20, wherein an affine flag at the block level is signaled in the bitstream to indicate whether affine motion compensation mode is enabled for the current video block, and if the affine flag indicates that the affine motion compensation mode is enabled for the current video block, whether the IBC mode is applied is not signaled.

[0349] 1.22. Any of Examples 1.17-1.19, wherein the particular coding mode includes a GBi mode, in which different weighting values ​​are associated with different reference pictures.

[0350] 1.23. The method of example 1.22, wherein an IBC indication is not signaled in response to the GBI mode being applied.

[0351] 1.24. The method of Example 1.23, wherein the IBC indication is presumed to be zero.

[0352] 1.25. Any of Examples 1.17-1.24, wherein the particular coding mode includes a merge with a motion vector difference (MMVD) mode, in which a motion vector for the current block is derived based on a merged motion candidate list and further refined by at least one motion vector difference.

[0353] 1.26. The method of Example 1.25, wherein an IBC indication is not signaled in response to the MMVD mode being applied in the current video block.

[0354] 1.27. The method of Example 1.26, wherein the IBC indication is presumed to be zero.

[0355] 1.28. Any of Examples 1.17-1.27, wherein the particular coding mode includes a combined inter-intra prediction (CIIP) mode, in which a prediction signal for the current video block is generated based on at least an intra prediction signal and an inter prediction signal.

[0356] 1.29. The method of Example 1.28, wherein an IBC flag for the current video block is not signaled in response to the inter-intra prediction mode being applied.

[0357] 1.30. The method of Example 1.29, wherein the IBC flag is presumed to be zero.

[0358] 1.31. A method for video processing, comprising: determining an affine mode for a current video block, where whether the current video block inherits an affine mode from a neighboring block depends on a reference list; and performing a conversion between the current video block of a video and a consistent bitstream representation of the video based on the determination.

[0359] 1.32. The method of example 1.31, wherein the neighboring blocks apply bi-prediction, a reference picture in reference list X is the current picture, and a reference picture in reference list (1-X) is not the current picture, where X is 0 or 1.

[0360] 1.33. The method of Example 1.32, wherein motion information of the neighboring blocks that reference the reference list X is not used to derive affine merging candidates in a sub-block merging candidate list.

[0361] 1.34. The method of Example 1.32 or 1.33, wherein motion information of the neighboring blocks referenced in the reference list (1-X) is used to derive the affine merging candidates in the sub-block merging candidate list.

[0362] 1.35. The method of Example 1.32, wherein motion information of the neighboring blocks that reference the reference list X is not used to derive affine AMVP candidates in an affine AMVP list.

[0363] 1.36. The method of Example 1.35, wherein the motion information of the neighboring blocks that reference the reference list X is not used to derive the affine AMVP candidates in the affine AMVP list for the reference list X.

[0364] 1.37. The method of Example 1.32, wherein motion information of the neighboring blocks referenced in the reference list (1-X) is used to derive affine AMVP candidates in an affine AMVP list.

[0365] 1.38. The method of Example 1.37, wherein motion information of the neighboring blocks that reference the reference list (1-X) is used to derive the affine AMVP candidates in the affine AMVP list for the reference list (1-X).

[0366] 1.39. A method for video processing comprising: performing a conversion between a current video block of a video and a bitstream representation of the current video block, wherein during the conversion, an Intra Block Copy (IBC) mode and a Merge with Motion Vector Difference (MMVD) mode are used, wherein in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is located, and in the MMVD mode, a motion vector of the video block is derived based on a merged motion candidate list and further refined by at least one motion vector difference.

[0367] 1.40. The method of Example 1.39, wherein the MMVD mode uses base MV candidates having at least one reference picture that references the current picture.

[0368] 1.41. The method of Example 1.40, wherein the MMVD is based on at least one MV that references the current picture, and MVs that do not reference the current picture, if present, are omitted.

[0369] 1.42. The method of example 1.41, wherein the distance signaled by the MMVD is an integer number.

[0370] 1.43. The method of Example 1.41, wherein the MMVD is unipredicted.

[0371] 1.44. The method of Example 1.39, wherein when the MMVD mode uses a base MV candidate having at least one reference picture that does not reference the current picture, the MMVD is based on at least one MV that does not reference the current picture, and a MV that references the current picture, if present, is omitted.

[0372] 1.45. A method for video processing, comprising: performing a conversion between a current video block of a video and a bitstream representation of the video, wherein during the conversion, an intra block copy (IBC) mode and an inter-intra prediction mode are used, wherein in the IBC mode, at least one reference picture used by the current video block is a current picture in which the current video block is located, and in the inter-intra prediction mode, a prediction signal for the current video block is generated based on at least an intra prediction signal and an inter prediction signal.

[0373] 1.46. The method of Example 1.45, wherein the inter-intra predicted merging candidates have at least one reference picture that references the current picture.

[0374] 1.47. The method of Example 1.46, wherein the inter-intra prediction is based on at least one MV that references the current picture, and MVs that do not reference the current picture, if any, are omitted.

[0375] 1.48. The method of Example 1.45, wherein the intra prediction portion of the inter-intra prediction mode excludes boundary filtering.

[0376] 1.49. The method of Example 1.45, wherein the intra prediction portion of the inter-intra prediction modes excludes position-dependent prediction combining (PDPC).

[0377] 1.50. The method of Example 1.45, wherein the intra prediction portion of the inter-intra prediction modes includes at least one of a most probable mode selection (MPM), a DC prediction mode, a horizontal prediction mode, or a vertical prediction mode.

[0378] 1.51. The method of example 1.45, wherein the inter prediction portion of the inter-intra prediction mode is uni-predicted.

[0379] 1.52. Any of the methods of Examples 1.45-1.51, in which if an inter-intra predicted merge candidate has at least one reference picture that does not reference the current picture, inter-intra prediction based on at least one MV that does not reference the current picture is performed, and an MV that references the current picture, if present, is omitted.

[0380] 1.53. A method for video processing, comprising: determining, during a conversion between a current video block and a bitstream representation of the current video block, decoded information of at least one coding method different from an IBC mode, in which at least one reference picture used by the current video block is a current picture in which the current video block is located; determining based on the decoded information whether to skip signaling of an IBC flag for the current video block in the bitstream representation of the current video block, the IBC flag being associated with an IB mode; and performing the conversion based on the determination.

[0381] 1.54. The method of Example 1.53, wherein the converting includes encoding the current video block into the bitstream representation.

[0382] 1.55. The method of Example 1.53, wherein the transforming includes decoding the bitstream representation of the current video block to generate pixel values ​​of the current video block.

[0383] 1.56. Any of the methods of Examples 1.53-1.55, wherein if in the determining step it is determined not to skip signaling of the IBC flag, signaling the IBC flag for the current video block in the bitstream representation, otherwise refraining from signaling the IBC flag for the current video block in the bitstream representation.

[0384] 1.57. Any of the methods of Examples 1.53-1.56, wherein it is determined to skip signaling of the IBC flag if the decoded information of the coding method includes at least one of an affine flag, an inter-intra prediction flag, a generalized bi-prediction (GBi) index, merge with motion vector difference (MMVD) information, and alternative temporal motion vector prediction (ATMVP) information.

[0385] 1.58. A method for video processing, wherein during a conversion between a current video block and a bitstream representation of the current video block, the current video block includes a luma component that is coded using a first coding structure tree and a chroma component that is coded using a second coding structure tree that is different from the first coding structure tree, the method comprising: deriving first motion vector information for chroma blocks from second motion vector information for one or more corresponding blocks of the luma component of the current video block; and performing the conversion based on the first motion vector information and the second motion vector information.

[0386] 1.59. The method of example 1.58, wherein the converting includes encoding the current video block into the bitstream representation.

[0387] 1.60. The method of Example 1.58, wherein the transforming includes decoding the bitstream representation of the current video block to generate pixel values ​​of the current video block.

[0388] 1.61. The method of Example 1.58, wherein the first motion vector information is used as an MV prediction or merging candidate to derive the second motion vector information.

[0389] 1.62. The method of any of Examples 1.58-1.61, wherein the one or more corresponding blocks of the luma component are at a top-left position, a top-right position, a bottom-left position, a bottom-right position, and / or a center position of the current video block.

[0390] 1.63. The method of Example 1.62, further comprising the steps of determining whether a color format of the chroma components is different from a 4:4:4 color format, and scaling the current video block to determine one or more corresponding blocks of the luma component if it is determined that the color format of the chroma components is different from a 4:4:4 color format.

[0391] 1.64. Any of the methods of Examples 1.58-1.60, wherein the current video block is coded using an intra block copy (IBC) mode.

[0392] 1.65. The method of Example 1.64, wherein the first set of constraints on the height and width of the current video block are different from the second set of constraints on the height and width of an intra-coded video block that is different from the current video block.

[0393] 1.66. The method of Example 1.65, wherein the width of the current video block is greater than 2 and the height is greater than 2 when the current video block is coded using an intra block copy mode, and the width of the current video block is greater than or equal to 2 and the height is greater than or equal to 2 when the current video block is coded using the IBC mode.

[0394] 1.67. An apparatus in a video system, comprising: a processor; and a non-transitory memory having instructions stored thereon, the instructions, when executed by the processor, causing the processor to perform any of the methods of Examples 1.1-1.66. Device.

[0395] 1.68. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing the method of any of Examples 1.1-1.66.

[0396] 2.1 A video processing method comprising: determining whether to apply an intra block copy mode (IBC) to the current video block, where at least one reference picture used by the current video block is the same as a picture in which the current video block is located; signaling whether the IBC mode is applied to the current video block; refraining from a signaling flag for a group of prediction modes based on determining that the IBC mode is applied to the current video block; and performing a transformation of the current video block based on the application of the IBC mode.

[0397] 2.2. The method of Example 2.1, wherein the transforming includes encoding the current video block into a bitstream representation.

[0398] 2.3. The method of example 2.1 or 2.2, wherein the group of prediction modes includes an affine mode, the affine mode using at least one control point motion vector.

[0399] 2.4. The method of Example 2.3, wherein the step of determining whether the IBC mode applies to the current video block is performed before an affine flag is signaled, and the method further includes the step of refraining from signaling the affine flag if it is determined that the IBC mode applies to the current video block, and the step of refraining from signaling the affine flag otherwise.

[0400] 2.5. The method of Example 2.4, where if the affine flag is not a signal, the affine flag is presumed to be zero.

[0401] 2.6. Any of the methods of Examples 2.1-2.5, wherein the group of prediction modes includes a generalized bi-prediction mode, and in the GBI mode, different weighting values ​​are associated with different reference pictures.

[0402] 2.7. The method of Example 2.6, wherein the step of determining whether the IBC mode applies to the current video block is performed before a GBi index is signaled, and the method further includes the step of refraining from signaling a GBi weighting index if it is determined that the IBC mode applies to the current video block, and refraining from signaling the GBi weighting index otherwise.

[0403] 2.8. The method of Example 2.7, wherein if the GBi weighting index is not signaled, the weighting value of the reference picture used by the GBi is presumed to be zero.

[0404] 2.9. A video processing method comprising: analyzing a bitstream to determine whether at least one reference picture used by a current video block is the same as the picture in which the current video block is located; and performing a transformation on the current video block; and a flag for a group of prediction modes is excluded from the bitstream based on determining that CPR is applied to the current video block.

[0405] 2.10. The method of Example 2.9, wherein the transforming includes decoding the bitstream representation of the current video block to generate pixel values ​​for the current video block.

[0406] 2.11. The method of example 2.9 or 2.10, wherein the prediction mode flag is estimated to be zero.

[0407] 2.12. Any of the methods of Examples 2.9-2.11, wherein the group of prediction modes includes at least one of an affine mode in which at least one control point motion vector is used, an intra prediction mode in which a prediction signal for the current video block is generated based on at least an intra prediction signal and an inter prediction signal, a generalized bi-predictive prediction mode with unequal weighting, a merge mode with motion vector difference in which a motion vector for a video block is derived based on a merge motion candidate list and further refined by at least one motion vector difference, and an alternative temporal motion vector prediction mode in which motion information of a sub-block of the current video block is determined based on a corresponding block determined by motion information of a spatial neighboring block of the current video block.

[0408] 2.13. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform any of the methods of Examples 2.1-2.12.

[0409] 2.14. A computer program product, stored on a non-transitory computer-readable medium, comprising program code for performing any of the methods of Examples 2.1-2.12.

[0410] 2.15. A video decoding apparatus including a processor configured to implement one or more of the methods of Examples 2.1-2.12.

[0411] 2.16. A video coding apparatus including a processor configured to implement one or more of the methods of Examples 2.1-2.12.

[0412] 3.1. A video processing method comprising: in a process of obtaining motion information of a current video block, during a conversion between the current video block and a bitstream representation of the current video block, the motion information of the current video block is based on at least one affine model of at least one neighboring block of the current video block, determining whether the neighboring block of the current video block uses an intra block copy mode (IBC) in which at least one reference picture used by the neighboring video block is the same as the picture in which the neighboring block is located; disabling deriving affine candidates from the neighboring blocks for the current video block based on determining that the neighboring video block uses the IBC mode; and performing the conversion based on the motion information of the current video block.

[0413] 3.2. The method of Example 3.1, wherein the transforming includes encoding the current video block into a bitstream representation.

[0414] 3.3. The method of Example 3.1, wherein the transforming includes decoding the bitstream representation of the current video block to generate pixel values ​​for the current video block.

[0415] 3.4. Any of the methods of Examples 3.1-3.3, the method further including deriving affine candidates for the current video block from the neighboring blocks based on a determination that the neighboring blocks do not use IBC mode.

[0416] 3.5. Any of the methods in Examples 3.1-3.4, where an affine candidate contains an affine merge candidate and the affine candidate is in the subblock merge candidate list.

[0417] 3.6. The method of example 3.5, in which a neighboring block is treated as unavailable based on determining that the neighboring block uses IBC mode.

[0418] 3.7. Any of the methods of Examples 3.1-3.4, where the affine candidate includes an affine AMVP candidate and the affine candidate is in an affine AMVP list.

[0419] 3.8. The method of Example 3.7 in which a neighboring block is treated as unavailable based on the neighboring block determining that it uses IBC mode.

[0420] 3.9. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform any of the methods of Examples 3.1-3.8.

[0421] 3.10. A computer program product stored on a non-temporal computer readable medium, the computer program product comprising program code for performing the method of any of Examples 3.1-3.8.

[0422] 3.11. A video decoding apparatus including a processor configured to implement one or more of the methods of Examples 3.1-3.8.

[0423] 3.12. A video coding apparatus including a processor configured to implement one or more of the methods of Examples 3.1-3.8.

[0424] 4.1. A video processing method comprising: dividing a current video block into sub-blocks during conversion between a bitstream representation of the current video block based on a sub-block based temporal motion vector prediction (SbTMVP) mode; identifying a corresponding block in a motion source picture having a temporal vector based on motion information of spatially neighboring blocks of the current video block; generating a motion vector and a reference index for the sub-block from the corresponding block of the sub-block based on coding mode information of the corresponding block of the sub-block; and performing the conversion based on the generated motion vector and reference index.

[0425] 4.2. The method of Example 4.1, wherein the motion source picture is a co-located reference picture of the current video block.

[0426] 4.3. Any of the methods of Examples 4.1-4.2, wherein the corresponding block is coded in intra block copy (IBC) mode and the reference block of the corresponding block includes at least one pixel in the picture that contains the corresponding block.

[0427] 4.4. The method of Example 4.3, further comprising treating the corresponding block as unavailable during the conversion.

[0428] 4.5. The method of Example 4.4, further comprising refraining from deriving the motion vector and the reference picture for the sub-block from the corresponding block.

[0429] 4.6. The method of Example 4.5, further comprising the step of assigning default motion information to the current video block.

[0430] 4.7. The method of Example 4.3, wherein the corresponding block is treated as available, and the motion vectors and the reference pictures of the sub-blocks are derived from the corresponding block.

[0431] 4.8. The method of Example 4.7, wherein the motion vectors of the subblocks are copied from those associated with the corresponding blocks.

[0432] 4.9. The method of Example 4.1, wherein the spatial neighboring block is treated as unavailable when using an IBC mode, and a reference block of the spatial neighboring block contains at least one pixel in the picture containing the spatial block.

[0433] 4.10. A video processing method comprising: a step of dividing a current video block into sub-blocks during conversion between a current video block and a bitstream representation of the current video block based on a sub-block-based temporal motion vector prediction (SbTMVP) mode; a step of identifying a corresponding block in a motion source picture having a temporal motion vector based on coding mode information of spatially neighboring blocks of the current video block; a step of generating a motion vector and a reference index for the sub-block from the corresponding block of the sub-block; and a step of performing the conversion based on the generated motion vector and reference index.

[0434] 4.11. The method of Example 4.10, wherein the motion source picture is a collocated reference picture of the current video block.

[0435] 4.12. Any of the methods of Examples 4.10-4.11, wherein the spatial neighboring block is coded in intra block copy (IBC) mode and the reference block of the spatial neighboring block comprises at least one pixel within the picture containing the spatial neighboring block.

[0436] 4.13. The method of Example 4.12, further comprising treating the spatially adjacent blocks as unavailable during the transformation.

[0437] 4.14. The method of Example 4.13, further comprising refraining from identifying the corresponding block based on the motion vectors of spatially neighboring blocks.

[0438] 4.15. The method of Example 4.14, further including assigning default motion information to identify the corresponding block.

[0439] 4.16. Any of the methods of Examples 4.1-4.15, wherein the transforming includes encoding the current video block into a bitstream representation.

[0440] 4.17. Any of the methods of Examples 4.1-4.15, wherein the transforming includes decoding a bitstream representation of the current video block to generate pixel values ​​for the current video block.

[0441] 4.18. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, the instructions, when executed by the processor, causing the processor to perform any of the methods of Examples 4.1-4.15.

[0442] 4.19. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing the method of any of Examples 4.1-4.15.

[0443] 4.20. A video decoding apparatus including a processor configured to implement one or more of the methods of Examples 4.1-4.15.

[0444] 4.21. A video coding apparatus including a processor configured to implement one or more of the methods of Examples 4.1-4.15.

[0445] From the foregoing, it will be understood that, although specific embodiments of the disclosed technology have been described herein for purposes of illustration, various modifications can be made without departing from the scope of the invention. Accordingly, the presently disclosed technology is not to be limited except as by the appended claims.

[0446] The implementation and functional operation of the subject matter described in this patent document can be implemented in various systems, digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The implementation of the subject matter described herein can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for controlling the operation of or by a data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter affecting a machine-readable propagated signal, or one or more combinations thereof. The term "data processing unit" or "data processing device" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. An apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0447] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in several coordinated files (e.g., a file that stores one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on several computers that are located on one computer or at one site, or distributed across several sites and interconnected by a communication network.

[0448] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus implemented in, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0449] Processors suitable for executing a computer program include, for example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer is also operatively coupled to receive data from, or transfer data to, one or more mass storage devices, e.g., magnetic disks, magneto-optical disks, or optical disks, for storing data. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by, or incorporated within, special purpose logic circuitry.

[0450] This specification, together with the drawings, are considered as merely illustrative, by way of example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0451] Although this patent document contains many details, these should not be construed as limiting the scope of any invention or what may be claimed, but as descriptions of features specific to particular embodiments of a particular invention. Certain features described in a patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features are described above as acting in a particular combination and may initially be claimed as such, one or more features from a claimed combination may, in some cases, be carved out of the combination. And, a claimed combination may be directed to a subcombination or a variation of the subcombination.

[0452] Similarly, although the figures depict acts in a particular order, this should not be construed as requiring that such acts be performed in a particular order or sequence, or that all of the acts illustrated be performed, to achieve desired results.Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0453] Only a few embodiments and examples have been described; other embodiments, extensions and variations may be made based on what is described and explained in this patent document.

Claims

1. 1. A method for processing video data, comprising the steps of: determining that an intra block copy (IBC) mode is applied to a current video block of a video, in which a reference sample from a video region containing the current video block is used; performing a conversion between the current video block and the video bitstream based on the determination; making a decision regarding disabling a particular encoding mode for the current video block; Including, the specific coding modes include affine modes and combined inter-intra prediction modes; The affine mode uses at least one control point motion vector; in the combined inter-intra prediction mode, a final prediction is generated based at least on a weighted sum of an intermediate intra prediction signal and an intermediate inter prediction signal; the IBC mode is applied to a video block having a dual coding tree, in which luma and chroma components are coded with separate coding structure trees; and the particular coding mode flag is not included in the bitstream in response to the IBC mode being used by the current video block; method.

2. If the flag is not included in the bitstream, the flag is presumed to be zero. The method of claim 1.

3. The particular coding mode includes bi-prediction using a CU-level weighting mode, in which different weighting values ​​are associated with different reference images in the prediction derivation process. The method of claim 1.

4. the bi-predictive weighting index using a CU-level weighting mode is not included in the bitstream in response to the IBC mode used in the current video block. The method according to claim 3.

5. If the weighting index is not included in the bitstream, the weighting index is presumed to be zero. The method according to claim 4.

6. The specific coding mode includes a merge with motion vector difference (MMVD) mode; In the MMVD mode, a motion vector for a video block is derived based on a merged motion candidate list and further refined by at least one motion vector offset. The method of claim 1.

7. the specific coding mode includes a sub-block-based temporal motion vector prediction mode; In the sub-block-based temporal motion vector prediction mode, motion information is derived based on a placement area determined by at least one temporal motion offset. The method of claim 1.

8. Prior to the performing step, the method further comprises: deriving a block vector for the current video block; and using at least one block vector difference included in the bitstream to refine the block vector. The method of claim 1.

9. The width of the current video block is greater than or equal to 2 and the height of the current video block is greater than or equal to 2; The method of claim 1.

10. the converting includes decoding the current video block from the bitstream. The method of claim 1.

11. the converting includes encoding the current video block into the bitstream. The method of claim 1.

12. 1. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions stored thereon, the apparatus comprising: The instructions, when executed by the processor, cause the processor to: determining that an intra block copy (IBC) mode is applied to a current video block of a video, in which a reference sample from a video region that includes the current video block is used; performing a conversion between the current video block and the bitstream of the video based on the determination; and making a decision regarding disabling a particular encoding mode for the current video block; Let them do so, the specific coding modes include affine modes and combined inter-intra prediction modes; The affine mode uses at least one control point motion vector; in the combined inter-intra prediction mode, a final prediction is generated based at least on a weighted sum of an intermediate intra prediction signal and an intermediate inter prediction signal; the IBC mode is applied to a video block having a dual coding tree, in which luma and chroma components are coded with separate coding structure trees; and the particular coding mode flag is not included in the bitstream in response to the IBC mode being used by the current video block; Device.

13. A non-transitory computer-readable storage medium storing instructions, comprising: The instructions, when executed, cause a processor to: determining that an intra block copy (IBC) mode is applied to a current video block of a video, in which a reference sample from a video region that includes the current video block is used; performing a conversion between the current video block and the bitstream of the video based on the determination; and making a decision regarding disabling a particular encoding mode for the current video block; Let them do so, the specific coding modes include affine modes and combined inter-intra prediction modes; The affine mode uses at least one control point motion vector; in the combined inter-intra prediction mode, a final prediction is generated based at least on a weighted sum of an intermediate intra prediction signal and an intermediate inter prediction signal; the IBC mode is applied to a video block having a dual coding tree, in which luma and chroma components are coded with separate coding structure trees; and the particular coding mode flag is not included in the bitstream in response to the IBC mode being used by the current video block; A non-transitory computer-readable storage medium.

14. 1. A method for archiving a video bitstream, comprising: The method comprises: determining that an intra block copy (IBC) mode is applied to a current video block of a video, in which a reference sample from a video region containing the current video block is used; generating the bitstream from the current video block based on the determination; making a decision regarding disabling a particular encoding mode for the current video block; storing the bitstream in a non-transitory computer readable storage medium; Including, the specific coding modes include affine modes and combined inter-intra prediction modes; The affine mode uses at least one control point motion vector; in the combined inter-intra prediction mode, a final prediction is generated based at least on a weighted sum of an intermediate intra prediction signal and an intermediate inter prediction signal; the IBC mode is applied to a video block having a dual coding tree, in which luma and chroma components are coded with separate coding structure trees; and the particular coding mode flag is not included in the bitstream in response to the IBC mode being used by the current video block; method.

Images