Motion candidate list construction for intra block copy mode

Abstract

A method of video processing, comprising: for a conversion between a current block of a current picture of a video and a bitstream representation of the video, constructing a list of motion candidates for a codec technique applicable to the current block, wherein the history-based motion candidates are derived based on information from previously processed video blocks. The codec technique uses reference samples from the current picture of the current block to predict at least a portion of the current block. The method also includes performing the conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is filed in order to timely claim the priority and benefits of International Patent Application No. PCT / CN2019 / 092438 filed on June 22, 2019, in accordance with applicable patent laws and / or the Paris Convention. The entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] This document deals with video and image encoding and decoding technologies. Background Art

[0004] Digital video accounts for the largest bandwidth usage on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth requirements for digital video usage will continue to grow. Summary of the invention

[0005] The disclosed techniques may be used by video or image decoder or encoder embodiments to perform intra-block copy partitioning techniques for encoding or decoding of a video bitstream at a sub-block level.

[0006] In one example aspect, a method of video processing is disclosed. The method includes: for a conversion between a current block of a current picture of a video and a bitstream representation of the video, constructing a list of motion candidates for a codec technique applicable to the current block, wherein the history-based motion candidates are derived based on information from previously processed video blocks. The codec technique uses reference samples from the current picture of the current block to predict at least a portion of the current block. The method also includes performing the conversion based on the determination.

[0007] In another example aspect, a method of video processing is disclosed. The method includes: for a conversion between a current block of video and a bitstream representation of the video, determining that the current block is divided into a plurality of sub-blocks. Encoding and decoding at least one of the plurality of blocks using a modified intra-block copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture of the current block. The method also includes performing the conversion based on the determination.

[0008] In another example aspect, a method of video processing is disclosed. The method includes: for a conversion between a current block of a video and a bitstream representation of the video, determining that the current block is divided into a plurality of sub-blocks. Encoding and decoding each of the plurality of sub-blocks in the encoded and decoded representation using a corresponding encoding and decoding technique according to a pattern. The method also includes performing the conversion based on the determination.

[0009] In another example aspect, a method of video processing is disclosed. The method includes: for a conversion between a current block of video and a bitstream representation of the video, determining an operation associated with a list of motion candidates based on a condition related to a characteristic of the current block. The list of motion candidates is constructed for a codec technology or based on information from a previously processed video block. The method also includes performing the conversion based on the determination.

[0010] In another example aspect, a method of video processing is disclosed. The method includes: for a conversion between a current block of a video and a bitstream representation of the video, determining that the current block encoded using an inter-frame coding technique based on temporal information is divided into a plurality of sub-blocks. Encoding and decoding at least one of the plurality of blocks using a modified intra-frame block copy (IBC) coding technique that uses reference samples from one or more video regions of a current picture including the current block. The method also includes performing the conversion based on the determination.

[0011] In another example aspect, a method of video processing is disclosed. The method includes determining to use a sub-block intra block copy (sbIBC) codec mode in a conversion between a current video block in a video region and a bitstream representation of the current video block, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded based on reference samples from the video region, wherein a size of the sub-block is based on a division rule and the conversion is performed on the plurality of sub-blocks using the sbIBC codec mode.

[0012] In another example aspect, a method of video processing is disclosed. The method includes determining to use a sub-block intra block copy (sbIBC) codec mode in a conversion between a current video block in a video region and a bitstream representation of the current video block, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded and decoded based on reference samples from the video region, and performing conversion on the plurality of sub-blocks using the sbIBC codec mode, wherein the conversion includes determining an initialization motion vector (initMV) of a given sub-block, identifying a reference block from the initMV, and deriving motion vector (MV) information of the given sub-block using MV information of the reference block.

[0013] In another example aspect, a method of video processing is disclosed. The method includes: determining to use a sub-block intra block copy (sbIBC) codec mode in a conversion between a current video block in a video region and a bitstream representation of the current video block, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded based on reference samples from the video region, and performing conversion on the plurality of sub-blocks using the sbIBC codec mode, wherein the conversion includes generating a sub-block IBC candidate.

[0014] In another example aspect, a method of video processing is disclosed. The method includes: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a sub-block intra block coding (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an intra coding mode.

[0015] In another example aspect, a method of video processing is disclosed. The method includes performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing all sub-blocks of the plurality of sub-blocks using an intra-frame codec mode.

[0016] In another example aspect, a method of video processing is disclosed. The method includes: performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing all of the plurality of sub-blocks using a palette encoding / decoding mode in which a palette of representative pixel values ​​is used to encode / decode each sub-block.

[0017] In another example aspect, a method of video processing is disclosed. The method includes: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a palette mode in which a palette of representative pixel values ​​is used to encode and decode the first sub-block and processing a second sub-block of the plurality of sub-blocks using an intra-block copy codec mode.

[0018] In another example aspect, a method of video processing is disclosed. The method includes: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a palette mode and processing a second sub-block of the plurality of sub-blocks using an intra-frame coding mode in which a palette of representative pixel values ​​is used to codec the first sub-block.

[0019] In another example aspect, a method of video processing is disclosed. The method includes: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a sub-block intra-block coding (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

[0020] In another example aspect, a method of video processing is disclosed. The method includes: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a sub-block intra-codec mode and processing a second sub-block of the plurality of sub-blocks using an inter-codec mode.

[0021] In another example aspect, a method of video processing is disclosed. The method includes: deciding to encode a current video block into a bitstream representation using the method of any one of the above claims; and including information indicating the decision in the bitstream representation at the following levels: decoder parameter set level or sequence parameter set level or video parameter set level or picture parameter set level or picture header level or slice header level or slice group header level or maximum codec unit level or codec unit level or maximum codec unit row level or LCU group level or transform unit level or prediction unit level or video codec unit level.

[0022] In another example aspect, a method of video processing is disclosed. The method includes: deciding to encode a current video block into a bitstream representation using the method described in any of the above claims based on an encoding condition; and performing encoding using the method described in any of the above claims, wherein the condition is based on one or more of the following: a codec unit, a prediction unit, a transform unit, a current video block, or a position of a video codec unit of the current video block.

[0023] In another example aspect, a method of video processing is disclosed. The method includes: determining to use an intra block copy mode and an inter prediction mode for conversion between a block in a video region and a bitstream representation of the video region; and performing conversion on the block in the video region using the intra block copy mode and the inter prediction mode.

[0024] In another example aspect, a method of video processing is disclosed. The method includes: during a conversion between a current video block and a bitstream representation of the current video block, performing a motion candidate list construction process and / or performing a table update process for updating a history-based motion vector predictor table as appropriate based on a codec condition, and performing the conversion based on the motion candidate list construction process and / or the table update process.

[0025] In another example aspect, the above method may be implemented by a video decoder device including a processor.

[0026] In another example aspect, the above method may be implemented by a video encoder apparatus including a processor.

[0027] In yet another example aspect, the methods may be implemented in the form of processor-executable instructions and stored on a computer-readable program medium.

[0028] These and other aspects are described further throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The export process for Merge candidate list construction is shown.

[0030] Figure 2 An example of the location of spatial merge candidates is shown.

[0031] Figure 3 An example of candidate pairs considered for redundancy checking of spatial Merge candidates is shown.

[0032] Figure 4 Example locations of the second PU for Nx2N and 2NxN partitions are shown.

[0033] Figure 5 An example of a diagram of motion vector scaling for a temporal Merge candidate is shown.

[0034] Figure 6 An example of candidate positions of the time-domain Merge candidates C0 and C1 is shown.

[0035] Figure 7 An example of a combined bi-predictive Merge candidate is shown.

[0036] Figure 8 An example of a process of deriving motion vector prediction candidates is shown.

[0037] Fig. 9 An example diagram showing motion vector scaling of spatial motion vector candidates is shown.

[0038] Fig.10 Example simplified affine motion models are shown for a 4-parameter affine mode (left) and a 6-parameter affine model (right).

[0039] Fig.11 An example of the affine motion vector field for each sub-block is shown.

[0040] Fig.12 Example candidate positions for the affine merge mode are shown.

[0041] Fig.13 An example of a modified Merge list building process is shown.

[0042] Fig.14 An example of inter prediction based on triangular partitioning is shown.

[0043] Fig.15 An example of a CU to which the first weighting factor group is applied is shown.

[0044] Fig.16 An example of motion vector storage is shown.

[0045] Fig.17 An example of a final motion vector expression (UMVE) search process is shown.

[0046] Fig.18 An example of a UMVE search point is shown.

[0047] Fig.19 An example of MVD (0, 1) mirrored between List 0 and List 1 in DMVR is shown.

[0048] Fig. 20 An example of MVs that can be checked in one iteration is shown.

[0049] Fig.21 An example of intra block copying is shown.

[0050] Fig. 22 is a block diagram of an example of a video processing device.

[0051] Fig.23 is a flow chart of an example of a video processing method.

[0052] Fig.24 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

[0053] Fig.25 is a flowchart representation of a method for video processing according to the present technology.

[0054] Fig.26 is a flowchart representation of another method for video processing according to the present technology.

[0055] Fig. 27 is a flowchart representation of another method for video processing according to the present technology.

[0056] Fig.28 is a flowchart representation of another method for video processing according to the present technology.

[0057] Fig.29 is a flowchart representation of yet another method for video processing in accordance with the present technology. DETAILED DESCRIPTION

[0058] This document provides various techniques that can be used by a decoder of an image or video bitstream to improve the quality of a decompressed or decoded digital video or image. For simplicity, the term "video" is used herein to include both a sequence of pictures (traditionally referred to as video) and a single image. In addition, a video encoder may also implement these techniques during the encoding process in order to reconstruct a decoded frame for further encoding.

[0059] The section headings used in this document are for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. Therefore, embodiments from one section can be combined with embodiments from other sections.

[0060] 1. Summary

[0061] This document relates to video codec technology. Specifically, it relates to intra-frame block copy (also known as current picture reference, CPR) codec. It can be applied to existing video codec standards such as HEVC or standards to be finalized (universal video codec). It can also be applied to future video codec standards or video codecs.

[0062] 2. Background

[0063] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced the H.261 and H.263 standards, ISO / IEC produced the MPEG-1 and MPEG-4 Visual standards, and the two organizations jointly produced the H.262 / MPEG-2 Video standard and the H.264 / MPEG-4 Advanced Video Coding (AVC) standard and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec structure that utilizes temporal prediction plus transform coding. In order to explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was established to work on the VVC standard with the goal of reducing the bit rate by 50% compared to HEVC.

[0064] 2.1 Inter-frame prediction in HEVC / H.265

[0065] For inter-coded codec units (CUs), they can be coded with one prediction unit (PU) or two PUs according to the partitioning mode. Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include motion vectors and reference picture indexes. Inter_pred_idc can also be used to signal the use of one of the two reference picture lists. The motion vector can be explicitly coded as a delta relative to the predictor.

[0066] When a CU is encoded using skip mode, a PU is associated with the CU and there are no significant residual coefficients, no coded motion vector increments or reference picture indices. A Merge mode is specified, whereby the motion parameters of the current PU are obtained from neighboring PUs, including spatial and temporal candidates. Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to Merge mode is explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference (MVD) compared to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the reference picture list usage are explicitly signaled for each PU. Such a mode is named Advanced Motion Vector Prediction (AMVP) in this disclosure.

[0067] When signaling indicates that one of the two reference picture lists is to be used, the PU is generated from one block of samples. This is called "unidirectional prediction". Unidirectional prediction can be used for P slices and B slices.

[0068] When signaling indicates that two reference picture lists are to be used, the PU is generated from two blocks of samples. This is called "bidirectional prediction". Bidirectional prediction is only available for B slices.

[0069] 2.1.1 Reference Image List

[0070] In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. As in H.264 / AVC, a picture can be predicted from multiple reference pictures. Reference pictures used for inter prediction are organized in one or more reference picture lists. A reference index identifies which reference pictures in the list should be used to create a prediction signal.

[0071] A single reference picture list, List 0, is used for P slices, and two reference picture lists, List 0 and List 1, are used for B slices. It should be noted that the reference pictures contained in List 0 / 1 can be pictures from both the past and the future in terms of capture / display order.

[0072] 2.1.2 Merge Mode

[0073] 2.1.2.1 Exporting Merge Mode Candidates

[0074] When using Merge mode to predict a PU, an index pointing to an entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized according to the order of the following steps:

[0075] Step 1: Initial Candidate Export

[0076] Step 1.1: Spatial Candidate Derivation

[0077] Step 1.2: Redundancy check of spatial candidates

[0078] Step 1.3: Time Domain Candidate Derivation

[0079] Step 2: Additional candidate insertions

[0080] Step 2.1: Create bidirectional prediction candidates

[0081] Step 2.2: Insert zero motion candidates

[0082] exist Figure 1 These steps are also schematically depicted in . For spatial domain Merge candidate derivation, up to four Merge candidates are selected from candidates located at five different positions. For temporal domain Merge candidate derivation, at most one Merge candidate is selected from two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, truncated unary binarization (TU) is used to encode the index of the best Merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

[0083] Hereinafter, operations associated with the aforementioned steps are described in detail.

[0084] 2.1.2.2 Spatial Candidate Export

[0085] In the export of spatial merge candidates, in Figure 2Up to four Merge candidates are selected from the candidates at the positions depicted in . The derived order is A1, B1, B0, A0, and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is not available (for example, because the PU belongs to another slice or piece) or is intra-coded. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only pairs with Figure 3 The arrows in the table link pairs, and only if the corresponding candidate for redundancy check has different motion information, then the candidate is added to the list. Another source of duplicate motion information is a "second PU" associated with a partition different from 2N×2N. As an example, Figure 4 The second PU is depicted for the N×2N and 2N×N cases, respectively. When the current PU is partitioned into N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate would result in two prediction units with the same motion information, which is redundant for having only one PU in the codec unit. Similarly, when the current PU is partitioned into 2N×N, position B1 is not considered.

[0086] 2.1.2.3 Time Domain Candidate Derivation

[0087] In this step, only one candidate is added to the list. In particular, in the derivation of the temporal Merge candidate, the scaled motion vector is derived based on the co-located PU in the co-located picture. Figure 5 As shown by the dashed line in the figure, a scaled motion vector for the temporal Merge candidate is obtained, which is scaled from the motion vector of the collocated PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC specification. For B slices, two motion vectors are obtained and combined to produce a bidirectional prediction Merge candidate, one of which is used for reference picture list 0 and the other for reference picture list 1.

[0088] 2.1.2.4 Collocation of pictures and PUs

[0089] When TMVP is enabled (e.g. slice_temporal_mvp_enabled_flag is equal to 1), the variable ColPic representing the collocated picture is derived as follows:

[0090] - If the current slice is a B slice and collocated_from_l0_flag is signaled to be equal to 0, ColPic is set equal to RefPicList1[collocated_ref_idx].

[0091] - Otherwise (slice_type is equal to B and collocated_from_l0_flag is equal to 1, or slice_type is equal to P), ColPic is set equal to RefPicList0[collocated_ref_idx].

[0092] Here, collocated_ref_idx and collocated_from_l0_flag are two syntax elements, which may be signaled in the slice header.

[0093] like Figure 6 As shown, in the collocated PU (Y) belonging to the reference frame, the position for the temporal candidate is selected between candidates C0 and C1. If the PU at position C0 is not available, is intra-coded, or is outside the current codec tree unit (CTU, ie, LCU, the largest codec unit) row, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal Merge candidate.

[0094] The relevant syntax elements are described as follows:

[0095] 7.3.6.1 General slice segment header syntax

[0096]

[0097] 2.1.2.5 Export of MVs for TMVP Candidates

[0098] More specifically, the following steps are performed to derive TMVP candidates:

[0099] (1) Set reference picture list X = 0, and the target reference picture is set to the reference picture with index equal to 0 in list X (e.g., curr_ref). Call the derivation process of the concatenated motion vector to obtain the MV of list X pointing to curr_ref.

[0100] (2) If the current slice is a B slice, set reference picture list X = 1, and the target reference picture is set to the reference picture (e.g., curr_ref) with index equal to 0 in list X. Call the derivation process of the concatenated motion vector to obtain the MV of list X pointing to curr_ref.

[0101] The derivation process of the collocated motion vector is described in the next subsection.

[0102] 2.1.2.5.1 Process of deriving concatenated motion vectors

[0103] For the collocated block, unidirectional prediction or bidirectional prediction can be used for intra-frame or inter-frame coding. If intra-frame coding is performed, the TMVP candidate is set to unavailable.

[0104] If it is unidirectional prediction from list A, the motion vector of list A is scaled to the target reference picture list X.

[0105] If it is bidirectional prediction and the target reference picture list is X, the motion vector of list A is scaled to the target reference picture list X, and A is determined according to the following rules:

[0106] - If none of the reference pictures has a larger POC value than the current picture, then A is set equal to X.

[0107] - Otherwise, set A equal to collocated_from_l0_flag.

[0108] Some relevant descriptions are as follows:

[0109] 8.5.3.2.9 Derivation process of concatenated motion vectors

[0110] The inputs to this process are:

[0111] - variable currPb, specifies the current prediction block,

[0112] - the variable colPb specifies the collocated prediction block within the collocated picture specified by ColPic,

[0113] - luma position (xColPb, yColPb), specifies the top left sample of the collocated luma prediction block specified by colPb relative to the top left luma sample of the collocated picture specified by ColPic,

[0114] - Reference index refIdxLX, X is 0 or 1.

[0115] The output of this process is:

[0116] - Motion vector prediction mvLXCol,

[0117] -Availability flag availableFlagLXCol.

[0118] The variable currPic specifies the current picture.

[0119] The arrays predFlagL0Col[x][y], mvL0Col[x][y], and refIdxL0Col[x][y] are set equal to PredFlagL0[x][y], MvL0[x][y], and RefIdxL0[x][y], respectively, of the collocated picture specified by ColPic, and the arrays predFlagL1Col[x][y], mvL1Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL1[x][y], MvL1[x][y], and RefIdxL1[x][y], respectively, of the collocated picture specified by ColPic.

[0120] The variables mvLXCol and availableFlagLXCol are exported as follows:

[0121] If colPb is encoded in intra prediction mode, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

[0122] Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows:

[0123] - If predFlagL0Col[xColPb][yColPb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL1Col[xColPb][yColPb], refIdxL1Col[xColPb][yColPb], and L1, respectively.

[0124] Otherwise, if predFlagL0Col[xColPb][yColPb] is equal to 1 and predFlagL1Col[xColPb][yColPb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL0Col[xColPb][yColPb], refIdxL0Col[xColPb][yColPb], and L0, respectively.

[0125] - Otherwise (predFlagL0Col[xColPb][yColPb] is equal to 1 and predFlagL1Col[xColPb][yColPb] is equal to 1), the following assignments are made:

[0126] - If NoBackwardPredFlag is equal to 1, mvCol, refIdxCol, and listCol are set equal to mvLXCol[xColPb][yColPb], refIdxLXCol[xColPb][yColPb], and LX, respectively.

[0127] - Otherwise, mvCol, refIdxCol, and listCol are set equal to mvLNCol[xColPb][yColPb], refIdxLNCol[xColPb][yColPb], and LN, respectively, where N is the value of collocated_from_l0_flag.

[0128] And, mvLXCol and availableFlagLXCol are derived as follows:

[0129] - If LongTermRefPic(currPic, currPb, refIdxLX, LX) is not equal to LongTermRefPic(ColPic, colPb, refIdxCol, listCol), both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

[0130] Otherwise, the variable availableFlagLXCol is set equal to 1, refPicListCol[refIdxCol] is set to the picture with reference index refIdxCol in the reference picture list listCol of the slice containing the prediction block colPb in the collocated picture specified by ColPic, and the following applies:

[0131] colPocDiff=DiffPicOrderCnt(ColPic, refPicListCol[refIdxCol]) (2-1)

[0132] currPocDiff=DiffPicOrderCnt(currPic, RefPicListX[refIdxLX]) (2-2)

[0133] - If RefPicListX[refIdxLX] is a long-term reference picture, or colPocDiff is equal to currPocDiff, then mvLXCol is derived as follows:

[0134] mvLXCol=mvCol (2-3)

[0135] Otherwise, export mvLXCol as a scaled version of the motion vector mvCol as follows:

[0136] tx=(16384+(Abs(td)>>1)) / td (2-4)

[0137] distScaleFactor=Clip3(-4096, 4095, (tb*tx+32)>>6) (2-5)

[0138] mvLXCol=Clip3(-32768, 32767, Sign(distScaleFactor*mvCol)*((Abs(distScaleFactor*mvCol)+127)>>8)) (2-6)

[0139] Among them, td and tb are derived as follows:

[0140] td=Clip3(-128, 127, colPocDiff) (2-7)

[0141] tb=Clip3(-128,127,currPocDiff) (2-8)

[0142] The definition of NoBackwardPredFlag is:

[0143] The variable NoBackwardPredFlag is exported as follows:

[0144] - If for each picture aPic in RefPicList0 or RefPicList1 of the current slice, DiffPicOrderCnt(aPic, CurrPic) is less than or equal to 0, then NoBackwardPredFlag is set equal to 1.

[0145] - Otherwise, NoBackwardPredFlag is set equal to 0.

[0146] 2.1.2.6 Additional Candidate Insertion

[0147] In addition to spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bidirectional prediction Merge candidates and zero Merge candidates. Combined bidirectional prediction Merge candidates are generated by utilizing spatial and temporal Merge candidates. Combined bidirectional prediction Merge candidates are only used for B slices. Combined bidirectional prediction candidates are generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they will form a new bidirectional prediction candidate. As an example, Figure 7 Depicted is a case where two candidates in the original list (on the left) with mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create a combined bi-predictive Merge candidate that is added to the final list (on the right). There are a number of rules about the combinations that are considered to generate these additional Merge candidates.

[0148] Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list and thus reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.

[0149] 2.1.3 AMVP

[0150] AMVP uses the spatial-temporal correlation of motion vectors and neighboring PUs, which is used for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is constructed by first checking the availability of neighboring PU positions in the time domain to the left and above, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the Merge index signaling notification, the index of the best motion vector candidate is encoded using truncated unary. The maximum value to be encoded in this case is 2 (see Figure 8 ). In the following sections, details about the derivation process of motion vector prediction candidates are provided.

[0151] 2.1.3.1 Export of AMVP Candidates

[0152] Figure 8 The derivation process for motion vector prediction candidates is summarized.

[0153] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For spatial motion vector candidate derivation, the final Figure 2 The motion vectors of each PU at the five different positions shown are used to derive two motion vector candidates.

[0154] For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates derived based on two different collocated positions. After making the first list of spatial-temporal candidates, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than 2, motion vector candidates whose reference picture index within the associated reference picture list is greater than 1 are removed from the list. If the number of spatial-temporal motion vector candidates is less than 2, an additional zero motion vector candidate is added to the list.

[0155] 2.1.3.2 Spatial Motion Vector Candidates

[0156] In the derivation of spatial motion vector candidates, at most two candidates are considered among five potential candidates, the five potential candidates being selected from the candidate located at Figure 2 PU derivation for the positions shown is the same as those of the motion Merge. The order of derivation for the left side of the current PU is defined as A0, A1 and scaled A0, scaled A1. The order of derivation for the upper side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not require the use of spatial scaling, and two use spatial scaling. The four different cases are summarized as follows.

[0157] * No airspace scaling

[0158] (1) Same reference picture list and same reference picture index (same POC)

[0159] (2) Different reference picture lists, but same reference pictures (same POC)*spatial scaling

[0160] (3) Same reference picture list, but different reference pictures (different POC)

[0161] (4) Different reference picture lists and different reference pictures (different POCs)

[0162] First check for no spatial scaling, then check for spatial scaling. Regardless of the reference picture list, consider spatial scaling when the POC is different between the reference picture of the neighbor PU and the reference picture of the current PU. If all PUs of the left candidate are unavailable or intra-coded, scaling of the upper motion vector is allowed to help parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

[0163] In the spatial domain scaling process, the motion vector of the neighboring PU is scaled in a similar way to the temporal domain scaling, such as Fig. 9 The main difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as that of temporal scaling.

[0164] 2.1.3.3 Temporal Motion Vector Candidates

[0165] Except for the reference picture index derivation, all derivation processes for the temporal Merge candidate are the same as those for the spatial motion vector candidate derivation (see Figure 6 ). The reference picture index is signaled to the decoder.

[0166] 2.2 Inter-frame prediction method in VVC

[0167] There are several new codec tools for inter prediction improvements, such as Adaptive Motion Vector Difference Resolution (AMVR) for signaling MVD, Merge with Motion Vector Difference (MMVD), Triangular Prediction Mode (TPM), Combined Intra-Inter Prediction (CIIP), Advanced TMVP (ATMVP, also known as SbTMVP), affine prediction mode, generalized bidirectional prediction (GBI), decoder-side motion vector refinement (DMVR) and bidirectional optical flow (BIO, also known as BDOF).

[0168] VVC supports three different Merge list construction processes:

[0169] (1) Sub-block Merge candidate list: It includes ATMVP and affine merge candidates. Both affine mode and ATMVP mode share a merge list construction process. Here, ATMVP and affine merge candidates can be added in sequence. The size of the sub-block merge list is signaled in the stripe header and the maximum value is 5.

[0170] (2) Conventional Merge List: For inter-coded blocks, a common Merge list construction process is shared. Here, spatial / temporal Merge candidates, HMVP, paired Merge candidates, and zero motion candidates can be inserted in sequence. The size of the conventional Merge list is signaled in the slice header, and the maximum value is 6. MMVD, TPM, and CIIP rely on the conventional Merge list.

[0171] (3) IBC Merge List: This is done in a similar way to the regular Merge List.

[0172] Similarly, VVC supports three AMVP lists:

[0173] (1) Affine AMVP candidate list

[0174] (2) Regular AMVP Candidate List

[0175] (3) IBC AMVP candidate list: The construction process is the same as that of the IBC Merge list

[0176] 2.2.1 Codec Block Structure in VVC

[0177] In VVC, a quadtree / binary tree / ternary tree (QT / BT / TT) structure is used to divide images into square or rectangular blocks.

[0178] In addition to QT / BT / TT, a separate tree (also called dual codec tree) is also used in VVC for I frames. The codec block structure is signaled for luma and chroma components separately through separate trees.

[0179] In addition, CU is set equal to PU and TU, except for blocks coded with a few special codecs (such as intra sub-partition prediction, where PU is equal to TU but smaller than CU, and sub-block transform for inter-coded blocks, where PU is equal to CU but TU is smaller than PU).

[0180] 2.2.2 Affine Prediction Model

[0181] In HEVC, only the translation motion model is applied for motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, simplified affine transformation motion compensation prediction is applied to 4-parameter affine model and 6-parameter affine model. Fig.10 As shown, the affine motion field of the block is described by two control point motion vectors (CPMVs) of the 4-parameter affine model and 3 CPMVs of the 6-parameter affine model.

[0182] The motion vector field (MVF) of a block is described by the following equations respectively with a 4-parameter affine model in equation (1) (where the 4-parameters are defined as variables a, b, e, and f) and a 6-parameter affine model in equation (2) (where the 4-parameters are defined as variables a, b, c, d, e, and f):

[0183]

[0184]

[0185] Where (mv h 0,mv h 0) is the motion vector of the upper left control point, and (mv h 1, mv h 1) is the motion vector of the upper right control point, and (mv h 2.mv h 2) is the motion vector of the lower left control point. All three motion vectors are called control point motion vectors (CPMVs). (x, y) represents the coordinates of the representative point relative to the upper left sample point in the current block, and (mv h (x,y),mv v (x, y)) is the motion vector derived for the sample at (x, y). The CP motion vector can be signaled (such as in affine AMVP mode) or derived on the fly (such as in affine Merge mode). w and h are the width and height of the current block. In practice, the division is performed by right shifting with an integer operation. In VTM, the representative point is defined as the center position of the sub-block, for example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample point in the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2). For each sub-block (e.g., 4×4 in VTM), the representative point is used to derive the motion vector for the entire sub-block.

[0186] To further simplify motion compensated prediction, sub-block based affine transform prediction is applied. To derive the motion vector of each M×N (both M and N are set to 4 in the current VVC) sub-block, as Fig.11 As shown, the motion vector of the center sample of each sub-block is calculated according to equations (1) and (2) and rounded to a fractional accuracy of 1 / 16. Then, a 1 / 16 pixel motion compensation interpolation filter can be applied to generate a prediction for each sub-block using the derived motion vector. The 1 / 16 pixel interpolation filter is introduced by the affine mode.

[0187] After MCP, the high-accuracy motion vector of each sub-block is rounded and saved with the same accuracy as the normal motion vector.

[0188] 2.2.3 MERGE for the entire block

[0189] 2.2.3.1 Merge list construction for normal merge mode of translation

[0190] 2.2.3.1.1 History-based Motion Vector Prediction (HMVP)

[0191] Different from the Merge list design, in VVC, a history-based motion vector prediction (HMVP) method is adopted.

[0192] In HMVP, the motion information of the previous codec is stored. The motion information of the previous codec block is defined as the HMVP candidate. Multiple HMVP candidates are stored in a table named HMVP table, and the table is maintained during the real-time encoding / decoding process. When encoding / decoding a new slice / LCU row / slice starts, the HMVP table is cleared. Whenever there is an inter-coded block and non-sub-block, non-TPM mode, the associated motion information is added to the last entry of the table as a new HMVP candidate. The entire encoding and decoding process is as follows Fig.12 shown.

[0193] 2.2.3.1.2 Conventional Merge List Construction Process

[0194] The construction of a regular Merge list (for translational motion) can be summarized according to the following sequence of steps:

[0195] Step 1: Derive airspace candidates

[0196] Step 2: Insert HMVP candidates

[0197] Step 3: Insert pairwise average candidates

[0198] Step 4: Default Motion Candidates

[0199] HMVP candidates can be used in the construction process of both AMVP and Merge candidate lists. Fig.13 A modified Merge candidate list construction process is depicted. When the Merge candidate list is not full after the TMVP candidate is inserted, the HMVP candidates stored in the HMVP table can be used to fill the Merge candidate list. Considering that a block is usually more correlated with the nearest neighbor block in terms of motion information, the HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first, and the first entry is added last. Similarly, redundancy removal is applied to the HMVP candidates. Once the total number of available Merge candidates reaches the maximum number of Merge candidates allowed to be signaled, the Merge candidate list construction process terminates.

[0200] Note that all spatial / temporal / HMVP candidates should be coded and decoded in non-IBC mode. Otherwise, they are not allowed to be added to the regular Merge candidate list.

[0201] The HMVP table contains up to 5 regular motion candidates, and each candidate is unique.

[0202] 2.2.3.1.2.1 Pruning process

[0203] Candidates are added to the list only if the corresponding candidates for redundancy check do not have the same motion information. Such a comparison process is called a pruning process.

[0204] The pruning process in the spatial domain candidates depends on the usage of the TPM of the current block.

[0205] When the current block is encoded and decoded in a TPM mode not using (eg, normal Merge, MMVD, CIIP), the HEVC pruning process (eg, five prunings) for spatial Merge candidates is utilized.

[0206] 2.2.4 Triangular Prediction Model (TPM)

[0207] In VCC, triangular partitioning mode is supported for inter prediction. Triangular partitioning mode is only applied to CUs that are 8×8 or larger and are encoded and decoded in Merge mode but not in MMVD or CIIP mode. For CUs that meet these conditions, a CU-level flag is signaled to indicate whether triangular partitioning mode is applied.

[0208] When using this mode, Fig.14 As shown, the CU is evenly divided into two triangular partitions using diagonal partitioning or anti-diagonal partitioning. Each triangular partition in the CU is inter-predicted using its own motion; each partition only allows unidirectional prediction, that is, each partition has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that each CU only requires two motion compensated predictions, the same as conventional bidirectional prediction.

[0209] If the CU level flag indicates that the current CU is encoded using a triangle partitioning mode, further signaling is used to indicate the direction of the triangle partitioning (diagonal or anti-diagonal) and two Merge indices (one for each partition). After predicting each triangle partition, a blending process with adaptive weights is used to adjust the sample values ​​along the diagonal or anti-diagonal edges. This is the prediction signal for the entire CU, and as in other prediction modes, the transform and quantization process will be applied to the entire CU. Finally, the motion field of the CU predicted using the triangle partitioning mode is stored in 4×4 units.

[0210] The regular Merge candidate list is reused for triangular partition Merge prediction without additional motion vector pruning. For each Merge candidate in the regular Merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangular prediction. In addition, the order of selecting L0 and L1 motion vectors is based on their Merge index parity. With this scheme, the regular Merge list can be used directly.

[0211] 2.2.4.1 TPM Merge List Construction Process

[0212] In some embodiments, the conventional Merge list construction process may include the following modifications:

[0213] (1) How the pruning process is performed depends on the use of the TPM of the current block

[0214] - If TPM is not used to encode or decode the current block, call HEVC 5 pruning applied to the spatial merge candidate.

[0215] - Otherwise (if the current block is encoded using TPM codec), full pruning is applied when adding new spatial merge candidates. In other words, B1 is compared with A1; B0 is compared with A1 and B1; A0 is compared with A1, B1 and B0; and B2 is compared with A1, B1, A0 and B0.

[0216] (2) The condition on whether to check the motion information from B2 depends on the use of the TPM of the current block

[0217] - If the current block is not encoded and decoded using TPM, before checking B2, B2 is accessed and checked only if there are less than 4 spatial merge candidates.

[0218] - Otherwise (if TPM is used to encode and decode the current block), before adding B2, always access and check B2 regardless of how many available merge candidates there are.

[0219] 2.2.4.2 Adaptive Weighting Process

[0220] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangular prediction units to derive the final prediction for the entire CU. Two groups of weighting factors are defined as follows:

[0221] The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luminance and chrominance samples respectively;

[0222] The second weighting factor group: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luma and chroma samples respectively.

[0223] The weighting factor set is selected based on the comparison of the motion vectors of the two triangular prediction units. The second weighting factor set is used if any of the following conditions is true:

[0224] - The reference pictures of the two triangular prediction units are different from each other

[0225] -The absolute value of the difference between the horizontal values ​​of two motion vectors is greater than 16 pixels.

[0226] -The absolute value of the difference between the vertical values ​​of two motion vectors is greater than 16 pixels.

[0227] Otherwise, the first set of weighting factors is used. Fig.15 An example is shown in FIG.

[0228] 2.2.4.3 Motion Vector Storage

[0229] Motion vector of the triangle prediction unit ( Fig.16 Mv1 and Mv2 in CU are stored in a 4×4 grid. For each 4×4 grid, a unidirectional prediction or bidirectional prediction motion vector is stored, depending on the position of the 4×4 grid in the CU. Fig.16 As shown, for a 4×4 grid located in a non-weighted area (i.e., not located at a diagonal edge), a unidirectional prediction motion vector is stored - Mv1 or Mv2. On the other hand, for a 4×4 grid located in a weighted area, a bidirectional prediction motion vector is stored. The bidirectional prediction motion vector is derived from Mv1 and Mv2 according to the following rules:

[0230] (1) In the case where Mv1 and Mv2 have motion vectors from different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector.

[0231] (2) In the case where both Mv1 and Mv2 come from the same L0 (or L1) direction,

[0232] - If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, then Mv2 is scaled to that picture. Mv1 and the scaled Mv2 are combined to form a bi-directional prediction motion vector.

[0233] - If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, then Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bidirectional prediction motion vector.

[0234] - Otherwise, only Mv1 is stored for the weighted region.

[0235] 2.2.4.4 Syntax, semantics, and decoding process of Merge mode

[0236] Additions are highlighted in bold, italic and underlined. Deletions are marked with [[ ]].

[0237] 7.3.5.1 General Strip Header Syntax

[0238]

[0239]

[0240] 7.3.7.5 Codec unit syntax

[0241]

[0242]

[0243] 7.3.7.7 Merge Data Syntax

[0244]

[0245]

[0246] 7.4.6.1 General Strip Header Semantics

[0247] six_minus_max_num_merge_cand specifies the maximum number of Merge motion vector prediction (MVP) candidates supported in slices subtracted from 6. The maximum number of Merge MVP candidates MaxNumMergeCand is derived as follows:

[0248] MaxNumMergeCand=6-six_minus_max_num_merge_cand (7-57)

[0249] The value of MaxNumMergeCand should be in the range of 1 to 6 (inclusive).

[0250] five_minus_max_num_subblock_merge_cand specifies the maximum number of sub-block based Merge motion vector prediction (MVP) candidates supported in a slice minus 5. When five_minus_max_num_subblock_merge_cand is not present, it is inferred to be equal to 5-sps_sbtmvp_enabled_flag. The maximum number of sub-block based Merge MVP candidates MaxNumSubblockMergeCand is derived as follows:

[0251] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand (7-58)

[0252] The value of MaxNumSubblockMergeCand should be in the range of 0 to 5 (inclusive).

[0253] 7.4.8.5 Codec unit semantics

[0254] pred_mode_flag equal to 0 specifies that the current codec unit is coded in inter prediction mode. pred_mode_flag equal to 1 specifies that the current codec unit is coded in intra prediction mode.

[0255] When pred_mode_flag is not present, it can be inferred as follows:

[0256] - If cbWidth is equal to 4 and cbHeight is equal to 4, pred_mode_flag is inferred to be equal to 1.

[0257] - Otherwise, pred_mode_flag is inferred to be equal to 1 when decoding an I slice, and to be equal to 0 when decoding a P or B slice, respectively.

[0258] For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, the variable CuPredMode[x][y] is derived as follows:

[0259] - If pred_mode_flag is equal to 0, CuPredMode [y] is set equal to MODE_INTER.

[0260] - Otherwise (pred_mode_flag is equal to 1), set CuPredMode[x][y] equal to MODE_INTRA.

[0261] pred_mode_ibc_flag equal to 1 specifies that the current codec unit is coded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current codec unit is not coded in IBC prediction mode.

[0262] When pred_mode_ibc_flag is not present, it can be inferred as follows:

[0263] - If cu_skip_flag[x0][y0] is equal to 1, and cbWidth is equal to 4, and cbHeight is equal to 4, then pred_mode_ibc_flag is inferred to be equal to 1.

[0264] - Otherwise, if cbWidth and cbHeight are both equal to 128, pred_mode_ibc_flag is inferred to be equal to 0.

[0265] - Otherwise, when decoding an I slice, pred_mode_ibc_flag is inferred to be equal to the value of sps_ibc_enabled_flag; and correspondingly when decoding a P or B slice, the value of pred_mode_ibc_flag is inferred to be equal to 0.

[0266] When pred_mode_ibc_flag is equal to 1, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the variable CuPredMode[x][y] is set equal to MODE_IBC.

[0267] general_merge_flag[x0][y0] specifies whether the inter prediction parameters of the current codec unit are inferred from the inter prediction partitions of the neighboring blocks. The array indices x0, y0 specify the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0268] When general_merge_flag[x0][y0] does not exist, it can be inferred as follows:

[0269] - If cu_skip_flag[x0][y0] is equal to 1, general_merge_flag[x0][y0] is inferred to be equal to 1.

[0270] Otherwise, general_merge_flag[x0][y0] is inferred to be equal to 0.

[0271] mvp_l0_flag[x0][y0] specifies the motion vector predictor index of list 0, where x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered codec block relative to the top-left luma picture sample of the picture.

[0272] When mvp_l0_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0273] mvp_l1_flag[x0][y0] has the same semantics as mvp_l0_flag, where l0 and list0 are replaced by l1 and list1 respectively.

[0274] inter_pred_idc[x0][y0] specifies whether list 0, list 1 or bi-directional prediction is used for the current codec unit according to Tables 7-10. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0275] Table 7-10 – Names associated with inter prediction modes

[0276]

[0277] When inter_pred_idc[x0][y0] is not present, it is inferred to be equal to PRED_L0.

[0278] 7.4.8.7 Merge Data Semantics

[0279] regular_merge_flag[x0][y0] equal to 1 specifies that the regular Merge mode is used to generate the inter prediction parameters of the current codec unit. The array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0280] When regular_merge_flag[x0][y0] does not exist, it can be inferred as follows:

[0281] - regular_merge_flag[x0][y0] is inferred to be equal to 1 if all of the following are true:

[0282] -sps_mmvd_enabled_flag is equal to 0.

[0283] -general_merge_flag[x0][y0] is equal to 1.

[0284] -cbWidth*cbHeight is equal to 32.

[0285] Otherwise, regular_merge_flag[x0][y0] is inferred to be equal to 0.

[0286] mmvd_merge_flag[x0][y0] equal to 1 specifies the use of Merge mode with motion vector difference to generate inter prediction parameters for the current codec unit. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0287] When mmvd_merge_flag[x0][y0] does not exist, it can be inferred as follows:

[0288] - mmvd_merge_flag[x0][y0] is inferred to be equal to 1 if all of the following are true:

[0289] -sps_mmvd_enabled_flag is equal to 1.

[0290] -general_merge_flag[x0][y0] is equal to 1.

[0291] -cbWidth*cbHeight is equal to 32.

[0292] -regular_merge_flag[x0][y0] is equal to 0.

[0293] Otherwise, mmvd_merge_flag[x0][y0] is inferred to be equal to 0.

[0294] mmvd_cand_flag[x0][y0] specifies whether to use the first (0) or second (1) candidate in the Merge candidate list with the motion vector difference derived from mmvd_distance_idx[x0][y0] and mmvd_direction_idx[x0][y0]. The array index x0, y0 specifies the position (x0, y0) of the top-left luma sample of the considered codec block relative to the top-left luma sample of the picture.

[0295] When mmvd_cand_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0296] mmvd_distance_idx[x0][y0] specifies the index used to derive MmvdDistance[x0][y0], as shown in Table 7-12. The array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0297] Table 7-12 – Definition of MmvdDistance[x0][y0] based on mmvd_distance_idx[x0][y0]

[0298]

[0299] mmvd_direction_idx[x0][y0] specifies the index used to derive MmvdSign[x0][y0] as specified in Table 7-13. The array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0300] Table 7-13 – Definition of MmvdSign[x0][y0] based on mmvd_direction_idx[x0][y0]

[0301]

[0302] The two components of Merge plus MVD offset MmvdOffset[x0][y0] are derived as follows:

[0303] MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][0](0-2)

[0304] MmvdOffset[x0][y0][1]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][1](0-3)

[0305] merge_subblock_flag[x0][y0] specifies whether to infer subblock-based inter prediction parameters for the current codec from neighboring blocks. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered codec block relative to the top-left luma sample of the picture. When merge_subblock_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0306] merge_subblock_idx[x0][y0] specifies the merge candidate index of the sub-block based merge candidate list, where x0, y0 specify the position (x0, y0) of the upper left luma sample of the considered codec block relative to the upper left luma sample of the picture.

[0307] When merge_subblock_idx[x0][y0] is not present, it is inferred to be equal to 0.

[0308] ciip_flag[x0][y0] specifies whether combined inter-picture Merge and intra-picture prediction is applied to the current codec unit. Array index x0, y0 specifies the position (x0, y0) of the top left luma sample of the considered codec block relative to the top left luma sample of the picture.

[0309] When ciip_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0310] When ciip_flag[x0][y0] is equal to 1, the variable IntraPredModeY[x][y] with x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1 is set equal to INTRA_PLANAR.

[0311] The variable MergeTriangleFlag[x0][y0] specifies whether triangle-based motion compensation is used to generate prediction samples for the current codec unit when decoding the B slice. The variable MergeTriangleFlag[x0][y0] is derived as follows:

[0312] - Set MergeTriangleFlag[x0][y0] equal to 1 if all of the following are true:

[0313] -sps_triangle_enabled_flag is equal to 1.

[0314] -slice_type is equal to B.

[0315] -general_merge_flag[x0][y0] is equal to 1.

[0316] -MaxNumTriangleMergeCand is greater than or equal to 2.

[0317] -cbWidth*cbHeight is greater than or equal to 64.

[0318] -regular_merge_flag[x0][y0] is equal to 0.

[0319] -mmvd_merge_flag[x0][y0] is equal to 0.

[0320] -merge_subblock_flag[x0][y0] is equal to 0.

[0321] -ciip_flag[x0][y0] is equal to 0.

[0322] Otherwise, set MergeTriangleFlag[x0][y0] equal to 0.

[0323] merge_triangle_split_dir[x0][y0] specifies the split direction of the Merge triangle mode. The array index x0, y0 specifies the position (x0, y0) of the upper left luma sample of the considered codec block relative to the upper left luma sample of the picture.

[0324] When merge_triangle_split_dir[x0][y0] is not present, it is inferred to be equal to 0.

[0325] merge_triangle_idx0[x0][y0] specifies the first merge candidate index of the triangle-based motion compensation candidate list, where x0, y0 specify the position (x0, y0) of the upper left luma sample of the considered codec block relative to the upper left luma sample of the picture.

[0326] When merge_triangle_idx0[x0][y0] is not present, it is inferred to be equal to 0.

[0327] merge_triangle_idx1[x0][y0] specifies the second Merge candidate index of the triangle-based motion compensation candidate list, where x0, y0 specify the position (x0, y0) of the upper left luma sample of the considered codec block relative to the upper left luma sample of the picture.

[0328] When merge_triangle_idx1[x0][y0] is not present, it is inferred to be equal to 0.

[0329] merge_idx[x0][y0] specifies the merge candidate index of the merge candidate list, where x0, y0 specify the position (x0, y0) of the upper left luma sample of the considered codec block relative to the upper left luma sample of the picture.

[0330] When merge_idx[x0][y0] does not exist, it can be inferred as follows:

[0331] - If mmvd_merge_flag[x0][y0] is equal to 1, merge_idx[x0][y0] is inferred to be equal to mmvd_cand_flag[x0][y0].

[0332] Otherwise (mmvd_merge_flag[x0][y0] is equal to 0), merge_idx[x0][y0] is inferred to be equal to 0.

[0333] 2.2.4.4.1 Decoding process

[0334] In some embodiments, the decoding process is defined as follows:

[0335] 8.5.2.2 Merge mode luminance motion vector derivation process

[0336] This process is called only when general_merge_flag[XCB][YCB] is equal to 1, where (xCb, yCb) specifies the top-left sample of the current luma codec block relative to the top-left luma sample of the current picture.

[0337] The inputs to this process are:

[0338] - The luminance position (xCb, yCb) of the luminance sample of the upper left sample of the current luminance codec block relative to the luminance sample of the upper left sample of the current picture,

[0339] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[0340] - The variable cbHeight specifies the height of the current codec block in luma samples.

[0341] The output of this process is:

[0342] -1 / 16 fractional sample accuracy of the luma motion vectors in mvL0[0][0] and mvL1[0][0],

[0343] - reference indices refIdxL0 and refIdxL1,

[0344] - prediction list uses flags predFlagL0[0][0] and predFlagL1[0][0],

[0345] - Bidirectional prediction weight index bcwIdx.

[0346] -Merge candidate list mergeCandList.

[0347] The bidirectional prediction weight index bcwIdx is set equal to 0.

[0348] The motion vectors mvL0[0][0] and mvL1[0][0], the reference indices refIdxL0 and refIdxL1 and the prediction utilization flags predFlagL0[0][0] and predFlagL1[0][0] are derived by the following ordered steps:

[0349] 1. The derivation process of the spatial Merge candidate from the neighboring codec unit specified in section 8.5.2.4 is called with the following conditions: the luma codec block position (xCb, yCb), the luma codec block width cbWidth and the luma codec block height cbHeight as input; and the output is the availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1 and availableFlagB2, the reference indexes refIdxLXA0, refIdxL XA1, refIdxLXB0, refIdxLXB1, and refIdxLXB2, prediction lists utilizing flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1, and predFlagLXB2, and motion vectors mvLXA0, mvLXA1, mvLXB0, mvLXB1, and mvLXB2, where X is 0 or 1, and bidirectional prediction weight indices bcwIdxA0, bcwIdxA1, bcwIdxB0, bcwIdxB1, and bcwIdxB2.

[0350] 2. The reference index refIdxLXCol (X is 0 or 1) and the bidirectional prediction weight index bcwIdxCol of the time domain Merge candidate Col are set to be equal to 0.

[0351] 3. The derivation process of temporal luma motion vector prediction specified in section 8.5.2.11 is called with the following conditions: luma position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight and variable refIdxL0Col as input; and the output is the availability flag availableFlagL0Col and the temporal motion vector mvL0Col. The available variables FlagCol, predFlagL0Col and predFlagL1Col are derived as follows:

[0352] availableFlagCol=availableFlagL0Col (8-263)

[0353] predFlagL0Col=availableFlagL0Col (8-264)

[0354] predFlagL1Col=0 (8-265)

[0355] 4. When slice_type is equal to B, the derivation process of temporal luma motion vector prediction specified in section 8.5.2.11 is called with the following conditions: luma position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight and variable refIdxL1Col as input; and the output is the availability flag availableFlagL1Col and the temporal motion vector mvL1Col. The variables availableFlagCol and predFlagL1Col are derived as follows:

[0356] availableFlagCol=availableFlagL0Col||availableFlagL1Col (8-266)

[0357] predFlagL1Col=availableFlagL1Col (8-267)

[0358] 5. The Merge candidate list mergeCandList is constructed as follows:

[0359]

[0360] 6. The variables numCurrMergeCand and numOrigMergeCand are set equal to the number of Merge candidates in mergeCandList.

[0361] 7. When numCurrMergeCand is less than (MaxNumMergeCand-1) and NumHmvpCand is greater than 0, the following applies:

[0362] - Invoke the history-based Merge candidate derivation process specified in 8.5.2.6 with: mergeCandList and numCurrMergeCand as input, and the modified mergeCandList and numCurrMergeCand as output.

[0363] -numOrigMergeCand is set equal to numCurrMergeCand.

[0364] 8. When numCurrMergeCand is less than MaxNumMergeCand and greater than 1, the following applies:

[0365] - The derivation process of the pairwise average Merge candidates specified in section 8.5.2.4 is called with the following conditions: mergeCandList, reference indices refIdxL0N and refIdxL1N, the prediction list uses flags predFlagL0N and predFlagL1N, the motion vectors mvL0N and mvL1N for each candidate N in mergeCandList, and numCurrMergeCand as input; and the output is assigned to mergeCandList, numCurrMergeCand, reference indices refIdxL0avgCand and refIdxL1avgCand, the prediction list uses flags predFlagL0avgCand and predFlagL1avgCand and the motion vectors mvL0avgCand and mvL1avgCand of the candidate avgCand added to mergeCandList. The bidirectional prediction weight index bcwIdx of the candidate avgCand added to mergeCandList is set equal to 0.

[0366] -numOrigMergeCand is set equal to numCurrMergeCand.

[0367] 9. The derivation process of the zero motion vector Merge candidate specified in section 8.5.2.5 is called with the following conditions: mergeCandList, reference indices refIdxL0N and refIdxL1N, prediction list utilization flags predFlagL0N and predFlagL1N, motion vectors mvL0N and mvL1N for each candidate N in mergeCandList, and numCurrMergeCand as input; and the output is assigned to mergeCandList, numCurrMergeCand, reference index refIdxL0zeroCand m and refIdxL1zeroCand m , the prediction list uses the flag predFlagL0zeroCand m and predFlagL1zeroCand m and each new candidate zeroCand is added to mergeCandList m The motion vector mvL0zeroCand m and mvL1zeroCand m Each new candidate zeroCand is added to mergeCandList mThe bidirectional prediction weight index bcwIdx is set equal to 0. The number of candidates added numSeroMergeCand is set equal to (numCurrMergeCand-numOrigMergeCand). When numZeroMergeCand is greater than 0, m ranges from 0 to numZeroMergeCand-1 (inclusive).

[0368] 10. Perform the following assignment, N is the candidate at position merge_idx[xCb][yCb] in the Merge candidate list mergeCandList (N=mergeCandList[merge_idx[xCb][yCb]]), and X is replaced by 0 or 1:

[0369] refIdxLX=refIdxLXN (8-269)

[0370] predFlagLX[0][0]=predFlagLXN (8-270)

[0371] mvLX[0][0][0]=mvLXN[0] (8-271)

[0372] mvLX[0][0][1]=mvLXN[1] (8-272)

[0373] bcwIdx=bcwIdxN (8-273)

[0374] 11. When mmvd_merge_flag[xCb][yCb] is equal to 1, the following applies:

[0375] -Call the Merge motion vector difference derivation process specified in 8.5.2.7 with the following conditions: luma position (xCb, yCb), reference indices refIdxL0, refIdxL1 and prediction list using flags predFlagL0[0][0] and predFlagL1[0][0] as input, and motion vector differences mMvdL0 and mMvdL1 as output.

[0376] -The motion vector difference mMvdLX is added to the Merge motion vector mvLX, where X is 0 and 1, as follows:

[0377] mvLX[0][0][0]+=mMvdLX[0] (8-274)

[0378] mvLX[0][0][1]+=mMvdLX[1] (8-275)

[0379] mvLX[0][0][0]=Clip3(-2 17 ,2 17 -1,mvLX[0][0][0]) (8-276)

[0380] mvLX[0][0][1]=Clip3(-2 17 ,2 17 -1,mvLX[0][0][1]) (8-277)

[0381] 2.2.3.3 MMVD

[0382] In some embodiments, a final motion vector representation (UMVE, also called MMVD) is proposed. UMVE uses a motion vector representation method for Skip mode or Merge mode.

[0383] UMVE reuses the same Merge candidates as those included in the regular Merge candidate list in VVC. Among the Merge candidates, a basic candidate can be selected and further extended by a motion vector expression method.

[0384] UMVE provides a new method to represent motion vector difference (MVD), in which the starting point, motion amplitude and motion direction are used to represent the MVD.

[0385] In some embodiments, the Merge candidate list is used as is. However, only candidates of the default Merge type (MRG_TYPE_DEFAULT_N) are considered for UMVE extension.

[0386] The base candidate index defines the starting point. The base candidate index indicates the best candidate among the candidates in the list, as follows.

[0387] Table 4: Basic candidate IDX

[0388]

[0389] If the number of basic candidates is equal to 1, the basic candidate IDX is not signaled.

[0390] The distance index is the motion amplitude information. The distance index indicates the predefined distance from the start point information. The predefined distances are as follows:

[0391] Table 5. Distance IDX

[0392]

[0393] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions, as shown below.

[0394] Table 6. Direction IDX

[0395] Directions to IDX 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0396] The UMVE flag is signaled immediately after the Skip flag or Merge flag is sent. If the Skip or Merge flag is true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. However, if it is not 1, the AFFINE flag is parsed. If the AFFINE flag is equal to 1, it is AFFINE mode; however, if it is not equal to 1, the Skip / Merge index will be parsed for the Skip / Merge mode of the VTM.

[0397] No additional line buffer is needed due to UMVE candidates. Because software's Skip / Merge candidates are used directly as base candidates. Using the input UMVE index, the complement of the MV can be determined before motion compensation. No need to keep a long line buffer for this.

[0398] Under the current general test conditions, the first or second Merge candidate in the Merge candidate list may be selected as the basic candidate.

[0399] UMVE is also known as Merge with MV Difference (MMVD).

[0400] 2.2.6 Combined Intra-Inter Prediction (CIIP)

[0401] In some embodiments, multi-hypothesis prediction is proposed, where combined intra-prediction and inter-prediction is one method to generate multiple hypotheses.

[0402] When multi-hypothesis prediction is applied to improve intra mode, multi-hypothesis prediction combines one intra prediction with one merge index prediction. In Merge CU, when the flag is true, a flag is signaled for Merge mode to select intra mode from the intra candidate list. For luma component, the intra candidate list is derived from only one intra prediction mode, e.g. planar mode. The weights applied to the prediction block from intra and inter predictions are determined by the codec mode (intra or non-intra) of the two neighboring blocks (A1 and B1).

[0403] 2.2.7 Sub-block-based Merge Technology

[0404] It is recommended to put all sub-block related motion candidates not only into the regular Merge list of non-sub-block Merge candidates, but also into a separate Merge list.

[0405] The motion candidates related to the sub-blocks are placed in a separate Merge list, which is named "sub-block Merge candidate list".

[0406] In one example, the sub-block Merge candidate list includes ATMVP candidates and affine Merge candidates.

[0407] Fill the sub-block Merge candidate list with candidates in the following order:

[0408] 1. ATMVP candidate (may be available or not);

[0409] 2. Affine Merge list (including inherited affine candidates and constructed affine candidates)

[0410] 3. Filled with zeros MV 4-parameter affine model

[0411] 2.2.7.1 ATMVP (also known as sub-block temporal motion vector predictor, SbTMVP)

[0412] The basic idea of ​​ATMVP is to derive multiple sets of temporal motion vector predictors for a block. Each sub-block is assigned a set of motion information. When generating ATMVP Merge candidates, full motion compensation is performed at the 8×8 level rather than the entire block level.

[0413] In the current design, ATMVP predicts the motion vector of a sub-CU within a CU in two steps, which are described in the following two subsections respectively.

[0414] 2.2.7.1.1 Derivation of Initialized Motion Vectors

[0415] The initialization motion vector is denoted by tempMv. When block A1 is available and is not intra-coded (eg, coded using inter or IBC mode), the following is applied to derive the initialization motion vector.

[0416] - tempMv is set equal to the motion vector of block A1 from List 1, denoted by mvL1A1, if all of the following conditions are true:

[0417] - the reference picture index of list 1 is available (not equal to -1) and has the same POC value as the collocated picture (e.g., DiffPicOrderCnt(ColPic, RefPicList[1][refIdxL1A1]) is equal to 0),

[0418] - none of the reference pictures has a larger POC than the current picture (e.g., for each picture aPic in each reference picture list of the current slice, DiffPicOrderCnt(aPic, currPic) is less than or equal to 0),

[0419] - Current stripe is equal to B stripe,

[0420] -collocated_from_l0_flag is equal to 0.

[0421] Otherwise, if all of the following conditions are true, tempMv is set equal to the motion vector of block A1 from list 0, denoted by mvL0A1:

[0422] - the reference picture index of list 0 is available (not equal to -1),

[0423] - Has the same POC value as the collocated picture (e.g., DiffPicOrderCnt(ColPic, RefPicList[0][refIdxL0A1]) equals 0).

[0424] - Otherwise, the zero motion vector is used as the initialization MV.

[0425] In the collocated picture signaled at the slice header with the initialization motion vector, the corresponding block is identified (the center position of the current block plus the rounded MV, clipped within a certain range if necessary).

[0426] If the block is inter-coded, go to step 2. Otherwise, the ATMVP candidate is set to unavailable.

[0427] 2.2.7.1.2 Sub-CU motion export

[0428] The second step is to divide the current CU into sub-CUs and obtain motion information of each sub-CU from a block corresponding to each sub-CU in a collocated picture.

[0429] If the corresponding block of the sub-CU is coded in inter-frame mode, the motion information is used to derive the final motion information of the current sub-CU by calling the derivation process for the collocated MV, which is different from the process used for the traditional TMVP process. Basically, if the corresponding block is predicted from the target list X for unidirectional prediction or bidirectional prediction, the motion vector is used; otherwise, if the corresponding block is predicted from list Y (Y=1-X) for unidirectional prediction or bidirectional prediction, and NoBackwardPredFlag is equal to 1, the MV of list Y is used. Otherwise, no motion candidate can be found.

[0430] If the block in the collocated picture identified by the initialization MV and the position of the current sub-CU is intra or IBC coded, or no motion candidate can be found as described above, the following further applies:

[0431] Will be used to extract the concatenated image R col The motion vector of the motion field in is denoted as MV col To minimize the impact of MV scaling, you can select the MV to be exported in the following ways: col MV in the spatial candidate list of: If the reference picture of the candidate MV is a collocated picture, then this MV is selected and used as the MV without any scaling col Otherwise, select the MV with the reference picture closest to the collocated picture to derive the MV using scaling col .

[0432] An example decoding process for the collocated motion vector derivation process is described as follows:

[0433] 8.5.2.12 Derivation process of concatenated motion vectors

[0434] The inputs to this process are:

[0435] - variable currCb, specifies the current codec block,

[0436] - variable colCb specifies the collocated codec block within the collocated picture specified by ColPic,

[0437] - luma position (xColCb, yColCb), specifies the top-left sample of the collocated luma codec block specified by colCb relative to the top-left luma sample of the collocated picture specified by ColPic,

[0438] -reference index refIdIdLX, X is 0 or 1,

[0439] - Flag sbFlag indicating the sub-block time domain Merge candidate.

[0440] The output of this process is:

[0441] - Motion vector prediction mvLXCol in 1 / 16 fractional sample accuracy,

[0442] -Availability flag availableFlagLXCol.

[0443] The variable currPic specifies the current picture.

[0444] The arrays predFlagL0Col[x][y], mvL0Col[x][y], and refIdxL0Col[x][y] are set equal to PredFlagL0[x][y], MvDmvrL0[x][y], and RefIdxL0[x][y], respectively, of the collocated picture specified by ColPic, and the arrays predFlagL1Col[x][y], mvL1Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL1[x][y], MvDmvrL1[x][y], and RefIdxL1[x][y], respectively, of the collocated picture specified by ColPic.

[0445] The variables mvLXCol and availableFlagLXCol are exported as follows:

[0446] If colCb is encoded in intra or IBC prediction mode, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

[0447] Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows:

[0448] - If sbFlag is equal to 0, availableFlagLXCol is set to 1 and the following applies:

[0449] - If predFlagL0Col[xColPb][yColPb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL1Col[xColPb][yColPb], refIdxL1Col[xColPb][yColPb], and L1, respectively.

[0450] Otherwise, if predFlagL0Col[xColPb][yColPb] is equal to 1 and predFlagL1Col[xColPb][yColPb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL0Col[xColPb][yColPb], refIdxL0Col[xColPb][yColPb], and L0, respectively.

[0451] - Otherwise (predFlagL0Col[xColPb][yColPb] is equal to 1 and predFlagL1Col[xColPb][yColPb] is equal to 1), the following assignments are made:

[0452] - If NoBackwardPredFlag is equal to 1, mvCol, refIdxCol, and listCol are set equal to mvLXCol[xColPb][yColPb], refIdxLXCol[xColPb][yColPb], and LX, respectively.

[0453] - Otherwise, mvCol, refIdxCol, and listCol are set equal to mvLNCol[xColPb][yColPb], refIdxLNCol[xColPb][yColPb], and LN, respectively, where N is the value of collocated_from_l0_flag.

[0454] - Otherwise (sbFlag equals 1), the following applies:

[0455] - If PredFlagLXCol[xColCb][yColCb] is equal to 1, then mvCol, refIdxCol, and listCol are set equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb], and LX, respectively, and availableFlagLXCol is set to 1.

[0456] - Otherwise (PredFlagLXCol[xColCb][yColCb] is equal to 0), the following applies:

[0457] - If DiffPicOrderCnt(aPic, currPic) for each picture aPic in each reference picture list of the current slice is less than or equal to 0, and PredFlagLYCol[xColCb][yColCb] is equal to 1, then mvCol, refIdxCol, and listCol are set to mvLYCol[xColCb][yColCb], refIdxLYCol[xColCb][yColCb], and LY, respectively, where Y is equal to !X, where X is the value of X at which this process was called. availableFlagLXCol is set to 1.

[0458] Both components of -mvLXCol are set to 0, and availableFlagLXCol is set equal to 0.

[0459] -When availableFlagLXCol is equal to TRUE, mvLXCol and availableFlagLXCol are derived as follows:

[0460] - If LongTermRefPic(currPic, currCb, refIdxLX, LX) is not equal to LongTermRefPic(ColPic, colCb, refIdxCol, listCol), both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

[0461] Otherwise, the variable availableFlagLXCol is set equal to 1, refPicList[listCol][refIdxCol] is set to the picture with reference index refIdxCol in the reference picture list listCol of the slice containing the codec block colCb in the collocated picture specified by ColPic, and the following applies:

[0462] colPocDiff=DiffPicOrderCnt(ColPic,refPicList[listCol][refIdxCol])(8-402)

[0463] currPocDiff=DiffPicOrderCnt(currPic,RefPicList[X][refIdxLX]) (8-403)

[0464] - Invoke the temporal motion buffer compression process for collocated motion vectors as specified in clause 8.5.2.15 with: mvCol as input and modified mvCol as output.

[0465] - If RefPicList[X][refIdxLX] is a long-term reference picture, or colPocDiff is equal to currPocDiff, then mvLXCol is derived as follows:

[0466] mvLXCol=mvCol (8-404)

[0467] Otherwise, mvLXCol is exported as a scaled version of the motion vector mvCol, as follows:

[0468] tx=(16384+(Abs(td)>>1)) / td (8-405)

[0469] distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6) (8-406)

[0470] mvLXCol=Clip3(-131072,131071,(distScaleFactor*mvCol+128-(distScaleFactor*mvCol>=0))>>8)) (8-407)

[0471] where td and tb are derived as follows:

[0472] td=Clip3(-128,127,colPocDiff) (8-408)

[0473] tb=Clip3(-128,127,currPocDiff) (8-409)

[0474] 2.2.8 Refinement of motion information

[0475] 2.2.8.1 Decoder-side Motion Vector Refinement (DMVR)

[0476] In the bidirectional prediction operation, for the prediction of a block area, two prediction blocks formed by using the motion vector (MV) of list 0 and the MV of list 1, respectively, are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined.

[0477] For DMVR in VVC, it is assumed that the MVD mirroring between list 0 and list 1 is as follows Fig.19 As shown, bilateral matching is performed to refine the MV, for example, to find the best MVD among several MVD candidates. The MVs of the two reference picture lists are represented by MVL0 (L0X, L0Y) and MVL1 (L1X, L1Y). The MVD represented by (MvdX, MvdY) of list 0 that can minimize the cost function (e.g., SAD) is defined as the best MVD. For the SAD function, it is defined as the SAD between the reference block of list 0 derived with the motion vector (L0X+MvdX, L0Y+MvdY) in the list 0 reference picture and the reference block of list 1 derived with the motion vector (L1X-MvdX, L1Y-MvdY) in the list 1 reference picture.

[0478] The motion vector refinement process can be iterated twice. In each iteration, up to 6 MVDs (with integer pixel precision) can be checked in two steps, such as Fig. 20As shown. In the first step, the MVD (0, 0), (-1, 0), (1, 0), (0, -1), (0, 1) is checked. In the second step, one of the MVDs (-1, -1), (-1, 1), (1, -1), or (1, 1) can be selected and further checked. Assume that the function Sad(x, y) returns the SAD value of MVD(x, y). The MVD represented by (MvdX, MvdY) checked in the second step is determined as follows:

[0479] MvdX = -1;

[0480] MvdY=-1;

[0481] If(Sad(1,0) <Sad(-1,0))

[0482] MvdX = 1;

[0483] If(Sad(0,1) <Sad(0,-1))

[0484] MvdY=1;

[0485] In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the best MVD selected in the first iteration. DMVR is applied only when one reference picture is the previous picture and the other reference picture is the next picture and both reference pictures have the same picture order count distance to the current picture.

[0486] To further simplify the DMVR process, in some embodiments, the DMVR design used has the following main features:

[0487] - When the SAD at position (0, 0) between list 0 and list 1 is less than the threshold, terminate early.

[0488] - Early termination when the SAD between list 0 and list 1 is zero for some positions.

[0489] - Block size of DMVR: W*H>=64&&H>=8, where W and H are the width and height of the block.

[0490] -For DMVR with CU size > 16*16, the CU is divided into multiple 16x16 sub-blocks. If only the width or height of the CU is greater than 16, it is divided only in the vertical or horizontal direction.

[0491] - Reference block size (W+7)*(H+7) (for luma).

[0492] - 25-point SAD-based integer pixel search (i.e., (+-)2 refinement search range, single stage)

[0493] -DMVR based on bilinear interpolation.

[0494] - Sub-pixel refinement based on the “parametric error surface equation”. This process is performed only if the minimum SAD cost is not equal to zero and the best MVD was (0, 0) in the previous MV refinement iteration.

[0495] - Luma / Chroma MC w / reference block padding (if needed).

[0496] - Refinement MV for MC and TMVP only.

[0497] 2.2.8.1.1 Use of DMVR

[0498] DMVR can be enabled when all of the following conditions are true:

[0499] -DMVR enabled flag in SPS (i.e., sps_dmvr_enabled_flag) equal to 1

[0500] -TPM flag, inter-frame affine flag, sub-block Merge flag (ATMVP or affine Merge), MMVD flag are all equal to 0

[0501] -Merge flag equal to 1

[0502] - The current block is bi-predicted, and the POC distance between the current picture and the reference picture in list 1 is equal to the POC distance between the reference picture and the current picture in list 0

[0503] - The height of the current CU is greater than or equal to 8

[0504] -The number of luma samples (CU width * height) is greater than or equal to 64

[0505] 2.2.8.1.2 Sub-pixel refinement based on the “parametric error surface equation”

[0506] The method is summarized as follows:

[0507] 1. The parameter error surface fit is calculated only if the center location is the best cost location in a given iteration.

[0508] 2. The cost at the center and the costs at (-1, 0), (0, -1), (1, 0), and (0, 1) from the center are used to fit a two-dimensional parabolic error surface equation of the following form

[0509] E(x,y)=A(x-x0) 2 +B(y-y0) 2 +C

[0510] Where (x0, y0) corresponds to the location with the lowest cost, and C corresponds to the minimum cost value. By solving the 5 equations in the 5 unknowns, (x0, y0) can be calculated as:

[0511] x0=(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0)))

[0512] y0=(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0)))

[0513] (x0, y0) can be calculated to any desired sub-pixel precision by adjusting the precision with which the division is performed (e.g., how many bits of the quotient are calculated). For 1 / 16 pixel accuracy, only 4 bits of the absolute value of the quotient need to be calculated, which facilitates the implementation of fast-shift-based subtraction for the 2 divisions required per CU.

[0514] 3. Add the calculated (x0, y0) to the integer distance refinement MV to obtain a sub-pixel accurate refinement delta MV.

[0515] 2.3 Intra-frame Block Copy

[0516] HEVC Screen Content Codec Extension (HEVC-SCC) and the current VVC Test Model (VTM-4.0) have adopted intra-frame block copy (IBC), also known as current picture reference. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Fig.21 As shown, when IBC is applied, the current block is predicted by a reference block in the same picture. Before the current block is encoded or decoded, the samples in the reference block must have been reconstructed. Although IBC is not efficient for most camera-captured sequences, it shows significant codec gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. IBC can effectively remove the redundancy between these repeated patterns. In HEVC-SCC, if the current picture is selected as its reference picture, the inter-frame codec unit (CU) can apply IBC. In this case, MV is renamed as block vector (BV), and BV always has integer pixel precision. In order to be compatible with the main configuration document HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (Decoded Picture Buffer, DPB). It should be noted that similarly, in the multi-view / 3D video codec standard, the inter-view reference picture is also marked as a "long-term" reference picture.

[0517] After the BV finds its reference block, a prediction can be generated by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Then, transforms and quantization can be applied as in other codec modes.

[0518] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to some constraints, some or all pixel values ​​are undefined. Basically, there are two solutions to this problem. One is to prohibit this situation, for example in bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0519] 2.3.1 IBC in VVC Test Model (VTM4.0)

[0520] In the current VVC test model (such as the VTM-4.0 design), the entire reference block should be together with the current codec tree unit (CTU) and not overlap with the current block. Therefore, there is no need to fill the reference or prediction block. The IBC flag is encoded and decoded as the prediction mode of the current CU. Therefore, there are a total of three prediction modes for each CU: MODE_INTRA\MODE_INTER and MODE_IBC.

[0521] 2.3.1.1 IBC Merge Mode

[0522] In IBC Merge mode, indices pointing to entries in the IBC Merge candidate list are parsed from the bitstream. The construction of the IBC Merge list can be summarized in the following sequence of steps:

[0523] Step 1: Derive airspace candidates

[0524] Step 2: Insert HMVP candidates

[0525] Step 3: Insert pairwise average candidates

[0526] In the derivation of spatial merge candidates, in the Figure 2 Up to four Merge candidates are selected from the candidates in the positions shown in the depicted A1, B1, B0, A0, and B2. The derived order is A1, B1, B0, A0, and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is unavailable (for example, because it belongs to another stripe or slice) or is not encoded and decoded in IBC mode. After adding the candidate at position A1, a redundancy check is performed on the insertion of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, so that the encoding and decoding efficiency is improved.

[0527] After inserting the spatial domain candidates, if the IBC Merge list size is still less than the maximum IBC Merge list size, the IBC candidates from the HMVP table can be inserted. A redundancy check is performed when inserting HMVP candidates.

[0528] Finally, the pairwise average candidates are inserted into the IBC Merge list.

[0529] When the reference block identified by a Merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstruction area, or is outside the valid area subject to certain constraints, the Merge candidate is called an invalid Merge candidate.

[0530] Note that invalid Merge candidates can be inserted into the IBC Merge list.

[0531] 2.3.1.2 IBC AMVP Model

[0532] In IBC AMVP mode, the AMVP index pointing to the entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized according to the following sequence of steps:

[0533] Step 1: Derive airspace candidates

[0534] - Check A0, A1 until a usable candidate is found.

[0535] - Check B0, B1, B2 until a usable candidate is found.

[0536] Step 2: Insert HMVP candidates

[0537] Step 3: Insert zero candidates

[0538] After inserting the spatial candidates, if the IBC AMVP list size is still less than the maximum IBC AMVP list size, the IBC candidates from the HMVP table may be inserted.

[0539] Finally, the zero candidate is inserted into the IBC AMVP list.

[0540] 2.3.1.3 Chroma IBC Mode

[0541] In the current VVC, motion compensation in chroma IBC mode is performed at the sub-block level. The chroma block will be split into several sub-blocks. Each sub-block determines whether the corresponding luminance block has a block vector and the validity (if any). There is an encoder constraint in the current VTM where the chroma IBC mode will be tested if all sub-blocks in the current chroma CU have valid luminance block vectors. For example, on YUV 420 video, the chroma block is NxM and then the concatenated luminance area is 2Nx2M. The sub-block size of the chroma block is 2x2. There are several steps to perform the chroma mv export followed by the block copy process.

[0542] (1) The chrominance block will first be split into (N>>1)*(M>>1) sub-blocks.

[0543] (2) Each sub-block with an upper left sample point with coordinates (x, y) retrieves the corresponding luminance block that covers the same upper left sample point with coordinates (2x, 2y).

[0544] (3) The encoder checks the block vector (bv) of the retrieved luminance block. If one of the following conditions is met, bv is considered invalid.

[0545] a. bv corresponding to the luminance block does not exist.

[0546] b. The prediction block identified by bv has not yet been reconstructed.

[0547] c. The prediction block identified by bv partially or completely overlaps with the current block.

[0548] (4) Set the chrominance motion vector of the sub-block to the motion vector of the corresponding luminance sub-block.

[0549] When valid bv is found for all sub-blocks, IBC mode is allowed at the encoder.

[0550] 2.3.2 Latest Developments of IBC

[0551] 2.3.2.1 Single BV List

[0552] In some embodiments, the BV predictors for Merge mode and AMVP mode in IBC share a common predictor list, which contains the following elements:

[0553] (1) Two spatial neighboring locations (such as Figure 2 A1, B1 in

[0554] (2) 5 HMVP entries

[0555] (3) Default zero vector

[0556] The number of candidates in the list is controlled by a variable derived from the stripe header. For Merge mode, up to the first 6 entries of this list can be used; for AMVP mode, the first 2 entries of this list can be used. And this list meets the shared Merge list region requirement (sharing the same list within SMR).

[0557] In addition to the above BV predictor candidate list, the pruning operation between the HMVP candidate and the existing Merge candidate (A1, B1) can be simplified. In the simplification, at most 2 pruning operations will be performed because it only compares the first HMVP candidate with (one or more) spatial Merge candidates.

[0558] 2.3.2.2 Size restrictions of IBC

[0559] In some embodiments, in previous VTM and VVC versions, a syntax constraint for disabling 128x128 IBC mode may be explicitly used on top of the current bitstream constraints, which makes the presence of the IBC flag dependent on the CU size < 128x128.

[0560] 2.3.2.3 IBC's shared merge list

[0561] In order to reduce decoder complexity and support parallel encoding, in some embodiments, all leaf codec units (CUs) of an ancestor node in the CU partition tree can share the same Merge candidate list to enable parallel processing of small skip / merge coded CUs. The ancestor node is named Merge shared node. Assuming that the Merge shared node is a leaf CU, a shared Merge candidate list is generated at the Merge shared node.

[0562] More specifically, the following may apply:

[0563] - If the block has no more than 32 luma samples and is divided into 2 4x4 child blocks, a shared Merge list between very small blocks (e.g., two adjacent 4x4 blocks) is used.

[0564] - If the block has more than 32 luma samples, but after partitioning, at least one sub-block is smaller than the threshold (32), all child blocks of the partition share the same Merge list (e.g., 16x4 or 4x16 ternary partition or 8x8 quad partition).

[0565] Such restrictions apply only to IBC Merge mode.

[0566] 3. Examples of technical problems solved by the embodiments

[0567] A block can be coded and decoded in IBC mode. However, different sub-regions within a block can have different contents. Further research is needed on how to explore the correlation with previously coded blocks within the current frame.

[0568] 4. Examples of Embodiments

[0569] In this document, intra block copy (IBC) may not be limited to the current IBC technology, but may be interpreted as a technology that uses reference samples within the current slice / slice / block / picture / other video unit (e.g., CTU row), but excludes traditional intra prediction methods.

[0570] To solve the above problems, a sub-block based IBC (sbIBC) coding method is proposed. In sbIBC, the video block (e.g., CU / PU / CB / PB) of the current IBC coding is divided into multiple sub-blocks. The size of each sub-block can be smaller than the size of the video block. For each corresponding sub-block from the multiple sub-blocks, the video codec can identify the reference block of the corresponding sub-block in the current picture / slice / slice / group block / slice group. The video codec can use the motion parameters of the identified reference block of the corresponding sub-block to determine the motion parameters of the corresponding sub-block.

[0571] In addition, IBC is not limited to being applied to unidirectional prediction codec blocks. Bidirectional prediction can also be supported, where both reference pictures are current pictures. Alternatively, bidirectional prediction can also be supported, where one reference picture is from the current picture and the other reference picture is from a different picture. In yet another example, multiple assumptions can also be applied.

[0572] The following list should be considered as an example to explain the general concept. These techniques should not be interpreted narrowly. In addition, these techniques can be combined in any way. Neighborhood blocks A0, A1, B0, B1 and B2 are as follows: Figure 2 shown.

[0573] 1. In sbIBC, a block of size equal to M×N can be divided into more than one sub-block.

[0574] a. In one example, the sub-block size is fixed to L×K, for example, L=K=4.

[0575] b. In one example, the sub-block size is fixed to the minimum decoding unit / prediction unit / transformation unit / motion information storage unit.

[0576] c. In one example, a block may be divided into multiple sub-blocks of different sizes or of equal sizes.

[0577] d. In one example, an indication of the sub-block size may be signaled.

[0578] e. In an example, the indication of sub-block size may vary from block to block, for example, depending on the block dimensions.

[0579] f. In one example, the sub-block size must be in the form of (N1×minW)×(N2×minH), where minW×minH represents the minimum decoding unit / prediction unit / transform unit / motion information storage unit, and N1 and N2 are positive integers.

[0580] g. In one example, the sub-block dimensions may depend on the color format and / or color components.

[0581] i. For example, the sub-block sizes of different color components can be different.

[0582] 1) Alternatively, the sub-block sizes of different color components may be the same.

[0583] ii. For example, when the color format is 4:2:0, a 2L×2K sub-block of a luma component may correspond to an L×K sub-block of a chroma component.

[0584] 1) Alternatively, when the color format is 4:2:0, four 2L×2K sub-blocks of the luma component may correspond to a 2L×2K sub-block of the chroma component.

[0585] iii. For example, when the color format is 4:2:2, a 2L×2K sub-block of a luma component may correspond to a 2L×K sub-block of a chroma component.

[0586] 1) Alternatively, when the color format is 4:2:2, two 2L×2K sub-blocks of the luma component may correspond to 2L×2K sub-blocks of the chroma component.

[0587] iv. For example, when the color format is 4:4:4, a 2L×2K sub-block of a luma component may correspond to a 2L×2K sub-block of a chroma component.

[0588] h. In one example, the MV of a sub-block of a first color component may be derived from a corresponding sub-block or multiple corresponding sub-blocks of a second color component.

[0589] i. For example, the MV of a sub-block of a first color component may be derived as an average MV of a plurality of corresponding sub-blocks of a second color component.

[0590] ii. Alternatively, furthermore, the above method may be applied when a single tree is utilized.

[0591] iii. Alternatively, also when for certain block sizes, such as 4×4 chroma blocks, the above method can be applied.

[0592] i. In one example, the sub-block size may depend on the codec mode, such as IBC Merge / AMVP mode.

[0593] j. In one example, the sub-blocks may be non-rectangular, such as triangular / wedge-shaped.

[0594] 2. Using two stages, including using an initialization motion vector (denoted as initMV) to identify a corresponding reference block, and deriving one or more motion vectors of the sub-CU based on the reference block to obtain motion information of the sub-CU, at least one of whose reference pictures is equal to the current picture.

[0595] a. In one example, the reference block may be in the current picture.

[0596] b. In one example, the reference block may be in a reference picture.

[0597] i. For example, it can be in a collocated reference picture.

[0598] ii. For example, it may be in a reference picture identified by using motion information of a collocated block or a neighboring block of the collocated block.

[0599] Phase 1.a: Setting of initMV(vx,vy)

[0600] c. In one example, initMV may be derived from one or more neighboring blocks (adjacent or non-adjacent) in the current block or current sub-block.

[0601] i. The neighboring block can be one of the same picture.

[0602] 1) Alternatively, it can be one of the reference pictures.

[0603] a. For example, it can be in a juxtaposed reference picture.

[0604] b. For example, it can be identified by using motion information of the collocated block or a neighboring block of the collocated block.

[0605] ii. In one example, it can be derived from the neighborhood block Z.

[0606] 1) For example, initMV can be set equal to the MV stored in the neighborhood block Z.

[0607] For example, the neighborhood block Z may be block A1.

[0608] iii. In one example, it can be derived from multiple blocks that are examined in sequence.

[0609] 1) In one example, the first identified motion vector associated with the current picture as a reference picture of the checked block may be set to initMV.

[0610] d. In one example, initMV can be derived from the motion candidate list.

[0611] i. In one example, it can be derived from the kth (eg, first) candidate in the IBC candidate list.

[0612] 1) In one example, the IBC candidate list is a Merge / AMVP candidate list.

[0613] 2) In one example, an IBC candidate list different from the existing IBC Merge candidate list construction process may be utilized, such as using a different spatial neighborhood block.

[0614] ii. In one example, it can be derived from the kth (eg, first) candidate in the IBC HMVP table.

[0615] e. In one example, it can be derived based on the position of the current block.

[0616] f. In one example, it can be derived depending on the dimension of the current block.

[0617] g. In one example, it can be set as a default value.

[0618] h. In one example, the indication of initMV may be signaled at a video unit level (such as slice / slice / picture / chunk / CTU row / CTU / CTB / CU / PU / TU, etc.).

[0619] i. The initial MVs of two different sub-blocks within the current block may be different.

[0620] j. How to derive the initial MV may vary from block to block, from slice to slice, from strip to strip, and so on.

[0621] Stage 1.b: Using initMV to identify the corresponding reference block of the sub-CU

[0622] k. In one example, initMV may be first converted to integer precision of 1 pixel, and the converted MV may be used to identify the corresponding block of the sub-block. The converted MV represented by (vx', vy') is represented.

[0623] i. In one example, if (vx, vy) is at an inter-frame accuracy of F pixels, then the MV of the transformation represented by (vx', vy') can be set to (vx*F, vy*F) (eg, F=2 or 4).

[0624] ii. Alternatively, (vx', vy') is directly set equal to (vx, vy).

[0625] l. Assume that the upper left position of a sub-block is (x, y) and the sub-block size is K × L. The corresponding block of the sub-block is set to the CU / CB / PU / PB covering the coordinates (x+offsetX+vx', y+offsetY+vy'), where offsetX and offsetY are used to indicate the selected coordinates relative to the current one.

[0626] i. In one example, offsetX and / or offsetY are set to 0.

[0627] ii. In one example, offsetX may be set to (L / 2) or (L / 2+1) or (L / 2−1), where L may be the width of the sub-block.

[0628] iii. In one example, offsetY may be set to (K / 2) or (K / 2+1) or (K / 2-1), where K may be the height of the sub-block.

[0629] iv. Alternatively, the horizontal and / or vertical offsets may be further clipped to a range, such as within a picture / strip / tile / block boundary / IBC reference area, etc.

[0630] Stage 2 is about deriving the motion vector of the sub-block (denoted by subMV (subMVx, subMVy)

[0631] m. The subMV of the sub-block is derived from the motion information of the corresponding block.

[0632] i. In one example, if the corresponding block has a motion vector pointing to the current picture, subMV is set equal to MV.

[0633] ii. In one example, if the corresponding block has a motion vector pointing to the current picture, subMV is set equal to MV plus initMV.

[0634] n. The exported subMV can be further clipped to a given range, or clipped to ensure it points to the IBC reference area.

[0635] o. In a conforming bitstream, the derived subMV must be a valid MV of the IBC of the sub-block.

[0636] 3. One or more IBC candidates with sub-block motion vectors may be generated, which may be denoted as sub-block IBC candidates.

[0637] 4. The sub-block IBC candidate can be inserted into the sub-block Merge candidate including ATMVP and affine Merge candidate.

[0638] a. In one example, it can be added before all other sub-block Merge candidates.

[0639] b. In one example, can be added after the ATMVP candidate.

[0640] c. In one example, it can be added after the inherited affine candidate or the constructed affine candidate.

[0641] d. In one example, it can be added to the IBC Merge / AMVP candidate list.

[0642] i. Alternatively, whether to add may depend on the mode information of the current block. For example, if it is IBC AMVP mode, it may not be added.

[0643] e. The candidate list to be added may depend on the partitioning structure, eg, dual tree or single tree.

[0644] f. Alternatively, multiple sub-block IBC candidates may be inserted into the sub-block Merge candidate.

[0645] 5. An IBC sub-block motion (eg, AMVP / Merge) candidate list may be constructed using at least one sub-block IBC candidate.

[0646] a. Alternatively, one or more sub-block IBC candidates may be inserted into the IBC Sub-block Merge candidate, for example, using different initialization MVs.

[0647] b. Alternatively, in addition, whether to construct an IBC sub-block motion candidate list or an existing IBC AMVP / Merge candidate list can be signaled by an indicator, or derived on the fly.

[0648] c. Alternatively, if the current block is encoded and decoded in IBC Merge mode, the index of the IBC sub-block Merge candidate list may be signaled.

[0649] d. Alternatively, in addition, if the current block is encoded and decoded in IBC AMVP mode, the index of the IBC sub-block AMVP candidate list may be signaled.

[0650] i. Alternatively, additionally, the signaled / derived MVD for IBC AMVP mode may be applied to one or more sub-blocks.

[0651] 6. The reference block and the sub-block of a sub-block may belong to the same color component.

[0652] Extending sbIBC by mixing other tools applied to different sub-blocks of the same block

[0653] 7. A block can be divided into multiple sub-blocks, where at least one sub-block is coded or decoded in IBC mode and at least one sub-block is coded or decoded in intra mode.

[0654] a. In one example, for a sub-block, a motion vector may not be derived. Instead, for a sub-block, one or more intra prediction modes may be derived.

[0655] b. Alternatively, the palette mode or / and the palette table may be exported.

[0656] c. In one example, one intra prediction mode may be derived for the entire block.

[0657] 8. A block can be divided into multiple sub-blocks, all of which are coded or decoded in intra mode.

[0658] 9. A block can be divided into multiple sub-blocks, where all sub-blocks are encoded and decoded in palette mode.

[0659] 10. A block may be divided into multiple sub-blocks, wherein at least one sub-block is encoded or decoded in IBC mode and at least one sub-block is encoded or decoded in palette mode.

[0660] 11. A block may be divided into a plurality of sub-blocks, wherein at least one sub-block is coded or decoded in intra mode and at least one sub-block is coded or decoded in palette mode.

[0661] 12. A block may be divided into multiple sub-blocks, wherein at least one sub-block is coded or decoded in IBC mode and at least one sub-block is coded or decoded in inter-frame mode.

[0662] 13. A block may be divided into a plurality of sub-blocks, wherein at least one sub-block is coded or decoded in intra mode and at least one sub-block is coded or decoded in inter mode.

[0663] Interaction with other tools

[0664] 14. When one or more of the above methods are applied, the IBC HMVP table may not be updated.

[0665] a. Alternatively, one or more of the motion vectors for the IBC codec sub-region may be used to update the IBC HMVP table.

[0666] 15. When one or more of the above methods are applied, non-IBC HMVP tables may not be updated.

[0667] b. Alternatively, one or more of the motion vectors for the inter-coded sub-region may be used to update the non-IBC HMVP table.

[0668] 16. The loop filtering process (eg, deblocking process) may depend on the use of the above methods.

[0669] a. In one example, when applying one or more of the above methods, sub-block boundaries may be filtered.

[0670] a. Alternatively, when applying one or more of the above methods, sub-block boundaries may be filtered.

[0671] b. In one example, a block encoded using the above method may be treated in a similar manner to a conventional IBC encoded block.

[0672] 17. For blocks using one or more of the above methods, certain coding methods can be disabled (for example, sub-block transform, affine motion prediction, multiple reference line intra prediction, matrix-based intra prediction, symmetric MVD coding, Merge using MVD decoder-side motion derivation / refinement, bidirectional best flow, simplified secondary transform, multiple transform sets, etc.).

[0673] 18. The indication of the use of the above methods and / or sub-block sizes may be signaled at sequence / picture / slice / slice group / slice / block / CTU / CTB / CU / PU / TU / other video unit level or derived on the fly.

[0674] a. In one example, one or more of the above methods can be considered as a special IBC mode.

[0675] i. Alternatively, additionally, if a block is coded in IBC mode, a further indication of using a conventional whole-block based IBC approach or sbIBC may be signaled or derived.

[0676] ii. In one example, the subsequent IBC codec block can use the motion information of the current sbIBC codec block as the MV predictor.

[0677] 1. Alternatively, subsequent IBC codec blocks may not be allowed to utilize the motion information of the current sbIBC codec block as MV predictor.

[0678] b. In one example, sbIBC may be indicated by a candidate index of a motion candidate list.

[0679] i. In one example, a specific candidate index is assigned to the sbIBC codec block.

[0680] c. In one example, IBC candidates may be divided into two categories: one for whole block coding and the other for sub-block coding. Whether a block is coded in sbIBC mode may depend on the category of the IBC candidate.

[0681] Use of tools

[0682] 19. Whether and / or how the above approach is applied may depend on the following information:

[0683] a. Messages signaled in DPS / SPS / VPS / PPS / APS / Picture Header / Slice Header / Slice Group Header / Largest Codec Unit (LCU) / Codec Unit (CU) / LCU Line / LCU Group / TU / PU Block / Video Codec Unit

[0684] b. Location of CU / PU / TU / block / video codec unit

[0685] c. Block dimensions of the current block and / or its neighboring blocks

[0686] d. Block shape of the current block and / or its neighboring blocks

[0687] e. Intra mode of the current block and / or its neighboring blocks

[0688] f. The motion / block vector of its neighboring blocks

[0689] g. Indication of color format (such as 4:2:0, 4:4:4)

[0690] h. Codec tree structure

[0691] i. Slice / slice group type and / or picture type

[0692] j. Color components (e.g., may apply only to chroma components or luma components)

[0693] k. Time domain layer ID

[0694] l. Standard configuration documents / levels / tiers

[0695] Thoughts on the Merge list building process and IBC usage

[0696] 20. IBC mode can be used together with inter prediction mode to inter-code blocks in a picture / slice / slice group / slice.

[0697] a. In one example, for IBC AMVP mode, a syntax element may be signaled to indicate whether the current block is predicted from both the current picture and a reference picture different from the current picture (denoted as a temporal reference picture).

[0698] i. Alternatively, in addition, if the current block can also be predicted from a temporal reference picture, a syntax element may be signaled to indicate which temporal reference picture is used and its associated MVP index, MVD, MV precision, etc.

[0699] ii. In one example, for IBC AMVP mode, one reference picture list may include only current pictures, while another reference picture list may include only temporal reference pictures.

[0700] b. In one example, for IBC Merge mode, motion vectors and reference pictures can be derived from neighboring blocks.

[0701] i. For example, if the neighboring block is predicted only from the current picture, the motion information derived from the neighboring block may refer only to the current picture.

[0702] ii. For example, if the neighbor block is predicted from both the current picture and the temporal reference picture, the derived motion information may refer to both the current picture and the temporal reference picture.

[0703] 1) Alternatively, the derived motion information may refer to the current picture only.

[0704] iii. For example, if a neighboring block is predicted only from a temporal reference picture, it may be considered “invalid” or “unavailable” when constructing an IBC Merge candidate.

[0705] c. In one example, a fixed weighting factor may be assigned to a reference block from a current picture and a reference block from a temporal reference picture for bidirectional prediction.

[0706] i. Alternatively, additionally, weighting factors may be signaled.

[0707] 21. The motion candidate list construction process (e.g., conventional Merge list, IBC Merge / AMVP list, sub-block Merge list, IBC sub-block candidate list) and / or whether / how to update the HMVP table may depend on the block dimension and / or Merge sharing condition. The width and height of the block are represented as W and H, respectively. Condition C may depend on the block dimension and / or codec information.

[0708] a. The motion candidate list construction process (eg, regular Merge list, IBC Merge / AMVP list, sub-block Merge list, IBC sub-block candidate list) and / or whether / how to update the HMVP table may depend on condition C.

[0709] b. In one example, condition C may depend on the coding information of the current block and / or its neighboring (adjacent or non-adjacent) blocks.

[0710] c. In one example, condition C may depend on the Merge share condition.

[0711] d. In one example, condition C may depend on the block dimension of the current block and / or the block dimension of a neighboring (adjacent or non-adjacent) block and / or the coding mode of the current and / or neighboring block.

[0712] e. In one example, if condition C is met, the derivation of the spatial domain merge candidate is skipped.

[0713] f. In one example, if condition C is satisfied, the derivation of candidates from spatial neighboring (adjacent or non-adjacent) blocks is skipped.

[0714] g. In one example, if condition C is satisfied, the derivation of candidates from certain spatial neighboring (adjacent or non-adjacent) blocks (eg, block B2) is skipped.

[0715] h. In one example, if condition C is met, the derivation of HMVP candidates is skipped.

[0716] i. In one example, if condition C is met, the derivation of pairwise Merge candidates is skipped.

[0717] j. In one example, if condition C is met, the number of maximum pruning operations is reduced or set to 0.

[0718] i. Alternatively, in addition, the pruning operations between spatial merge candidates can be reduced or removed.

[0719] ii. Alternatively, in addition, the pruning operations between the HMVP candidate and other Merge candidates can be reduced or removed.

[0720] k. In one example, if condition C is met, the update of the HMVP candidate is skipped.

[0721] i. In one example, HMVP candidates can be directly added to the motion list without being pruned.

[0722] 1. In one example, if condition C is met, no default motion candidate is added (eg, zero motion candidate in the IBC Merge / AVMP list).

[0723] m. In one example, when condition C is met, a different checking order (eg, from first to last instead of from last to first) and / or a different number of HMVP candidates will be checked / added.

[0724] n. In one example, when W*H is greater than or not less than a threshold value (eg, 1024), condition C may be satisfied.

[0725] o. In one example, condition C may be satisfied when W and / or H is greater than or not less than a threshold value (eg, 32).

[0726] p. In one example, when W is greater than or not less than a threshold value (eg, 32), condition C may be satisfied.

[0727] q. In one example, when H is greater than or not less than a threshold value (eg, 32), condition C may be satisfied.

[0728] r. In one example, when W*H is greater than or not less than a threshold (eg, 1024) and the current block is encoded or decoded in IBC AMVP and / or Merge mode, condition C may be satisfied.

[0729] s. In one example, condition C may be satisfied when W*H is less than or not greater than a threshold (eg, 16 or 32 or 64) and the current block is encoded or decoded in IBCAMVP and / or Merge mode.

[0730] i. Alternatively, in addition, when condition C is satisfied, the IBC motion list construction process may include candidates from spatial neighboring blocks (eg, A1, B1) and default candidates, that is, skip the insertion of HMVP candidates.

[0731] ii. Alternatively, in addition, when condition C is satisfied, the IBC motion list construction process may include candidates of HMVP candidates from the IBCHMVP table and default candidates, i.e., skip the insertion of candidates from the spatial neighboring blocks.

[0732] iii. Alternatively, in addition, if condition C is satisfied, after decoding the block, the updating of the IBC HMVP table is skipped.

[0733] iv. Alternatively, condition C may be satisfied when one / some / all of the following are true:

[0734] 1) When W*H is equal to or less than T1 (e.g. 16) and the current block is encoded or decoded in IBC AMVP and / or Merge mode

[0735] 2) When W is equal to T2 and H is equal to T3 (for example, T2=4, T3=8), the block above it is available and has a size equal to AxB; and the current block and the block above it are encoded and decoded in a certain mode

[0736] a. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), the block above it is available, in the same CTU and has a size equal to AxB, and both the current block and the block above it are coded and decoded in the same mode

[0737] b. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), the block above it is available and its size is equal to AxB, and both the current block and its block above it are encoded and decoded in the same mode

[0738] c. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), the blocks above it are unavailable.

[0739] d. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), the upper block is not available or the upper block is not outside the current CTU

[0740] 3) When W is equal to T4 and H is equal to T5 (for example, T4=8, T5=4), its left block is available and its size is equal to AxB; both the current block and its left block are encoded and decoded in a certain mode

[0741] a. Alternatively, when W is equal to T4 and H is equal to T5 (e.g., T4=8, T5=4), then its left block is unavailable.

[0742] 4) When W*H is not greater than T1 (eg, 32), the current block is encoded and decoded in IBC AMVP and / or Merge mode; its upper and left neighboring blocks are available and have a size equal to AxB and are encoded and decoded in a certain mode.

[0743] a. When W*H is not greater than T1 (for example, 32), the current block is encoded and decoded in a certain mode; its left neighbor block is available, the size is equal to AxB and is encoded and decoded in IBC; and its upper neighbor block is available, in the same CTU and the size is equal to AxB, and is encoded and decoded in the same mode.

[0744] b. When W*H is not greater than T1 (for example, 32), the current block is encoded and decoded in a certain mode; its left neighbor block is not available; and its upper neighbor block is available, in the same CTU and with a size equal to AxB, and is encoded and decoded in the same mode.

[0745] c. When W*H is not greater than T1 (eg, 32), the current block is encoded and decoded in a certain mode; its left neighboring block is unavailable; and its upper neighboring block is unavailable.

[0746] d. When W*H is not greater than T1 (eg, 32), the current block is encoded or decoded in a certain mode; its left neighboring block is available, has a size equal to AxB, and is encoded or decoded in the same mode; and its upper neighboring block is not available.

[0747] e. When W*H is not greater than T1 (eg, 32), the current block is encoded and decoded in a certain mode; its left neighboring block is not available; and its upper neighboring block is not available or is outside the current CTU.

[0748] f. When W*H is not greater than T1 (e.g., 32), the current block is encoded and decoded in a certain mode; its left neighboring block is available, has a size equal to AxB, and is encoded and decoded in the same mode; and its upper neighboring block is unavailable or outside the current CTU.

[0749] 5) In the above example, the "certain mode" is the IBC mode.

[0750] 6) In the above example, the "certain mode" is the inter-frame mode.

[0751] 7) In the above example, "AxB" can be set to 4x4.

[0752] 8) In the above example, “the neighborhood block size is equal to AxB” can be replaced by “the neighborhood block size is not greater than or less than AxB”.

[0753] 9) In the above example, the above and left neighbor blocks are the two blocks visited for spatial Merge candidate derivation.

[0754] a. In one example, assume that the coordinates of the upper left sample point in the current block are (x, y), and the left block is a block covering (x-1, y+H-1).

[0755] b. In one example, assume that the coordinates of the upper left sample point in the current block are (x, y), and the left block is a block covering (x+W-1, y-1).

[0756] t. The above thresholds may be predefined or signaled.

[0757] i. Alternatively, additionally, the threshold may depend on codec information of the block, such as a codec mode.

[0758] u. In one example, when the current block is under a shared node and the current block is encoded and decoded in IBC AMVP and / or Merge mode, condition C is satisfied.

[0759] i. Alternatively, in addition, when condition C is satisfied, the IBC motion list construction process may include candidates from spatial neighboring blocks (eg, A1, B1) and default candidates, that is, skip the insertion of HMVP candidates.

[0760] ii. Alternatively, in addition, when condition C is satisfied, the IBC motion list construction process may include candidates of HMVP candidates from the IBCHMVP table and default candidates, i.e., skip the insertion of candidates from the spatial neighboring blocks.

[0761] iii. Alternatively, in addition, if condition C is satisfied, after decoding the block, the updating of the IBC HMVP table is skipped.

[0762] v. In one example, the condition C may be adaptively changed, such as according to the codec information of the block.

[0763] i. In one example, the condition C may be defined based on the codec mode (IBC or non-IBC mode) and the block dimension.

[0764] w. Whether to apply the above method may depend on the codec information of the block, such as whether it is an IBC codec block.

[0765] i. In one example, when the block is encoded and decoded in IBC, the above method can be applied.

[0766] IBC Sports List

[0767] 22. It is recommended that motion candidates in the IBC HMVP table be stored with integer pixel precision rather than 1 / 16 pixel precision.

[0768] a. In one example, all motion candidates are stored with 1 pixel precision.

[0769] b. In one example, when using motion information from spatial neighboring (neighboring or non-neighboring) blocks and / or from the IBC HMVP table, the rounding process of the MV is skipped.

[0770] 23. It is recommended that an IBC motion list may only contain motion candidates from one or more IBC HMVP tables.

[0771] a. Alternatively, additionally, the signaling of candidates in the IBC motion list may depend on the number of HMVP candidates available in the HMVP table.

[0772] b. Alternatively, additionally, the signaling of candidates in the IBC motion list may depend on the maximum number of HMVP candidates in the HMVP table.

[0773] c. Alternatively, without pruning, the HMVP candidates in the HMVP table are added to the list in order.

[0774] i. In one example, the order is based on ascending order of the table's entry indexes.

[0775] ii. In one example, the order is based on descending order of the table's entry indexes.

[0776] iii. In one example, the first N entries in the table may be skipped.

[0777] iv. In one example, the last N entries in the table may be skipped.

[0778] v. In one example, entries with invalid BV(s) may be skipped.

[0779] vi.

[0780] d. Alternatively, in addition, the motion candidates derived from the HMVP candidates of one or more HMVP tables may be further modified, such as by adding an offset to a horizontal vector and / or adding an offset to a vertical vector.

[0781] i. HMVP candidates with invalid BV(s) may be modified to provide valid BV(s).

[0782] e. Alternatively, additionally, a default motion candidate may be added after or before one or more HMVP candidates.

[0783] f. How / whether to add HMVP candidates to the IBC motion list may depend on the dimensions of the block.

[0784] i. For example, when the block dimensions (W and H represent width and height) satisfy condition C, the IBC motion list may only contain motion candidates from one or more HMVP tables.

[0785] 1) In one example, the condition C is W<=T1 and H<=T2, for example, T1=T2=4.

[0786] 2) In one example, the condition C is W<=T1 or H<=T2, for example, T1=T2=4.

[0787] 3) In one example, condition C is W*H<=T, for example, T=16.

[0788] 5. Examples

[0789] Additions are highlighted in bold, italic and underlined. Deletions are marked with [[ ]].

[0790] 5.1 Example #1

[0791] When the current block is under a shared node, the HMVP table is not updated, and only a single IBC HMVP table is used for blocks under a shared node.

[0792] 7.4.8.5 Codec unit semantics

[0793] [[Update the history-based motion vector predictor list for the shared Merge candidate list area by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i], where i=0...NumHmvpIbcCand-1, when all of the following conditions are true:

[0794] IsInSmr[x0][y0] is equal to true (TRUE).

[0795] SmrX[x0][y0] is equal to x0.

[0796] SmrY[x0][y0] is equal to y0. ]]

[0797] 8.6.2 Derivation process of motion vector components of IBC blocks

[0798] 8.6.2.1 Overview

[0799] The inputs to this process are:

[0800] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0801] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[0802] - variable cbHeight, specifies the height of the current codec block in luminance samples,

[0803] The output of this process is:

[0804] - Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0805] The luminance motion vector mvL is derived as follows:

[0806] - Call the derivation process of the IBC luma motion vector prediction specified in section 8.6.2.2 with the following conditions: luma position (xCb, yCb), variables cbWidth and cbHeight as input; and the output is the luma motion vector mvL.

[0807] - When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0808] 1. The variable mvd is exported as follows:

[0809] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[0810] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0811] 2. Call the motion vector rounding process specified in Section 8.5.2.14 with the following conditions: mvX is set equal to mvL, rightShift is set equal to MvShift+2, and leftShift is set equal to MvShift+2 as input; and the rounded mvL is used as output.

[0812] 3. The luminance motion vector mvL is modified as follows:

[0813] u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885)

[0814] mvL[0]=(u[0]>=2 17 )? (u[0]-2 18 ):u[0] (8-886)

[0815] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[0816] mvL[1]=(u[1]>=2 17 )? (u[1]-2 18 ):u[1] (8-888)

[0817] NOTE 1 – The resulting values ​​of mvL[0] and mvL[1] as specified above will always be between -2 17 To 2 17 The value is in the range of -1 (inclusive).

[0818] When IsInSmr[xCb][yCb] is false, the history-based motion vector predictor list update process specified in section 8.6.2.6 is invoked with the luma motion vector mvL.

[0819] The upper left position inside the reference block (xRefTL, yRefTL) and the lower right position inside the reference block (xRefBR, yRefBR) are derived as follows:

[0820] (xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889)

[0821] (xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890)

[0822] Bitstream conformance requires that the luma motion vector mvL should obey the following constraints:

[0823] -……

[0824] 8.6.2.4 Motion vector candidate derivation process based on IBC history

[0825] The inputs to this process are:

[0826] -motion vector candidate list mvCandList,

[0827] - The number of motion vector candidates available in the list, numCurrCand.

[0828] The output of this process is:

[0829] -Modified motion vector candidate list mvCandList,

[0830] -[[Variable isInSmr, specifies whether the current codec unit is inside the shared Merge candidate area]]

[0831] - The number of modified motion vector candidates in the list, numCurrCand.

[0832] The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE.

[0833] The array smrHmvpIbcCandList and the variable smrNumHmvpIbcCand are exported as follows:

[0834] [[smr]]HmvpIbcCandList=[[isInSmr? HmvpSmrIbcCandList:]]HmvpIbcCandList (8-906)

[0835] [[smr]]NumHmvpIbcCand=[[isInSmr? NumHmvpSmrIbcCand:]]NumHmvpIbcCand(8-907)

[0836] For each candidate in smrHmvpIbcCandList[hMvpIdx], where index hMvpIdx = 1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand is equal to MaxNumMergeCand:

[0837] 1. The variable sameMotion is derived as follows:

[0838] - If all of the following conditions are true for any motion vector candidate N, where N is A1 or B1, then both sameMotion and isPrunedN are set equal to true:

[0839] - hMvpIdx is less than or equal to 1.

[0840] - Candidate [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand - hMvpIdx] is equal to motion vector candidate N.

[0841] - isPrunedN is equal to false.

[0842] - Otherwise, sameMotion is set equal to false.

[0843] 2. When sameMotion is false, add candidate [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand - hMvpIdx] to the motion vector candidate list as follows:

[0844] mvCandList[numCurrCand++] = [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand - hMvpIdx](8 - 908)

[0845] 5.2 Example #2

[0846] When the block size meets certain conditions (such as Width * Height < K), remove the check for空域Merge / AMVP candidates in the IBC motion list construction process. In the following description, the threshold K can be predefined, such as 16.

[0847] 7.4.8.2 Coding and Decoding Tree Unit Semantics

[0848] The CTU is the root node of the coding and decoding tree structure.

[0849] [[The array IsInSmr[x][y] that specifies whether the sample point at (x, y) is located inside the shared Merge candidate list area is initialized as follows, where x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[0850] IsInSmr[x][y]=FALSE (7-96)]]

[0851] 7.4.8.4 Codec Tree Semantics

[0852] [[IsInSmr[x][y] is set equal to true when all of the following conditions are true, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight-1.

[0853] -IsInSmr[x0][y0] is equal to false

[0854] -cbWidth*cbHeight / 4 is less than 32

[0855] -treeType is not equal to DUAL_TREE_CHROMA

[0856] When IsInSmr[x0][y0] is equal to true, the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[0857] SmrX[x][y]=x0 (7-98)

[0858] SmrY[x][y]=y0 (7-99)

[0859] SmrW[x][y]=cbWidth(7-100)

[0860] SmrH[x][y]=cbHeight (7-101)

[0861] IsInSmr[x][y] is set equal to true when all of the following conditions are true, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[0862] -IsInSmr[x0][y0] is equal to false

[0863] - One of the following conditions is true:

[0864] -mtt_split_cu_binary_flag is equal to 1, and cbWidth*cbHeight / 2 is less than 32

[0865] -mtt_split_cu_binary_flag is equal to 0, and cbWidth*cbHeight / 4 is less than 32

[0866] -treeType is not equal to DUAL_TREE_CHROMA

[0867] When IsInSmr[x0][y0] is equal to true, the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[0868] SmrX[x][y]=x0 (7-102)

[0869] SmrY[x][y]=y0 (7-103)

[0870] SmrW[x][y]=cbWidth (7-104)

[0871] SmrH[x][y]=cbHeight (7-105)]]

[0872] 7.4.8.5 Codec unit semantics

[0873] [[Update the history-based motion vector predictor list for the shared Merge candidate list area by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i], where i=0...NumHmvpIbcCand-1, when all of the following conditions are true:

[0874] IsInSmr[x0][y0] is equal to true.

[0875] SmrX[x0][y0] is equal to x0.

[0876] SmrY[x0][y0] is equal to y0. ]]

[0877] The following assignments are made, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[0878] CbPosX[x][y]=x0 (7-106)

[0879] CbPosY[x][y]=y0 (7-107)

[0880] CbWidth[x][y]=cbWidth (7-108)

[0881] CbHeight[x][y]=cbHeight (7-109)

[0882] 8.6.2 Derivation process of motion vector components of IBC blocks

[0883] 8.6.2.1 Overview

[0884] The inputs to this process are:

[0885] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0886] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[0887] - variable cbHeight, specifies the height of the current codec block in luminance samples,

[0888] The output of this process is:

[0889] - Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0890] The luminance motion vector mvL is derived as follows:

[0891] - Call the derivation process of the IBC luma motion vector prediction specified in section 8.6.2.2 with the following conditions: luma position (xCb, yCb), variables cbWidth and cbHeight as input; and the output is the luma motion vector mvL.

[0892] - When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0893] 4. The variable mvd is exported as follows:

[0894] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[0895] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0896] 5. Call the motion vector rounding process specified in Section 8.5.2.14 with the following conditions: mvX is set equal to mvL, rightShift is set equal to MvShift+2, and leftShift is set equal to MvShift+2 as input; and the rounded mvL is used as output.

[0897] 6. The luminance motion vector mvL is modified as follows:

[0898] u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885)

[0899] mvL[0]=(u[0]>=2 17 )? (u[0]-2 18 ):u[0] (8-886)

[0900] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[0901] mvL[1]=(u[1]>=2 17 )? (u[1]-2 18 ):u[1] (8-888)

[0902] NOTE 1 – The resulting values ​​of mvL[0] and mvL[1] as specified above will always be between -2 17 To 2 17 The value is in the range of -1 (inclusive).

[0903] When smrWidth*smrHeight is greater than K , calling the history-based motion vector predictor list update process specified in section 8.6.2.6 with the luma motion vector mvL.

[0904] The upper left position inside the reference block (xRefTL, yRefTL) and the lower right position inside the reference block (xRefBR, yRefBR) are derived as follows:

[0905] (xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889)

[0906] (xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890)

[0907] Bitstream conformance requires that the luma motion vector mvL should obey the following constraints:

[0908] -……

[0909] 8.6.2.2 IBC Luma Motion Vector Prediction Derivation Process

[0910] This process is called only when CuPredMode[xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma codec block relative to the top-left luma sample of the current picture.

[0911] The inputs to this process are:

[0912] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0913] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[0914] - The variable cbHeight specifies the height of the current codec block in luma samples.

[0915] The output of this process is:

[0916] - Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0917] The variables xSmr, ySmr, smrWidth, smrHeight, and smrNumHmvpIbcCand are derived as follows:

[0918] xSmr=[[IsInSmr[xCb][yCb]? SmrX[xCb][yCb]:]]xCb (8-895)

[0919] ySmr=[[IsInSmr[xCb][yCb]? SmrY[xCb][yCb]:]]yCb (8-896)

[0920] smrWidth=[[IsInSmr[xCb][yCb]? SmrW[xCb][yCb]:]]cbWidth (8-897)

[0921] smrHeight=[[IsInSmr[xCb][yCb]? SmrH[xCb][yCb]:]]cbHeight (8-898)

[0922] smrNumHmvpIbcCand=[[IsInSmr[xCb][yCb]? NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-899)

[0923] The luminance motion vector mvL is derived by the following sequential steps:

[0924] 1. When smrWidth*smrHeight is greater than K , the derivation process of spatial motion vector candidates from neighboring codec units specified in section 8.6.2.3 is called with the following conditions: the luma codec block position (xCb, yCb) is set equal to (xSmr, ySmr), the luma codec block width cbWidth and the luma codec block height cbHeight are set equal to smrWidth and smrHeight as input; and the outputs are the availability flags availableFlagA1, availableFlagB1 and the motion vectors mvA1 and mvB1.

[0925] 2. When smrWidth*smrHeight is greater than K , the motion vector candidate list mvCandList is constructed as follows:

[0926]

[0927] 3. When smrWidth*smrHeight is greater than K , the variable numCurrCand is set equal to the number of Merge candidates in mvCandList.

[0928] 4. When numCurrCand is less than MaxNumMergeCandand and smrNumHmvpIbcCand is greater than 0, call the IBC history-based motion vector candidate derivation process specified in 8.6.2.4 with the following conditions: mvCandList, isInSmr set equal to IsInSmr[xCb][yCb] and numCurrCand as input; and the modified mvCandList and numCurrCand as output.

[0929] 5. When numCurrCand is less than MaxNumMergeCand, the following applies until numCurrCand equals MaxNumMergeCand:

[0930] 1. mvCandList[numCurrCand][0] is set equal to 0.

[0931] 2. mvCandList[numCurrCand][1] is set equal to 0.

[0932] 3. numCurrCand increases by 1.

[0933] 6. The variable mvIdx is exported as follows:

[0934] mvIdx=general_merge_flag[xCb][yCb]? merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb](8-901)

[0935] 7. Make the following allocations:

[0936] mvL[0]=mergeCandList[mvIdx][0] (8-902)

[0937] mvL[1]=mergeCandList[mvIdx][1] (8-903)

[0938] 8.6.2.4 Motion vector candidate derivation process based on IBC history

[0939] The inputs to this process are:

[0940] -motion vector candidate list mvCandList,

[0941] - The number of motion vector candidates available in the list, numCurrCand.

[0942] The output of this process is:

[0943] -Modified motion vector candidate list mvCandList,

[0944] -[[Variable isInSmr, specifies whether the current codec unit is inside the shared Merge candidate area]]

[0945] - The number of modified motion vector candidates in the list, numCurrCand.

[0946] The variables isPrunedA1 and isPrunedB1 are both set equal to false.

[0947] The array smrHmvpIbcCandList and the variable smrNumHmvpIbcCand are exported as follows:

[0948] [[smr]]HmvpIbcCandList=[[isInSmr? HmvpSmrIbcCandList:]]HmvpIbcCandList (8-906)

[0949] [[smr]]NumHmvpIbcCand=[[isInSmr? NumHmvpSmrIbcCand:]]NumHmvpIbcCand(8-907)

[0950] For each candidate in smrHmvpIbcCandList[hMvpIdx], where index hMvpIdx = 1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand equals MaxNumMergeCand:

[0951] 1. The variable sameMotion is exported as follows:

[0952] -if smrWidth*smrHeight is greater than K , for any motion vector candidate N, if all of the following conditions are true, where N is A1 or B1, then both sameMotion and isPrunedN are set equal to true:

[0953] -hMvpIdx is less than or equal to 1.

[0954] -Candidate[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] is equal to motion vector candidate N.

[0955] -isPrunedN is equal to false.

[0956] Otherwise, sameMotion is set equal to false.

[0957] 2. When sameMotion is equal to false, add the candidate [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] to the motion vector candidate list as follows:

[0958] mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

[0959] 5.3 Example #3

[0960] Remove the check of spatial Merge / AMVP candidates in the IBC motion list construction process when the block size meets certain conditions (such as the current block is under a shared node and the HMVP table is not updated).

[0961] 8.6.2.2 IBC Luma Motion Vector Prediction Derivation Process

[0962] This process is called only when CuPredMode[xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma codec block relative to the top-left luma sample of the current picture.

[0963] The inputs to this process are:

[0964] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[0965] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[0966] - The variable cbHeight specifies the height of the current codec block in luma samples.

[0967] The output of this process is:

[0968] - Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[0969] The variables xSmr, ySmr, smrWidth, smrHeight, and smrNumHmvpIbcCand are derived as follows:

[0970] xSmr=[[IsInSmr[xCb][yCb]? SmrX[xCb][yCb]:]]xCb (8-895)

[0971] ySmr=[[IsInSmr[xCb][yCb]? SmrY[xCb][yCb]:]]yCb (8-896)

[0972] smrWidth=[[IsInSmr[xCb][yCb]? SmrW[xCb][yCb]:]]cbWidth (8-897)

[0973] smrHeight=[[IsInSmr[xCb][yCb]? SmrH[xCb][yCb]:]]cbHeight (8-898)

[0974] smrNumHmvpIbcCand=[[IsInSmr[xCb][yCb]? NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-899)

[0975] The luminance motion vector mvL is derived by the following sequential steps:

[0976] 1. When IsInSmr[xCb][yCb] is false, the derivation process of spatial motion vector candidates from neighboring codec units specified in section 8.6.2.3 is called with the following conditions: the luma codec block position (xCb, yCb) is set equal to (xSmr, ySmr), the luma codec block width cbWidth and the luma codec block height cbHeight are set equal to smrWidth and smrHeight as input; and the outputs are the availability flags availableFlagA1, availableFlagB1 and the motion vectors mvA1 and mvB1.

[0977] 2. When IsInSmr[xCb][yCb] is false , the motion vector candidate list mvCandList is constructed as follows:

[0978]

[0979] 3. When IsInSmr[xCb][yCb] is false , the variable numCurrCand is set equal to the number of Merge candidates in mvCandList.

[0980] 4. When numCurrCand is less than MaxNumMergeCand and smrNumHmvpIbcCand is greater than 0, call the IBC history-based motion vector candidate derivation process specified in 8.6.2.4 with the following conditions: mvCandList, isInSmr set equal to IsInSmr[xCb][yCb] and numCurrCand as input; and the modified mvCandList and numCurrCand as output.

[0981] 5. When numCurrCand is less than MaxNumMergeCand, the following applies until numCurrCand equals MaxNumMergeCand:

[0982] 1. mvCandList[numCurrCand][0] is set equal to 0.

[0983] 2. mvCandList[numCurrCand][1] is set equal to 0.

[0984] 3. numCurrCand increases by 1.

[0985] 6. The variable mvIdx is exported as follows:

[0986] mvIdx=general_merge_flag[xCb][yCb]? merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-901)

[0987] 7. Make the following allocations:

[0988] mvL[0]=mergeCandList[mvIdx][0] (8-902)

[0989] mvL[1]=mergeCandList[mvIdx][1] (8-903)

[0990] 8.6.2.4 Motion vector candidate derivation process based on IBC history

[0991] The inputs to this process are:

[0992] -motion vector candidate list mvCandList,

[0993] - The number of motion vector candidates available in the list, numCurrCand.

[0994] The output of this process is:

[0995] -Modified motion vector candidate list mvCandList,

[0996] -[[Variable isInSmr, specifies whether the current codec unit is inside the shared Merge candidate area]]

[0997] - The number of modified motion vector candidates in the list, numCurrCand.

[0998] The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE.

[0999] The array smrHmvpIbcCandList and the variable smrNumHmvpIbcCand are exported as follows:

[1000] [[smr]]HmvpIbcCandList=[[isInSmr? HmvpSmrIbcCandList:]]HmvpIbcCandList (8-906)

[1001] [[smr]]NumHmvpIbcCand=[[isInSmr? NumHmvpSmrIbcCand:]]NumHmvpIbcCand(8-907)

[1002] For each candidate in smrHmvpIbcCandList[hMvpIdx], where index hMvpIdx = 1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand equals MaxNumMergeCand:

[1003] 1. The variable sameMotion is exported as follows:

[1004] -if isInSmr is false And for any motion vector candidate N all of the following are true, where N is A1 or B1, then both sameMotion and isPrunedN are set equal to true:

[1005] -hMvpIdx is less than or equal to 1.

[1006] -Candidate[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] is equal to motion vector candidate N.

[1007] -isPrunedN is equal to false.

[1008] Otherwise, sameMotion is set equal to false.

[1009] 2. When sameMotion is equal to false, add the candidate [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] to the motion vector candidate list as follows:

[1010] mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

[1011] 8.6.2 Derivation process of motion vector components of IBC blocks

[1012] 8.6.2.1 Overview

[1013] The inputs to this process are:

[1014] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1015] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1016] - variable cbHeight, specifies the height of the current codec block in luminance samples,

[1017] The output of this process is:

[1018] - Luma motion vector mvL with 1 / 16 fractional sample accuracy.

[1019] The luminance motion vector mvL is derived as follows:

[1020] - Call the derivation process of the IBC luma motion vector prediction specified in section 8.6.2.2 with the following conditions: luma position (xCb, yCb), variables cbWidth and cbHeight as input; and the output is the luma motion vector mvL.

[1021] - When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[1022] 7. The variable mvd is exported as follows:

[1023] mvd[0]=MvdL0[xCb][yCb][0] (8-883)

[1024] mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[1025] 8. Call the motion vector rounding process specified in Section 8.5.2.14 with the following conditions: mvX is set equal to mvL, rightShift is set equal to MvShift+2, and leftShift is set equal to MvShift+2 as input; and the rounded mvL is used as output.

[1026] 9. The luminance motion vector mvL is modified as follows:

[1027] u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885)

[1028] mvL[0]=(u[0]>=2 17 )? (u[0]-2 18 ):u[0] (8-886)

[1029] u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887)

[1030] mvL[1]=(u[1]>=2 17 )? (u[1]-2 18):u[1] (8-888)

[1031] NOTE 1 – The resulting values ​​of mvL[0] and mvL[1] as specified above will always be between -2 17 To 2 17 The value is in the range of -1 (inclusive).

[1032] When IsInSmr[xCb][yCb] is false , calling the history-based motion vector predictor list update process specified in section 8.6.2.6 with the luma motion vector mvL.

[1033] The upper left position inside the reference block (xRefTL, yRefTL) and the lower right position inside the reference block (xRefBR, yRefBR) are derived as follows:

[1034] (xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889)

[1035] (xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890)

[1036] Bitstream conformance requires that the luma motion vector mvL should obey the following constraints:

[1037] -……

[1038] 5.4 Example #4

[1039] When the block size satisfies certain conditions (such as Width*Height<=K; or Width=N, Height=4, and the left neighbor block is 4×4 and is encoded and decoded in IBC mode; or Width=4, Height=N, and the upper neighbor block is 4x4 and is encoded and decoded in IBC mode; and the HMVP table is not updated), the check of spatial Merge / AMVP candidates in the IBC motion list construction process is removed. In the following description, a threshold K may be predefined, such as 16, and N may be predefined, such as 8.

[1040] 7.4.9.2 Codec Tree Unit Semantics

[1041] CTU is the root node of the codec tree structure.

[1042] The array IsAvailable[cIdx][x][y] that specifies whether the sample at (x, y) is available for the derivation process of the neighborhood block availability specified in section 6.4.4 is initialized as follows, where cIdx = 0..2, x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[1043] IsAvailable[cIdx][x][y]=FALSE (7-123)

[1044] [[The array IsInSmr[x][y] that specifies whether the sample point at (x, y) is located inside the shared Merge candidate list region is initialized as follows, where x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[1045] IsInSmr[x][y]=FALSE (7-124)]]

[1046] 7.4.9.4 Codec Tree Semantics

[1047] [[IsInSmr[x][y] is set equal to true when all of the following conditions are true, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[1048] -IsInSmr[x0][y0] is equal to false

[1049] -cbWidth*cbHeight / 4 is less than 32

[1050] -treeType is not equal to DUAL_TREE_CHROMA

[1051] When IsInSmr[x0][y0] is equal to true, the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[1052] SmrX[x][y]=x0 (7-126)

[1053] SmrY[x][y]=y0 (7-127)

[1054] SmrW[x][y]=cbWidth(7-128)

[1055] SmrH[x][y]=cbHeight (7-129)]]

[1056] 8.6.2 Derivation process of block vector components of IBC blocks

[1057] 8.6.2.1 Overview

[1058] The inputs to this process are:

[1059] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1060] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1061] - variable cbHeight, specifies the height of the current codec block in luminance samples,

[1062] The output of this process is:

[1063] - Luma block vector bvL with 1 / 16 fractional sample accuracy.

[1064] The luminance block vector bvL is derived as follows:

[1065] - Call the derivation process of the IBC luma block vector prediction specified in section 8.6.2.2 with the following conditions: luma position (xCb, yCb), variables cbWidth and cbHeight as input; and the output is the luma block vector bvL.

[1066] - When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[1067] 1. The variable bvd is exported as follows:

[1068] bvd[0]=MvdL0[xCb][yCb][0] (8-900)

[1069] bvd[1]=MvdL0[xCb][yCb][1] (8-901)

[1070] 2. The motion vector rounding process specified in section 8.5.2.14 is called with the following conditions: mvX is set equal to bvL, rightShift is set equal to AmvrShift, and leftShift is set equal to AmvrShift as input; and the rounded bvL is used as output.

[1071] 3. The luminance block vector bvL is modified as follows:

[1072] u[0]=(bvL[0]+bvd[0]+2 18 )%2 18 (8-902)

[1073] bvL[0]=(u[0]>=2 17 )? (u[0]-2 18 ):u[0] (8-903)

[1074] u[1]=(bvL[1]+bvd[1]+2 18 )%2 18 (8-904)

[1075] bvL[1]=(u[1]>=2 17 )? (u[1]-2 18 ):u[1] (8-905)

[1076] NOTE 1 – The resulting values ​​of bvL[0] and bvL[1] as specified above will always be between -2 17 To 2 17 The value is in the range of -1 (inclusive).

[1077] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1078] If IsLgrBlk is true, then when CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1079] If IsLgrBlk is true, then when CbWidth is equal to 4 and CbHeight is equal to N and the upper neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1080] (or alternatively:

[1081] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1082] If IsLgrBlk is true, then when CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1083] If IsLgrBlk is true, then when CbWidth is equal to 4 and CbHeight is equal to N and the upper neighbor block is available IsLgrBlk is set to false when encoding or decoding in IBC mode and 4x4.

[1084] when IsLgrBlk is true , [[IsInSmr[xCb][yCb] is equal to false]], the history-based block vector predictor list update process specified in section 8.6.2.6 is called with the luminance block vector bvL.

[1085] Bitstream conformance requires that the luma block vector bvL shall obey the following constraints:

[1086] -CtbSizeY is greater than or equal to ((yCb+(bvL[1]>>4))&(CtbSizeY-1))+cbHeight.

[1087] -IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] should not be equal to -1, where x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1.

[1088] 8.6.2.2 IBC Luma Block Vector Prediction Derivation Process

[1089] This process is called only when CuPredMode[0][xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top left sample of the current luma codec block relative to the top left luma sample of the current picture.

[1090] The inputs to this process are:

[1091] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1092] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1093] - The variable cbHeight specifies the height of the current codec block in luma samples.

[1094] The output of this process is:

[1095] - Luma block vector bvL with 1 / 16 fractional sample accuracy.

[1096] [[The variables xSmr, ySmr, smrWidth, and smrHeight are derived as follows:

[1097] xSmr=IsInSmr[xCb][yCb]? SmrX[xCb][yCb]:xCb (8-906)

[1098] ySmr=IsInSmr[xCb][yCb]? SmrY[xCb][yCb]:yCb (8-907)

[1099] smrWidth=IsInSmr[xCb][yCb]? SmrW[xCb][yCb]:cbWidth (8-908)

[1100] smrHeight=IsInSmr[xCb][yCb]? SmrH[xCb][yCb]:cbHeight (8-909)]]

[1101] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1102] If IsLgrBlk is true, then when CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1103] If IsLgrBlk is true, then when CbWidth is equal to 4 and CbHeight is equal to N and the upper neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1104] (or alternatively:

[1105] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1106] If IsLgrBlk is true, then when CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1107] If IsLgrBlk is true, then when CbWidth is equal to 4 and CbHeight is equal to N and the upper neighbor block is available IsLgrBlk is set to false when encoding or decoding in IBC mode and 4x4.

[1108] The luminance block vector bvL is derived by the following sequential steps:

[1109] 1. When IslgrBlk is true , the derivation process of the spatial block vector candidate from the neighboring codec unit specified in Section 8.6.2.3 is called with the following conditions: the luma codec block position (xCb, yCb) is set equal to ( xCb,yCb [[xSmr, ySmr]]), the luma codec block width cbWidth and luma codec block height cbHeight are set equal to [[smr]] Cb Width and [[smr]] Cb Height as input; and the outputs are the availability flags availableFlagA1, availableFlagB1 and the block vectors bvA1 and bvB1.

[1110] 2. When IslgrBlk is true , the block vector candidate list bvCandList is constructed as follows:

[1111]

[1112]

[1113] 3. When IslgrBlk is true , the variable numCurrCand is set equal to the number of Merge candidates in bvCandList.

[1114] 4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the derivation process of block vector candidates based on IBC history specified in 8.6.2.4 is called with the following conditions: bvCandList, and IsLgrBlk , and numCurrCand as input; and the modified bvCandList and numCurrCand as output.

[1115] 5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

[1116] 1. bvCandList[numCurrCand][0] is set equal to 0.

[1117] 2. bvCandList[numCurrCand][1] is set equal to 0.

[1118] 3. numCurrCand increases by 1.

[1119] 6. The variable bvIdx is derived as follows:

[1120] bvIdx=general_merge_flag[xCb][yCb]? merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

[1121] 7. Make the following allocations:

[1122] bvL[0]=bvCandList[mvIdx][0] (8-912)

[1123] bvL[1]=bvCandList[mvIdx][1] (8-913)

[1124] 8.6.2.3 IBC Spatial Block Vector Candidate Derivation Process

[1125] The inputs to this process are:

[1126] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1127] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1128] - The variable cbHeight specifies the height of the current codec block in luma samples.

[1129] The output of this process is:

[1130] - the availability flags availableFlagA1 and availableFlagB1 of the neighboring codec unit,

[1131] - the block vectors bvA1 and bvB1 of the neighborhood codec with 1 / 16 fractional sample accuracy,

[1132] For the exports of availableFlagA1 and mvA1, the following applies:

[1133] -The luma position (xNbA1, yNbA1) inside the neighborhood luma codec block is set equal to (xCb-1, yCb+cbHeight-1).

[1134] - Call the derivation process for neighborhood block availability specified in Section 6.4.4 with the following conditions: the current luma position (xCurr, yCurr) is set equal to (xCb, yCb), the neighborhood luma position (xNbA1, yNbA1), checkPredModeY is set equal to true, and cIdx is set equal to 0 as input; and assign the output to the block availability flag availableA1.

[1135] -Variables availableFlagA1 and bvA1 are exported as follows:

[1136] - If availableA1 is equal to false, availableFlagA1 is set equal to 0 and both components of bvA1 are set equal to 0.

[1137] - Otherwise, availableA1 is set equal to 1 and the following assignments are made:

[1138] bvA1=MvL0[xNbA1][yNbA1] (8-914)

[1139] For the exports of availableFlagB1 and bvB1, the following applies:

[1140] -The luma position (xNbB1, yNbB1) inside the neighborhood luma codec block is set equal to (xCb+cbWidth-1, yCb-1).

[1141] - Call the derivation process for neighborhood block availability specified in Section 6.4.4 with the following conditions: the current luma position (xCurr, yCurr) is set equal to (xCb, yCb), the neighborhood luma position (xNbB1, yNbB1), checkPredModeY is set equal to true, and cIdx is set equal to 0 as input; and assign the output to the block availability flag availableB1.

[1142] -Variables availableFlagB1 and bvB1 are exported as follows:

[1143] - availableFlagB1 is set equal to 0 and both components of bvB1 are set equal to 0 if one or more of the following conditions are true:

[1144] -availableB1 is equal to false.

[1145] -availableA1 is equal to true and luma positions (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vector.

[1146] - Otherwise, availableFlagB1 is set equal to 1 and the following assignments are made:

[1147] bvB1=MvL0[xNbB1][yNbB1] (8-915)

[1148] 8.6.2.4 Derivation process of block vector candidates based on IBC history

[1149] The inputs to this process are:

[1150] - block vector candidate list bvCandList,

[1151] -[[Variable isInSmr, specifies whether the current codec unit is inside the shared Merge candidate area]]

[1152] The variable IslgrBlk indicating a non-small block,

[1153] - The number of block vector candidates available in the list, numCurrCand.

[1154] The output of this process is:

[1155] - modified block vector candidate list bvCandList,

[1156] - The number of modified motion vector candidates in the list, numCurrCand.

[1157] The variables isPrunedA1 and isPrunedB1 are both set equal to false.

[1158] For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], where index hMvpIdx = 1.[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand equals MaxNumIbcMergeCand:

[1159] 1. The variable sameMotion is exported as follows:

[1160] -if IsLgrBlk is true And for any block vector candidate N all of the following conditions are true, where N is A1 or B1, then sameMotion and isPrunedN are both set equal to true:

[1161] -hMvpIdx is less than or equal to 1.

[1162] -CandidateHmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is equal to block vector candidateN.

[1163] -isPrunedN is equal to false.

[1164] Otherwise, sameMotion is set equal to false.

[1165] 2. When sameMotion is equal to false, add the candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] to the block vector candidate list as follows:

[1166] bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx](8-916)

[1167] 5.5 Example #5

[1168] When the block size meets certain conditions (such as Width = N, Height = 4, and the left neighbor block is 4×4 and is encoded and decoded in IBC mode; or Width = 4, Height = N, and the upper neighbor block is 4×4 and is encoded and decoded in IBC mode), remove the check of spatial Merge / AMVP candidates in the IBC motion list construction process. In the following description, N can be predefined, such as 4 or 8.

[1169] 7.4.9.2 Codec Tree Unit Semantics

[1170] CTU is the root node of the codec tree structure.

[1171] The array IsAvailable[cIdx][x][y] that specifies whether the sample at (x, y) is available for the derivation process of the neighborhood block availability specified in section 6.4.4 is initialized as follows, where cIdx = 0..2, x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[1172] IsAvailable[cIdx][x][y]=FALSE (7-123)

[1173] [[The array IsInSmr[x][y] that specifies whether the sample point at (x, y) is located inside the shared Merge candidate list region is initialized as follows, where x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[1174] IsInSmr[x][y]=FALSE (7-124)]]

[1175] 7.4.9.4 Codec Tree Semantics

[1176] [[IsInSmr[x][y] is set equal to true when all of the following conditions are true, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[1177] -IsInSmr[x0][y0] is equal to false

[1178] -cbWidth*cbHeight / 4 is less than 32

[1179] -treeType is not equal to DUAL_TREE_CHROMA

[1180] When IsInSmr[x0][y0] is equal to true, the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[1181] SmrX[x][y]=x0 (7-126)

[1182] SmrY[x][y]=y0 (7-127)

[1183] SmrW[x][y]=cbWidth(7-128)

[1184] SmrH[x][y]=cbHeight (7-129)]]

[1185] 8.6.2 Derivation process of block vector components of IBC blocks

[1186] 8.6.2.1 Overview

[1187] The inputs to this process are:

[1188] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1189] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1190] - variable cbHeight, specifies the height of the current codec block in luminance samples,

[1191] The output of this process is:

[1192] - Luma block vector bvL with 1 / 16 fractional sample accuracy.

[1193] The luminance block vector mvL is derived as follows:

[1194] - Call the derivation process of the IBC luma block vector prediction specified in section 8.6.2.2 with the following conditions: luma position (xCb, yCb), variables cbWidth and cbHeight as input; and the output is the luma block vector bvL.

[1195] - When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[1196] 1. The variable bvd is exported as follows:

[1197] bvd[0]=MvdL0[xCb][yCb][0] (8-900)

[1198] bvd[1]=MvdL0[xCb][yCb][1] (8-901)

[1199] 2. The motion vector rounding process specified in section 8.5.2.14 is called with the following conditions: mvX is set equal to bvL, rightShift is set equal to AmvrShift, and leftShift is set equal to AmvrShift as input; and the rounded bvL is used as output.

[1200] 3. The luminance block vector bvL is modified as follows:

[1201] u[0]=(bvL[0]+bvd[0]+2 18 )%2 18 (8-902)

[1202] bvL[0]=(u[0]>=2 17 )? (u[0]-2 18 ):u[0] (8-903)

[1203] u[1]=(bvL[1]+bvd[1]+2 18 )%2 18 (8-904)

[1204] bvL[1]=(u[1]>=2 17 )? (u[1]-2 18 ):u[1] (8-905)

[1205] NOTE 1 – The resulting values ​​of bvL[0] and bvL[1] as specified above will always be between -217 To 2 17 The value is in the range of -1 (inclusive).

[1206] The variable IslgrBlk is set to true when one of the following conditions is true.

[1207] When CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 and is encoded and decoded in IBC mode hour.

[1208] When CbWidth is 4 and CbHeight is N and the upper neighbor block is 4x4 and is encoded and decoded in IBC mode hour.

[1209] Otherwise, IsLgrBlk is set to false.

[1210] (or alternatively:

[1211] The variable IslgrBlk is set to (cbWidth×cbHeight greater than 16? True: False), and the following is further checked:

[1212] If IsLgrBlk is true, then when CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1213] If IsLgrBlk is true, then when CbWidth is equal to 4 and CbHeight is equal to N and the upper neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false. )

[1214] when IsLgrBlk is true , [[IsInSmr[xCb][yCb] is equal to false]], the history-based block vector predictor list update process specified in section 8.6.2.6 is called with the luminance block vector bvL.

[1215] Bitstream conformance requires that the luma block vector bvL shall obey the following constraints:

[1216] -CtbSizeY is greater than or equal to ((yCb+(bvL[1]>>4))&(CtbSizeY-1))+cbHeight.

[1217] -IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] should not be equal to -1, where x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1.

[1218] 8.6.2.2 IBC Luma Block Vector Prediction Derivation Process

[1219] This process is called only when CuPredMode[0][xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top left sample of the current luma codec block relative to the top left luma sample of the current picture.

[1220] The inputs to this process are:

[1221] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1222] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1223] - The variable cbHeight specifies the height of the current codec block in luma samples.

[1224] The output of this process is:

[1225] - Luma block vector bvL with 1 / 16 fractional sample accuracy.

[1226] [[The variables xSmr, ySmr, smrWidth, and smrHeight are derived as follows:

[1227] xSmr=IsInSmr[xCb][yCb]? SmrX[xCb][yCb]:xCb (8-906)

[1228] ySmr=IsInSmr[xCb][yCb]? SmrY[xCb][yCb]:yCb (8-907)

[1229] smrWidth=IsInSmr[xCb][yCb]? SmrW[xCb][yCb]:cbWidth (8-908)

[1230] smrHeight=IsInSmr[xCb][yCb]? SmrH[xCb][yCb]:cbHeight (8-909)]]

[1231] The variable IslgrBlk is set to true when one of the following conditions is true.

[1232] When CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 and is encoded and decoded in IBC mode hour.

[1233] When CbWidth is 4 and CbHeight is N and the upper neighbor block is 4x4 and is encoded and decoded in IBC mode hour.

[1234] Otherwise, IsLgrBlk is set to false.

[1235] (or alternatively:

[1236] The variable IslgrBlk is set to (cbWidth×cbHeight greater than 16? True: False), and the following is further checked:

[1237] If IsLgrBlk is true, then when CbWidth is equal to N and CbHeight is equal to 4 and the left neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false.

[1238] If IsLgrBlk is true, then when CbWidth is equal to 4 and CbHeight is equal to N and the upper neighbor block is 4x4 When encoding or decoding in IBC mode, IsLgrBlk is set to false. )

[1239] The luminance block vector bvL is derived by the following sequential steps:

[1240] 1. When IslgrBlk is true , the derivation process of spatial block vector candidates from the neighborhood codec unit specified in section 8.6.2.3 is called with the following conditions: As input, the luma codec block position (xCb, yCb) is set equal to ( xCb, Yb [[xSmr, ySmr]]), the luma codec block width cbWidth and luma codec block height cbHeight are set equal to [[smr]] Cb Width and [[smr]] Cb Height; and the outputs are the availability flags availableFlagA1, availableFlagB1 and the block vectors bvA1 and bvB1.

[1241] 2. When IslgrBlk is true , the block vector candidate list bvCandList is constructed as follows:

[1242]

[1243] 3. When IslgrBlk is true , the variable numCurrCand is set equal to the number of Merge candidates in bvCandList.

[1244] 4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the derivation process of block vector candidates based on IBC history specified in 8.6.2.4 is called with the following conditions: bvCandList, and IsLgrBlk , and numCurrCand as input; and the modified bvCandList and numCurrCand as output.

[1245] 5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

[1246] 1. bvCandList[numCurrCand][0] is set equal to 0.

[1247] 2. bvCandList[numCurrCand][1] is set equal to 0.

[1248] 3. numCurrCand increases by 1.

[1249] 6. The variable bvIdx is derived as follows:

[1250] bvIdx=general_merge_flag[xCb][yCb]? merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

[1251] 7. Make the following allocations:

[1252] bvL[0]=bvCandList[mvIdx][0] (8-912)

[1253] bvL[1]=bvCandList[mvIdx][1] (8-913)

[1254] 8.6.2.3 IBC Spatial Block Vector Candidate Derivation Process

[1255] The inputs to this process are:

[1256] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1257] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1258] - The variable cbHeight specifies the height of the current codec block in luma samples.

[1259] The output of this process is:

[1260] - the availability flags availableFlagA1 and availableFlagB1 of the neighboring codec unit,

[1261] - the block vectors bvA1 and bvB1 of the neighborhood codec with 1 / 16 fractional sample accuracy,

[1262] For the exports of availableFlagA1 and mvA1, the following applies:

[1263] -The luma position (xNbA1, yNbA1) inside the neighborhood luma codec block is set equal to (xCb-1, yCb+cbHeight-1).

[1264] - Call the derivation process for neighborhood block availability specified in Section 6.4.4 with the following conditions: the current luma position (xCurr, yCurr) is set equal to (xCb, yCb), the neighborhood luma position (xNbA1, yNbA1), checkPredModeY is set equal to true, and cIdx is set equal to 0 as input; and assign the output to the block availability flag availableA1.

[1265] -Variables availableFlagA1 and bvA1 are exported as follows:

[1266] - If availableA1 is equal to false, availableFlagA1 is set equal to 0 and both components of bvA1 are set equal to 0.

[1267] - Otherwise, availableA1 is set equal to 1 and the following assignments are made:

[1268] bvA1=MvL0[xNbA1][yNbA1] (8-914)

[1269] For the exports of availableFlagB1 and bvB1, the following applies:

[1270] -The luma position (xNbB1, yNbB1) inside the neighborhood luma codec block is set equal to (xCb+cbWidth-1, yCb-1).

[1271] - Call the derivation process for neighborhood block availability specified in Section 6.4.4 with the following conditions: the current luma position (xCurr, yCurr) is set equal to (xCb, yCb), the neighborhood luma position (xNbB1, yNbB1), checkPredModeY is set equal to true, and cIdx is set equal to 0 as input; and assign the output to the block availability flag availableB1.

[1272] -Variables availableFlagB1 and bvB1 are exported as follows:

[1273] - availableFlagB1 is set equal to 0 and both components of bvB1 are set equal to 0 if one or more of the following conditions are true:

[1274] -availableB1 is equal to false.

[1275] -availableA1 is equal to true and luma positions (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vector.

[1276] - Otherwise, availableFlagB1 is set equal to 1 and the following assignments are made:

[1277] bvB1=MvL0[xNbB1][yNbB1] (8-915)

[1278] 8.6.2.4 Derivation of Block Vector Candidates Based on IBC History

[1279] The inputs to this process are:

[1280] - block vector candidate list bvCandList,

[1281] -[[Variable isInSmr, specifies whether the current codec unit is inside the shared Merge candidate area]]

[1282] The variable IslgrBlk indicating a non-small block,

[1283] - The number of block vector candidates available in the list, numCurrCand.

[1284] The output of this process is:

[1285] - modified block vector candidate list bvCandList,

[1286] - The number of modified motion vector candidates in the list, numCurrCand.

[1287] The variables isPrunedA1 and isPrunedB1 are both set equal to false.

[1288] For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], where index hMvpIdx = 1.[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand equals MaxNumIbcMergeCand:

[1289] 1. The variable sameMotion is exported as follows:

[1290] -if IsLgrBlk is true And for any block vector candidate N all of the following conditions are true, where N is A1 or B1, then sameMotion and isPrunedN are both set equal to true:

[1291] -hMvpIdx is less than or equal to 1.

[1292] -CandidateHmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is equal to block vector candidateN.

[1293] -isPrunedN is equal to false.

[1294] Otherwise, sameMotion is set equal to false.

[1295] 2. When sameMotion is equal to false, add the candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] to the block vector candidate list as follows:

[1296] bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx](8-916)

[1297] 5.6 Example #6

[1298] When the block size meets certain conditions (such as Width*Height<=K and the left neighbor block or the upper neighbor block is 4×4 and encoded and decoded in IBC mode), the check of spatial Merge / AMVP candidates in the IBC motion list construction process is removed. In the following description, a threshold K can be predefined, such as 16 or 32.

[1299] 7.4.9.2 Codec Tree Unit Semantics

[1300] CTU is the root node of the codec tree structure.

[1301] The array IsAvailable[cIdx][x][y] that specifies whether the sample at (x, y) is available for the derivation process of the neighborhood block availability specified in section 6.4.4 is initialized as follows, where cIdx = 0..2, x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[1302] IsAvailable[cIdx][x][y]=FALSE (7-123)

[1303] [[The array IsInSmr[x][y] that specifies whether the sample point at (x, y) is located inside the shared Merge candidate list region is initialized as follows, where x = 0..CtbSizeY–1 and y = 0..CtbSizeY-1:

[1304] IsInSmr[x][y]=FALSE (7-124)]]

[1305] 7.4.9.4 Codec Tree Semantics

[1306] [[IsInSmr[x][y] is set equal to true when all of the following conditions are true, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[1307] -IsInSmr[x0][y0] is equal to false

[1308] -cbWidth*cbHeight / 4 is less than 32

[1309] -treeType is not equal to DUAL_TREE_CHROMA

[1310] When IsInSmr[x0][y0] is equal to true, the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows, where x = x0..x0+cbWidth–1 and y = y0..y0+cbHeight–1:

[1311] SmrX[x][y]=x0 (7-126)

[1312] SmrY[x][y]=y0 (7-127)

[1313] SmrW[x][y]=cbWidth(7-128)

[1314] SmrH[x][y]=cbHeight (7-129)]]

[1315] 8.6.2 Derivation process of block vector components of IBC blocks

[1316] 8.6.2.1 Overview

[1317] The inputs to this process are:

[1318] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1319] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1320] - variable cbHeight, specifies the height of the current codec block in luminance samples,

[1321] The output of this process is:

[1322] - Luma block vector bvL with 1 / 16 fractional sample accuracy.

[1323] The luminance block vector mvL is derived as follows:

[1324] - Call the derivation process of the IBC luma block vector prediction specified in section 8.6.2.2 with the following conditions: luma position (xCb, yCb), variables cbWidth and cbHeight as input; and the output is the luma block vector bvL.

[1325] - When general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[1326] 1. The variable bvd is exported as follows:

[1327] bvd[0]=MvdL0[xCb][yCb][0] (8-900)

[1328] bvd[1]=MvdL0[xCb][yCb][1] (8-901)

[1329] 2. The motion vector rounding process specified in section 8.5.2.14 is called with the following conditions: mvX is set equal to bvL, rightShift is set equal to AmvrShift, and leftShift is set equal to AmvrShift as input; and the rounded bvL is used as output.

[1330] 3. The luminance block vector bvL is modified as follows:

[1331] u[0]=(bvL[0]+bvd[0]+2 18 )%2 18 (8-902)

[1332] bvL[0]=(u[0]>=2 17 )? (u[0]-2 18 ):u[0] (8-903)

[1333] u[1]=(bvL[1]+bvd[1]+2 18 )%2 18 (8-904)

[1334] bvL[1]=(u[1]>=2 17 )? (u[1]-2 18 ):u[1] (8-905)

[1335] NOTE 1 – The resulting values ​​of bvL[0] and bvL[1] as specified above will always be between -2 17 To 2 17 The value is in the range of -1 (inclusive).

[1336] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1337] If IsLgrBlk is true, IsLgrBlk sets the left neighbor block to 4x4 when encoded or decoded in IBC mode. is false.

[1338] If IsLgrBlk is true, IsLgrBlk sets the upper neighbor block to 4x4 when encoded or decoded in IBC mode. is false.

[1339] (or alternatively:

[1340] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1341] If IsLgrBlk is true, IsLgrBlk sets the left neighbor block to 4x4 when encoded or decoded in IBC mode. is false.

[1342] If IsLgrBlk is true, the upper neighbor block is in the same CTU as the current block and is 4x4 and in IBC mode. IsLgrBlk is set to false when using the .

[1343] when IsLgrBlk is true , [[IsInSmr[xCb][yCb] is equal to false]], the history-based block vector predictor list update process specified in section 8.6.2.6 is called with the luminance block vector bvL.

[1344] Bitstream conformance requires that the luma block vector bvL shall obey the following constraints:

[1345] -CtbSizeY is greater than or equal to ((yCb+(bvL[1]>>4))&(CtbSizeY-1))+cbHeight.

[1346] -IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] should not be equal to -1, where x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1.

[1347] 8.6.2.2 IBC Luma Block Vector Prediction Derivation Process

[1348] This process is called only when CuPredMod[0][xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma codec block relative to the top-left luma sample of the current picture.

[1349] The inputs to this process are:

[1350] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1351] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1352] - The variable cbHeight specifies the height of the current codec block in luma samples.

[1353] The output of this process is:

[1354] - Luma block vector bvL with 1 / 16 fractional sample accuracy.

[1355] [[The variables xSmr, ySmr, smrWidth, and smrHeight are derived as follows:

[1356] xSmr=IsInSmr[xCb][yCb]? SmrX[xCb][yCb]:xCb (8-906)

[1357] ySmr=IsInSmr[xCb][yCb]? SmrY[xCb][yCb]:yCb (8-907)

[1358] smrWidth=IsInSmr[xCb][yCb]? SmrW[xCb][yCb]:cbWidth (8-908)

[1359] smrHeight=IsInSmr[xCb][yCb]? SmrH[xCb][yCb]:cbHeight (8-909)]]

[1360] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1361] If IsLgrBlk is true, IsLgrBlk sets the left neighbor block to 4x4 when encoded or decoded in IBC mode. is false.

[1362] If IsLgrBlk is true, IsLgrBlk sets the upper neighbor block to 4x4 when encoded or decoded in IBC mode. is false.

[1363] (or alternatively:

[1364] The variable IslgrBlk is set to (cbWidth×cbHeight is greater than K? True: False).

[1365] If IsLgrBlk is true, IsLgrBlk sets the left neighbor block to 4x4 when encoded or decoded in IBC mode. is false.

[1366] If IsLgrBlk is true, the upper neighbor block is in the same CTU as the current block and is 4x4 and in IBC mode. IsLgrBlk is set to false when using the .

[1367] The luminance block vector bvL is derived by the following sequential steps:

[1368] 1. When IslgrBlk is true , the derivation process of the spatial block vector candidate from the neighboring codec unit specified in Section 8.6.2.3 is called with the following conditions: the luma codec block position (xCb, yCb) is set equal to ( xCb,yCb[[xSmr, ySmr]]), the luma codec block width cbWidth and luma codec block height cbHeight are set equal to [[smr]] Cb Width and [[smr]] Cb Height as input; and the outputs are the availability flags availableFlagA1, availableFlagB1 and the block vectors bvA1 and bvB1.

[1369] 2. When IslgrBlk is true , the block vector candidate list bvCandList is constructed as follows:

[1370]

[1371] 3. When IslgrBlk is true , the variable numCurrCand is set equal to the number of Merge candidates in bvCandList.

[1372] 4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the derivation process of block vector candidates based on IBC history specified in 8.6.2.4 is called with the following conditions: bvCandList, and IsLgrBlk , and numCurrCand as input; and the modified bvCandList and numCurrCand as output.

[1373] 5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

[1374] 1. bvCandList[numCurrCand][0] is set equal to 0.

[1375] 2. bvCandList[numCurrCand][1] is set equal to 0.

[1376] 3. numCurrCand increases by 1.

[1377] 6. The variable bvIdx is derived as follows:

[1378] bvIdx=general_merge_flag[xCb][yCb]? merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

[1379] 7. Make the following allocations:

[1380] bvL[0]=bvCandList[mvIdx][0] (8-912)

[1381] bvL[1]=bvCandList[mvIdx][1] (8-913)

[1382] 8.6.2.3 IBC Spatial Block Vector Candidate Derivation Process

[1383] The inputs to this process are:

[1384] - The luminance position (xCb, yCb) of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture,

[1385] - variable cbWidth, specifies the width of the current codec block in luminance samples,

[1386] - The variable cbHeight specifies the height of the current codec block in luma samples.

[1387] The output of this process is:

[1388] - the availability flags availableFlagA1 and availableFlagB1 of the neighboring codec unit,

[1389] - the block vectors bvA1 and bvB1 of the neighborhood codec with 1 / 16 fractional sample accuracy,

[1390] For the exports of availableFlagA1 and mvA1, the following applies:

[1391] -The luma position (xNbA1, yNbA1) inside the neighborhood luma codec block is set equal to (xCb-1, yCb+cbHeight-1).

[1392] - Call the derivation process for neighborhood block availability specified in Section 6.4.4 with the following conditions: the current luma position (xCurr, yCurr) is set equal to (xCb, yCb), the neighborhood luma position (xNbA1, yNbA1), checkPredModeY is set equal to true, and cIdx is set equal to 0 as input; and assign the output to the block availability flag availableA1.

[1393] -Variables availableFlagA1 and bvA1 are exported as follows:

[1394] - If availableA1 is equal to false, availableFlagA1 is set equal to 0 and both components of bvA1 are set equal to 0.

[1395] - Otherwise, availableA1 is set equal to 1 and the following assignments are made:

[1396] bvA1=MvL0[xNbA1][yNbA1] (8-914)

[1397] For the exports of availableFlagB1 and bvB1, the following applies:

[1398] -The luma position (xNbB1, yNbB1) inside the neighborhood luma codec block is set equal to (xCb+cbWidth-1, yCb-1).

[1399] - Call the derivation process for neighborhood block availability specified in Section 6.4.4 with the following conditions: the current luma position (xCurr, yCurr) is set equal to (xCb, yCb), the neighborhood luma position (xNbB1, yNbB1), checkPredModeY is set equal to true, and cIdx is set equal to 0 as input; and assign the output to the block availability flag availableB1.

[1400] -Variables availableFlagB1 and bvB1 are exported as follows:

[1401] - availableFlagB1 is set equal to 0 and both components of bvB1 are set equal to 0 if one or more of the following conditions are true:

[1402] -availableB1 is equal to false.

[1403] -availableA1 is equal to true and luma positions (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vector.

[1404] - Otherwise, availableFlagB1 is set equal to 1 and the following assignments are made:

[1405] bvB1=MvL0[xNbB1][yNbB1] (8-915)

[1406] 8.6.2.4 Derivation of Block Vector Candidates Based on IBC History

[1407] The inputs to this process are:

[1408] - block vector candidate list bvCandList,

[1409] -[[Variable isInSmr, specifies whether the current codec unit is inside the shared Merge candidate area]]

[1410] The variable IslgrBlk indicating a non-small block,

[1411] - The number of block vector candidates available in the list, numCurrCand.

[1412] The output of this process is:

[1413] - modified block vector candidate list bvCandList,

[1414] - The number of modified motion vector candidates in the list, numCurrCand.

[1415] The variables isPrunedA1 and isPrunedB1 are both set equal to false.

[1416] For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], where index hMvpIdx = 1.[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand equals MaxNumIbcMergeCand:

[1417] 1. The variable sameMotion is exported as follows:

[1418] -if IsLgrBlk is true And for any block vector candidate N all of the following conditions are true, where N is A1 or B1, then sameMotion and isPrunedN are both set equal to true:

[1419] -hMvpIdx is less than or equal to 1.

[1420] -CandidateHmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is equal to block vector candidateN.

[1421] -isPrunedN is equal to false.

[1422] Otherwise, sameMotion is set equal to false.

[1423] 2. When sameMotion is equal to false, add the candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] to the block vector candidate list as follows:

[1424] bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx](8-916)

[1425] Fig. 22 2200. The apparatus 2200 may be used to implement one or more methods described herein. The apparatus 2200 may be implemented in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 2200 may include one or more processors 2202, one or more memories 2204, and video processing hardware 2206. The processor(s) 2202 may be configured to implement one or more methods described in this document. The memory(s) 2204 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 2206 may be used to implement some of the techniques described herein in hardware circuits. The video processing hardware 2206 may be included in part or in whole in the form of dedicated hardware or a graphics processing unit (GPU) or a specialized signal processing block within the processor(s) 2202.

[1426] Fig.23 23 is a flow chart of an example of a video processing method 2300. The method 2300 includes: at operation 2302, determining to use a sub-block intra block copy (sbIBC) codec mode. The method 2300 also includes: at operation 2304, performing conversion using the sbIBC codec mode.

[1427] Some embodiments may be described using the following clause-based description.

[1428] The following clauses illustrate example embodiments of the techniques discussed in Item 1 in the previous section.

[1429] 1. A method of video processing, comprising: determining to use a sub-block intra block copy (sbIBC) codec mode in a conversion between a current video block in a video region and a bitstream representation of the current video block, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded and decoded based on reference samples from the video region, wherein the size of the sub-blocks is based on a division rule, and the conversion is performed on the plurality of sub-blocks using the sbIBC codec mode.

[1430] 2. The method of clause 1, wherein the current video block is an MxN block, wherein M and N are integers, and wherein the partitioning rule specifies that each sub-block has the same size.

[1431] 3. A method according to clause 1, wherein the bitstream representation includes syntax elements indicating a partitioning rule or size of the sub-blocks.

[1432] 4. A method according to any of clauses 1-3, wherein the partitioning rule specifies the size of the sub-blocks depending on the color components of the current video block.

[1433] 5. A method according to any of clauses 1-4, wherein a sub-block of a first colour component may derive motion vector information for the sub-block of the first colour component from a sub-block of a second colour component corresponding to the current video block.

[1434] The following clauses illustrate example embodiments of the techniques discussed in Items 2 and 6 in the previous section.

[1435] 1. A method of video processing, comprising: determining to use a sub-block intra block copy (sbIBC) codec mode in a conversion between a current video block in a video region and a bitstream representation of the current video block, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded and decoded based on reference samples from the video region, and performing the conversion on the plurality of sub-blocks using the sbIBC codec mode, wherein the conversion comprises: determining an initialization motion vector (initMV) of a given sub-block, identifying a reference block from the initMV, and deriving motion vector (MV) information of the given sub-block using MV information of the reference block.

[1436] 2. A method according to clause 1, wherein determining the initMV includes determining the initMV from one or more neighboring blocks of the given sub-block.

[1437] 3. A method according to clause 2, wherein the one or more neighboring blocks are checked in sequence.

[1438] 4. The method of clause 1, wherein said determining said initMV comprises deriving said initMV from a motion candidate list.

[1439] 5. The method according to any one of clauses 1 to 4, wherein the identifying the reference block comprises converting the initMV to an accuracy of one pixel and identifying the reference block based on the converted initMV.

[1440] 6. A method according to any one of clauses 1-4, wherein the identifying the reference block comprises: applying the initMV to an offset position within the given block, wherein the offset position is expressed as an offset (offsetX, offsetY) from a predetermined position of the given sub-block.

[1441] 7. A method according to any of clauses 1-6, wherein deriving the MV of the given sub-block comprises cropping MV information of the reference block.

[1442] 8. A method according to any of clauses 1-7, wherein the reference block has different color components than the current video block.

[1443] The following clauses illustrate example embodiments of the techniques discussed in items 3, 4, and 5 in the previous section.

[1444] 1. A method of video processing, comprising: determining to use a sub-block intra block copy (sbIBC) codec mode in a conversion between a current video block in a video region and a bitstream representation of the current video block, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded and decoded based on reference samples from the video region; and performing the conversion on the plurality of sub-blocks using the sbIBC codec mode, wherein the conversion includes generating a sub-block IBC candidate.

[1445] 2. A method according to clause 1, wherein the sub-block IBC candidate is added to a candidate list, the candidate list comprising alternative motion vector predictor candidates.

[1446] 3. The method of clause 1, wherein the sub-block IBC candidate is added to a list, the list comprising affine merge candidates.

[1447] The following clauses illustrate example embodiments of the techniques discussed in items 7, 8, 9, 10, 11, 12, and 13 in the previous section.

[1448] 1. A method of video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing a first sub-block of the plurality of sub-blocks using a sub-block intra block coding (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an intra coding mode.

[1449] 2. A method of video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing all of the plurality of sub-blocks using an intra-frame coding mode.

[1450] 3. A method for video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing all of the plurality of sub-blocks using a palette encoding and decoding mode, in which a palette of representative pixel values ​​is used to encode and decode each sub-block.

[1451] 4. A method for video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing a first sub-block of the plurality of sub-blocks using a palette mode, and processing a second sub-block of the plurality of sub-blocks using an intra-block copy coding and decoding mode, wherein a palette of representative pixel values ​​in the palette mode is used to encode and decode the first sub-block.

[1452] 5. A method for video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing a first sub-block of the plurality of sub-blocks using a palette mode, and processing a second sub-block of the plurality of sub-blocks using an intra-frame coding mode, wherein a palette of representative pixel values ​​in the palette mode is used to encode and decode the first sub-block.

[1453] 6. A method of video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing a first sub-block of the plurality of sub-blocks using a sub-block intra-block coding (sbIBC) mode, and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

[1454] 7. A method of video processing, comprising: performing conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion comprises processing a first sub-block of the plurality of sub-blocks using a sub-block intra-codec mode, and processing a second sub-block of the plurality of sub-blocks using an inter-codec mode.

[1455] The following clauses illustrate example embodiments of the techniques discussed in Item 14 in the previous section.

[1456] 8. A method according to any of clauses 1-7, wherein the method further comprises avoiding updating the IBC history based motion vector predictor table after conversion of the current video block.

[1457] The following clauses illustrate example embodiments of the techniques discussed in Item 15 in the previous section.

[1458] 9. The method of any one or more of clauses 1-7, further comprising avoiding updating a non-IBC history based motion vector predictor table after conversion of the current video block.

[1459] The following clauses illustrate example embodiments of the techniques discussed in Item 16 in the previous section.

[1460] 10. A method according to any of clauses 1-7, wherein the converting comprises selective use of a loop filter based on the processing.

[1461] The following clauses illustrate example embodiments of the techniques discussed in Item 1 in the previous section.

[1462] 17. A method according to any one of clauses 1-7, wherein performing the conversion includes performing the conversion by disabling a certain codec mode for the current video block due to the use of the method, wherein the certain codec mode includes one or more of the following: sub-block transform, affine motion prediction, multiple reference line intra-frame prediction, matrix-based intra-frame prediction, symmetric motion vector difference (MVD) encoding and decoding, Merge using MVD decoder-side motion derivation / refinement, bidirectional best flow, simplified secondary transform, or multiple transform sets.

[1463] The following clauses illustrate example embodiments of the techniques discussed in Item 18 in the previous section.

[1464] 1. A method according to any of the preceding clauses, wherein the indicator in the bitstream representation comprises information on how to apply the method to the current video block.

[1465] The following clauses illustrate example embodiments of the techniques discussed in Item 19 in the previous section.

[1466] 1. A method for video encoding, comprising: deciding to use the method described in any one of the above clauses to encode the current video block into the bitstream representation; and including information indicating the decision in the bitstream representation at the following levels: decoder parameter set level or sequence parameter set level or video parameter set level or picture parameter set level or picture header level or slice header level or slice group header level or maximum codec unit level or codec unit level or maximum codec unit row level or LCU group level or transform unit level or prediction unit level or video codec unit level.

[1467] 2. A method of video encoding, comprising: deciding to use the method described in any one of the above clauses to encode the current video block into the bitstream representation based on an encoding condition; and performing the encoding using the method described in any one of the above clauses, wherein the condition is based on one or more of the following: a codec unit, a prediction unit, a transform unit, the current video block, or a position of a video codec unit of the current video block,

[1468] the block dimensions of the current video block and / or its neighboring blocks,

[1469] the block shapes of the current video block and / or its neighboring blocks,

[1470] the intra mode of the current video block and / or its neighboring blocks,

[1471] The motion / block vector of the neighboring blocks of the current video block;

[1472] The color format of the current video block;

[1473] Codec tree structure;

[1474] The slice or slice group type or picture type of the current video block;

[1475] The color component of the current video block;

[1476] The temporal layer ID of the current video block;

[1477] The profile or level or standard used for the bitstream representation.

[1478] The following clauses illustrate example embodiments of the techniques discussed in Item 20 in the previous section.

[1479] 1. A method of video processing, comprising: determining to use an intra block copy mode and an inter prediction mode for a conversion between a block in a video region and a bitstream representation of the video region; and performing the conversion on the block in the video region using the intra block copy mode and the inter prediction mode.

[1480] 2. The method according to clause 1, wherein the video region comprises a video picture or a video strip or a video slice group or a video slice.

[1481] 3. A method according to any of clauses 1-2, wherein the inter prediction mode uses the Alternative Motion Vector Predictor (AMVP) codec mode.

[1482] 4. A method according to any of clauses 1-3, wherein performing the conversion comprises deriving Merge candidates for the intra block copy mode from neighboring blocks.

[1483] The following clauses illustrate example embodiments of the techniques discussed in Item 21 in the previous section.

[1484] 1. A method for video processing, comprising: during a conversion between a current video block and a bitstream representation of the current video block, performing a motion candidate list construction process and / or performing a table update process for updating a history-based motion vector predictor table as appropriate based on a coding condition, and performing the conversion based on the motion candidate list construction process and / or the table update process.

[1485] 2. The method according to clause 1, wherein different processes may be applied when the encoding and decoding conditions are met or not met.

[1486] 3. A method according to clause 1, wherein when the encoding condition is met, updating of the history-based motion vector predictor table is not applied.

[1487] 4. A method according to clause 1, wherein when the encoding and decoding condition is met, the derivation of candidates from spatial neighboring (adjacent or non-adjacent) blocks is skipped.

[1488] 5. The method of clause 1, wherein when the encoding and decoding condition is met, derivation of HMVP candidates is skipped.

[1489] 6. The method according to clause 1, wherein the encoding and decoding condition includes that the block width multiplied by the height is not greater than 16 or 32 or 64.

[1490] 7. The method of clause 1, wherein the encoding and decoding condition comprises that the block is encoded and decoded in IBC mode.

[1491] 8. A method according to clause 1, wherein the encoding and decoding conditions are as described in item 21.bsiv in the previous chapter.

[1492] A method according to any of the preceding clauses, wherein said converting comprises generating said bitstream representation from said current video block.

[1493] A method according to any of the preceding clauses, wherein the converting comprises generating samples of the current video block from the bitstream representation.

[1494] A video processing device comprises a processor, wherein the processor is configured to implement the method described in any one or more of the above clauses.

[1495] A computer readable medium having codes stored thereon, which when executed cause a processor to implement the method described in any one or more of the above clauses.

[1496] Fig.24 2400 . 2400 may include some or all of the components of system 2400. System 2400 may include an input 2402 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be received in a compressed or encoded format. Input 2402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[1497] System 2400 may include a codec component 2404 that can implement various codecs or encoding methods described in this document. Codec component 2404 can reduce the average bit rate of the video from input 2402 to the output of codec component 2404 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 2404 can be stored or sent via the connected communication, as represented by component 2406. The bitstream (or codec) representation of the storage or communication of the video received at input 2402 can be used by component 2408 to generate pixel values ​​or displayable video sent to display interface 2410. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations of the reverse codec results will be performed by the decoder.

[1498] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be implemented in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[1499] Fig.25 is a flowchart representation of a method 2500 for video processing according to the present technology. The method 2500 includes, at operation 2510, for conversion between a current block of video and a bitstream representation of the video, determining that the current block is divided into a plurality of sub-blocks. Encoding and decoding at least one of the plurality of blocks using a modified intra-block copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture of the current block. The method 2500 includes, at operation 2520, performing the conversion based on the determination.

[1500] In some embodiments, the video region includes a current picture, a slice, a slice, a block, or a slice group. In some embodiments, where the current block has dimensions of M×N (M and N are integers), the current block is divided into a plurality of sub-blocks. In some embodiments, the plurality of sub-blocks have the same size L×K, where L and K are integers. In some embodiments, L=K. In some embodiments, L=4 or K=4.

[1501] In some embodiments, the plurality of sub-blocks have different sizes. In some embodiments, the plurality of sub-blocks have non-rectangular shapes. In some embodiments, the plurality of sub-blocks have triangular or wedge shapes.

[1502] In some embodiments, the size of at least one of the plurality of sub-blocks is determined based on the size of a minimum decoding unit, a minimum prediction unit, a minimum transform unit, or a minimum unit for motion information storage. In some embodiments, the size of at least one sub-block is represented as (N1×minW)×(N2×minH), where minW×minH represents the size of a minimum decoding unit, a prediction unit, a transform unit, or a unit for motion information storage, and where N1 and N2 are positive integers. In some embodiments, the size of at least one of the plurality of sub-blocks is based on a coding mode in which the current block is coded in the bitstream representation. In some embodiments, the coding mode includes at least an intra-block copy (IBC) Merge mode or a sub-block temporal motion vector prediction mode. In some embodiments, the size of at least one of the plurality of sub-blocks is signaled in the bitstream representation.

[1503] In some embodiments, the method includes determining that a subsequent block of the video is divided into a plurality of sub-blocks for conversion, wherein a first sub-block in the current block has a different size than a second sub-block in the subsequent block. In some embodiments, the size of the first sub-block is different from the size of the second sub-block according to the dimension of the current block and the dimension of the subsequent block. In some embodiments, the size of at least one of the plurality of sub-blocks is based on a color format or a color component of the video. In some embodiments, the first sub-block is associated with a first color component of the video and the second sub-block is associated with a second color component of the video, and the first sub-block and the second sub-block have different dimensions. In some embodiments, when the color format of the video is 4:2:0, the dimension of the first sub-block associated with the luma component is 2L×2K, and the dimension of the second sub-block associated with the chroma component is L×K. In some embodiments, when the color format of the video is 4:2:2, the dimension of the first sub-block associated with the luma component is 2L×2K, and the dimension of the second sub-block associated with the chroma component is 2L×K. In some embodiments, the first sub-block is associated with a first color component of the video and the second sub-block is associated with a second color component of the video, and the first sub-block and the second sub-block have the same dimension. In some embodiments, where the color format of the video is 4:2:0 or 4:4:4, the dimensions of the first sub-block associated with the luma component are 2L×2K, and the dimensions of the second sub-block associated with the chroma component are 2L×2K.

[1504] In some embodiments, a motion vector of a first sub-block associated with a first color component of the video is determined based on one or more sub-blocks associated with a second color component of the video. In some embodiments, the motion vector of the first sub-block is an average of motion vectors of one or more sub-blocks associated with the second color component. In some embodiments, the current block is partitioned into a plurality of sub-blocks based on a single tree partitioning structure. In some embodiments, the current block is associated with a chroma component of the video. In some embodiments, the size of the current block is 4×4.

[1505] In some embodiments, the motion information of at least one of the plurality of sub-blocks is determined based on: identifying a reference block based on an initial motion vector, and determining the motion information of the sub-block based on the reference block. In some embodiments, the reference block is located in a current picture. In some embodiments, the reference block is located in a reference picture of one or more reference pictures. In some embodiments, at least one of the one or more reference pictures is a current picture. In some embodiments, the reference block is located in a collocated reference picture, which is collocated with one of the one or more reference pictures according to temporal information. In some embodiments, the reference picture is determined based on motion information of a collated block or a neighboring block of the collated block. The collocated block is collocated with the current block according to temporal information. In some embodiments, the initial motion vector of the reference block is determined based on one or more neighboring blocks of the current block or one or more neighboring blocks of the sub-block. In some embodiments, the one or more neighboring blocks include neighboring blocks and / or non-neighboring blocks of the current block or sub-block. In some embodiments, at least one of the one or more neighboring blocks and the reference block is located in the same picture. In some embodiments, at least one of the one or more neighboring blocks is located in a reference picture of the one or more reference pictures. In some embodiments, at least one of the one or more neighboring blocks is located in a collocated reference picture, the collocated reference picture being collocated with one of the one or more reference pictures according to temporal information. In some embodiments, the reference picture is determined based on motion information of the sorted block or a neighboring block of the sorted block, wherein the collocated block is collocated with the current block according to temporal information.

[1506] In some embodiments, the initial motion vector is equal to a motion vector stored in one or more neighboring blocks. In some embodiments, the initial motion vector is determined based on the order in which one or more neighboring blocks are checked for conversion. In some embodiments, the initial motion vector is the first identified motion vector associated with the current picture. In some embodiments, the initialization motion vector of the reference block is determined based on a list of motion candidates. In some embodiments, the list of motion candidates includes a list of intra-block copy (IBC) candidates, a list of Merge candidates, a list of sub-block temporal motion vector prediction candidates, or a list of history-based motion candidates determined based on past motion prediction results. In some embodiments, the initialization motion vector is determined based on the selected candidate in the list. In some embodiments, the selected candidate is the first candidate in the list. In some embodiments, the list of candidates is constructed based on a process using spatial neighborhood blocks that are different from the spatial neighborhood blocks in the traditional construction process.

[1507] In some embodiments, an initial motion vector of a reference block is determined based on the position of the current block in the current picture. In some embodiments, an initial motion vector of a reference block is determined based on the dimension of the current block. In some embodiments, the initial motion vector is set to a default value. In some embodiments, the initial motion vector is indicated in a bitstream representation at a video unit level. In some embodiments, the video unit includes a slice, a stripe, a picture, a group block, a row of a codec tree unit (CTU), a CTU, a codec tree block (CTB), a codec unit (CU), a prediction unit (PU), or a transform unit (TU). In some embodiments, the initial motion vector for a sub-block is different from another initial motion block for a second sub-block of the current block. In some embodiments, the initial motion vectors of multiple sub-blocks of the current block are determined differently according to the video unit. In some embodiments, the video unit includes a block, a slice, or a stripe.

[1508] In some embodiments, before identifying the reference block, the initial motion vector is converted to integer precision of F pixels, where F is a positive integer greater than or equal to 1. In some embodiments, F is 1, 2, or 4. In some embodiments, the initial motion vector is represented as (vx, vy), and wherein the converted motion vector (vx', vy') is represented as (vx×F, vy×F). In some embodiments, the upper left position of the sub-block is represented as (x, y), and the size of the sub-block is L×K, where L is the width of the current sub-block and K is the height of the sub-block. The reference block is identified as covering the area of ​​(x+offsetX+vx', y+offsetY+vy'), where offsetX and offset are non-negative values. In some embodiments, offsetX is 0 and / or offsetY is 0. In some embodiments, offsetX is equal to L / 2, L / 2+1, or L / 2-1. In some embodiments, offsetY is equal to K / 2, K / 2+1, or K / 2-1. In some embodiments, offsetX and / or offsetY is clipped to a range that includes a picture, a slice, a slice, a block, or an intra block copy reference area.

[1509] In some embodiments, the motion vector of the sub-block is further determined based on the motion information of the reference block. In some embodiments, in the case where the motion vector of the reference block points to the current picture, the motion vector of the sub-block is the same as the motion vector of the reference block. In some embodiments, in the case where the motion vector of the reference block points to the current picture, the motion vector of the sub-block is determined based on adding the initial motion vector to the motion vector of the reference block. In some embodiments, the motion vector of the sub-block is clipped within a certain range so that the motion vector of the sub-block points to the intra block copy reference area. In some embodiments, the motion vector of the sub-block is a valid motion vector of the intra block copy candidate of the sub-block.

[1510] In some embodiments, one or more intra block copy candidates for a sub-block are determined to determine motion information of the sub-block. In some embodiments, one or more intra block copy candidates are added to a motion candidate list, the list comprising one of the following: a Merge candidate for a sub-block, a sub-block temporal motion vector prediction candidate for a sub-block, or an affine Merge candidate for a sub-block. In some embodiments, one or more intra block copy candidates are positioned before any Merge candidate for a sub-block in the list. In some embodiments, one or more intra block copy candidates are positioned after any sub-block temporal motion vector prediction candidate for a sub-block in the list. In some embodiments, one or more intra block copy candidates are positioned after any inherited or constructed affine candidate for a sub-block in the list. In some embodiments, whether to add one or more intra block copy candidates to the list of motion candidates is based on the encoding and decoding mode of the current block. In some embodiments, when the current block is encoded and decoded using an intra block copy (IBC) sub-block temporal motion vector prediction mode, one or more intra block copy candidates are excluded from the list of motion candidates.

[1511] In some embodiments, whether to add one or more intra block copy candidates to the list of motion candidates is based on the partition structure of the current block. In some embodiments, one or more intra block copy candidates are added as Merge candidates for sub-blocks in the list. In some embodiments, one or more intra block copy candidates are added to the list of motion candidates based on different initial motion vectors. In some embodiments, whether to add one or more intra block copy candidates to the list of motion candidates is indicated in the bitstream representation. In some embodiments, whether to signal the index of the list of motion candidates in the bitstream representation is based on the encoding and decoding mode of the current block. In some embodiments, in the case of encoding and decoding the current block using the intra block copy Merge mode, the index of the list of motion candidates indicating the intra block copy Merge candidate is signaled in the bitstream representation. In some embodiments, in the case of using the intra block copy sub-block temporal motion vector prediction mode, the index of the list of motion candidates indicating the intra block copy sub-block temporal motion vector prediction candidate is signaled in the bitstream representation. In some embodiments, the motion vector difference of the intra block copy sub-block temporal motion vector prediction mode is applied to multiple sub-blocks.

[1512] In some embodiments, the reference block and the sub-block are associated with the same color component of the video. In some embodiments, whether the current block is divided into a plurality of sub-blocks is based on a codec characteristic associated with the current block. In some embodiments, the codec characteristic includes a syntax flag in the following bitstream representation: a decoder parameter set, a sequence parameter set, a video parameter set, a picture parameter set, an APS, a picture header, a slice header, a slice group header, a maximum codec unit (LCU), a codec unit (CU), a row of an LCU, a group of LCUs, a transform unit, a prediction unit, a prediction unit block, or a video codec unit. In some embodiments, the codec characteristic includes the position of the codec unit, the prediction unit, the transform unit, the block, or the video codec unit. In some embodiments, the codec characteristic includes the dimension of the current block or the neighboring block of the current block. In some embodiments, the codec characteristic includes the shape of the current block or the neighboring block of the current block. In some embodiments, the codec characteristic includes the intra-frame codec mode of the current block or the neighboring block of the current block. In some embodiments, the codec characteristic includes the motion vector of the neighboring block of the current block. In some embodiments, the codec characteristic includes an indication of the color format of the video. In some embodiments, the codec characteristic includes the codec tree structure of the current block. In some embodiments, the codec characteristics include a slice type, a slice group type, or a picture type associated with the current block. In some embodiments, the codec characteristics include a color component associated with the current block. In some embodiments, the codec characteristics include a temporal layer identifier associated with the current block. In some embodiments, the codec characteristics include a profile, level, or hierarchy of a standard for bitstream representation.

[1513] Fig.26 26 is a flowchart representation of a method 2600 for video processing according to the present technology. The method 2600 includes, at operation 2610, for conversion between a current block of video and a bitstream representation of the video, determining that the current block is divided into a plurality of sub-blocks. Each of the plurality of sub-blocks is encoded and decoded in the encoded and decoded representation using a corresponding encoding and decoding technique according to a pattern. The method also includes, at operation 2620, performing conversion based on the determination.

[1514] In some embodiments, the pattern specifies that a first sub-block of the plurality of sub-blocks is encoded and decoded using a modified intra-block copy (IBC) codec technique using reference samples from a video region. In some embodiments, the pattern specifies that a second sub-block of the plurality of sub-blocks is encoded and decoded using an intra-prediction codec technique using samples from the same sub-block. In some embodiments, the pattern specifies that a second sub-block of the plurality of sub-blocks is encoded and decoded using a palette codec technique using a palette of representative pixel values. In some embodiments, the pattern specifies that a second sub-block of the plurality of sub-blocks is encoded and decoded using an inter-codec technique using temporal information.

[1515] In some embodiments, the pattern specifies that a first sub-block of the plurality of sub-blocks is encoded and decoded using an intra-prediction encoding and decoding technique using samples from the same sub-block. In some embodiments, the pattern specifies that a second sub-block of the plurality of sub-blocks is encoded and decoded using a palette encoding and decoding technique using a palette of representative pixel values. In some embodiments, the pattern specifies that a second sub-block of the plurality of sub-blocks is encoded and decoded using an inter-frame encoding and decoding technique using temporal information.

[1516] In some embodiments, the pattern specifies that a single codec technique is used to encode and decode all of the multiple sub-blocks. In some embodiments, the single codec technique includes an intra-prediction codec technique in which samples from the same sub-block are used to encode and decode the multiple sub-blocks. In some embodiments, the single codec technique includes a palette codec technique in which a palette of representative pixel values ​​is used to encode and decode the multiple sub-blocks.

[1517] In some embodiments, where the pattern of one or more codecs is applied to the current block, the history-based motion candidate table for the sub-block temporal motion vector prediction mode remains the same for the conversion, the history-based motion candidate table being determined based on motion information in past conversions. In some embodiments, the history-based table is used for either the IBC codec or the non-IBC codec.

[1518] In some embodiments, when the pattern specifies that at least one of the plurality of sub-blocks is encoded and decoded using the IBC codec technique, one or more motion vectors for the at least one sub-block are used to update a history-based motion candidate table for an IBC sub-block temporal motion vector prediction mode, the history-based motion candidate table being determined based on motion information in past conversions. In some embodiments, when the pattern specifies that at least one of the plurality of sub-blocks is encoded and decoded using the inter-frame codec technique, one or more motion vectors for the at least one sub-block are used to update a history-based motion candidate table for a non-IBC sub-block temporal motion vector prediction mode, the history-based motion candidate table being determined based on motion information in past conversions.

[1519] In some embodiments, the use of a filtering process for filtering boundaries of a plurality of sub-blocks is based on the use of at least one coding technique according to a pattern. In some embodiments, the filtering process for filtering boundaries of a plurality of sub-blocks is applied when at least one coding technique is applied. In some embodiments, the filtering process for filtering boundaries of a plurality of sub-blocks is omitted when at least one coding technique is applied.

[1520] In some embodiments, the second codec technique is disabled for the current block converted according to the pattern. In some embodiments, the second codec technique includes at least one of the following: a sub-block transform codec technique, an affine motion prediction codec technique, a multiple reference line intra prediction codec technique, a matrix-based intra prediction codec technique, a symmetric motion vector difference (MVD) codec technique, a Merge codec technique using decoder-side motion derivation or refinement, a bidirectional best stream codec technique, a secondary transform codec technique with reduced dimension based on the current block dimension, or a multiple transform set codec technique.

[1521] In some embodiments, use of at least one codec technique according to a pattern is signaled in a bitstream representation. In some embodiments, use is signaled at a sequence level, a picture level, a slice level, a slice group level, a slice level, a chunk level, a codec tree unit (CTU) level, a codec tree block (CTB) level, a codec unit (CU) level, a prediction unit (PU) level, a transform unit (TU), or another video unit level. In some embodiments, at least one codec technique comprises a modified IBC codec technique, and the modified IBC codec technique is indicated in the bitstream representation based on an index value indicating a candidate in a motion candidate list. In some embodiments, a predefined value is assigned to a current block encoded using the modified IBC codec technique.

[1522] In some embodiments, use of at least one coding technique according to the pattern is determined during conversion. In some embodiments, use of an intra-block copy (IBC) coding technique for coding the current block is signaled in the bitstream, in which a reference sample from the current block is used. In some embodiments, use of an intra-block copy (IBC) coding technique for coding the current block is determined during conversion, in which a reference sample from the current block is used.

[1523] In some embodiments, motion information of the plurality of sub-blocks of the current block is used as a motion vector predictor for conversion between subsequent blocks of the video and a bitstream representation. In some embodiments, motion information of the plurality of sub-blocks of the current block is not allowed to be used for conversion between subsequent blocks of the video and a bitstream representation. In some embodiments, the current block is divided into a plurality of sub-blocks encoded and decoded using at least one codec technique based on whether the motion candidate is a candidate applicable to the video block or a sub-block within the block.

[1524] Fig. 2727 is a flowchart representation of a method 2700 for video processing according to the present technology. The method 2700 includes, at operation 2710, for a conversion between a current block of video and a bitstream representation of the video, determining an operation associated with a motion candidate list based on conditions related to characteristics of the current block. The list of motion candidates is constructed for a codec technique or based on information from previously processed video blocks. The method 2700 also includes, at operation 2720, performing the conversion based on the determination.

[1525] In some embodiments, the coding technique includes a Merge coding technique, an intra block copy (IBC) sub-block temporal motion vector prediction coding technique, a sub-block Merge coding technique, an IBC coding technique, or a modified IBC coding technique, which uses reference samples from a video area of ​​the current block to encode and decode at least one sub-block of the current block.

[1526] In some embodiments, the dimension of the current block is W×H, where W and H are positive integers. The condition is related to the dimension of the current block. In some embodiments, the condition is related to the codec information of the current block or the codec information of the neighboring block of the current block. In some embodiments, the condition is related to the Merge sharing condition for sharing a list of motion candidates between the current block and another block.

[1527] In some embodiments, the operation includes using Merge encoding and decoding technology to derive spatial Merge candidates for a list of motion candidates. In some embodiments, the operation includes deriving motion candidates for a list of motion candidates based on spatial neighboring blocks of the current block. In some embodiments, the spatial neighboring blocks include adjacent blocks or non-adjacent blocks of the current block.

[1528] In some embodiments, the operations include deriving motion candidates for a list of motion candidates constructed based on information from previously processed video blocks. In some embodiments, the operations include deriving pairwise Merge candidates for the list of motion candidates. In some embodiments, the operations include one or more pruning operations that remove redundant entries in the list of motion candidates. In some embodiments, the one or more pruning operations are used for spatial Merge candidates in the list of motion candidates.

[1529] In some embodiments, the operation includes updating a motion candidate list constructed based on information from previously processed video blocks after conversion. In some embodiments, the updating includes adding the derived candidate to the list of motion candidates without performing a pruning operation that removes redundancy in the list of motion candidates. In some embodiments, the operation includes adding a default motion candidate to the list of motion candidates. In some embodiments, the default motion candidate includes a zero motion candidate using an IBC sub-block temporal motion vector prediction codec technique. In some embodiments, the operation is skipped if a condition is met.

[1530] In some embodiments, the operation includes checking the motion candidates in the list of motion candidates in a predefined order. In some embodiments, the operation includes checking a predefined number of motion candidates in the list of motion candidates. In some embodiments, the condition is satisfied when W×H is greater than or equal to a threshold. In some embodiments, the condition is satisfied when W×H is greater than or equal to a threshold and the current block is encoded and decoded using an IBC sub-block temporal motion vector prediction encoding and decoding technique or a Merge encoding and decoding technique. In some embodiments, the threshold is 1024.

[1531] In some embodiments, the condition is satisfied when W and / or H is greater than or equal to a threshold. In some embodiments, the threshold is 32. In some embodiments, the condition is satisfied when W×H is less than or equal to a threshold and the current block is encoded and decoded using an IBC sub-block temporal motion vector prediction encoding and decoding technique or a Merge encoding and decoding technique. In some embodiments, the threshold is 16. In some embodiments, the threshold is 32 or 64. In some embodiments, when the condition is satisfied, the operation includes skipping inserting a candidate determined based on a spatial neighboring block into the list of motion candidates.

[1532] In some embodiments, the condition is satisfied when W is equal to T2, H is equal to T3, T2 and T3 are positive integers, and the neighboring block above the current block is available and is encoded and decoded using the same encoding and decoding technology as the current block. In some embodiments, the condition is satisfied when the neighboring block and the current block are in the same encoding and decoding tree unit.

[1533] In some embodiments, the condition is satisfied when W is equal to T2, H is equal to T3, T2 and T3 are positive integers, and the neighboring block above the current block is not available or is outside the current codec tree unit where the current block is located. In some embodiments, T2 is 4 and T3 is 8. In some embodiments, the condition is satisfied when W is equal to T4, H is equal to T5, T4 and T5 are positive integers, and the neighboring block to the left of the current block is available and is encoded and decoded using the same codec technology as the current block. In some embodiments, the condition is satisfied when W is equal to T4, H is equal to T5, T4 and T5 are positive integers, and the neighboring block to the left of the current block is not available. In some embodiments, T4 is 8 and T5 is 4.

[1534] In some embodiments, when W×H is less than or equal to a threshold, the current block is encoded and decoded using the IBC sub-block temporal motion vector prediction codec technology or the Merge codec technology, and the first neighboring block above the current block and the second neighboring block to the left of the current block are encoded and decoded using the same codec technology, the condition is met. In some embodiments, the first and second neighboring blocks are available and are encoded and decoded using the IBC codec technology, and wherein the second neighboring block is in the same codec tree unit as the current block. In some embodiments, the first neighboring block is not available, and wherein the second neighboring block is available and in the same codec tree unit as the current block. In some embodiments, the first and second neighboring blocks are not available. In some embodiments, the first neighboring block is available and the second neighboring block is not available. In some embodiments, the first neighboring block is not available and the second neighboring block is outside the codec tree unit where the current block is located. In some embodiments, the first neighboring block is available and the second neighboring block is outside the codec tree unit where the current block is located. In some embodiments, the threshold is 32. In some embodiments, the first and second neighboring blocks are used to derive spatial Merge candidates. In some embodiments, the upper left sample point of the current block is located at (x, y), and wherein the second neighboring block covers the sample point located at (x-1, y+H-1). In some embodiments, the upper left sample point of the current block is located at (x, y), and wherein the second neighboring block covers the sample point located at (x+W-1, y-1).

[1535] In some embodiments, the same coding technology includes IBC coding technology. In some embodiments, the same coding technology includes inter-frame coding technology. In some embodiments, the dimension of the neighboring block of the current block is equal to A×B. In some embodiments, the dimension of the neighboring block of the current block is greater than A×B. In some embodiments, the dimension of the neighboring block of the current block is less than A×B. In some embodiments, A×B is equal to 4×4. In some embodiments, the threshold is predefined. In some embodiments, the threshold is signaled in the bitstream representation. In some embodiments, the threshold is based on the coding characteristics of the current block, which includes a coding mode for coding and decoding the current block.

[1536] In some embodiments, the condition is satisfied when the current block has a parent node of a list of shared motion candidates and the current block is encoded and decoded using IBC sub-block temporal motion vector prediction encoding and decoding technology or Merge encoding and decoding technology. In some embodiments, the condition is adaptively changed according to the encoding and decoding characteristics of the current block.

[1537] Fig.28 is a flowchart representation of a method 2800 for video processing according to the present technology. The method 2800 includes, at operation 2810, for conversion between a current block of video and a bitstream representation of the video, determining that the current block encoded using an inter-frame coding technique based on temporal information is divided into a plurality of sub-blocks. At least one of the plurality of blocks is encoded and decoded using a modified intra-frame block copy (IBC) coding technique that uses reference samples from one or more video regions of a current picture including the current block. The method 2800 includes, at operation 2820, performing the conversion based on the determination.

[1538] In some embodiments, the video region includes a current picture, a slice, a slice, a block, or a slice group. In some embodiments, the inter-frame coding technique includes a sub-block temporal motion vector coding technique, and wherein one or more syntax elements are included in the bitstream representation, the one or more syntax elements indicating whether the current block is coded based on both the current picture and the reference picture different from the current picture. In some embodiments, in the case where the current block is coded based on both the current picture and the reference picture, the one or more syntax elements indicate the reference picture used to code the current block. In some embodiments, the one or more syntax elements also indicate motion information associated with the reference picture, the motion information including at least a motion vector prediction index, a motion vector difference, or a motion vector accuracy. In some embodiments, the first reference picture list includes only the current picture, and the second reference picture list includes only the reference picture. In some embodiments, the inter-frame coding technique includes a temporal Merge coding technique, and the motion information is determined based on the neighboring blocks of the current block, the motion information including at least a motion vector or a reference picture. In some embodiments, in the case where the neighboring blocks are determined based only on the current picture, the motion information is only applicable to the current picture. In some embodiments, in the case where the neighboring blocks are determined based on both the current picture and the reference picture, the motion information is applicable to both the current picture and the reference picture. In some embodiments, the motion information is applicable to the current picture only when the neighboring blocks are determined based on both the current picture and the reference picture. In some embodiments, when the neighboring blocks are determined based only on the reference picture, the neighboring blocks are discarded to determine the Merge candidate.

[1539] In some embodiments, fixed weighting factors are assigned to reference blocks from the current picture and reference blocks from reference pictures.In some embodiments, the fixed weighting factors are signaled in the bitstream representation.

[1540] Fig.29 is a flowchart representation of a method 2900 for video processing according to the present technology. The method 2900 includes, at operation 2910, for a conversion between a current block of a current picture of a video and a bitstream representation of the video, constructing a list of motion candidates for a codec technique applicable to the current block, wherein the history-based motion candidates are derived based on information from previously processed video blocks. The codec technique uses reference samples from the current picture of the current block to predict at least a portion of the current block. The method 2900 also includes, at operation 2920, performing the conversion based on the determination.

[1541] In some embodiments, the history-based motion candidates are organized as entries in at least one table. In some embodiments, constructing includes checking only the history-based motion candidates organized in the at least one table. In some embodiments, the constructed list of motion candidates includes only history-based motion candidates.

[1542] In some embodiments, the history-based motion candidates are added to the list of motion candidates according to an order associated with the table without performing a pruning step to remove redundancy in the list. In some embodiments, the order specifies that the history-based motion candidates are added to the list based on an ascending order of the entry indexes of the history-based motion candidates in the table. In some embodiments, the order specifies that the history-based motion candidates are added to the list based on a descending order of the entry indexes of the history-based motion candidates in the table. In some embodiments, the order specifies that the first N entries in the skip table are skipped, where N is a non-negative integer. In some embodiments, the order specifies that the last N entries in the skip table are skipped, where N is a non-negative integer. In some embodiments, the order specifies that entries in the skip table have invalid block vectors.

[1543] In some embodiments, the construction is determined based on the dimensions of the block. In some embodiments, the block has a width W and a height H, and the list of motion candidates includes only motion candidates based on history when the dimensions of the block satisfy the condition. In some embodiments, the condition is satisfied when W≤T1 and H≤T2, and T1 and T2 are positive integers. In some embodiments, the condition is satisfied when W≤T1 or H≤T2, and T1 and T2 are positive integers. In some embodiments, T1=T2=4. In some embodiments, the condition is satisfied when W×H≤T, and T is a positive integer. In some embodiments, T=16.

[1544] In some embodiments, whether to signal in the bitstream representation whether the candidates in the motion candidate list are based on the number of history-based motion candidates available in the table. In some embodiments, whether to signal in the bitstream representation whether the candidates in the motion candidate list are based on the maximum number of history-based motion candidates in the table. In some embodiments, the history-based motion candidate is modified before being added to the list of motion candidates. In some embodiments, the history-based motion candidate is modified by adding an offset to the horizontal vector or the vertical vector of the history-based motion candidate. In some embodiments, the history-based motion candidate with an invalid block vector is modified to a valid block vector.

[1545] In some embodiments, a default motion candidate is added to the list before or after the history-based motion candidate is added. In some embodiments, at least one history-based motion candidate is stored in the list of motion candidates with integer pixel precision. In some embodiments, all history-based motion candidates in the list of motion candidates are stored with integer pixel precision. In some embodiments, in the case where motion information from spatial neighboring blocks or previously processed video blocks is used to construct the motion candidate list, the step of performing rounding of motion candidates is skipped.

[1546] In some embodiments, performing the conversion includes generating a bitstream representation based on the video blocks. In some embodiments, performing the conversion includes generating the video blocks from the bitstream representation.

[1547] It will be appreciated that techniques for video encoding or video decoding are disclosed. A video encoder or decoder can employ these techniques to use intra-block replication and sub-block-based video processing together to achieve higher encoding and decoding efficiency and performance.

[1548] Some embodiments of the disclosed technology include deciding or determining to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a video block, but will not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from a video block to a bitstream representation of a video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, a decoder will process the bitstream based on the video processing tool or mode in the case where it is known that the bitstream has been modified. That is, the conversion from the bitstream representation of a video to a video block will be performed using a video processing tool or mode enabled based on a decision or determination.

[1549] Some embodiments of the disclosed technology include deciding or determining to disable a video processing tool or mode. In one example, when the video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion of video blocks to a bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream using the video processing tool or mode enabled based on the decision or determination, knowing that the bitstream has not been modified.

[1550] The disclosed and other schemes, examples, embodiments, modules and functional operations described in this document may be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or in a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, for example, one or more computer program instruction modules encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material composition that affects a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" covers all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal, such as a machine-generated electrical signal, an optical signal or an electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[1551] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including stand-alone programs or modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers located on one site or distributed on multiple sites and interconnected by a communications network.

[1552] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[1553] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include or be operably coupled to one or more mass storage devices (such as magnetic disks, magneto-optical disks, or optical disks) for storing data, to receive data from the one or more mass storage devices, or to transfer data to the one or more mass storage devices, or both to receive and transfer data. However, the computer need not have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

[1554] Although this patent document contains many details, these details should not be interpreted as limitations on any subject matter or the scope of protection that can be claimed, but rather as descriptions of features specific to specific embodiments of a particular technology. In this patent document, certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although the above features may be described as working in certain combinations and even initially claimed as such, in some cases, one or more features from the combination may be removed from the claimed combination, and the claimed combination may point to a sub-combination or a variation of the sub-combination.

[1555] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all of the illustrated operations be performed, in order to achieve the desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1556] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For conversion between a current block of a current picture of a video and a bitstream of the video, determining to apply a first prediction mode on the current block; constructing a block motion list for the current block, wherein the block motion list only includes one or more motion candidates from a history-based motion candidate list; and performing the conversion based on the construction; wherein the history-based motion candidate list includes motion candidates derived based on information from previously processed blocks of the video, and Wherein, in the first prediction mode, reference samples from the current picture of the current block are used to predict at least a part of the current block.

2. The method according to claim 1, wherein: The one or more motion candidates from the history-based motion candidate list are added to the block motion list without pruning.

3. The method according to claim 1, wherein: Addition of the first N entries from the history-based motion candidate list to the block motion list is skipped, N being a non-negative integer.

4. The method according to claim 1, wherein: Addition of the last M entries from the history-based motion candidate list to the block motion list is skipped, M being a non-negative integer.

5. The method according to claim 1, wherein: Addition of invalid block vectors from the history-based motion candidate list to the block motion list is skipped.

6. The method according to claim 1, wherein: Based on a dimension of the block corresponding to the motion candidate, determining to add the motion candidate from the history-based motion candidate list to the block motion list.

7. The method according to claim 6, wherein: In response to a size of the block corresponding to the motion candidate satisfying a condition, it is determined to add the motion candidate from the history-based motion candidate list to the block motion list.

8. The method according to claim 1, wherein: In response to a size of the block corresponding to the motion candidate satisfying a condition, the block motion list includes only the one or more motion candidates from the history-based motion candidate list.

9. The method according to claim 8, wherein: In response to W*H not being greater than a threshold, the block motion list includes only the one or more motion candidates from the history-based motion candidate list, where W represents a width of the current block and H represents a height of the current block.

10. The method according to claim 9, wherein: The threshold is equal to 16.

11. The method according to claim 1, wherein: The converting includes encoding the current block into the bitstream.

12. The method according to claim 1, wherein: The converting includes decoding the current block from the bitstream.

13. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to: For conversion between a current block of a current picture of a video and a bitstream of the video, determining to apply a first prediction mode on the current block; constructing a block motion list for the current block, wherein the block motion list only includes one or more motion candidates from a history-based motion candidate list; and performing the conversion based on the construction; wherein the history-based motion candidate list includes motion candidates derived based on information from previously processed blocks of the video, and Wherein, in the first prediction mode, reference samples from the current picture of the current block are used to predict at least a part of the current block.

14. The device according to claim 13, wherein: One or more motion candidates from the history-based motion candidate list are added to the block motion list without pruning.

15. The device according to claim 13, wherein: skip adding the first N entries from the history-based motion candidate list to the block motion list, N being a non-negative integer, or Wherein, the addition of the last M entries from the history-based motion candidate list to the block motion list is skipped, and M is a non-negative integer.

16. The device according to claim 13, wherein: Addition of invalid block vectors from the history-based motion candidate list to the block motion list is skipped.

17. The device according to claim 13, wherein: In response to a size of the block corresponding to the motion candidate satisfying a condition, it is determined to add the motion candidate from the history-based motion candidate list to the block motion list.

18. The device according to claim 13, wherein: In response to W*H being not greater than 16, the block motion list includes only the one or more motion candidates from the history-based motion candidate list, where W represents the width of the current block and H represents the height of the current block.

19. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For conversion between a current block of a current picture of a video and a bitstream of the video, determining to apply a first prediction mode on the current block; Construct a block motion list for the current block, where: The block motion list includes only one or more motion candidates from the history-based motion candidate list; as well as performing the conversion based on the construction; wherein the history-based motion candidate list includes motion candidates derived based on information from previously processed blocks of the video, and Wherein, in the first prediction mode, reference samples from the current picture of the current block are used to predict at least a part of the current block.

20. A method for storing a bitstream of a video, the method comprising: Determining to apply a first prediction mode to a current block of a current picture of the video; constructing a block motion list for the current block, wherein the block motion list only includes one or more motion candidates from a history-based motion candidate list; generating the bitstream based on the construction; and storing the bitstream in a non-transitory computer-readable recording medium; wherein the history-based motion candidate list includes motion candidates derived based on information from previously processed blocks of the video, and Wherein, in the first prediction mode, reference samples from the current picture of the current block are used to predict at least a part of the current block.

21. A computer readable medium having stored thereon code which, when executed, causes a processor to implement the method of any one of claims 2 to 12.

Images