Constructing motion candidate lists for video coding.

Abstract

To provide a method for processing a picture and a picture processing device for coding or decoding an intra block copy division technique of a picture bit stream at a sub block level.SOLUTION: The method for processing a picture includes determining an operation related to a motion candidate list on the basis of a condition related to a feature of a current block for a conversion between a current block of a picture and a bit stream expression of a picture. The motion list is formed by a coding technique or on the basis of information from a block of a picture processed last time. The method also includes executing a conversion on the basis of the determination.SELECTED DRAWING: Figure 27

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Books This application claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 090409, filed on June 6, 2019, International Patent Application No. PCT / CN2019 / 092438, filed on June 22, 2019, and International Patent Application No. PCT / CN2019 / 105180, filed on September 10, 2019. main Stretch , based on Japanese Patent Application No. PCT / CN2020 / 094865 filed on June 8, 2020. The entire disclosure of the above application is three By lighting Here It is incorporated herein by reference.

[0002] This specification relates to video and image encoding and decoding techniques. [Background technology]

[0003] Digital video accounts for the largest bandwidth usage on the Internet and other digital communications networks, and the bandwidth demands for digital video use are expected to continue to grow as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention

[0004] The disclosed techniques may be used by video or image decoder or encoder embodiments to encode or decode intra block copy partitioning techniques of a video bitstream at a sub-block level.

[0005] In one exemplary aspect, a method of video processing is disclosed. The method includes determining to divide a current block into a plurality of sub-blocks for conversion between the current block of the video and a bitstream representation of the video. At least one of the plurality of blocks is encoded using a modified Intra Block Copy (IBC) encoding technique that uses reference samples from one or more video regions from a current picture of the current block. The method also includes performing the conversion based on the determination.

[0006] In another exemplary aspect, a method of video processing is disclosed. The method includes determining to divide a current block into a plurality of sub-blocks for conversion between the current block of the video and a bitstream representation of the video. Each of the plurality of sub-blocks is coded into a coded representation using a corresponding coding technique according to a pattern. The method also includes performing the conversion based on the determination.

[0007] In another exemplary aspect, a method of video processing is disclosed. The method includes determining an operation associated with a list of motion candidates for conversion between a current block of video and a bitstream representation of the video based on a condition related to a characteristic of the current block. The list of motion candidates is constructed for an encoding technique or based on information from a previously processed block of the video. The method also includes performing the conversion based on the determination.

[0008] In another exemplary aspect, a method of video processing is disclosed. The method includes determining, based on temporal information, to divide a current block coded with an inter-coding technique into a plurality of sub-blocks for conversion between the current block of the video and a bitstream representation of the video. At least one of the plurality of blocks is coded with a modified Intra Block Copy (IBC) coding technique that uses reference samples from one or more video regions from a current picture that includes the current block. The method also includes performing the conversion based on the determination.

[0009] In another exemplary aspect, a method of video processing is disclosed that includes determining to use a sub-block Intra Block Copy (sbIBC) coding mode in converting between a current video block in a video domain and a bitstream representation of the video block, where the current video block is divided into a plurality of sub-blocks, each sub-block being coded based on reference samples from the video domain, a size of the sub-blocks being based on a partitioning rule, and performing the conversion using the sbIBC coding mode on the plurality of sub-blocks.

[0010] In another exemplary aspect, a method of video processing is disclosed. The method includes determining to use a sub-block Intra Block Copy (sbIBC) coding mode in converting between a current video block in a video domain and a bitstream representation of the video block, where the current video block is divided into a plurality of sub-blocks, each sub-block being coded based on a reference sample from the video domain, and performing the conversion using the sbIBC coding mode on the plurality of sub-blocks, the conversion including determining an initialized Motion Vector (initMV) for a given sub-block, identifying a reference block from the initMV, and deriving Motion Vector (MV) information for the given sub-block using the MV information for the reference block.

[0011] In another example aspect, a method of video processing is disclosed that includes determining to use a sub-block Intra Block Copy (sbIBC) coding mode in converting between a current video block in a video domain and a bitstream representation of the video block, where the current video block is divided into a plurality of sub-blocks, each sub-block being coded based on reference samples from the video domain, and performing a conversion using the sbIBC coding mode on the plurality of sub-blocks, where the conversion includes generating sub-block IBC candidates.

[0012] In another example aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a sub-block Intra Block Copy (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an intra-coding mode.

[0013] In another exemplary aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing all sub-blocks of the plurality of sub-blocks using an intra-coding mode.

[0014] In another exemplary aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing all of the plurality of sub-blocks using a palette coding mode in which a palette of representative pixel values ​​is used to code each sub-block.

[0015] In another example aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a palette mode in which a palette of representative pixel values ​​is used to code the first sub-block, and processing a second sub-block of the plurality of sub-blocks using an intra-block mode.

[0016] In another example aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a palette mode in which a palette of representative pixel values ​​is used to code the first sub-block, and processing a second sub-block of the plurality of sub-blocks using an intra-block mode.

[0017] In another example aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a sub-block Intra Block Copy (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

[0018] In another example aspect, a method of video processing is disclosed that includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a sub-block intra-coding mode and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

[0019] In another exemplary aspect, a method of video processing is disclosed, the method including: determining to use a method according to any of the above claims to encode a current video block into a bitstream representation, and including information indicative of the decision in the bitstream representation at a decoder parameter set level, a sequence parameter set level, or a video parameter set level, or a picture parameter set level, or a picture header level, or a slice header level, or a tile group header level, or a maximum coding unit level, or a coding unit level, or a maximum coding unit row level, or a group of LCU levels, or a transform unit level, or a prediction unit level, or a video coding unit level.

[0020] In another exemplary aspect, a method of video processing is disclosed, the method including: determining to use a method as set forth in any of the above claims to encode a current video block into a bitstream representation based on an encoding condition, the condition being based on one or more of a location of a coding unit, a prediction unit, a transform unit, a current video block, or a video coding unit of the current video block.

[0021] In another example aspect, a method of video processing is disclosed that includes determining to use an intra block copy mode and an inter prediction mode for converting between a block in a video domain and a bitstream representation of the video domain, and performing the conversion using the intra block copy mode and the inter prediction mode for the block in the video domain.

[0022] In another exemplary aspect, a method of video processing is disclosed that includes performing a motion candidate list construction process and / or a table update process for updating a history-based motion vector predictor table based on an encoding condition during conversion between a current video block and a bitstream representation of the current video block, and performing a conversion based on the motion candidate list construction process and / or the table update process.

[0023] In another exemplary aspect, the methods described above may be implemented by a video decoder including a processor.

[0024] In another exemplary aspect, the methods described above may be implemented by a video encoder including a processor.

[0025] In yet another exemplary aspect, the methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.

[0026] These and other aspects are further described herein. [Brief description of the drawings]

[0027] [Figure 1] FIG. 1 illustrates the derivation process for constructing a merge candidate list. [Diagram 2] FIG. 2 shows an example of the locations of spatial merging candidates. [Diagram 3] FIG. 3 shows examples of candidate pairs that are considered for redundancy checking of spatial merge candidates. [Figure 4] FIG. 4 shows the locations for the second PU for N×2N and 2N×N divisions. [Diagram 5] FIG. 5 shows an example of an illustration of motion vector scaling for temporal merging candidates. [Figure 6] FIG. 6 shows example candidate positions for temporal merge candidates, C0 and C1. [Figure 7] FIG. 7 shows an example of joint bi-predictive merge candidates. [Figure 8] FIG. 8 shows an example of a process for deriving motion vector prediction candidates. [Figure 9] FIG. 9 shows an example of an illustration of motion vector scaling for spatial motion vector candidates. [Figure 10] FIG. 10 shows examples of simplified affine motion models for the four-parameter affine model (left) and the six-parameter affine model (right). [Figure 11] FIG. 11 shows an example of an affine motion vector field for each sub-block. [Figure 12] FIG. 12 shows examples of candidate positions for the affine merge mode. [Figure 13] FIG. 13 illustrates the modified merge list construction process. [Figure 14] FIG. 14 shows an example of inter prediction based on triangulation. [Figure 15] FIG. 15 shows an example of a CU to which the first weighting coefficient group is applied. [Figure 16] FIG. 16 shows an example of motion vector storage. [Figure 17] FIG. 17 shows an example of an Ultimate Motion Vector Expression (UMVE) search process. [Figure 18] FIG. 18 shows an example of a starting point for the UMVE. [Figure 19] FIG. 19 shows an example of an MVD(0,1) mirrored between list 0 and list 1 in a DMVR. [Figure 20] FIG. 20 shows the MVs that may be checked in one iteration. [Figure 21] FIG. 21 shows an example of intra block copy. [Figure 22] FIG. 22 is a block diagram illustrating an example of a video processing device. [Figure 23] FIG. 23 is a flowchart illustrating an example of a video processing method. [Figure 24] FIG. 24 is a block diagram illustrating an example video processing system in which the disclosed techniques can be implemented. [Diagram 25] FIG. 25 is a flowchart showing a video processing method according to the present technology. [Figure 26] FIG. 26 is a flowchart showing another image processing method according to the present technology. [Figure 27] FIG. 27 is a flowchart showing another video processing method according to the present technology. [Figure 28] FIG. 28 is a flowchart showing yet another image processing method according to the present technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] This patent specification provides various techniques that can be used by a decoder of an image or video bitstream to improve the quality of the decompressed or decoded digital video or image. For simplicity, the term "video" is used herein to include both a series of pictures (conventionally called video) and individual images. Furthermore, a video encoder may implement these techniques during the encoding process to reconstruct the decoded frames that are used for further encoding.

[0029] This patent specification uses section headings for ease of understanding and is not intended to limit embodiments disclosed in one section to only that section, and thus embodiments in one section may be combined with embodiments in other sections.

[0030] 1. Summary of the Invention

[0031] This specification relates to video coding technology. In particular, it relates to intra block copy (also known as CPR (Current Picture Referencing)) coding. It may be applied to existing video coding standards such as HEVC, or to finalize standards (Versatile Video Coding). The present invention is also applicable to future video coding standards or video codecs.

[0032] 2.Background technology

[0033] Video coding standards have evolved mainly through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, ISO / IEC created MPEG-1 and MPEG-4 Visual, and the two organizations jointly created H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC standards. Since H.262, video coding standards have been based on hybrid video coding structures where temporal prediction and transform coding are utilized. In order to explore future video coding technologies beyond HEVC, in 2015, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET). Since then, many new methods have been adopted by JVET and incorporated into reference software called JEM (Joint Exploration Mode). In April 2018, the Joint Video Expert Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG), and is working on formulating the VVC standard with the goal of reducing the bitrate by 50% compared to HEVC.

[0034] 2.1 Inter Prediction in HEVC / H.265

[0035] For an inter-coded coding unit (CU), it may be coded with one prediction unit (PU), two PUs according to the partition mode. Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists may be signaled using inter_pred_idc. The motion vector may be explicitly coded as a differential to the predictor.

[0036] When one CU is coded in skip mode, one PU is associated with the CU, there are no significant residual coefficients, and there are no coded motion vector differentials or reference picture indexes. A merge mode is defined, whereby motion parameters for the current PU are obtained from neighboring PUs, including spatial and temporal candidates. The merge mode can be applied to any inter-predicted PU, not just to skip mode. As an alternative to the merge mode, there is an explicit transmission of motion parameters, where the motion vectors (more precisely, the Motion Vector Differences (MVD) compared to the motion vector predictor), the corresponding reference picture index of each reference picture list, and the use of the reference picture list are explicitly signaled to each PU. Such a mode is referred to as Advanced Motion Vector Prediction (AMVP) in this disclosure.

[0037] If the signaling indicates to use one of the two reference picture lists, the PU is generated from one block of samples. This is called "uni-prediction." Uni-prediction is available for both P and B slices.

[0038] If the signaling indicates to use both reference picture lists, the PU is generated from two blocks of samples. This is called "bi-prediction." Bi-prediction is only available for B slices.

[0039] 2.1.1 Reference Picture Buffer

[0040] In HEVC, the term inter prediction is used to indicate a prediction derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. Similar to H.264 / AVC, an image can be predicted from multiple reference pictures. The reference pictures used for inter prediction are organized into one or more reference picture lists. A reference index identifies which reference picture in the list to use to generate the prediction signal.

[0041] One reference picture list, List0, is used for P slices, and two reference picture lists, List0 and List1, are used for B slices, where the reference pictures included in List0 / 1 may be from past and future pictures in terms of capture / display order.

[0042] 2.2.1 Merge Mode

[0043] 2.1.2.1 Deriving Merge Mode Candidates

[0044] When predicting a PU using merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to look up the motion information. The construction of this list is specified in the HEVC standard and can be summarized based on the following sequence of steps:

[0045] Step 1: Derive initial candidates

[0046] Step 1.1: Derive spatial candidates

[0047] Step 1.2: Redundancy check of spatial candidates

[0048] Step 1.3: Deriving Temporal Candidates

[0049] Step 2: Insert additional candidates

[0050] Step 2.1: Creating bidirectional prediction candidates

[0051] Step 2.2: Inserting zero motion candidates

[0052] These steps are also shown diagrammatically in Fig. 1. For the derivation of spatial merge candidates, select up to four merge candidates among the candidates at five different positions. For the derivation of temporal merge candidates, select up to one merge candidate among the two candidates. Since we assume a constant number of candidates per PU on the decoder side, additional candidates are generated if the number of candidates obtained in step 1 does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, we use truncated unary binarization (TU) to encode the index of the best merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share one merge candidate list, which is the same as the merge candidate list of the 2N × 2N prediction units.

[0053] The operations associated with the above steps are described in detail below.

[0054] 2.1.2.2 Deriving Spatial Candidates

[0055] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any of the PUs at positions A1, B1, B0, A0 are not available (e.g., because they belong to another slice or tile) or are intra-coded. After adding the candidate at position A1, the remaining candidates are subjected to a redundancy check, which ensures that candidates with the same motion information are removed from the list, improving coding efficiency. To reduce computational complexity, the aforementioned redundancy check does not consider all possible candidate pairs. Instead, only the pairs linked by arrows in FIG. 3 are considered, and a candidate is added to the list only if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a partition different from 2N×2N. As an example, FIG. 4 shows the second PU for N×2N and 2N×N cases, respectively. If the current PU is divided into N×2N, the candidate at position A1 is not considered in the list construction. In fact, adding this candidate would lead to two prediction units with the same motion information, which is redundant for having only one PU in the coding unit. Similarly, if the current PU is divided into 2N×N, the candidate at position B1 is not considered.

[0056] 2.1.2.3 Temporal Candidate Derivation

[0057] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the co-located PU in the co-located picture. As shown by the dotted line in FIG. 5, a scaled motion vector of the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located PU using POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located PU and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. The practical realization of the scaling process is described in the HEVC specification. For B slices, two motion vectors, one for reference picture list 0 and the other for reference picture list 1, are obtained and combined to form a bi-predictive merge candidate.

[0058] 2.1.2.4 Co-located pictures and co-located PUs

[0059] When TMVP is enabled (eg, slice_temporal_mvp_enabled_flag is equal to 1), the variable ColPic representing the co-located picture is derived as follows:

[0060] - If the current slice is a B slice and the signaled collocated_from_l0_flag is equal to 0, then ColPic is set equal to RefPicList1[collocated_ref_idx].

[0061] - Otherwise (slice_type is equal to B and collocated_from_l0_flag is equal to 1 or slice_type is equal to P), ColPic is set equal to RefPicList0[collocated_ref_idx].

[0062] Here, collocated_ref_idx and collocated_from_l0_flag are two syntax elements that may be signaled in the slice header.

[0063] For the co-located PU(Y) belonging to the reference frame, the location for the temporal candidate is selected between candidate C0 and candidate C1, as shown in Figure 6. If the PU at location C0 is not available, is intra-coded, or is outside the current Coding Tree Unit (CTU) (aka Largest Coding Unit (LCU)) row, location C1 is used. Otherwise, location C0 is used in deriving the temporal merge candidate.

[0064] The relevant syntax elements are described below.

[0065] 7.3.6.1 General slice segment header syntax

[0066] [Table 1]

[0067] 2.1.2.5 Deriving MVs for TMVP Candidates

[0068] Specifically, to derive TMVP candidates, the following operations are performed.

[0069] (1) In list X, set reference picture list X=0, and set the target reference picture to a reference picture with an index equal to 0 (e.g., curr_ref). Invoke the process of deriving a motion vector at the same position to obtain the MV of list X that points to curr_ref.

[0070] (2) If the current slice is a B slice, set reference picture list X=1, and the target reference picture is the reference picture with index 0 in list X (e.g., curr_ref). Invoke the co-located motion vector derivation process to obtain the MV of list X that points to curr_ref.

[0071] The process of derivation of co-located motion vectors is described in the next subsection.

[0072] 2.1.2.5.1 Derivation of motion vectors at the same position

[0073] For a co-located block, it may be intra-coded with uni- or bi-prediction, or inter-coded. If it is intra-coded, TMVP candidates are set to unavailable.

[0074] If it is uni-prediction from list A, the motion vectors of list A are scaled to reference picture list X.

[0075] If it is bi-predictive and the object reference picture list is X, then the motion vectors of list A are scaled to the referenced picture list X, where A is determined according to the following rules:

[0076] If none of the reference pictures has a larger POC value compared to the current picture, then A is set equal to X.

[0077] - Otherwise, A is set equal to collocated_from_l0_flag.

[0078] The relevant explanation is given below.

[0079] 8.5.3.2.9 Co-located motion vector derivation process The inputs to this process are: - the variable currPb that defines the current prediction block, - A variable colPb that specifies a predicted block at the same position in a picture at the same position specified by ColPic, - Luminance position (xColPb, yColPb) that specifies the top left sample of the luminance prediction block at the same position specified by colPb relative to the top left luminance sample of the picture at the same position specified by ColPic - A reference index refIdxLX, with X equal to 0 or 1.

[0080] The output of this process is: - Motion vector prediction mvLXCol, -Availability flag availableFlagLXCol.

[0081] The variable currPic defines the current picture. The arrays predFlagL0Col[x], mlvL0Col[x], and refIdxL0Col[x][y] are set equal to PredFlagL0[x], MlvL0[x], and RefIdxL0[x][y], respectively, of the picture at the same location specified by ColPic, and the arrays predFlagL1Col[x][y], mvL1Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL1[x], MvL1[x], and RefIdxL1[x][y], respectively, of the picture at the same location specified by ColPic.

[0082] The variables mvLXCol and availableFlagLXCol are derived as follows. If colPb is coded in an intra prediction mode, then both components of mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0. - Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows: -If predFlagL0Col[xColPb][yColPb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL1Col[xColPb][yColPb], refIdxL1Col[xColPb][yColPb], and L1, respectively. - Otherwise, if predFlagL0Col[xColPb][yColPb] is equal to 1 and predFlagL1Col[xColPb][yColPb][yColPb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL0Col[xColPb][yColPb], refIdxL0Col[xColPb][yColPb], and L0, respectively. - Otherwise (if predFlagL0Col[xColPb][yColPb] is equal to 1 and predFlagL1Col[xColPb][yColPb] is equal to 1), make the following assignment: - If NoBackwardPredFlag is equal to 1, then mvCol, refIdxCol, and listCol are set equal to mvLXCol[xColPb][yColPb], refIdxLXCol[xColPb][yColPb], and LX, respectively. - Otherwise, mvCol, refIdxCol, and listCol are set equal to mvLNCol[xColPb][yColPb], refIdxLNCol[xColPb][yColPb], and LN, respectively, where N is the value of collocated_from_l0_flag.

[0083] Moreover, mvLXCol and availableFlagLXCol are derived as follows. - If LongTermRefPic(currPic, currPb, refIdxLX, LX) is not equal to LongTermRefPic(ColPic, colPb, refIdxCol, listCol), then both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0. - Otherwise, the variable availableFlagLXCol is set equal to 1, refPicListCol[refIdxCol] is set to be the picture with reference index refIdxCol in the reference picture list listCol of the slice containing the predictive block colPb in the co-located picture defined by ColPic, and the following applies: colPocDiff=DiffPicOrderCnt(ColPic,refPicListCol[refIdxCol]) (2-1) currPocDiff=DiffPicOrderCnt(currPic,RefPicListX[refIdxLX]) (2-2) - If RefPicListX[refIdxLX] is a long-term reference picture or if colPocDiff is equal to currPocDiff, mvLXCol is derived as follows: mvLXCol=mvCol (2-3) Otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol as follows: tx=(16384+(Abs(td)>>1)) / td (2-4) distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6) (2-5) mvLXCol=Clip3(-32768,32767,Sign(distScaleFactor*mvCol)*((Abs(distScaleFactor*mvCol)+127)>>8)) (2-6) Here, td and tb are derived as follows: td=Clip3(-128,127,colPocDiff) (2-7) tb=Clip3(-128,127,currPocDiff) (2-8)

[0084] The definition of NoBackwardPredFlag is: The variable NoBackwardPredFlag is derived as follows. - For each picture aPic in RefPicList0 or RefPicList1 of the current slice, if DiffPicOrderCnt(aPic,CurrPic) is less than or equal to 0, then NoBackwardPredFlag is set equal to 1. Otherwise, NoBackwardPredFlag is set equal to 0.

[0085] 2.1.2.6 Inserting additional candidates

[0086] Besides the spatial-temporal merge candidates, there are two additional types of merge candidates: joint bi-predictive merge candidates and zero merge candidates. The joint bi-predictive merge candidates are generated using the spatial-temporal merge candidates. The joint bi-predictive merge candidates are used only for B slices. The joint bi-predictive candidate is generated by combining the first reference picture list motion parameters of the first candidate with the second reference picture list motion parameters of another candidate. If these two tuples provide different motion hypotheses, they form a new bi-predictive candidate. As an example, FIG. 7 shows the case where two candidates with mvL0 and refIdxL0, or mvL1 and refIdxL1 in the original list (left side) are used to generate the joint bi-predictive merge candidate that is added to the final list (right side). There are various rules for the combinations considered to generate these additional merge candidates.

[0087] The MaxNumMergeCand capacity is hit by inserting zero motion candidates and filling the remaining entries in the merge candidate list. These candidates have a spatial displacement of zero and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.

[0088] 2.1.3 AMVP

[0089] AMVP exploits the spatial-temporal correlation between motion vectors and neighboring PUs and uses it for explicit transmission of motion parameters. For each reference picture list, we first check the availability of the left and upper temporal neighboring PU positions, and then construct a motion vector candidate list by removing redundant candidates and adding zero vectors to keep the length of the candidate list constant. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the merge index signaling, the index of the best motion vector candidate is coded using a truncated unary. The maximum value to be coded in this case is 2 (see Figure 8). The following sections will explain the details of the motion vector prediction candidate derivation process.

[0090] 2.1.3.1 Derivation of AMVP Candidates

[0091] FIG. 8 summarizes the process of deriving motion vector predictor candidates.

[0092] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. To derive spatial motion vector candidates, as shown in Figure 2, we finally derive two motion vector candidates based on the motion vectors of each PU at five different positions.

[0093] To derive a temporal motion vector candidate, select one motion vector candidate from two candidates derived based on two different co-located positions. After creating an initial list of spatial-temporal candidates, remove duplicate motion vector candidates in the list. If the number of candidates is greater than two, remove from the list the motion vector candidates whose reference picture index in the associated reference picture list is greater than 1. If the number of spatial-temporal motion vector candidates is less than two, add an additional zero motion vector candidate to the list.

[0094] 2.1.3.2 Spatial Motion Vector Candidates

[0095] In deriving spatial motion vector candidates, we consider up to two candidates among five possible candidates derived from PUs located as shown in Fig. 2, which are in the same position as the motion merge. The order of derivation for the left side of the current PU is specified as A0, A1, scaled A0, scaled A1. The order of derivation for the top side of the current PU is specified as B0, B1, B2, scaled B0, scaled B1, scaled B2. So, for each edge, there are four possible cases for motion vector candidates: two cases that do not require spatial scaling and two cases that use spatial scaling. The four different cases can be summarized as follows:

[0096] *No spatial scaling

[0097] (1) The same reference picture list and the same reference picture index (same POC)

[0098] (2) Different reference picture lists and the same reference picture (same POC)

[0099] *Spatial scaling

[0100] (3) Same reference picture list but different reference pictures (different POC)

[0101] (4) Different reference picture lists and different reference pictures (different POCs)

[0102] First check the non-spatial scaling case, then do spatial scaling. Regardless of the reference picture list, if the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, spatial scaling is considered. If all PUs of the left candidate are not available or are intra-coded, scaling of the top motion vector helps in the parallel derivation of the left and top MV candidates. Otherwise, spatial scaling is not allowed for the top motion vector.

[0103] In the spatial scaling process, the motion vectors of neighboring PUs are scaled in a similar manner to the temporal scaling, as shown in Figure 9. The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as that of the temporal scaling.

[0104] 2.1.3.3 Temporal Motion Vector Candidates

[0105] Except for deriving the reference picture index, all the processes for deriving the temporal merge candidates are the same as those for deriving the spatial motion vector candidates (see FIG. 6). The reference picture index is signaled to the decoder.

[0106] 2.2 Inter Prediction Method in VVC

[0107] New coding tools to improve inter prediction include Adaptive Motion Vector difference Resolution (AMVR) for signaling MVD, Merge with Motion Vector Differences (MMVD), Triangular Prediction Mode (TPM), Combined Intra-Inter Prediction (CIIP), Advanced TMVP (ATMVP) (aka SbTMVP), Affine Prediction Mode, Generalized BI-prediction (GBI), Decode-side Motion Vector Refinement (DMVR), and BI-directional Optical flow (BIO) (aka BDOF).

[0108] There are three different merge list configuration processes supported by VVC:

[0109] (1) Sub-block merge candidate list: includes ATMVP and affine merge candidates. One merge list construction process is shared for both affine and ATMVP modes. Note that ATMVP and affine merge candidates may be added in sequence. The size of the sub-block merge list is signaled in the slice header, and its maximum value is 5.

[0110] (2) Regular merge list: For inter-coded blocks, we share one merge list construction process. Here, spatial / temporal merge candidates, HMVP, pairwise merge candidates, and zero motion candidates may be inserted in order. The size of the regular merge list is signaled in the slice header, and its maximum value is 6. MMVD, TPM, and CIIP depend on the regular merge list.

[0111] (3) IBC merge list: This is performed in the same way as a normal merge list.

[0112] Similarly, there are three AMVP lists supported by VVC:

[0113] (1) Affine AMVP Candidate List

[0114] (2) Regular AMVP candidate list

[0115] (3) IBC AMVP candidate list: Same composition process as IBC merge list

[0116] 2.2.1 Coding block structure in VVC

[0117] In VVC, a quadtree / binarytree / ternarytree (QT / BT / TT) structure is adopted to divide a picture into square or rectangular blocks.

[0118] Besides QT / BT / TT, a separate tree (also known as a dual coding tree) is also adopted in VVC for I-frames. Using separate trees, the coding block structure is signaled separately for luma and chroma components.

[0119] In addition, the CU is set equal to the PU and the TU, except for blocks coded with two specific coding methods (e.g., intra sub-partition prediction, where the PU is equal to the TU but is smaller than the CU, and sub-block transformation of inter-coded blocks, where the PU is equal to the CU but the TU is smaller than the PU).

[0120] 2.2.2 Affine Prediction Mode

[0121] In HEVC, only translational motion model is applied to MCP (Motion Compensation Prediction). Meanwhile, in the real world, there are various kinds of motion, such as zoom in / zoom out, rotation, perspective motion, and other irregular motion. In VVC, a 4-parameter affine model and a 6-parameter affine model are used to apply simplified affine transformation motion compensation prediction. As shown in FIG. 10, the affine motion field of a block is represented by two CPMVs (Control Point Motion Vectors) in the case of the 4-parameter affine model, and by three CPMVs in the case of the 6-parameter affine model.

[0122] The MVF (Motion Vector Field) of a block is expressed by the following equation using the four-parameter affine model in equation (1) (where the four parameters are defined as variables a, b, e, and f) and the six-parameter affine model in equation (2) (where the four parameters are defined as variables a, b, c, d, e, and f), respectively.

[0123]

number

[0124]

number

[0125] Here, (mv h 0,mv h 0) is the motion vector of the upper left corner control point, and (mv h 1,mv h 1) is the motion vector of the control point in the upper right corner, (mv h 2,mv h 2) is the motion vector of the control point in the lower left corner. All three motion vectors are called CPMV (Control Point Motion Vector). (x,y) represents the coordinates of the representative point relative to the upper left sample in the current block. (mvh (x,y),mv v (x,y) is the motion vector derived for the sample located at (x,y). The CP motion vector may be signaled (as in affine AMVP mode) or derived on the fly (as in affine merge mode). w and h are the width and height of the current block. In practice, this division is implemented by a right shift with rounding operation. In VTM, the representative point is defined as the center position of a subblock, e.g., if the coordinates of the top-left corner of the subblock relative to the top-left sample in the current block are (xs,ys), the coordinates of the representative point are (xs+2,ys+2). For each subblock (e.g., 4×4 in VTM), the representative point is used to derive the motion vector for the entire subblock.

[0126] To further simplify the motion compensation prediction, subblock-based affine transformation prediction is applied. To derive the motion vectors of each M×N (in current VVC, both M and N are set to 4) subblock, the motion vectors of the center samples of each subblock are calculated according to equations (1) and (2) and rounded to 1 / 16 decimal precision, as shown in Figure 11. Then, a 1 / 16 pixel motion compensation interpolation filter is applied and the derived motion vectors are used to generate the predictions of each subblock. The 1 / 16 pixel interpolation filter is introduced in affine mode.

[0127] After MCP, the high-precision motion vectors of each sub-block are rounded and stored with the same precision as the normal motion vectors.

[0128] 2.2.3 Merging Whole Blocks

[0129] 2.2.3.1 Merge list configuration for normal translational merge mode

[0130] 2.2.3.1.1 HMVP(History-based Motion Vector Prediction)

[0131] Unlike the merge list design, VVC employs the History-based Motion Vector Prediction (HMVP) method.

[0132] HMVP stores the previously coded motion information. Define the motion information of the previously coded block as an HMVP candidate. Store multiple HMVP candidates in a table denoted as HMVP table, which is maintained on-the-fly during the encoding / decoding process. When starting to encode / decode a new tile / LCU row / slice, the HMVP table is emptied. Whenever there are inter-coded blocks and non-sub-blocks in non-TPM mode, add the associated motion information to the last entry of the table as a new HMVP candidate. The overall coding flow is shown in Figure 12.

[0133] 2.2.3.1.2 Normal merge list construction process

[0134] The construction of a typical merge list (for translational movements) can be summarized as the following sequence of steps:

[0135] Step 1: Derive spatial candidates

[0136] Step 2: Inserting HMVP candidates

[0137] Step 3: Inserting pairwise average candidates

[0138] Step 4: Default movement candidates

[0139] HMVP candidates can be used in both AMVP and merge candidate list construction process. Figure 13 shows the modified merge candidate list construction process. If the merge candidate list is not full after inserting the TMVP candidates, the HMVP candidates stored in the HMVP table can be used to fill the merge candidate list. Considering that a block usually has a high correlation with its nearest neighbors in terms of motion information, the HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first, and the first entry is added at the end. Similarly, redundancy elimination is applied to the HMVP candidates. When the total number of available merge candidates reaches the maximum number of mergeable merge candidates that can be signaled, the merge candidate list construction process is terminated.

[0140] Note that all spatial / temporal / HMVP candidates are coded in non-IBC modes, otherwise they are not allowed to be added to the regular merge candidate list.

[0141] The HMVP table contains up to five canonical motion candidates, each unique.

[0142] 2.2.3.1.2.1 Pruning Process

[0143] A candidate is added to the list only if the corresponding candidate used in the redundancy check does not have the same motion information. Such a comparison process is called a pruning process.

[0144] The pruning process among spatial candidates depends on the use of the TPM of the current block.

[0145] If the current block is not coded in a TPM mode (e.g., regular merging, MMVD, CIIP), the HEVC pruning process for spatial merging candidates (e.g., 5 pruning) is utilized.

[0146] 2.2.4 TPM (Triangular Prediction Mode)

[0147] In VVC, triangulation mode is supported for inter prediction. Triangulation mode applies only to CUs that are 8x8 or larger, are coded in merge mode, and are not coded in MMVD or CIIP mode. For CUs that meet these conditions, a CU level flag is signaled to indicate whether triangulation mode applies.

[0148] When using this mode, a CU is divided equally into two triangular partitions using either diagonal or anti-diagonal partitioning, as shown in Figure 14. Each triangular partition in a CU is inter-predicted using its own motion, and only uni-prediction is allowed for each partition; that is, each partition has one motion vector and one reference index. As with conventional bi-prediction, a uni-prediction motion constraint is applied to require only two motion-compensated predictions per CU.

[0149] If the CU level flag indicates that the current CU is coded using triangulation mode, a flag indicating the direction of the triangular partition (diagonal or anti-diagonal) and two merge indices (one for each partition) are further signaled. After predicting each of the triangular partitions, the sample values ​​along the diagonal or anti-diagonal edges are adjusted using a blending process with adaptive weights. This is the prediction signal for the entire CU, and like other prediction modes, the transform and quantization process is performed on the entire CU. Finally, the motion field of the CU predicted using the triangulation mode is stored in 4x4 units.

[0150] The normal merge candidate list is reused for triangulation merge prediction without extra pruning of motion vectors. For each merge candidate in the normal merge candidate list, only one of its L0 or L1 motion vectors is used for triangulation prediction. And the order of selecting L0 vs. L1 motion vectors is based on their merge index parity. According to this scheme, the normal merge list can be used directly.

[0151] 2.2.4.1 TPM Merge List Configuration Processing

[0152] In some embodiments, the normal merge list construction process may include the following modifications.

[0153] (1) How the pruning process is performed depends on the TPM usage of the current block.

[0154] - If the current block is not TPM encoded, HEVC5 pruning applied to spatial merging candidates is invoked.

[0155] - Otherwise (if the current block is TPM coded), full pruning is applied when adding new spatial merge candidates: B1 is compared to A1, B0 is compared to A1 and B1, A0 is compared to A1, B1, and B0, and B2 is compared to A1, B1, A0, and B0.

[0156] (2) The condition for checking the motion information from B2 depends on the TPM usage for the current block.

[0157] - If the current block is not TPM encoded, B2 is accessed and B2 is checked only if there are less than 4 spatial merge candidates before checking B2.

[0158] - Otherwise (if the current block is TPM encoded), B2 is always accessed and checked before adding it, regardless of the number of available spatial merging candidates.

[0159] 2.2.4.2 Adaptive Weighting

[0160] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edge between two triangular prediction units to derive the final prediction for the entire CU. The two weighting factor groups are defined as follows:

[0161] A first group of weighting factors of {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for the luma and chroma samples, respectively.

[0162] A second group of weighting factors of {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for the luma and chroma samples, respectively.

[0163] A weighting factor group is selected based on a comparison of the motion vectors of the two triangular prediction units. The second weighting factor group is used if any one of the following conditions is true:

[0164] The reference pictures of the two triangular prediction units are different from each other.

[0165] The absolute value of the difference between the horizontal values ​​of the two motion vectors is greater than 16 pixels.

[0166] The absolute value of the difference between the vertical values ​​of the two motion vectors is greater than 16 pixels.

[0167] Otherwise, the first weighting factor group is used. An example is shown in FIG.

[0168] 2.2.4.3 Motion Vector Storage

[0169] The motion vectors of the triangular prediction units (Mv and Mv2 in FIG. 16) are stored in a 4×4 grid. For each 4×4 grid, a uni-predictive or bi-predictive motion vector is stored based on the location of the 4×4 grid in the CU. As shown in FIG. 16, for a 4×4 grid located in a non-weighted area (i.e., not located on a diagonal edge), a uni-predictive motion vector of either Mv1 or Mv2 is stored. Meanwhile, for a 4×4 grid located in a weighted area, a bi-predictive motion vector is stored. A bi-predictive motion vector is derived from Mv1 and Mv2 according to the following rules:

[0170] (1) If Mv1 and Mv2 have motion vectors in different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional predictive motion vector.

[0171] (2) If both Mv1 and Mv2 are coming from the same L0 (or L1) direction,

[0172] - If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, Mv2 is scaled to a picture. Mv1 and the scaled Mv2 are combined to form a bi-predictive motion vector.

[0173] - If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, Mv1 is scaled to a picture. The scaled Mv1 and Mv2 are combined to form a bi-predictive motion vector.

[0174] - Otherwise, only Mv1 is stored for the weighting region.

[0175] 2.2.4.4 Syntax Tables, Semantics and Decoding Process for Merge Mode

[0176] Added changes are highlighted in bold italics and underlined. Deletions are marked with [[]].

[0177] 7.3.5.1 General slice segment header syntax

[0178] [Table 2]

[0179] [Table 3]

[0180] 7.3.7.5 Coding Unit Syntax

[0181] [Table 4]

[0182] [Table 5]

[0183] 7.3.7.7 Merge Data Syntax

[0184] [Table 6]

[0185] [Table 7]

[0186] 7.4.6.1 General slice header semantics six_minus_max_num_merge_cand specifies the maximum number of merge Motion Vector Prediction (MVP) candidates supported by a slice, subtracted from 6. The maximum number of merge MVP candidates, MaxNumMergeCand, is derived as follows: MaxNumMergeCand=6-six_minus_max_num_merge_cand (7-57) The value of MaxNumMergeCand is in the range of 1 to 6.

[0187] five_minus_max_num_subblock_merge_cand specifies the maximum number of subblock-based merge Motion Vector Prediction (MVP) candidates supported in a slice, subtracted from 5. If five_minus_max_num_subblock_merge_cand is not present, it is inferred to be equal to 5-sps_sbtmvp_enabled_flag. The maximum number of subblock-based merge MVP candidates, MaxNumSubblockMergeCand, is derived as follows: MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand (7-58) The value of MaxNumSubblockMergeCand is in the range of 1 to 5.

[0188] 7.4.8.5 Coding Unit Syntax pred_mode_flag equal to 0 specifies that the current coding unit is coded in inter prediction mode. pred_mode_flag equal to 1 specifies that the current coding unit is coded in intra prediction mode. If pred_mode_flag is not present, it is inferred as follows: - If cbWidth is equal to 4 and cbHeight is equal to 4, pred_mode_flag is inferred to be equal to 1. - Otherwise, pred_mode_flag is inferred to be equal to 1 when decoding an I slice, and to 0 when decoding a P or B slice, respectively.

[0189] The variables CuPredMode[x][y], for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, are derived as follows: If -pred_mode_flag is equal to 0, CuPredMode[x][y] is set equal to MODE_INTER. Otherwise (if pred_mode_flag is equal to 1), CuPredMode[x][y] is set equal to MODE_INTRA. pred_mode_ibc_flag equal to 1 specifies that the current coding unit is coded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not coded in IBC prediction mode.

[0190] If pred_mode_ibc_flag is not present, it is inferred as follows: - If cu_skip_flag[x0][y0] is equal to 1, and cbWidth is equal to 4, and cbHeight is equal to 4, then pred_mode_ibc_flag is inferred to be equal to 1. Otherwise, if both cbWidth and cbHeight are equal to 128, pred_mode_ibc_flag is inferred to be equal to 0. - Otherwise, pred_mode_ibc_flag is inferred to be equal to the value of sps_ibc_enabled_flag when decoding an I slice, and to 0 when decoding a P or B slice, respectively. If pred_mode_ibc_flag is equal to 1, then the variable CuPredMode[x][y] is set equal to MODE_IBC for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1.

[0191] general_merge_flag[x0][y0] specifies whether to infer inter prediction parameters in the current coding unit from neighboring inter prediction intervals. The array indexes x0,y0 specify the position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If general_merge_flag[x0][y0] is not present, it is inferred as follows: - If cu_skip_flag[x0][y0] is equal to 1, general_merge_flag[x0][y0] is inferred to be equal to 1. - Otherwise, general_merge_flag[x0][y0] is inferred to be equal to 0.

[0192] mvp_l0_flag[x0][y0] specifies the motion vector predictor index for list0, where x0,y0 specifies the position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If mvp_l0_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0193] mvp_l1_flag[x0][y0] has the same semantics as mvp_l0_flag, with l0 and list0 replaced by l1 and list1, respectively. inter_pred_idc[x0][y0] specifies whether list0, list1 or bidirectional prediction is used for the current coding unit according to Table 7-10. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

[0194] [Table 8]

[0195] If inter_pred_idc[x0][y0] is not present, it is inferred to be equal to PRED_L0.

[0196] 7.4.8.7 Merge Data Semantics regular_merge_flag[x0][y0] equal to 1 specifies that the regular merge mode is used to generate inter prediction parameters for the current coding unit. The array indexes x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If regular_merge_flag[x0][y0] is not present, it is inferred as follows: - regular_merge_flag[x0][y0] is inferred to be equal to 1 if all of the following conditions are true: -sps_mmvd_enabled_flag equals 0. -general_merge_flag[x0][y0] equals 1. -cbWidth*cbHeight is equal to 32. - Otherwise, regular_merge_flag[x0][y0] is inferred to be equal to 0.

[0197] mmvd_merge_flag[x0][y0] equal to 1 specifies to use the merge mode with motion vector differential to generate inter prediction parameters for the current coding unit. The array indexes x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

[0198] If mmvd_merge_flag[x0][y0] is not present, it is inferred as follows: - mmvd_merge_flag[x0][y0] is inferred to be equal to 1 if all of the following conditions are true: -sps_mmvd_enabled_flag equals 1. -general_merge_flag[x0][y0] equals 1. -cbWidth*cbHeight is equal to 32. -regular_merge_flag[x0][y0] is equal to 0. - Otherwise, mmvd_merge_flag[x0][y0] is inferred to be equal to 0.

[0199] mmvd_cand_flag[x0][y0] specifies whether to use the first (0) or second (1) candidate in the merge candidate list for the motion vector differential derived from mmvd_distance_idx[x0][y0] and mmvd_direction_idx[x0][y0]. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If mmvd_cand_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0200] mmvd_distance_idx[x0][y0] specifies the indexes used to derive MmvdDistance[x0][y0] as specified in Table 7-12. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

[0201] [Table 9]

[0202] mmvd_distance_idx[x0][y0] specifies the indexes used to derive MmvdDistance[x0][y0] as specified in Table 7-13. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

[0203] [Table 10]

[0204] Both components of the merge+MVD offset MmvdOffset[x0][y0] are derived as follows: MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][0] (7-124) MmvdOffset[x0][y0][1]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][1] (7-125)

[0205] merge_subblock_flag[x0][y0] specifies whether to infer subblock-based inter prediction parameters for the current coding unit from neighboring blocks. The array indexes x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If merge_subblock_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0206] merge_subblock_idx[x0][y0] specifies the merge candidate index in the subblock based merge candidate list, where x0, y0 specifies the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If merge_subblock_idx[x0][y0] is not present, it is inferred to be equal to 0.

[0207] ciip_flag[x0][y0] specifies whether to apply a combination of inter-picture merging and intra-picture prediction for the current coding unit. The array indexes x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If ciip_flag[x0][y0] is not present, it is inferred to be equal to 0. If ciip_flag[x0][y0] is equal to 1, the variable IntraPredModeY[x][y] is set equal to INTRA_PLANAR, where x=xCb..xCb+cbWidth-1, and y=yCb..yCb+cbHeight-1.

[0208] The variable MergeTriangleFlag[x0][y0] specifies whether to use motion compensation based on the triangle shape to generate the prediction samples of the current coding unit when decoding a B slice, and is derived as follows: - If all of the following conditions are true, then MergeTriangleFlag[x0][y0] is set equal to 1: -sps_triangle_enabled_flag equals 1. -slice_type equals B. -general_merge_flag[x0][y0] equals 1. -MaxNumTriangleMergeCand is 2 or greater. -cbWidth*cbHeight is 64 or greater. -regular_merge_flag[x0][y0] is equal to 0. -mmvd_merge_flag[x0][y0] equals 0. -merge_subblock_flag[x0][y0] equals 0. -ciip_flag[x0][y0] is equal to 0. - Otherwise, MergeTriangleFlag[x0][y0] is equal to 0.

[0209] merge_triangle_split_dir[x0][y0] specifies the split direction for the merge triangle mode. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If merge_triangle_split_dir[x0][y0] is not present, it is inferred to be equal to 0.

[0210] merge_triangle_idx0[x0][y0] specifies the first merge candidate index in the triangle-based motion compensation candidate list, where x0, y0 specifies the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If merge_triangle_idx0[x0][y0] is not present, it is inferred to be equal to 0.

[0211] merge_triangle_idx1[x0][y0] specifies the second merge candidate index of the triangle-based motion compensation candidate list, where x0, y0 specifies the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If merge_triangle_idx1[x0][y0] is not present, it is inferred to be equal to 0.

[0212] merge_idx[x0][y0] specifies the merge candidate index in the merge candidate list, where x0, y0 specifies the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If merge_idx[x0][y0] does not exist, it is inferred as follows: - if mmvd_merge_flag[x0][y0] is equal to 1, merge_idx[x0][y0] is inferred to be equal to mmvdmmvd_cand_flag[x0][y0]. - Otherwise (mmvd_merge_flag[x0][y0] is equal to 0), merge_idx[x0][y0] is inferred to be equal to 0.

[0213] 2.2.4.4.1 Decryption Process

[0214] In some embodiments, the decoding process is defined as follows:

[0215] 8.5.2.2 Luminance motion vector derivation process for merge mode This process is only invoked if general_merge_flag[xCb][yCb] is equal to 1, where (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

[0216] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0217] The output of this process is: - Luma motion vectors at 1 / 16 fractional sample precision mvL0[0][0] and mvL1[0][0], - reference indices refIdxL0 and refIdxL1, - prediction list usage flags predFlagL0[0][0] and predFlagL1[0][0], - Bidirectional prediction weight index bcwIdx. - Merge candidate list mergeCandList. The bi-prediction weight index bcwIdx is set equal to zero. The motion vectors mvL0[0][0], mvL1[0], reference indices refIdxL0, refIdxL1, and prediction usage flags predFlagLagL0[0][0], predFlagL1[0][0] are derived in the following sequential steps.

[0218] The process of deriving spatial merge candidates from neighboring coding units, as specified in Section 1.8.5.2.2, is called with the inputs of the position (xCb, yCb) of the luma coding block, the width cbWidth of the luma coding block, and the height cbHeight of the luma coding block, and outputs the availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1, and availableFlagB2, the reference indices refIdxLXA0, refIdxLXA1, refIdxLXA2, refIdxLXA3, refIdxLXA4, refIdxLXA5, refIdxLXA6, refIdxLXA7, refIdxLXA8, refIdxLXA9, refIdxLXA11, refIdxLXA12, refIdxLXA13, refIdxLXA14, refIdxLXA15, refIdxLXA16, refIdxLXA17, refIdxLXA18, refIdxLXA19, refIdxLXA21, refIdxLXA11, refIdxLXA12, refIdxLXA13, refIdxLXA14, refIdxLXA15, refIdxLXA16, refIdxLXA17, refIdxLXA18, refIdxLXA19, refIdxLXA11, refIdxLXA11, refIdxLXA12, refIdxLXA13, refIdxLXA14, refIdxLXA15, refIdxLXA16 dxLXA1, refIdxLXB0, refIdxLXB1 and refIdxLXB2, prediction list usage flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1 and predFlagLXB2, motion vectors mvLXA0, mvLXA1, mvLXB0, mvLXB1 and mvLXB2 (X is 0 or 1), and bidirectional prediction weight indexes bcwIdxA0, bcwIdxA1, bcwIdxB0, bcwIdxB1, bcwIdxB2.

[0219] 2. The reference index refIdxLXCol (X is 0 or 1) and the bidirectional prediction weight index bcwIdxCol of the temporal merge candidate Col are set equal to 0.

[0220] The derivation process of the temporal luma motion vector prediction specified in section 3.8.5.2.11 is called with the luma position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight, and the variable refIdxL0Col as inputs, and the output is the availability flag availabilityFlagL0Col and the temporal motion vector mvL0Col. The variables availableFlagCol, predFlagL0Col, and predFlagL1Col are derived as follows: availableFlagCol=availableFlagL0Col (8-263) predFlagL0Col=availableFlagL0Col (8-264) predFlagL1Col=0 (8-265)

[0221] 4. If slice_type is equal to B, the derivation process of the temporal luma motion vector prediction specified in section 8.5.2.11 is called with the luma position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight, and the variable refIdxL1Col as inputs, and the output is the availability flag availabilityFlagL1Col, and the temporal motion vector mvL1Col. The variables availableFlagCol and predFlagL1Col are derived as follows: availableFlagCol=availableFlagL0Col||availableFlagL1Col (8-266) predFlagL1Col=availableFlagL1Col (8-267)

[0222] 5. The merge candidate list, mergeCandList, is constructed as follows: i=0 if(availableFlagA1) mergeCandList[i++]=A1 if(availableFlagB1) mergeCandList[i++]=B1 if(availableFlagB0) mergeCandList[i++]=B0 if(availableFlagA0) mergeCandList[i++]=A0 if(availableFlagB2) mergeCandList[i++]=B2 if(availableFlagCol) mergeCandList[i++]=Col (8-268)

[0223] 6. The variables numCurrMergeCand and numOrigMergeCand are set equal to the number of merge candidates in mergeCandList.

[0224] 7. If numCurrMergeCand is less than (MaxNumMergeCand-1) and NumHmvpCand is greater than 0, the following applies: - The history-based merge candidate derivation process specified in Section 8.5.2.6 is called with mergeCandList and numCurrMergeCand as inputs and the modified mergeCandList and numCurrMergeCand as outputs. -numOrigMergeCand is set equal to numCurrMergeCand.

[0225] 8. If numCurrMergeCand is less than MaxNumMergeCand and greater than 1, the following applies: The pairwise average merge candidate derivation process specified in section 8.5.2.4 is called with inputs mergeCandList, reference indices refIdxL0N and refIdxL1N, prediction list usage flags predFlagL0N and preDFlagL1N, motion vectors mvL0N and mlvL1N of all candidates N in mergeCandList, and numCurrMergeCand, and the outputs are assigned to mergeCandList, numCurrMergeCand, reference indices refIdxL0avgCand and refIdxL1avgCand, prediction list usage flags predFlagL0avgCand and predFlagL1avgCand, and motion vectors mvL0avgCand and mvL1avgCand of the candidate avgCand added to mergeCandList. The bidirectional prediction weight index bcwIdx of the candidate avgCand added to mergeCandList is set equal to 0. -numOrigMergeCand is set equal to numCurrMergeCand.

[0226] The zero motion vector merge candidate derivation process specified in Section 9.8.5.2.5 is called with mergeCandList, reference indices refIdxL0N and refIdxL1N, prediction list usage flags predFlagL0N and predFlagL1N, motion vectors mvL0N and mvL1N of all candidates N in mergeCandList, and numCurrMergeCand as inputs, and outputs mergeCandList, numCurrMergeCand, reference indices refIdxL0zeroCand, m and refIdxL1zeroCand m , Prediction list usage flag predFlagL0zeroCand m and predFlagL1zeroCand m , and all new candidates zeroCand are added to mergeCandList m Motion vector mvL0zeroCand m and mvL1zeroCand m All new candidates added to mergeCandList are assigned to zeroCand m The bidirectional prediction weight index bcwIdx of is set equal to 0. The number of candidates to be added, numZeroMergeCand, is set equal to (numCurrMergeCand-numOrigMergeCand). If numZeroMergeCand is greater than 0, m ranges from 0 to numZeroMergeCand-1.

[0227] 10. Let the candidate at position merge_idx[xCb][yCb] in the merge candidate list mergeCandList be N (N=mergeCandList[merge_idx[xCb][yCb]), and by replacing X with 0 or 1, the following assignment is made. refIdxLX=refIdxLXN (8-269) predFlagLX[0][0]=predFlagLXN (8-270) mvLX[0][0][0]=mvLXN[0] (8-271) mvLX[0][0][1]=mvLXN[1] (8-272) bcwIdx=bcwIdxN (8-273)

[0228] 11.If mmvd_merge_flag[xCb][yCb] is equal to 1, the following applies: The process of deriving the merge motion vector differential as specified in -8.5.2.7 is invoked with the luminance position (xCb, yCb), reference indices refIdxL0, refIdxL1, and prediction list usage flags predFlagL0[0][0] and predFlagL1[0][0] as input, and the motion vector differentials mMvdL0 and mMvdL1 as output. The motion vector differential mMvdLX is added to the merge motion vector mvLX for X=0 and 1 as follows: mvLX[0][0][0]+=mMvdLX[0] (8-274) mvLX[0][0][1]+=mMvdLX[1] (8-275) mvLX[0][0][0]=Clip3(-2 17 ,2 17 -1,mvLX[0][0][0]) (8-276) mvLX[0][0][1]=Clip3(-2 17 ,2 17 -1,mvLX[0][0][1]) (8-277)

[0229] 2.2.5 MMVD

[0230] In some embodiments, UMVE (Ultimate Motion Vector Expression, also known as MMVD) is provided, which is a motion vector representation method used in either skip mode or merge mode.

[0231] UMVE reuses merge candidates similar to those in the regular merge candidate list in VVC. Among the merge candidates, a base candidate can be selected, which is further extended by the motion vector representation method.

[0232] UMVE provides a new Motion Vector Difference (MVD) representation method, where a starting point, a motion magnitude, and a motion direction are used to represent the MVD.

[0233] In some embodiments, the merge candidate list can be used as is, but for the sake of extending the UMVE, we only consider candidates of the default merge type (MRG_TYPE_DEFAULT_N).

[0234] The base candidate index defines a starting point. The base candidate index indicates the best candidate among the candidates in the list as follows:

[0235] [Table 11]

[0236] If the number of base candidates is equal to 1, the base candidate IDX is not signaled.

[0237] The distance index is information on the magnitude of the movement. The distance index indicates a predefined distance from the starting point information. The predefined distance is as follows:

[0238] [Table 12]

[0239] The direction index represents the direction of the MVD relative to the starting point. The direction index can represent four directions, as shown below.

[0240] [Table 13]

[0241] Immediately after sending the skip or merge flag, the UMVE flag is signaled. If the skip or merge flag is true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. But if it is not 1, the AFFINE flag is parsed. If the AFFINE flag is equal to 1, i.e., AFFINE mode, but not 1, the skip / merge index is parsed for the skip / merge mode of the VTM.

[0242] No additional line buffers due to UMVE candidates are required, because the software skip / merge candidates are used directly as base candidates. The input UMVE index is used to determine the MV completion just before motion compensation. There is no need to maintain long line buffers for this.

[0243] Under the current common test conditions, either the first or second merge candidate in the merge candidate list may be selected as the base candidate.

[0244] UMVE is also known as MMVD (Merge with MV Differences).

[0245] 2.2.6 CIIP(Combined Intra-Inter Prediction)

[0246] In some embodiments, multiple hypothesis prediction using combined intra and inter prediction is proposed as one way of generating multiple hypotheses.

[0247] When applying multiple hypothesis prediction to improve intra mode, multiple hypothesis prediction combines one intra prediction and one merge index prediction. In merge CU, if the flag is true, one flag is signaled for merge mode and one intra mode is selected from the intra candidate list. For luma component, the intra candidate list is derived from only one intra prediction mode, e.g., planar mode. The weight applied to the predicted block from intra prediction and inter prediction is determined by the coding mode (intra or non-intra) of the two neighboring blocks (A1 and B1).

[0248] 2.2.7 Merging for Sub-Block-Based Techniques

[0249] Note that, in addition to the regular merge list of non-subblock merge candidates, it is recommended to put all subblock related motion candidates into a separate merge list.

[0250] Sub-block related motion candidates are put into a separate merge list, called the "sub-block merge candidate list."

[0251] In one example, the sub-block merging candidate list includes ATMVP candidates and affine merging candidates.

[0252] The subblock merging candidate list is filled with candidates in the following order:

[0253] 1. ATMVP candidate (maybe available or not).

[0254] 2. Affine merge list (including inherited affine candidates and constituent affine candidates)

[0255] 3. Padding as a 4-parameter affine model with zero MV

[0256] 2.2.7.1 ATMVP (also known as SbTMVP (Sub-block Temporal Motion Vector Predictor))

[0257] The basic idea of ​​ATMVP is to derive multiple temporal motion vector predictors for one block. Each subblock is assigned one set of motion information. When generating ATMVP merge candidates, we perform motion compensation at the 8x8 level instead of the whole block level.

[0258] In the current design, ATMVP predicts motion vectors for sub-CUs within a CU in two steps, which are described in the following two subsections, respectively.

[0259] 2.2.7.1.1 Deriving the Initial Motion Vector

[0260] Now, let the initialized motion vector be tempMv. If block A1 is available and is non-intra coded (e.g., inter coded or coded in IBC mode), the following applies to derive the initialized motion vector:

[0261] - If all of the following conditions are true, tempMv is set equal to the motion vector of block A1 from list 1, represented by mvL1A1:

[0262] - the reference picture index in list 1 is available (is not -1) and has the same POC value as the co-located picture (e.g. DiffPicOrderCnt(ColPic,RefPicList[1][equals refIdxL1A1),

[0263] - all reference pictures have no greater POC than the current picture (e.g., DiffPicOrderCnt(aPic, currPic) is less than or equal to 0 for all pictures aPic in all reference picture lists of the current slice);

[0264] - current slice is equal to B slice,

[0265] -collocated_from_l0_flag is equal to 0.

[0266] - Otherwise, if all of the following conditions are true, tempMv is set equal to the motion vector of block A1 from list 0, represented by mvL0A1:

[0267] - the reference picture index in list 0 is available (is not equal to -1),

[0268] - has the same POC value as the co-located picture (e.g. DiffPicOrderCnt(ColPic, RefPicList[0][refIdxL0A1]) equals 0).

[0269] - Otherwise, a zero motion vector is used as the initialized MV.

[0270] In the co-located picture signaled in the slice header with the initialized motion vector, the corresponding block (center position of the current block and rounded MV, clipped to a certain range if necessary) is identified.

[0271] If the block is inter-coded, proceed to the second step. Otherwise, set ATMVP candidate equal to unavailable.

[0272] 2.2.7.1.2 Sub-CU motion derivation

[0273] The second step is to divide the current CU into sub-CUs, and obtain the motion information of each sub-CU from its corresponding block in the co-located picture.

[0274] If the corresponding block for the sub-CU is coded in inter mode, the motion information is used to derive the final motion information of the current sub-CU by invoking the derivation process for the co-located MV, which is not different from the case of conventional TMVP processing. Basically, if the corresponding block is predicted from target list X for uni- or bi-prediction, the motion vector is utilized; otherwise, if it is predicted from list Y (Y=1-X) for uni- or bi-prediction, and NoBackwardPredFlag is equal to 1, the MV for list Y is utilized. Otherwise, no motion candidate can be found.

[0275] If the block in the picture at the same location identified by the initialized MV and the position of the current sub-CU is intra-coded or IBC-coded, or if no motion candidate can be found as described above, the following further applies.

[0276] Picture R at the same position col The motion vectors used to extract the motion regions in MV col In order to minimize the effect of MV scaling, MV col The MVs in the spatial candidate list used to derive MV are selected in the following manner: If the reference picture of a candidate MV is a co-located picture, this MV is selected and the MV is derived without scaling. col Otherwise, the MV with the closest reference picture to the co-located picture is selected and scaled to obtain the MV col Derive:

[0277] An exemplary decoding process for the co-located motion vector derivation process is described below.

[0278] 8.5.2.12 Co-located motion vector derivation process

[0279] The inputs to this process are: a variable currCb that defines the current coding block, - A variable colCb that specifies an encoding block at the same position in a picture at the same position specified by ColPic, - a luminance position (xColCb, yColCb) specifying the top left sample of the co-located luminance coding block specified by colCb relative to the top left luminance sample of the co-located picture specified by ColPic, - Reference index refIdxLX, where X is 0 or 1, - Flag indicating sub-block temporal merging candidate sbFlag.

[0280] The output of this process is: - Motion vector prediction mvLXCol with 1 / 16 fractional sample accuracy, -Availability flag availableFlagLXCol. The variable currPic defines the current picture.

[0281] The arrays predFlagL0Col[x], mlvL0Col[x], and refIdxL0Col[x][y] are set equal to PredFlagL0[x][y], MvDmvrL0[x][y], and RefIdxL0[x][y], respectively, of the co-located picture specified by ColPic, and the arrays predFlagL1Col[x][y], mvL1Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL1[x][y], MvDmvrL1[x][y], and RefIdxL1[x][y], respectively, of the co-located picture specified by ColPic.

[0282] The variables mvLXCol and availableFlagLXCol are derived as follows. If colCb is coded in intra or IBC prediction mode, then both components of mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0. - Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows: If -sbFlag is equal to 0, availableFlagLXCol is set equal to 1 and the following applies: -If predFlagL0Col[xColCb][yColCb] is equal to 0, then mvCol, refIdxCol, and listCol are set equal to mvL1Col[xColCb][yColCb], refIdxL1Col[xColCb][yColCb], and L1, respectively. - Otherwise, if predFlagL0Col[xColCb][yColCb] is equal to 1 and predFlagL1Col[xColCb][yColCb][yColCb] is equal to 0, then mvCol, refxIdCol, and listCol are set equal to mvL0Col[xColCb][yColCb], refIdxL0Col[xColCb][yColCb], and L0, respectively. - Otherwise (predFlagL0Col[xColCb][yColCb] is equal to 1 and predFlagL1Col[xColCb][yColCb] is equal to 1), make the following assignments: - If NoBackwardPredFlag is equal to 1, then mvCol, refIdxCol, and listCol are set equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb], and LX, respectively. - Otherwise, mvCol, refIdxCol, and listCol are set equal to mvLNCol[xColCb][yColCb], refIdxLNCol[xColCb][yColCb], and LN, respectively, where N is the value of collocated_from_l0_flag.

[0283] - Otherwise (sbFlag is equal to 1), the following applies: - If PredFlagLXCol[xColCb][yColCb] is equal to 1, then mvCol, refxIdCol, and listCol are set equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb], and LX, respectively, and availableFlagLXCol is set equal to 1. - Otherwise (PredFlagLXCol[xColCb][yColCb] is equal to 0), the following applies: -If DiffPicOrderCnt(aPic, currPic) is less than or equal to 0 in all reference picture lists aPic of the current slice and PredFlagLYCol[xColCb][yColCb] is equal to 1, then mvCol, refxIdCol, and listCol are set equal to mvLYCol[xColCb][yColCb], refIdxLYCol[xColCb][yColCb], and LY, respectively, where Y is !X, where X is the value of X for which this process is called, and availableFlagLXCol is set equal to 1. - Both components of mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0.

[0284] If -availableFlagLXCol is TRUE, then mvLXCol and availableFlagLXCol are derived as follows: - If LongTermRefPic(currPic, currCb, refIdxLX, LX) is not equal to LongTermRefPic(ColPic, colCb, refIdxCol, listCol), then both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0. - Otherwise, the variable availableFlagLXCol is set equal to 1 and refPicList[listCol][refIdxCol] is set to the picture with the reference index refIdxCol in the reference picture list listCol of the slice containing the coding block colb in the co-located picture specified in ColPic, and the following applies: colPocDiff=DiffPicOrderCnt(ColPic,refPicList[listCol][refIdxCol]) (8-402) currPocDiff=DiffPicOrderCnt(currPic,RefPicList[X][refIdxLX]) (8-403) - The co-located motion vector temporal motion buffer compression process as specified in subclause 8.5.2.15 is called with mvCol as input and the modified mvCol as output. - If RefPicList[X][refIdxLX] is a long-term reference picture or if colPocDiff is currPocDiff, then mvLXCol is derived as follows: mvLXCol=mvCol (8-404) Otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol as follows: tx=(16384+(Abs(td)>>1)) / td (8-405) distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6) (8-406) mvLXCol=Clip3(-131072,131071,(distScaleFactor*mvCol+128-(distScaleFactor*mvCol>=0))>>8)) (8-407) Here, td and tb are derived as follows: td=Clip3(-128,127,colPocDiff) (8-408) tb=Clip3(-128,127,currPocDiff) (8-409)

[0285] 2.2.8 Improvement of movement information

[0286] 2.2.8.1 DMVR(Decoder-side Motion Vector Refinement)

[0287] In the bidirectional prediction operation, to predict one block region, the bidirectional prediction blocks constructed using the MVs (Motion Vectors) in list 0 and MVs in list 1 are combined to form one prediction signal. In the Decoder-side Motion Vector Refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined.

[0288] For DMVR in VVC, as shown in FIG. 19, we assume mirroring between list 0 and list 1, and improve the MV by performing bilateral matching. For example, find the best MVD among several MVD candidates. The MVs of two reference picture lists are represented by MVL0(L0X, L0Y) and MVL1(L1X, L1Y). We define the MVD represented by (MvdX, MvdY) for list 0 that can minimize a cost function (e.g., SAD) as the best MVD. The SAD function is defined as the SAD between the reference block of list 0 derived by the motion vector (L0X+MvdX, L0Y+MvdY) in the reference picture of list 0 and the reference block of list 1 derived by the motion vector (L1X-MvdX, L1Y-MvdY) in the reference picture of list 1.

[0289] The motion vector refinement process may be iterated twice. In each iteration, up to six MVDs (integer pixel accuracy) may be checked in two steps, as shown in FIG. 20. In the first step, MVDs (0,0), (-1,0), (1,0), (0,-1), (0,1) are checked. In the second step, one of MVDs (-1,-1), (-1,1), (1,-1), or (1,-1) may be selected and checked further. Let function Sad(x,y) return the SAD value of MVD(x,y). The MVD, denoted by (MvdX,MvdY) checked in the second step is determined as follows: MvdX=-1; MvdY=-1; If(Sad(1,0) <Sad(-1,0)) MvdX=1; If(Sad(0,1) <Sad(0,-1)) MvdY=1;

[0290] In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the best MVD selected in the first iteration. DMVR is applied only if one reference picture is a previous picture and the other reference picture is a later picture, and the two reference pictures have the same picture order count distance from the current picture.

[0291] To further simplify the processing of the DMVR, in some embodiments, the adopted DMVR design has the following main features:

[0292] - If the (0,0) position SAD between list0 and list1 is less than a threshold, then terminate early.

[0293] - If at some position the SAD between list0 and list1 is 0, then terminate early.

[0294] -DMVR: Block size W*H>=64&H>=8, where W and H are the width and height of the block.

[0295] - For DMVR where the size of the CU is larger than 16*16, divide the CU into multiple sub-blocks of 16x16. If only the width or height of the CU is larger than 16, the CU is only divided vertically or horizontally.

[0296] - Reference block size (W+7)*(H+7) (for luminance).

[0297] -25 point SAD based integer pixel search (e.g. (+-)2 improvement search range, single stage)

[0298] -Bilinear Interpolation Based DMVR

[0299] - Sub-pixel refinement based on the "parameter error surface equation". This procedure is performed only if, in the previous MV refinement iteration, the minimum SAD cost is not equal to zero and the best MVD is (0,0).

[0300] - Luma / chroma MC with reference block padding (if necessary).

[0301] - Improved MV used only for MC and TMVP.

[0302] 2.2.8.1.1 Using DMVR

[0303] The DMVR may be enabled if all of the following conditions are true:

[0304] - The DMVR enabled flag in the SPS (e.g., sp_dmvr_enabled_flag) is equal to 1.

[0305] - The TPM flag, the inter-affine flag, the sub-block merge flag (ATMVP or affine merge), and the MMVD flag are all equal to 0.

[0306] - Merge flag equals 1.

[0307] The current block is bidirectionally predicted and the POC distance between the current picture and the reference picture in list 1 is equal to the POC distance between the reference picture in list 0 and the current picture.

[0308] - The height of the current CU is 8 or more.

[0309] -The number of luminance samples (CU width * height) is 64 or more.

[0310] 2.2.8.1.2 Sub-pixel refinement based on the "Parametric Error Surface Equation"

[0311] The method is summarized below.

[0312] 1. Compute the parametric error surface fit only if the center location is the best cost location at a given iteration.

[0313] 2. Using the central location cost and the costs at locations (-1,0), (0,-1), (1,0), and (0,1) from the central location, fit the following 2D parabolic error surface equation: E(x,y)=A(x-x0) 2 +B(y-y0) 2 +C

[0314] where (x0,y0) corresponds to the position with the smallest cost and C corresponds to the smallest cost value. By solving five equations in five unknowns, (x0,y0) is calculated as follows: x0=(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0))) y0=(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0)))

[0315] (x0,y0) can be calculated to any required sub-pixel precision by adjusting the precision with which the division is done (e.g. how many bits of the quotient are calculated). To get 1 / 16 pixel precision, the absolute value of the quotient only needs to be calculated with 4 bits, which is suitable for a fast shift-and-subtract based implementation requiring two divisions per CU.

[0316] 3. Add the calculated (x0, y0) to the integer refinement MV to obtain the sub-pixel accurate refinement differential MV.

[0317] 2.3 Intra-block copy

[0318] Intra Block Copy (IBC), also known as current picture reference, is adopted in HEVC Screen Content Coding extensions (HEVC-SCC) and the current VVC Test Model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in FIG. 21, the current block is predicted by a reference block in the same picture when IBC is applied. Before coding or decoding the current block, samples in the reference block must already be reconstructed. IBC is not very efficient for most camera-captured sequences, but for screen content, it shows significant coding gain. The reason is that there are many repeating patterns, such as icons, characters, etc., in screen content pictures. IBC can effectively remove redundancy between these repeating patterns. In HEVC-SCC, an inter-coded coding unit (CU) can apply IBC if it selects the current picture as its reference picture. In this case, MV is renamed as block vector (BV), and BV always has integer pixel accuracy. To comply with the Main Profile HEVC, the current picture is marked as a "long-term" reference picture in the Decoded Picture Buffer (DPB), just as in multiple view / 3D video coding standards, inter-view reference pictures are also marked as "long-term" reference pictures.

[0319] After the BV finds its reference block, it can generate a prediction by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Then, transforms and quantization can be applied as in other coding modes.

[0320] However, if the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed region, or is outside the valid region limited by some constraint, some or all of the pixel values ​​are undefined. Basically, there are two solutions to deal with such problems. One is to not allow such situations, e.g. bitstream conformance. The other is to apply padding to these undefined pixel values. The following sub-sections will explain the solutions in detail.

[0321] 2.3.1 IBC in the VVC test model (VTM4.0)

[0322] In the current VVC test model, e.g., VTM-4.0 design, the entire reference block should have the current Coding Tree Unit (CTU) and does not overlap with the current block. Therefore, there is no need to pad the reference or prediction block. The IBC flag is coded as the prediction mode of the current CU. Thus, there are a total of three prediction modes for each CU: MODE_INTRA, MODE_INTER, and MODE_IBC.

[0323] 2.3.1.1 IBC Merge Mode

[0324] In IBC merge mode, indices pointing to entries in the IBC merge candidate list are parsed from the bitstream. The construction of the IBC merge list can be summarized according to the following sequence of steps:

[0325] Step 1: Derive spatial candidates

[0326] Step 2: Inserting HMVP candidates

[0327] Step 3: Inserting pairwise average candidates

[0328] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at positions A1, B1, B0, A0 and B2 shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any PU at positions A1, B1, B0, A0 is not available (e.g., because it belongs to another slice or tile) or is not coded in IBC mode. After adding the candidate at position A1, the insertion of the remaining candidates is subjected to a redundancy check, which can ensure that candidates with the same motion information are removed from the list, improving coding efficiency.

[0329] If after inserting the spatial candidates, the IBC merge list size is still less than the maximum IBC merge list size, then an IBC candidate from the HMVP table can be inserted. Upon inserting an HMVP candidate, a redundancy check is performed.

[0330] Finally, we insert the pairwise average candidates into the IBC merge list.

[0331] If the reference block identified by a merge candidate is outside the picture, or if it overlaps with the current block, or if it is outside the reconstructed region, or if it is outside the valid region limited by some constraint, the merge candidate is called an invalid merge candidate.

[0332] Note that invalid merge candidates may be inserted into the IBC merge list.

[0333] 2.3.1.2 IBC AMVP mode

[0334] In IBC AMVP mode, an AMVP index that points to an entry in an IBC AMVP list is parsed from the bitstream. The construction of an IBC AMVP list can be summarized according to the following sequence of steps:

[0335] Step 1: Derive spatial candidates

[0336] -A until an available candidate is found 0、 Check A1.

[0337] -Check B0, B1, B2 until an available candidate is found.

[0338] Step 2: Inserting HMVP candidates

[0339] Step 3: Inserting zero candidates

[0340] If after inserting the spatial candidates, the IBC AMVP list size is still less than the maximum IBC AMVP list size, then IBC candidates from the HMVP table may be inserted.

[0341] Finally, the zero candidate is inserted into the IBC AMVP list.

[0342] 2.3.1.3 Saturation IBC Mode

[0343] In the current VVC, motion compensation in chroma IBC mode is done at the sub-block level. A chroma block is divided into multiple sub-blocks. Each sub-block checks whether the corresponding luma block has a block vector, and if so, checks its validity. The current VTM has an encoder constraint to test chroma IBC mode whether all sub-blocks in the current chroma CU have valid luma block vectors. For example, in a YUV420 video, the chroma block is NxM, and the co-located luma region is 2Nx2M. The sub-block size of the chroma block is 2x2. There are several steps to perform the chroma mv derivation and then the block copy process.

[0344] (1) The chroma block is first divided into (N>>1)*(M>>1) sub-blocks.

[0345] (2) For each sub-block whose top-left sample is located at (x,y), fetch the corresponding luma block that contains the same top-left sample located at (2x,2y).

[0346] (3) The encoder checks the block vector (bv) of the fetched luma block. If it meets one of the following conditions, bv is considered invalid:

[0347] a. There is no corresponding luminance block bv.

[0348] The predicted block identified by b.bv has not yet been reconstructed.

[0349] The predicted block identified in c.bv overlaps partially or completely with the current block.

[0350] (4) The chrominance motion vector of a subblock is set to the motion vector of the corresponding luma subblock.

[0351] If all sub-blocks find a valid bv, then IBC mode is allowed in the encoder.

[0352] 2.3.2 Recent progress in IBC

[0353] 2.3.2.1 Single BV List

[0354] In some embodiments, the BV predictors for merge mode and AMVP mode in IBC share a common predictor list consisting of the following elements:

[0355] (1) Two spatially adjacent locations (A1, B1, like P2)

[0356] (2) Five HMVP entries

[0357] (3) Zero vector by default

[0358] The number of candidates in the list is controlled by a variable derived from the slice header. In merge mode, up to the first six entries of this list can be used, in AMVP mode, the first two entries of this list can be used, and the list complies with the requirements for a shared merge list area (sharing the same list within an SMR).

[0359] In addition to the above BV predictor candidate list, the pruning operation between the HMVP candidate and the existing merge candidates (A1, B1) can be simplified by only comparing the first HMVP candidate with the spatial merge candidates, so that the pruning operation is limited to a maximum of two times.

[0360] 2.3.2.2 Size Restrictions for IBCs

[0361] In some embodiments, in previous VTM and VVC versions, in addition to the current bitstream constraints, a syntax constraint to disable IBC mode at 128x128 can be explicitly used, making the presence of the IBC flag dependent on CU size < 128x128.

[0362] 2.3.2.3 Shared merge list for IBC

[0363] To reduce decoder complexity and support parallel coding, in some embodiments, the same merge candidate list can be shared for all leaf coding units (CUs) of one ancestor node in a CU partition tree to enable parallel processing of small skip / merge coded CUs. The ancestor node is called a merge-share node. A shared merge candidate list is generated in the merge-share node, making it appear as if the merge-share node is a leaf CU.

[0364] Specifically, the following may apply:

[0365] - A shared merge list is used between very small blocks (e.g., two adjacent 4x4 blocks) if the block has 32 or fewer luma samples and is split into two 4x4 child blocks.

[0366] -However, if the luma samples of a block are larger than 32, then after the splitting, at least one child block will be smaller than the threshold (32) and all child blocks of this split will share the same merge list (e.g. 16x4 or 4x16 splits in 3, or 8x8 in 4).

[0367] Such restrictions apply only to the IBC merge mode.

[0368] 3. Problems to be Solved by the Embodiments

[0369] A block may be coded in IBC mode. However, different sub-regions within the block may have different contents. We need to consider how to further check the correlation with the previously coded block in the current frame.

[0370] 4. Example of embodiment

[0371] In this specification, IBC (Intra Block Copy) is not limited to current IBC techniques, but may be interpreted as a technique that uses reference samples within the current slice / tile / brick / picture / other video unit (e.g., a CTU row), excluding conventional intra prediction methods.

[0372] To solve the above-mentioned problems, a sub-block-based IBC (sbIBC) coding method is proposed. In sbIBC, a current IBC coded video block (e.g., CU / PU / CB / PB) is divided into multiple sub-blocks. Each sub-block may have a size smaller than the size of the video block. For each sub-block from the multiple sub-blocks, the video coder may identify a reference block for the respective sub-block in the current picture / slice / tile / brick / tile group. The video coder may use motion parameters of the identified reference block for each sub-block to determine motion parameters for each sub-block.

[0373] Also, IBC is not limited to being applied only to uni-predictively coded blocks. Bi-directional prediction may be supported with both of the two reference pictures being the current picture. Alternatively, bi-directional prediction with one from the current picture and the other from a different picture may be supported. In yet another example, multiple hypotheses may be applied.

[0374] The following should be considered as examples to illustrate the general concept. These techniques should not be construed in a narrow sense. Moreover, these techniques can be combined in any manner. Neighborhood blocks A0, A1, B0, B1, B2 are shown in FIG. 2.

[0375] 1. In sbIBC, a block of size equal to M×N may be divided into two or more sub-blocks. In one example, the size of the sub-blocks is fixed at L×K, for example, L=K=4. b. In one example, the size of the sub-blocks is fixed to the smallest coding unit / prediction unit / transform unit / unit for motion information storage. c. In one example, a block may be divided into multiple sub-blocks of different or equal size. d. In one example, an indication of sub-block size may be signaled. e. In one example, the indication of the size of the sub-blocks may vary from block to block, for example according to the block dimensions. f. In one example, the size of the sub-blocks should be in the form of (N1×minW)×(N2×minH), where minW×minH represents the minimum coding unit / prediction unit / transform unit / unit for motion information storage, and N1 and N2 are positive integers. g. In one example, the sub-block dimensions may depend on the color format and / or color components. For example, the sub-block sizes of different color components may be different. 1) Alternatively, the sub-block sizes of different color components may be the same. ii. For example, if the color format is 4:2:0, a 2L×2K sub-block of the luma component may correspond to an L×K sub-block of the chroma component. 1) Alternatively, if the color format is 4:2:0, four 2L×2K sub-blocks of the luma component may correspond to a 2L×2K sub-block of the chroma component. iii. For example, if the color format is 4:2:2, a 2L×2K sub-block of the luma component may correspond to a 2L×K sub-block of the chroma component. 1) Alternatively, if the color format is 4:2:2, two 2L×2K sub-blocks of the luma component may correspond to a 2L×2K sub-block of the chroma component. iv. For example, if the color format is 4:4:4, a 2L×2K sub-block of the luma component may correspond to a 2L×2K sub-block of the chroma component. h. In one example, the MV of a sub-block of a first color component may be derived from a corresponding sub-block or multiple corresponding sub-blocks of a second color component. i. For example, the MV of a sub-block of a first color component may be derived as the average MV of multiple corresponding sub-blocks of a second color component. ii. Additionally, alternatively, if a single tree is utilized, the above method may be applied. iii. Additionally, alternatively, the above method may also be applied to a specific block size, for example a 4x4 chroma block. i. In one example, the size of the sub-blocks may depend on the coding mode, such as IBC merge / AMVP mode. j. In one example, the sub-blocks may be non-rectangular, such as triangular / wedge shaped.

[0376] 2. Obtain motion information of a sub-CU, at least one of whose reference pictures is the current picture, using two stages including identifying a corresponding reference block using an initialized motion vector (denoted by initMV) and deriving one or more motion vectors for the sub-CU according to the reference block. In one example, the reference block may be within the current picture. b. In one example, the reference block may be within a reference picture. i. For example, it may be in a co-located reference picture. ii. For example, it may be in a reference picture that is identified using motion information of the co-located block or a block in a neighborhood of the co-located block.

[0377] Stage 1.a: Setting initMV(vx,vy) c. In one example, initMV may be derived from one or more neighboring blocks (adjacent or non-adjacent) of the current block or the current sub-block. i. The neighboring block may be one in the same picture. 1) Alternatively, the neighboring block may be one in the reference picture. For example, the neighboring blocks may be in the same located reference picture. b. For example, neighboring blocks may be identified using motion information of the co-located block or blocks neighboring the co-located block. ii. In one example, initMV may be derived from the neighboring blocks Z. 1) For example, initMV may be set equal to the MV stored in neighboring block Z, which may be block A1, for example. iii. In one example, initMV may be derived from multiple blocks checked in order. 1) In one example, the first identified motion vector associated with the current picture as a reference picture from the checked block may be set to initMV.

[0378] d. In one example, initMV may be derived from a motion candidate list. i. In one example, initMV may be derived from the kth (eg, first) candidate in the IBC candidate list. 1) In one example, the IBC candidate list is a merged / AMVP candidate list. 2) In one example, a different IBC candidate list may be used than existing IBC merge candidate list construction processes, such as using different spatially neighboring blocks. ii. In one example, initMV may be derived from the kth (eg, first candidate) in the IBC HMVP table. e. In one example, it may be derived based on the position of the current block. f. In one example, it may be derived based on the dimensions of the current block. g. In one example, it may be set to a default value. h. In one example, the initMV indication may be signaled at a video unit level, such as tile / slice / picture / brick / CTU row / CTU / CTB / CU / PU / TU, etc. i. The initial MV may be different for two different sub-blocks within the current block. j. How the initial MV is derived may vary on a block-by-block, tile-by-tile, slice-by-slice basis.

[0379] Stage 1.b on identification of corresponding reference blocks of sub-CUs using initMV In one example, initMV may first be converted to one pixel integer precision, and the converted MV may be used to identify the corresponding block of the subblock. Let the converted MV be (vx', vy'). i. In one example, if (vx, vy) is F pixel-to-pixel precision, the transformed MV denoted by (vx', vy') may be set to (vx*F, vy*F) (e.g., F=2 or 4). ii. Alternatively, (vx',vy') is set directly equal to (vx,vy). l. Let the top-left position of one sub-block be (x,y) and the size of the sub-block be K×L. The corresponding block of the sub-block is set equal to the CU / CB / PU / PB that covers the coordinate (x+offsetX+vx',y+offsetY+vy'), and offsetX and offsetY are used to indicate the selected coordinate for the current sub-block. i. In one example, offsetX and / or offsetY are set equal to 0. ii. In one example, offsetX may be set equal to (L / 2) or (L / 2+1) or (L / 2-1), where L may be the width of the sub-block. iii. In one example, offsetY may be set equal to (K / 2) or (K / 2+1) or (K / 2-1), where K may be the height of the sub-block. iv. Alternatively, the horizontal and / or vertical offset may be further clipped to the extents of the picture / slice / tile / brick boundary / IBC reference region etc.

[0380] Stage 2 involves deriving a motion vector (denoted subMV(subMVx, subMVy)) of a sub-block using the motion information of the identified corresponding reference block. The sub-MVs of m sub-blocks are derived from the motion information of the corresponding blocks. i. In one example, if the corresponding block has a motion vector that points to the current picture, then subMV is set equal to MV. ii. In one example, if the corresponding block has a motion vector that points to the current picture, then subMV is set equal to MV plus initMV. n. The derived subMV may be further clipped to a given range and / or to ensure that it points to the IBC reference region. o. In a conforming bitstream, the derived subMV must be a valid MV of the IBC for the sub-block.

[0381] 3. One or more IBC candidates with sub-block motion vectors may be generated, which may be denoted as sub-block IBC candidates.

[0382] 4. Sub-block IBC candidates may be inserted into sub-block merging candidates, including ATMVP, affine merging candidates. a. In one example, a sub-block IBC candidate may be added before all other sub-block merging candidates. b. In one example, a sub-block IBC candidate may be added after the ATMVP candidate. c. In one example, the sub-block IBC candidates may be added after the inherited or constructed affine candidates. d. In one example, the sub-block IBC candidate may be added to the IBC Merge / AMVP candidate list. Alternatively, whether to add may be based on the mode information of the current block, e.g., if it is in IBC AMVP mode, it may not be added. e. Which candidate list to populate may be based on the partition structure, for example dual tree or single tree. f. Alternatively, multiple sub-block IBC candidates may be inserted into the sub-block merging candidate.

[0383] 5. An IBC sub-block motion (eg, AMVP / merge) candidate list may consist of at least one sub-block IBC candidate. Alternatively, for example, using different initialized MVs, one or more sub-block IBC candidates may be inserted into the IBC sub-block merging candidates. b. Additionally, alternatively, whether to construct an IBC sub-block motion candidate list or an existing IBC AMVP / merge candidate list may be signaled by an indicator or derived on the fly. c. Additionally, alternatively, if the current block is coded in IBC merge mode, an index into the IBC sub-block merging candidate list may be signaled. d. Additionally, alternatively, if the current block is coded in IBC AMVP mode, an index into the IBC sub-block AMVP candidate list may be signaled. i. Additionally, alternatively, the signaled / derived MVD for IBC AMVP mode may be applied to one or more sub-blocks.

[0384] 6. The reference block of a sub-block and the sub-block may belong to the same color component.

[0385] Expanding sbIBC by intermixing other tools that apply to different subblocks within the same block

[0386] 7. A block may be divided into multiple sub-blocks, at least one of which is coded with IBC and at least one of which is coded in intra mode. In one example, for a sub-block, a motion vector may not be derived. Instead, one or more intra-prediction modes may be derived for the sub-block. b. Alternatively, a palette mode and / or palette table may be derived. c. In one example, one intra-prediction mode may be derived for the entire block.

[0387] 8. A block may be divided into multiple sub-blocks with all the sub-blocks coded in intra mode.

[0388] 9. A block may be divided into multiple sub-blocks with all the sub-blocks coded in palette mode.

[0389] 10. A block may be divided into multiple sub-blocks, with at least one sub-block coded in IBC mode and at least one sub-block coded in palette mode.

[0390] 11. A block may be divided into multiple sub-blocks, with at least one sub-block coded in intra mode and at least one sub-block coded in palette mode.

[0391] 12. A block may be divided into multiple sub-blocks, with at least one sub-block coded in IBC mode and at least one sub-block coded in inter mode.

[0392] 13. A block may be divided into multiple sub-blocks, with at least one sub-block coded in intra mode and at least one sub-block coded in inter mode.

[0393] Interaction with other tools

[0394] 14. When applying one or more of the above methods, the IBC HMVP table does not need to be updated. a. Alternatively, one or more of the motion vectors for the IBC coded sub-regions may be used to update the IBC HMVP table.

[0395] 15. When applying one or more of the above methods, the non-IBC HMVP table does not need to be updated. b. Alternatively, one or more of the motion vectors for the inter-coded sub-regions may be used to update the non-IBC HMVP table.

[0396] 16. In-loop filtering processes (eg, deblocking procedures) may rely on the use of the above methods. In one example, when one or more of the above methods are applied, sub-block boundaries may be filtered. a. Alternatively, when one or more of the above methods are applied, the sub-block boundaries may be filtered. b. In one example, blocks encoded in the above manner may be treated the same as conventional IBC encoded blocks.

[0397] 17. For blocks coded with one or more of the methods described above, certain coding methods (e.g., sub-block transform, affine motion prediction, intra prediction of multiple reference lines, matrix-based intra prediction, symmetric MVD coding, merging with MVD decoder-side motion derivation / refinement, bidirectional optimal flow, reduced secondary transform, multiple transform sets, etc.) may be disabled.

[0398] 18. An indication of the use of the above-mentioned methods and / or sub-block sizes may be signaled at the sequence / picture / slice / tile group / tile / brick / CTU / CTB / CU / PU / TU / other video unit level or may be derived on the fly. In one example, one or more of the above methods may be treated as special IBC modes. i. Additionally, alternatively, if one block is coded as IBC mode, a further indication of using a conventional full-block based IBC method or sbIBC may be signaled or derived. ii. In one example, a subsequent IBC coded block may use the motion information of the current sbIBC coded block as an MV predictor. 1. Alternatively, subsequent IBC coded blocks may not be allowed to utilize the motion information of the current sbIBC coded block as an MV predictor. b. In one example, sbIBC may be indicated by a candidate index into a motion candidate list. i. In one example, a specific candidate index is assigned to an sbIBC coded block. c. In one example, IBC candidates may be classified into two categories, one for whole block coding and one for sub-block coding. Whether a block is coded in sbIBC mode may depend on the category of the IBC candidate.

[0399] Using the tool

[0400] 19. Whether and / or how to apply the above methods may depend on the following information: a. Messages signaled in DPS / SPS / VPS / PPS / APS / picture header / slice header / tile group header / LCU (Largest Coding Unit) / CU (Coding Unit) / LCU row / LCU group / TU / PU block / video coding unit b. CU / PU / TU / block / video coding unit location c. Block dimensions of the current block and / or its neighboring blocks d. Block shapes of the current block and / or its neighboring blocks e. Intra mode of the current block and / or its neighboring blocks f. Neighboring block movement / block vector g. Color format indication (e.g. 4:2:0, 4:4:4) h. Coding tree structure i. Slice / tile group type and / or picture type j. Color components (e.g., may be applied only to chroma or luma components) k. Time layer ID l. Profile / Level / Tier of Standard

[0401] Merge list construction process and IBC usage concepts

[0402] 20. IBC modes may be used in conjunction with inter prediction modes for blocks in inter-coded pictures / slices / tile groups / tiles. a. In one example, for IBC AMVP mode, a syntax element may be signaled to indicate whether the current block is predicted from both the current picture and a reference picture that is not identical to the current picture (denoted as a temporal reference picture). i. Additionally, alternatively, if the current block is also predicted from a temporal reference picture, syntax elements may be signaled to indicate which temporal reference picture is used and its associated MVP index, MVD, MV precision, etc. ii. In one example, for IBC AMVP mode, one reference picture list may contain only the current picture, and the other reference picture list may contain only temporal reference pictures. b. In one example, for IBC merge mode, motion vectors and reference pictures may be derived from neighboring blocks. i. For example, if a neighboring block is predicted only from the current picture, then the motion information derived from the neighboring block may only refer to the current picture. ii. For example, if a neighboring block is predicted from both the current picture and a temporal reference picture, the derived motion information may refer to both the current picture and the temporal reference picture. 1) Alternatively, the derived motion information may only refer to the current picture. iii. For example, if a neighboring block is predicted only from temporal reference pictures, it may be considered as "invalid" or "unavailable" when constructing IBC merging candidates. c. In one example, fixed weighting factors may be assigned to reference blocks from the current picture and reference blocks from the temporal reference picture for bi-directional prediction. i. Additionally or alternatively, the weighting factors may be signaled.

[0403] 21. Whether / how to update the motion candidate list construction process (e.g., normal merge list, IBC merge / AMVP list, sub-block merge list, IBC sub-block candidate list) and / or HMVP table may depend on the block dimensions and / or the shared condition of merging. Let W and H be the width and height of a block, respectively. Condition C may depend on the block dimensions and / or the coded information. a. How to update the motion candidate list construction process (eg, normal merge list, IBC merge / AMVP list, sub-block merge list, IBC sub-block candidate list) and / or the HMVP table may depend on condition C. b. In one example, condition C may depend on the coded information of the current block and / or its neighboring (adjacent or non-adjacent) blocks. c. In one example, condition C may depend on a merge share condition. d. In one example, condition C may depend on the block size of the current block and / or the block size of neighboring (adjacent or non-adjacent) blocks and / or the coding mode of the current block and / or the neighboring blocks. e. In one example, if condition C is satisfied, the derivation of spatial merging candidates is skipped. f. In one example, if condition C is met, the derivation of candidates from spatially nearby (adjacent or non-adjacent) blocks is skipped.

[0404] g. In one example, if condition C is met, derivation of candidates from a particular spatially nearby (adjacent or non-adjacent) block (eg, block B2) is skipped. h. In one example, if condition C is satisfied, the derivation of HMVP candidates is skipped. i. In one example, if condition C is satisfied, then the derivation of pairwise merging candidates is skipped. j. In one example, if condition C is met, the number of max pruning operations is reduced or set to 0. i. Additionally, alternatively, pruning operations between spatial merging candidates may be reduced or eliminated. ii. Additionally, alternatively, pruning operations between HMVP candidates and other merging candidates may be reduced or eliminated. k. In one example, if condition C is met, the update of the HMVP candidate is skipped. i. In one example, HMVP candidates may be added directly to the motion list without being pruned. l. In one example, if condition C is met, then the default motion candidate (eg, zero motion candidate in the IBC merge / AVMP list) is not added. m. In one example, if condition C is met, a different check order (e.g., from first to last instead of last to first) and / or a different number of HMVP candidates to be checked / added.

[0405] In one example, condition C may be met if W*H is greater than or equal to a threshold value (eg, 1024). o. In one example, condition C may be met if W and / or H is greater than or equal to a threshold value (eg, 32). p. In one example, condition C may be met if W is greater than or equal to a threshold value (eg, 32). q. In one example, condition C may be met if H is greater than or equal to a threshold value (eg, 32). r. In one example, condition C may be met if W*H is greater than or equal to a threshold (eg, 1024) and the current block is encoded in IBC AMVP and / or merge mode.

[0406] s. In one example, condition C may be met if W*H is less than or equal to a threshold (e.g., 16 or 32 or 64) and the current block is coded in IBC AMVP and / or merge mode. i. Additionally, alternatively, if condition C is met, the IBC motion list construction process may include candidates from spatially neighboring blocks (e.g., A1, B1) and the default candidate, i.e., the insertion of the HMVP candidate is skipped. ii. Additionally, alternatively, if condition C is met, the IBC motion list construction process may include candidates from the HMVP candidates in the IBC HMVP table and default candidates, i.e., the insertion of candidates from spatially neighboring blocks is skipped. iii. Additionally, alternatively, after decoding a block for which condition C is satisfied, updating the IBC HMVP table is skipped. iv. Alternatively, condition C may be satisfied if one / some / all of the following cases are true: 1) If W*H is less than or equal to T1 (e.g., 16) and the current block is coded in IBC AMVP and / or merge mode. 2) If W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), then the block above is available and its size is equal to A×B, and both the current block and the block above it are coded with a particular mode. a. Alternatively, if W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), the block above is available, is in the same CTU, and has size equal to A×B, and both the current block and the block above it are coded with the same mode. b. Alternatively, if W is equal to T2 and H is equal to T3 (e.g., T2=4, T3=8), then the block above is available and has a size equal to A×B, and both the current block and the block above it are coded with the same mode. c. Alternatively, if W is equal to T2 and H is equal to T3 (eg, T2=4, T3=8), then the block above is unavailable. d. Alternatively, if W is equal to T2 and H is equal to T3 (eg, T2=4, T3=8), then the block above is unavailable or the block above is outside the current CTU. 3) If W is equal to T4 and H is equal to T5 (e.g., T4=8, T5=4), then a block to its left is available and has a size equal to A×B, and both the current block and the block to its left are coded with a particular mode. Alternatively, if W is equal to T4 and H is equal to T5 (eg, T4=8, T5=4), then the block to its left is unavailable. 4) If W*H is not greater than T1 (e.g., 32), the current block is coded with IBC AMVP and / or merge mode, and both its upper and left neighboring blocks are available and of size equal to A×B and coded with the specified mode. If aW*H is not greater than T1 (e.g., 32), the current block is coded in a particular mode, its left neighboring block is available, is of size A×B, and is IBC coded, and its upper neighboring block is available, is in the same CTU, is of size equal to A×B, and is coded in the same mode. If bW*H is not greater than T1 (e.g., 32), then the current block is coded in a particular mode, its left neighbor is unavailable, and its upper neighbor is available, is in the same CTU, has a size equal to A×B, and is coded in the same mode. If cW*H is not greater than T1 (eg, 32), then the current block is coded in a particular mode, its left neighbor is unavailable, and its upper neighbor is unavailable. If dW*H is not greater than T1 (e.g., 32), then the current block is coded in a particular mode, its left neighbor is available, its size is equal to A×B, it is coded in the same mode, and its upper neighbor is unavailable. If eW*H is not greater than T1 (e.g., 32), then the current block is coded in a particular mode, its left neighbor is unavailable, and its upper neighbor is unavailable or is outside the current CTU. If fW*H is not greater than T1 (e.g., 32), then the current block is coded in a particular mode, its left neighbor is available, its size is equal to A×B, it is coded in the same mode, and its upper neighbor is unavailable or is outside the current CTU. 5) In the above example, the "specific mode" is the IBC mode. 6) In the above example, the "specific mode" is the inter mode. 7) In the above example, "A×B" may be set to 4×4. 8) In the above example, "the neighboring block size is A×B" may be replaced with "the neighboring block size is less than or equal to A×B, or greater than or equal to A×B." 9) In the above example, the upper and left neighboring blocks are the two blocks that are accessed for the derivation of spatial merging candidates. In one example, assume that the coordinates of the top-left sample in the current block are (x,y) and the left block covers (x-1,y+H-1). b. In one example, assume that the coordinates of the top-left sample in the current block are (x,y) and the left block covers (x+W-1,y-1).

[0407] t. The above mentioned thresholds may be predefined or signaled. i. Additionally or alternatively, the threshold may depend on the coding information of the block, for example the coding mode. u. In one example, condition C is met if the current block is under a shared node and the current block is encoded in IBC AMVP and / or merge mode. i. Additionally, alternatively, if condition C is met, the IBC motion list construction process may include candidates from spatially neighboring blocks (e.g., A1, B1) and the default candidate, i.e., the insertion of the HMVP candidate is skipped. ii. Additionally, alternatively, if condition C is met, the IBC motion list construction process may include candidates from the HMVP candidates in the IBC HMVP table and default candidates, i.e., the insertion of candidates from spatially neighboring blocks is skipped. iii. Additionally, alternatively, after decoding a block for which condition C is satisfied, updating the IBC HMVP table is skipped.

[0408] v. In one example, condition C may be adaptively changed, such as according to the coded information of the block. i. In one example, condition C may be defined based on the coding mode (IBC mode or non-IBC mode), block size. w. Whether to apply the above method may depend on the coding information of the block, for example, whether it is an IBC coded block or not. i. In one example, if the block is IBC coded, the above method may be applied.

[0409] IBC Move List

[0410] 22. It is proposed that the motion candidates in the IBC HMVP table be stored with integer pixel precision instead of 1 / 16 pixel precision. In one example, all motion candidates are stored with one pixel accuracy. b. In one example, when using motion information from spatially neighboring (adjacent or non-adjacent) blocks and / or an IBC HMVP table, MV rounding is skipped.

[0411] 23. It is proposed that an IBC motion list may only contain motion candidates from one or more IBC HMVP tables. Additionally, alternatively, the signaling of candidates in the IBC motion list may depend on the number of available HMVP candidates in the HMVP table. b. Additionally, alternatively, the signaling of candidates in the IBC motion list may depend on the maximum number of HMVP candidates in the HMVP table. c. Alternatively, the HMVP candidates in the HMVP table are added to the list in order without pruning. i. In one example, the order is based on ascending order of entry index into the table. ii. In one example, the order is based on descending order of entry index into the table. iii. In one example, the first N entries in the table may be skipped. iv. In one example, the last N entries in the table may be skipped. v. In one example, entries with invalid BV(s) may be skipped. vi. d. Alternatively, the motion candidates derived from the HMVP candidates in one or more HMVP tables may be further modified, such as by adding an offset to the horizontal vector and / or adding an offset to the vertical vector. i. HMVP candidates with invalid BVs may be modified to provide valid BVs. e. Additionally, alternatively, a default motion candidate may be added after or before one or more HMVP candidates. f. How / whether to add HMVP candidates to the IBC motion list may depend on the block size. i. For example, an IBC motion list may only include motion candidates from one or more HMVP tables if the block dimensions (W and H, representing width and height) satisfy condition C. 1) In one example, condition C is W<=T1 and H<=T2, for example, T1=T2=4. 2) In one example, condition C is W<=T1 or H<=T2, for example, T1=T2=4. 3) In one example, condition C is W*H<=T, for example, T=16.

[0412] 5. Embodiment

[0413] Added changes are highlighted in bold italics and underlined. Deletions are marked with [[]].

[0414] 5.1 Embodiment #1

[0415] If the current block is under a shared node, no HMVP table updates are done, and we only use a single IBC HMVP table for blocks under a shared node.

[0416] 7.4.8.5 Coding Unit Syntax [[If all of the following conditions are true, the history-based motion vector predictor for shared merge candidate list regions is updated by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i], for i = 0..NumHmvpIbcCand-1: -IsInSmr[x0][y0] is equal to TRUE. - SmrX[x0][y0] is equal to x0. -SmrY[x0][y0] is equal to y0.

[0417] 8.6.2 Motion Vector Component Derivation Process for IBC Blocks 8.6.2.1 General

[0418] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples. The output of this process is: - Luma motion vectors at 1 / 16 fractional sample precision mvL.

[0419] The luminance motion vector mvL is derived as follows. The IBC luma motion vector prediction derivation process specified in Section -8.6.2.2 is invoked with the luma position (xCb, yCb) and the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL. - when general_merge_flag[xCb][yCb] is equal to 0, the following applies: 1. The variable mvd is derived as follows: mvd[0]=MvdL0[xCb][yCb][0] (8-883) mvd[1]=MvdL0[xCb][yCb][1] (8-884) The motion vector rounding process as specified in Section 2.8.5.2.14 is invoked with mvX set equal to mvL, rightShift set equal to MvShift+2, leftShift set equal to MvShift+2 as input, and the rounded mvL as output. 3. The luma motion vector mvL is modified as follows: u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885) mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886) u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887) mvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-888) NOTE 1 - The result value of mvL[0] and mvL[1] specified above is always -2. 17 ~2 17 Included in the range -1.

[0420] If IsInSmr[xCb][yCb] is false, The history-based motion vector predictor list update process as specified in section 8.6.2.6 is invoked using the luma motion vector mvL. The top left position (xRefTL, yRefTL) within the reference block and the bottom right position (xRefBR, yRefBR) within the reference block are derived as follows. (xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889) (xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890) It is a bitstream compatibility requirement that the luma motion vectors mvL should obey the following constraints: -...

[0421] 8.6.2.4 Motion Vector Candidate Derivation Process Based on IBC History

[0422] The inputs to this process are: - Motion vector candidate list mvCandList, - The number of available motion vector candidates in the list numCurrCand. The inputs to this process are: - modified motion vector candidate list mvCandList, -[[isInSmr variable that specifies whether the current coding unit is in the shared merge candidate region]] - the number of corrections to motion vector candidates in the list numCurrCand The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE.

[0423] The array smrHmvpIbcCandList and the variable smrNumHmvpIbcCand are derived as follows. [[smr]]HmvpIbcCandList=[[isInSmr?HmvpSmrIbcCandList:]]HmvpIbcCandList (8-906) [[smr]]NumHmvpIbcCand=[[isInSmr?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-907) For each candidate in smrHmvpIbcCandList[hMvpIdx] with index hMvpIdx=1..[[smr]]]NumHmvpIbcCand, the following ordered steps are repeated until numCurrCand is equal to MaxNumMergeCand. 1. The variable sameMotion is derived as follows: - For any candidate motion vector N, where N is A1 or B1, if all of the following conditions are TRUE, then sameMotion and isPrunedN are both set equal to TRUE: -hMvpIdx is 1 or less. -candidate[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] equals motion vector candidate N. -isPrunedN equals FALSE. Otherwise, sameMotion is set equal to FALSE. 2.If sameMotion is equal to FALSE, the candidate [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] is added to the motion vector candidate list as follows: mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

[0424] 5.2 Embodiment #2

[0425] When the block size satisfies a specific condition, for example, Width*Height<K, the spatial merge / AMVP candidate check in the IBC motion list construction process is removed. In the following description, the threshold value K may be predefined, for example, 16.

[0426] 7.4.8.2 Encoding Tree Unit Syntax The CTU is the root node of the encoding tree structure. [[The array IsInSmr[x][y] that defines whether the sample at (x,y) is located within the shared merge candidate list area is initialized as follows for x = 0..CtbSizeY-1 and y = 0..CtbSizeY-1: IsInSmr[x][y]=FALSE (7-96)]]

[0427] 7.4.8.4 Encoding Tree Syntax [[When all of the following conditions are true, IsInSmr[x][y] is set equal to TRUE for x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1. - isInSmr[x0][y0] is equal to FALSE - cbWidth*cbHeight / 4 is less than 32 - treeType is not equal to DUAL_TREE_CHROMA When IsInSmr[x0][y0] is equal to TRUE, the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows for x = x0..x0+cbWidth-1, and y = y0..y0+cbHeight-1. SmrX[x][y]=x0 (7-98) SmrY[x][y]=y0 (7-99) SmrW[x][y]=cbWidth (7-100) SmrH[x][y]=cbHeight (7-101) IsInSmr[x][y] is set equal to TRUE for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1 if all of the following conditions are true: -isInSmr[x0][y0] equals FALSE. -One of the following conditions is true: -mtt_split_cu_binary_flag is equal to 1 and cbWidth*cbHeight / 2 is less than 32 -mtt_split_cu_binary_flag is equal to 0 and cbWidth*cbHeight / 4 is less than 32 -treeType is not equal to DUAL_TREE_CHROMA If IsInSmr[x0][y0] is equal to TRUE, then the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1: SmrX[x][y]=x0 (7-102) SmrY[x][y]=y0 (7-103) SmrW[x][y]=cbWidth (7-104) SmrH[x][y]=cbHeight (7-105)]

[0428] 7.4.8.5 Coding Unit Syntax [[If all of the following conditions are true, the history-based motion vector predictor for shared merge candidate list regions is updated by setting NumHmvpSmrIbcCand equal to NumHmvpIbcCand and setting HmvpSmrIbcCandList[i] equal to HmvpIbcCandList[i], for i = 0..NumHmvpIbcCand-1: -IsInSmr[x0][y0] is equal to TRUE. - SmrX[x0][y0] is equal to x0. -SmrY[x0][y0] is equal to y0. For x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the following assignments are made: CbPosX[x][y]=x0 (7-106) CbPosY[x][y]=y0 (7-107) CbWidth[x][y]=cbWidth (7-108) CbHeight[x][y]=cbHeight (7-109)

[0429] 8.6.2 Motion Vector Component Derivation Process for IBC Blocks 8.6.2.1 General

[0430] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0431] The output of this process is: - Luma motion vectors at 1 / 16 fractional sample precision mvL.

[0432] The luminance motion vector mvL is derived as follows. The IBC luma motion vector prediction derivation process specified in Section -8.6.2.2 is invoked with the luma position (xCb, yCb) and the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL. - if general_merge_flag[xCb][yCb] is equal to 0, the following applies: 4. The variable mvd is derived as follows: mvd[0]=MvdL0[xCb][yCb][0] (8-883) mvd[1]=MvdL0[xCb][yCb][1] (8-884) The motion vector rounding process as specified in Section 5.8.5.2.14 is invoked with mvX set equal to mvL, rightShift set equal to MvShift+2, leftShift set equal to MvShift+2 as input, and the rounded mvL as output. 6. The luminance motion vector mvL is modified as follows: u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885) mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886) u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887) mvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-888) NOTE 1 - The resulting value of mvL[0] and mvL[1] as specified above is always -2. 17 ~2 17 Included in the range -1.

[0433] If smrWidth*smrHeight is greater than K ,The history-based motion vector predictor list update process as specified in Section 8.6.2.6 is invoked using the luma motion vector mvL. The top left position within the reference block (xRefTL, yRefTL) and the bottom right position within the reference block (xRefBR, yRefBR) are derived as follows. (xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889) (xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890) It is a bitstream compatibility requirement that the luma motion vectors mvL should obey the following constraints: -...

[0434] 8.6.2.2 Derivation process for IBC luma motion vector prediction This process is only called if CuPredMode[xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

[0435] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0436] The output of this process is: - Luma motion vectors at 1 / 16 fractional sample precision mvL. The variables xSmr, ySmr, smrWidth, smrHeight, and smrNumHmvpIbcCand are derived as follows: xSmr=[[IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:]]xCb (8-895) ySmr=[[IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:]]yCb (8-896) smrWidth=[[IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:]]cbWidth (8-897) smrHeight=[[IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:]]cbHeight (8-898) smrNumHmvpIbcCand=[[IsInSmr[xCb][yCb]?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-899)

[0437] The luma motion vector mvL is derived by the following sequential steps: 1. If smrWidth*smrHeight is greater than K, The process of derivation of spatial motion vector candidates from neighboring coding units as specified in clause 8.6.2.3 is called with input the position (xCb, yCb) of the luma coding block set equal to (xSmr, ySmr), the width cbWidth of the luma coding block and the height cbHeight of the luma coding block set equal to smrWidth and smrHeight, and the outputs are the availability flags availableFlagA1, availableFlagB1, and the motion vectors mvA1 and mvB1.

[0438] 2. If smrWidth*smrHeight is greater than K, The motion vector candidate list mvCandList is constructed as follows. i=0 if(availableFlagA1) mvCandList[i++]=mvA1 if(availableFlagB1) mvCandList[i++]=mvB1(8-900)

[0439] 3. If smrWidth*smrHeight is greater than K, the variable numCurrCand is set equal to the number of merge candidates in mvCandList.

[0440] 4. If numCurrCand is less than MaxNumMergeCand and smrNumHmvpIbcCand is greater than 0, then the IBC history-based motion vector candidate derivation process as specified in 8.6.2.4 is called with mvCandList, isInSmr set equal to IsInSmr[xCb][yCb], and numCurrCand as inputs, and the modified mvCandList and numCurrCand as outputs.

[0441] 5. If numCurrCand is less than MaxNumMergeCand, the following applies until numCurrCand is equal to MaxNumMergeCand: 1. mvCandList[numCurrCand][0] is set equal to 0. 2. mvCandList[numCurrCand][1] is set equal to 0. 3.numCurrCand is incremented by 1.

[0442] 6. The variable mvIdx is derived as follows: mvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-901)

[0443] 7. The following allocations will be made: mvL[0]=mergeCandList[mvIdx][0] (8-902) mvL[1]=mergeCandList[mvIdx][1] (8-903)

[0444] 8.6.2.4 IBC History-Based Motion Vector Candidate Derivation Process The inputs to this process are: - Motion vector candidate list mvCandList, - The number of available motion vector candidates in the list numCurrCand.

[0445] The inputs to this process are: - modified motion vector candidate list mvCandList, -[[variable isInSmr, which specifies whether the current coding unit is within the shared merge candidate region]] - The number of modified motion vector candidates in the list numCurrCand.

[0446] The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE. The array smrHmvpIbcCandList and the variable smrNumHmvpIbcCand are derived as follows. [[smr]]HmvpIbcCandList=[[isInSmr?HmvpSmrIbcCandList:]]HmvpIbcCandList (8-906) smrNumHmvpIbcCand=[[isInSmr?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-907)

[0447] For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], with index hMvpIdx=1..smrNumHmvpIbcCand, repeat the following ordered steps until numCurrCand is equal to MaxNumMergeCand. 1. The variable sameMotion is derived as follows: - smrWidth*smrHeight is greater than K, and For any motion vector candidate N, where N is A1 or B1, if all of the following conditions are TRUE, then sameMotion and isPrunedN are both set equal to TRUE: -hMvpIdx is 1 or less. -candidate[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] equals motion vector candidate N. -PrisPrunedN equals FALSE. Otherwise, sameMotion is set equal to FALSE. 2. If sameMotion is equal to FALSE, the candidate [[smr]]HmvpIbcCandList[smrNumHmvpIbcCand-hMvpIdx] is added to the motion vector candidate list as follows: mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

[0448] 5.3 Example #3

[0449] If the block size meets certain conditions, such as the current block is under a shared node and no HMVP table updates have been performed, the check for spatial merging / AMVP candidates in the IBC motion list construction process is removed.

[0450] 8.6.2.2 Derivation process for IBC luma motion vector prediction This process is only called if CuPredMode[xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

[0451] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0452] The output of this process is: - Luma motion vectors at 1 / 16 fractional sample precision mvL. The variables xSmr, ySmr, smrWidth, smrHeight, and smrNumHmvpIbcCand are derived as follows: xSmr=[[IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:]]xCb (8-895) ySmr=[[IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:]]yCb (8-896) smrWidth=[[IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:]]cbWidth (8-897) smrHeight=[[IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:]]cbHeight (8-898) smrNumHmvpIbcCand=[[IsInSmr[xCb][yCb]?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-899)

[0453] The luma motion vector mvL is derived by the following sequential steps: 1. If IsInSmr[xCb][yCb] is false, The process of derivation of spatial motion vector candidates from neighboring coding units as specified in clause 8.6.2.3 is called with input the position (xCb, yCb) of the luma coding block set equal to (xSmr, ySmr), the width cbWidth of the luma coding block and the height cbHeight of the luma coding block set equal to smrWidth and smrHeight, and the outputs are the availability flags availableFlagA1, availableFlagB1, and the motion vectors mvA1 and mvB1.

[0454] 2. If IsInSmr[xCb][yCb] is false, The motion vector candidate list mvCandList is constructed as follows. i=0 if(availableFlagA1) mvCandList[i++]=mvA1 if(availableFlagB1) mvCandList[i++]=mvB1(8-900)

[0455] 3. If IsInSmr[xCb][yCb] is false, The variable numCurrCand is set equal to the number of merge candidates in mvCandList.

[0456] 4. If numCurrCand is less than MaxNumMergeCand and smrNumHmvpIbcCand is greater than 0, then the IBC history-based motion vector candidate derivation process as specified in 8.6.2.4 is called with mvCandList, isInSmr set equal to IsInSmr[xCb][yCb], and numCurrCand as inputs, and the modified mvCandList and numCurrCand as outputs.

[0457] 5. If numCurrCand is less than MaxNumMergeCand, the following applies until numCurrCand is equal to MaxNumMergeCand: 1. mvCandList[numCurrCand][0] is set equal to 0. 2. mvCandList[numCurrCand][1] is set equal to 0. 3.numCurrCand is incremented by 1.

[0458] 6. The variable mvIdx is derived as follows: mvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-901)

[0459] 7. The following allocations will be made: mvL[0]=mergeCandList[mvIdx][0] (8-902) mvL[1]=mergeCandList[mvIdx][1] (8-903)

[0460] 8.6.2.4 Motion Vector Candidate Derivation Process Based on IBC History The inputs to this process are: - Motion vector candidate list mvCandList, - The number of available motion vector candidates in the list numCurrCand.

[0461] The inputs to this process are: - modified motion vector candidate list mvCandList, -[[isInSmr variable that specifies whether the current coding unit is in the shared merge candidate region]] - the number of corrections to motion vector candidates in the list numCurrCand

[0462] The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE. The array smrHmvpIbcCandList and the variable smrNumHmvpIbcCand are derived as follows. [[smr]]HmvpIbcCandList=[[isInSmr?HmvpSmrIbcCandList:]]HmvpIbcCandList (8-906) [[smr]]NumHmvpIbcCand=[[isInSmr?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-907) For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], with index hMvpIdx=1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand is equal to MaxNumMergeCand.

[0463] 3. The variable sameMotion is derived as follows: - isInSmr is false, andFor any candidate motion vector N, where N is A1 or B1, sameMotion and isPrunedN are both set equal to TRUE if all of the following conditions are true: -hMvpIdx is 1 or less. -candidate[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] equals motion vector candidate N. -isPrunedN equals FALSE. Otherwise, sameMotion is set equal to FALSE.

[0464] 4.If sameMotion is equal to FALSE, the candidate [[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] is added to the motion vector candidate list as follows: mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

[0465] 8.6.2 Motion Vector Component Derivation Process for IBC Blocks 8.6.2.1 General

[0466] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0467] The output of this process is: - Luma motion vectors at 1 / 16 fractional sample precision mvL.

[0468] The luminance motion vector mvL is derived as follows. The IBC luma motion vector prediction derivation process specified in Section -8.6.2.2 is invoked with the luma position (xCb, yCb) and the variables cbWidth and cbHeight as input, and the output is the luma motion vector mvL. - if general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0469] 7. The variable mvd is derived as follows: mvd[0]=MvdL0[xCb][yCb][0] (8-883) mvd[1]=MvdL0[xCb][yCb][1] (8-884)

[0470] The motion vector rounding process as specified in clause 8.8.5.2.14 is invoked with mvX set equal to mvL, rightShift set equal to MvShift+2, leftShift set equal to MvShift+2 as input, and the rounded mvL as output.

[0471] 9. The luma motion vector mvL is modified as follows: u[0]=(mvL[0]+mvd[0]+2 18 )%2 18 (8-885) mvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-886) u[1]=(mvL[1]+mvd[1]+2 18 )%2 18 (8-887) mvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-888) NOTE 1 - The result value of mvL[0] and mvL[1] specified above is always -2. 17 ~2 17 Included in the range -1.

[0472] If IsInSmr[xCb][yCb] is false, The history-based motion vector predictor list update process, as specified in Section 8.6.2.6, uses the luma motion vector mvL. and is called. The top left position (xRefTL, yRefTL) in the reference block and the bottom right position (xRefBR, yRefBR) in the reference block are derived as follows. (xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889) (xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890) It is a bitstream compatibility requirement that the luma motion vectors mvL should obey the following constraints: -...

[0473] 5.4 Example #4

[0474] If the block size meets a certain condition, such as Width*Height<=K or Width=N, Height=4 and the left neighbor block is 4x4 and coded in IBC mode, or Width=4, Height=N and the top neighbor block is 4x4 and coded in IBC mode, and there is no update of the HMVP table, the check of spatial merge / AMVP candidates in the IBC motion list construction process is removed. In the following description, the threshold K may be predefined, e.g., 16, and N may be predefined, e.g., 8.

[0475] 7.4.9.2 Coding Tree Unit Syntax The CTU is the root node of the coding tree structure. In the process of deriving neighborhood block availability as specified in Section 6.4.4, the array IsAvailable[cIdx][x][y] specifying whether a sample at (x,y) is available is initialized as follows for cIdx=0...2, x=0...CtbSizeY-1, and y=0...CtbSizeY-1: IsAvailable[cIdx][x][y]=FALSE (7-123) [[Array IsInSmr[x][y], which specifies whether a sample at (x,y) lies within the shared merge candidate list region, is initialized as follows for x=0..CtbSizeY-1 and y=0..CtbSizeY-1: IsInSmr[x][y]=FALSE (7-124)

[0476] 7.4.9.4 Coding Tree Syntax [[IsInSmr[x][y] is set equal to TRUE for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1 if all of the following conditions are true: -isInSmr[x0][y0] equals FALSE -cbWidth*cbHeight / 4 is less than 32 -treeType is not equal to DUAL_TREE_CHROMA The arrays SmrX[x][y], SmrY[x][y], SmrW[x][y] and SmrH[x][y], with IsInSmr[x0][y0] equal to TRUE, are derived as follows for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1: SmrX[x][y]=x0 (7-126) SmrY[x][y]=y0 (7-127) SmrW[x][y]=cbWidth (7-128) SmrH[x][y]=cbHeight (7-129)]

[0477] 8.6.2 Derivation of block vector components of IBC blocks 8.6.2.1 General

[0478] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0479] The output of this process is: The luma block vector at -1 / 16 fractional sample precision bvL.

[0480] The luminance block vector mvL is derived as follows: The IBC luma block vector prediction derivation process specified in Section -8.6.2.2 is invoked with the luma position (xCb, yCb) and the variables cbWidth and cbHeight as input, and the output is the luma block vector bvL. - if general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0481] 1. The variable bvd is derived as follows: bvd[0]=MvdL0[xCb][yCb][0] (8-900) bvd[1]=MvdL0[xCb][yCb][1] (8-901)

[0482] The motion vector rounding process as specified in Section 2.8.5.2.14 is invoked with mvX set equal to bvL, rightShift set equal to AMvrShift, leftShift set equal to AMvrShift, and the rounded bvL as output.

[0483] 3. The luminance block vector bvL is modified as follows: u[0]=(bvL[0]+bvd[0]+2 18 )%2 18 (8-902) bvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-903) u[1]=(bvL[1]+bvd[1]+2 18 )%2 18 (8-904) bvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-905) NOTE 1 - The result value of bvL[0] and bvL[1] specified above is always -2. 17 ~2 17 Included in the range -1.

[0484] The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true : false ). If IsLgrBlk is true, then if CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor block is 4x4 and is coded in IBC mode, then IsLgrBlk is set to false. If IsLgrBlk is true, then if CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is 4x4 and coded in IBC mode, then IsLgrBlk is set to false. (or alternatively, The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true : false ). If IsLgrBlk is true, then if CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor block is 4x4 and is coded in IBC mode, then IsLgrBlk is set to false. If IsLgrBlk is true, then if CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is available, is 4x4, and is coded in IBC mode, then IsLgrBlk is set equal to false. ) IsLgrBlk is true If [[IsInSmr[xCb][yCb] is false]], the history-based block vector predictor list update procedure as specified in Section 8.6.2.6 is invoked using the luma block vector bvL.

[0485] It is a bitstream compatibility requirement that the luma block vector bvL should obey the following constraints: -CtbSizeY is greater than or equal to ((yCb+(bvL[1]>4))&(CtbSizeY-1)+cbHeight. -IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] must not be equal to -1 for x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1.

[0486] 8.6.2.2 IBC Luminance Block Vector Prediction Derivation Process This process is only called if CuPredMode[0][xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

[0487] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0488] The output of this process is: The luma block vector at -1 / 16 fractional sample precision bvL. [[The variables xSmr, ySmr, smrWidth, and smrHeight are derived as follows: xSmr=IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:xCb (8-906) ySmr=IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:yCb (8-907) smrWidth=IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:cbWidth (8-908) smrHeight=IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:cbHeight (8-909)]]

[0489] The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true: false). If IsLgrBlk is true, then if CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor block is 4x4 and is coded in IBC mode, then IsLgrBlk is set to false. If IsLgrBlk is true, then if CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is 4x4 and coded in IBC mode, then IsLgrBlk is set to false. (or alternatively, The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true: false). If IsLgrBlk is true, then if CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor block is 4x4 and is coded in IBC mode, then IsLgrBlk is set to false. If IsLgrBlk is true, then if CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is available, is 4x4, and is coded in IBC mode, then IsLgrBlk is set to false. )

[0490] The luminance block vector bvL is derived by the following sequential steps: 1. If IslgrBlk is true, The process of deriving spatial block vector candidates from neighboring coding units as specified in Section 8.6.2.3 is as follows: xCb,yCb The position (xCb,yCb) of the luminance coding block set equal to [[xSmr,ySmr]]), [[smr]] Cb Width and [[smr]] Cb It is called with inputs cbWidth and cbHeight of the luma coding block set equal to Height, and the outputs are the availability flags availableFlagA1, availableFlagB1, and the block vectors bvA1 and bvB1.

[0491] 2. If IslgrBlk is true, The block vector candidate list bvCandList is constructed as follows: i=0 if(availableFlagA1) bvCandList[i++]=bvA1 if(availableFlagB1) bvCandList[i++]=bvB1(8-910)

[0492] 3. If IslgrBlk is true, The variable numCurrCand is set equal to the number of merge candidates in bvCandList.

[0493] 4. If numCurrCand is less than MaxNumIbcMergeCand and smrNumHmvpIbcCand is greater than 0, the IBC history-based block vector candidate derivation process as specified in 8.6.2.4 is performed using mvCandList, and IsLgrBlk , and numCurrCand as inputs, and the modified bvCandList and numCurrCand as outputs.

[0494] 5. If numCurrCand is less than MaxNumIbcMergeCand, the following is applied until numCurrCand is equal to MaxNumIbcMergeCand: 1. bvCandList[numCurrCand][0] is set equal to 0. can be. 2. bvCandList[numCurrCand][1] is set equal to 0. 3.numCurrCand is incremented by 1.

[0495] 6. The variable bvIdx is derived as follows: bvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

[0496] 7. The following allocations will be made: bvL[0]=bvCandList[mvIdx][0] (8-912) bvL[1]=bvCandList[mvIdx][1] (8-913)

[0497] 8.6.2.3 IBC spatial block vector candidate derivation process The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0498] The output of this process is: - The availability flags availableFlagA1 and availableFlagB1 of the neighboring coding units. - 1 / 16 fractional sample precision of neighboring coding units bvA1 and bvB1.

[0499] For the derivation of availableFlagA1 and mvA1 the following applies: - The luminance position (xNbA1, yNbA1) in the neighboring luminance coding block is set equal to (xCb-1, yCb+cbHeight-1). The derivation process for neighboring block availability as specified in clause 6.4.4 is called with inputs current luminance position (xCurr, yCurr) set equal to (xCb, yCb), neighboring luminance positions (xNbA1, yNbA1), checkPredModeY set equal to TRUE, cIdx set equal to 0, and the output is assigned to the block availability flag availableA1. The variables availableFlagA1 and bvA1 are derived as follows: If -availableA1 is equal to FALSE, then availableFlagA1 is set equal to 0, and both components of bvA1 are set equal to 0. - Otherwise, availableFlagA1 is set equal to 1 and the following allocation is made: bvA1=MvL0[xNbA1][yNbA1] (8-914)

[0500] For the derivation of availableFlagB1 and bvB1 the following applies: - The luminance position (xNbB1, yNbB1) in the neighboring luma coding block is set equal to (xCb+cbWidth-1, yCb-1). - The derivation process for neighboring block availability as specified in clause 6.4.4 is called with inputs current luminance position (xCurr, yCurr) set equal to (xCb, yCb), neighboring luminance positions (xNbB1, yNbB1), checkPredModeY set equal to TRUE, cIdx set equal to 0, and the output is assigned to the block availability flag availableB1. The variables availableFlagB1 and bvB1 are derived as follows: If one or more of the following conditions are true, then availableFlagB1 is set equal to 0 and both components of bvB1 are set to 0: -availableB1 equals FALSE. -availableA1 is equal to TRUE and luminance positions (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vector. - Otherwise, availableFlagB1 is set equal to 1 and the following allocation is made: bvB1=MvL0[xNbB1][yNbB1] (8-915)

[0501] 8.6.2.4 Deriving block vector candidates based on IBC history The inputs to this process are: -Block vector candidate list bvCandList, -[[isInSmr variable that specifies whether the current coding unit is in the shared merge candidate region]] - A variable indicating the non-small block IslgrBlk, - The number of available block vector candidates in the list numCurrCand. The inputs to this process are: - modified block vector candidate list bvCandList, - The number of modified motion vector candidates in the list numCurrCand. The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE.

[0502] For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], with index hMvpIdx=1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand is equal to MaxNumMergeCand. 1. The variable sameMotion is derived as follows: - IsLgrBlk is true, For any candidate block vector N, where N is A1 or B1, sameMotion and isPrunedN are both set equal to TRUE if all of the following conditions are true: -hMvpIdx is 1 or less. - Candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is equal to the block vector candidate N. -PrisPrunedN equals FALSE. - Otherwise, set sameMotion equal to FALSE. 2. If sameMotion is equal to FALSE, the candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is added to the block vector candidate list as follows: bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] (8-916)

[0503] 5.5 Embodiment #5

[0504] If the block size meets a certain condition, e.g., Width=N, Height=4, the left neighbor block is 4×4 and coded in IBC mode, or Width=4, Height=N, the top neighbor block is 4×4 and coded in IBC mode, remove the check of spatial merge / AMVP candidates in the IBC motion list construction process and delete the update of the HMVP table. In the following description, N may be predefined, e.g., 4 or 8.

[0505] 7.4.9.2 Coding Tree Unit Syntax The CTU is the root node of the coding tree structure. In the process of deriving neighborhood block availability as specified in Section 6.4.4, the array IsAvailable[cIdx][x][y] specifying whether a sample at (x,y) is available is initialized as follows for cIdx=0...2, x=0...CtbSizeY-1, and y=0...CtbSizeY-1: IsAvailable[cIdx][x][y]=FALSE (7-123) [[Array IsInSmr[x][y], which specifies whether a sample at (x,y) lies within the shared merge candidate list region, is initialized as follows for x=0..CtbSizeY-1 and y=0..CtbSizeY-1: IsInSmr[x][y]=FALSE (7-124)

[0506] 7.4.9.4 Coding Tree Syntax [[IsInSmr[x][y] is set equal to TRUE for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1 if all of the following conditions are true: -isInSmr[x0][y0] equals FALSE -cbWidth *cbHeight / 4 is less than 32 -treeType is not equal to DUAL_TREE_CHROMA If IsInSmr[x0][y0] is equal to TRUE, then the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1: SmrX[x][y]=x0 (7-126) SmrY[x][y]=y0 (7-127) SmrW[x][y]=cbWidth (7-128) SmrH[x][y]=cbHeight (7-129)]

[0507] 8.6.2 Derivation of block vector components of IBC blocks 8.6.2.1 General

[0508] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0509] The output of this process is: The luma block vector at -1 / 16 fractional sample precision bvL.

[0510] The luminance block vector mvL is derived as follows: The IBC luma block vector prediction derivation process specified in Section -8.6.2.2 is invoked with the luma position (xCb, yCb) and the variables cbWidth and cbHeight as input, and the output is the luma block vector bvL. - if general_merge_flag[xCb][yCb] is equal to 0, the following applies:

[0511] 1. The variable bvd is derived as follows: bvd[0]=MvdL0[xCb][yCb][0] (8-900) bvd[1]=MvdL0[xCb][yCb][1] (8-901)

[0512] The motion vector rounding process as specified in Section 2.8.5.2.14 is invoked with mvX set equal to bvL, rightShift set equal to AMvrShift, leftShift set equal to AMvrShift, and the rounded bvL as output.

[0513] 3. The luminance block vector bvL is modified as follows: u[0]=(bvL[0]+bvd[0]+2 18 )%2 18 (8-902) bvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-903) u[1]=(bvL[1]+bvd[1]+2 18 )%2 18 (8-904) bvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-905) NOTE 1 - The result value of bvL[0] and bvL[1] specified above is always -2. 17 ~2 17 Included in the range -1.

[0514] The variable IslgrBlk is set to true if one of the following conditions is true: - If CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor is 4x4 and is coded in IBC mode. - If CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is 4x4 and is coded in IBC mode. Otherwise, IsLgrBlk is set equal to false. (or alternatively, The variable IslgrBlk is set to (CbWidth*CbHeight>16? true:false) and then the following is checked: If IsLgrBlk is true, then if CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor block is 4x4 and is coded in IBC mode, then IsLgrBlk is set to false. If IsLgrBlk is true, then if CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is 4x4 and coded in IBC mode, then IsLgrBlk is set to false.

[0515] IsLgrBlk is true If [[IsInSmr[xCb][yCb] is equal to false]], the history-based block vector predictor list update procedure as specified in Section 8.6.2.6 is invoked with the luma block vector bvL. It is a bitstream compatibility requirement that the luma block vector bvL should obey the following constraints: -CtbSizeY is greater than or equal to ((yCb+(bvL[1]>4))&(CtbSizeY-1))+cbHeight. -IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] must not be equal to -1 for x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1.

[0516] 8.6.2.2 Derivation process for IBC luma block vector prediction This process is only called if CuPredMode[0][xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

[0517] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0518] The output of this process is: The luma block vector at -1 / 16 fractional sample precision bvL. [[The variables xSmr, ySmr, smrWidth, and smrHeight are derived as follows: xSmr=IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:xCb (8-906) ySmr=IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:yCb (8-907) smrWidth=IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:cbWidth (8-908) smrHeight=IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:cbHeight (8-909)]]

[0519] The variable IslgrBlk is set to true if one of the following conditions is true: - If CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor is 4x4 and is coded in IBC mode. - If CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is 4x4 and is coded in IBC mode. Otherwise, IsLgrBlk is set to false. (or alternatively, The variable IslgrBlk is set to (CbWidth*CbHeight>16? true:false) and then the following is checked: If IsLgrBlk is true, then if CbWidth is equal to N, CbHeight is equal to 4, and the left neighbor block is 4x4 and is coded in IBC mode, then IsLgrBlk is set to false. If IsLgrBlk is true, then if CbWidth is equal to 4, CbHeight is equal to N, and the upper neighbor block is 4x4 and coded in IBC mode, then IsLgrBlk is set to false.

[0520] The luminance block vector bvL is derived by the following sequential steps: 1. If IslgrBlk is true, The process of deriving spatial block vector candidates from neighboring coding units as specified in Section 8.6.2.3 is as follows: xCb,yCb The position (xCb,yCb) of the luminance coding block set equal to [[xSmr,ySmr]]), [[smr]] Cb Width and [[smr]] Cb It is called with inputs cbWidth and cbHeight of the luma coding block set equal to Height, and the outputs are the availability flags availableFlagA1, availableFlagB1, and the block vectors bvA1 and bvB1.

[0521] 2. If IslgrBlk is true,The block vector candidate list bvCandList is constructed as follows. i=0 if(availableFlagA1) bvCandList[i++]=bvA1 if(availableFlagB1) bvCandList[i++]=bvB1(8-910)

[0522] 3. If IslgrBlk is true, The variable numCurrCand is set equal to the number of merge candidates in bvCandList.

[0523] 4. If numCurrCand is less than MaxNumIbcMergeCand and smrNumHmvpIbcCand is greater than 0, the IBC history-based block vector candidate derivation process as specified in 8.6.2.4 is performed using mvCandList, and IsLgrBlk , and numCurrCand as inputs, and the modified bvCandList and numCurrCand as outputs.

[0524] 5. If numCurrCand is less than MaxNumIbcMergeCand, the following is applied until numCurrCand is equal to MaxNumIbcMergeCand: 1. bvCandList[numCurrCand][0] is set equal to 0. 2. bvCandList[numCurrCand][1] is set equal to 0. 3.numCurrCand is incremented by 1.

[0525] 6. The variable bvIdx is derived as follows: bvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

[0526] 7. The following allocations will be made: bvL[0]=bvCandList[mvIdx][0] (8-912) bvL[1]=bvCandList[mvIdx][1] (8-913)

[0527] 8.6.2.3 IBC spatial block vector candidate derivation process The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0528] The output of this process is: - the availability flags availableFlagA1 and availableFlagB1 of the neighboring coding units, - 1 / 16 fractional sample precision of neighboring coding units bvA1 and bvB1. For the derivation of availableFlagA1 and mvA1 the following applies: - The luminance position (xNbA1, yNbA1) in the neighboring luminance coding block is set equal to (xCb-1, yCb+cbHeight-1). - The derivation process for neighboring block availability as specified in clause 6.4.4 is called with inputs current luminance position (xCurr, yCurr) set equal to (xCb, yCb), neighboring luminance positions (xNbA1, yNbA1), checkPredModeY set equal to TRUE, cIdx set equal to 0, and the output is assigned to the block availability flag availableA1. The variables availableFlagA1 and bvA1 are derived as follows: If -availableA1 is equal to FALSE, then availableFlagA1 is set equal to 0, and both components of bvA1 are set equal to 0. - Otherwise, availableFlagA1 is set equal to 1 and the following allocation is made: bvA1=MvL0[xNbA1][yNbA1] (8-914)

[0529] For the derivation of availableFlagB1 and bvB1 the following applies: - The luminance position (xNbB1, yNbB1) in the neighboring luma coding block is set equal to (xCb+cbWidth-1, yCb-1). - The derivation process for neighboring block availability as specified in clause 6.4.4 is called with inputs current luminance position (xCurr, yCurr) set equal to (xCb, yCb), neighboring luminance positions (xNbB1, yNbB1), checkPredModeY set equal to TRUE, cIdx set equal to 0, and the output is assigned to the block availability flag availableB1. The variables availableFlagB1 and bvB1 are derived as follows: If one or more of the following conditions are true, then availableFlagB1 is set to 0 and both components of bvB1 are set equal to 0: -availableB1 equals FALSE. -availableA1 is equal to TRUE and luminance positions (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vector. - Otherwise, availableFlagB1 is set to 1 and the following allocation is made: bvB1=MvL0[xNbB1][yNbB1] (8-915)

[0530] 8.6.2.4 Deriving block vector candidates based on IBC history The inputs to this process are: -Block vector candidate list bvCandList, -[[isInSmr variable that specifies whether the current coding unit is in the shared merge candidate region]] - A variable indicating the non-small block IslgrBlk, - The number of available block vector candidates in the list numCurrCand.

[0531] The outputs to this process are: - modified block vector candidate list bvCandList, - the number of corrections to motion vector candidates in the list numCurrCand

[0532] The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE. For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], with index hMvpIdx=1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand is equal to MaxNumMergeCand.

[0533] 1. The variable sameMotion is derived as follows: - IsLgrBlk is true and For any candidate block vector N, where N is A1 or B1, sameMotion and isPrunedN are both set equal to TRUE if all of the following conditions are true: -hMvpIdx is 1 or less. Candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is the block vector candidate N. -isPrunedN equals FALSE. Otherwise, sameMotion is set equal to FALSE.

[0534] 2. If sameMotion is equal to FALSE, the candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is added to the block vector candidate list as follows: bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] (8-916)

[0535] 5.6 Example #6

[0536] If the block size meets a certain condition, e.g., Width*Height<=K, and the left or top neighbor block is 4×4 and coded in IBC mode, remove the check for spatial merging / AMVP candidates in the IBC motion list construction process and remove the update of the HMVP table. In the following description, the threshold K may be predefined, e.g., 16 or 32.

[0537] 7.4.9.2 Coding Tree Unit Syntax The CTU is the root node of the coding tree structure. In the process of deriving neighborhood block availability as specified in Section 6.4.4, the array IsAvailable[cIdx][x][y] specifying whether a sample at (x,y) is available is initialized as follows for cIdx=0...2, x=0...CtbSizeY-1, and y=0...CtbSizeY-1: IsAvailable[cIdx][x][y]=FALSE (7-123) The array IsInSmr[x][y], which specifies whether a sample at [x,y] lies within the shared merge candidate list region, is initialized as follows for x=0..CtbSizeY-1 and y=0..CtbSizeY-1: IsInSmr[x][y]=FALSE (7-124)

[0538] 7.4.9.4 Coding Tree Syntax [[IsInSmr[x][y] is set equal to TRUE for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1 if all of the following conditions are true: -isInSmr[x0][y0] equals FALSE -cbWidth*cbHeight / 4 is less than 32 -treeType is not equal to DUAL_TREE_CHROMA If IsInSmr[x0][y0] is equal to TRUE, then the arrays SmrX[x][y], SmrY[x][y], SmrW[x][y], and SmrH[x][y] are derived as follows for x=x0..x0+cbWidth-1, and y=y0..y0+cbHeight-1: SmrX[x][y]=x0 (7-126) SmrY[x][y]=y0 (7-127) SmrW[x][y]=cbWidth (7-128) SmrH[x][y]=cbHeight (7-129)]

[0539] 8.6.2 Derivation of block vector components of IBC blocks 8.6.2.1 General

[0540] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0541] The output of this process is: The luma block vector at -1 / 16 fractional sample precision bvL.

[0542] The luminance block vector mvL is derived as follows: The IBC luma block vector prediction derivation process specified in Section -8.6.2.2 is invoked with the luma position (xCb, yCb) and the variables cbWidth and cbHeight as input, and the output is the luma block vector bvL. - if general_merge_flag[xCb][yCb] is equal to 0, the following applies: 1. The variable bvd is derived as follows: bvd[0]=MvdL0[xCb][yCb][0] (8-900) bvd[1]=MvdL0[xCb][yCb][1] (8-901) The motion vector rounding process as specified in Section 2.8.5.2.14 is invoked with mvX set equal to bvL, rightShift set equal to AMvrShift, leftShift set equal to AMvrShift, and the rounded bvL as output. 3. The luminance block vector bvL is modified as follows: u[0]=(bvL[0]+bvd[0]+2 18 )%2 18 (8-902) bvL[0]=(u[0]>=2 17 )?(u[0]-2 18 ):u[0] (8-903) u[1]=(bvL[1]+bvd[1]+2 18 )%2 18 (8-904) bvL[1]=(u[1]>=2 17 )?(u[1]-2 18 ):u[1] (8-905) NOTE 1 - The result value of bvL[0] and bvL[1] specified above is always -2. 17 ~2 17 Included in the range -1.

[0543] The variable IslgrBlk is set to (Is cbWidth x cbHeight greater than K? true: false). If IsLgrBlk is true, then if the left neighboring block is 4x4 and is coded in IBC mode, then IsLgrBlk is set equal to false. If IsLgrBlk is true, then if the upper neighboring block is 4x4 and coded in IBC mode, IsLgrBlk is set equal to false. (or alternatively, The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true: false). If IsLgrBlk is true, then if the left neighboring block is 4x4 and is coded in IBC mode, then IsLgrBlk is set equal to false. If IsLgrBlk is true, then if the upper neighboring block is in the same CTU as the current block, is 4x4, and is coded in IBC mode, then IsLgrBlk is set equal to false. )

[0544] IsLgrBlk is true If [[IsInSmr[xCb][yCb] is equal to false]], the history-based block vector predictor list update procedure as specified in Section 8.6.2.6 is invoked with the luma block vector bvL. It is a bitstream compatibility requirement that the luma block vector bvL should obey the following constraints: -CtbSizeY is greater than or equal to ((yCb+(bvL[1]>4))&(CtbSizeY-1))+cbHeight. -IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] must not be equal to -1 for x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1.

[0545] 8.6.2.2 Derivation process for IBC luma block vector prediction This process is only called if CuPredMode[0][xCb][yCb] is equal to MODE_IBC, where (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

[0546] The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0547] The output of this process is: The luma block vector at -1 / 16 fractional sample precision bvL. [[The variables xSmr, ySmr, smrWidth, and smrHeight are derived as follows: xSmr=IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:xCb (8-906) ySmr=IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:yCb (8-907) smrWidth=IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:cbWidth (8-908) smrHeight=IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:cbHeight (8-909)]]

[0548] The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true: false). If IsLgrBlk is true, then if the left neighboring block is 4x4 and is coded in IBC mode, then IsLgrBlk is set equal to false. If IsLgrBlk is true, then if the upper neighboring block is 4x4 and coded in IBC mode, IsLgrBlk is set equal to false. (or alternatively, The variable IslgrBlk is set to (is cbWidth x cbHeight greater than K? true: false). If IsLgrBlk is true, then if the left neighboring block is 4x4 and is coded in IBC mode, then IsLgrBlk is set equal to false. If IsLgrBlk is true, then if the upper neighboring block is in the same CTU as the current block, is 4x4, and is coded in IBC mode, then IsLgrBlk is set equal to false. )

[0549] The luminance block vector bvL is derived by the following sequential steps: 1. If IslgrBlk is true, The process of deriving spatial block vector candidates from neighboring coding units as specified in Section 8.6.2.3 is as follows: xCb,yCb The position (xCb,yCb) of the luminance coding block set equal to [[xSmr,ySmr]]), [[smr]] Cb Width and [[smr]] Cb It is called with inputs cbWidth and cbHeight of the luma coding block set equal to Height, and the outputs are the availability flags availableFlagA1, availableFlagB1, and the block vectors bvA1 and bvB1.

[0550] 2. If IslgrBlk is true, The block vector candidate list bvCandList is constructed as follows. i=0 if(availableFlagA1) bvCandList[i++]=bvA1 if(availableFlagB1) bvCandList[i++]=bvB1(8-910)

[0551] 3. If IslgrBlk is true, The variable numCurrCand is set equal to the number of merge candidates in bvCandList.

[0552] 4. If numCurrCand is less than MaxNumIbcMergeCand and smrNumHmvpIbcCand is greater than 0, the IBC history-based block vector candidate derivation process as specified in 8.6.2.4 shall be performed using mvCandList, IsLgrBlk , and numCurrCand as inputs, and the modified bvCandList and numCurrCand as outputs.

[0553] 5. If numCurrCand is less than MaxNumIbcMergeCand, the following is applied until numCurrCand is equal to MaxNumIbcMergeCand: 1. bvCandList[numCurrCand][0] is set equal to 0. 2. bvCandList[numCurrCand][1] is set equal to 0. 3.numCurrCand is incremented by 1.

[0554] 6. The variable bvIdx is derived as follows: bvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

[0555] 7. The following allocations will be made: bvL[0]=bvCandList[mvIdx][0] (8-912) bvL[1]=bvCandList[mvIdx][1] (8-913)

[0556] 8.6.2.3 IBC spatial block vector candidate derivation process The inputs to this process are: the luminance position (xCb, yCb) of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture, a variable cbWidth that specifies the width of the current coding block in luma samples, A variable cbHeight that specifies the height of the current coding block in luma samples.

[0557] The output of this process is: - the availability flags availableFlagA1 and availableFlagB1 of the neighboring coding units, - precisions bvA1 and bvB1 in 1 / 16 fractional samples of neighboring coding units.

[0558] For the derivation of availableFlagA1 and mvA1 the following applies: - The luminance position (xNbA1, yNbA1) in the neighboring luminance coding block is set equal to (xCb-1, yCb+cbHeight-1). - The derivation process for neighboring block availability specified in clause 6.4.4 is called with inputs current luminance position (xCurr, yCurr) set equal to (xCb, yCb), neighboring luminance positions (xNbA1, yNbA1), checkPredModeY set equal to TRUE, cIdx set equal to 0, and the output is assigned to the block availability flag availableA1. The variables availableFlagA1 and bvA1 are derived as follows: If -availableA1 is equal to FALSE, then availableFlagA1 is set equal to 0, and both components of bvA1 are set equal to 0. - Otherwise, availableFlagA1 is set equal to 1 and the following allocation is made: bvA1=MvL0[xNbA1][yNbA1] (8-914)

[0559] For the derivation of availableFlagB1 and bvB1 the following applies: - The luminance position (xNbB1, yNbB1) in the neighboring luma coding block is set equal to (xCb+cbWidth-1, yCb-1). - The derivation process for neighboring block availability as specified in clause 6.4.4 is called with inputs current luminance position (xCurr, yCurr) set equal to (xCb, yCb), neighboring luminance positions (xNbB1, yNbB1), checkPredModeY set equal to TRUE, cIdx set equal to 0, and the output is assigned to the block availability flag availableB1. The variables availableFlagB1 and bvB1 are derived as follows: If one or more of the following conditions are true, then availableFlagB1 is set equal to 0 and both components of bvB1 are set equal to 0: -availableB1 equals FALSE. -availableA1 is equal to TRUE and luminance positions (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vector. - Otherwise, availableFlagB1 is set equal to 1 and the following allocation is made: bvB1=MvL0[xNbB1][yNbB1] (8-915)

[0560] 8.6.2.4 IBC History-Based Block Vector Candidate Derivation Process The inputs to this process are: -Block vector candidate list bvCandList, -[[variable isInSmr, which specifies whether the current coding unit is within the shared merge candidate region]] - A variable indicating the non-small block IslgrBlk, - The number of available block vector candidates in the list numCurrCand.

[0561] The outputs to this process are: - modified block vector candidate list bvCandList, - the number of corrections to motion vector candidates in the list numCurrCand The variables isPrunedA1 and isPrunedB1 are both set equal to FALSE.

[0562] For each candidate in [[smr]]HmvpIbcCandList[hMvpIdx], with index hMvpIdx=1..[[smr]]NumHmvpIbcCand, repeat the following ordered steps until numCurrCand is equal to MaxNumMergeCand. 1. The variable sameMotion is derived as follows: - IsLgrBlk is true and For any candidate block vector N, where N is A1 or B1, sameMotion and isPrunedN are both set equal to TRUE if all of the following conditions are true: -hMvpIdx is 1 or less. - Candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is equal to the block vector candidate N. -isPrunedN is set equal to FALSE. - Otherwise, set sameMotion equal to FALSE. 2. If sameMotion is equal to FALSE, the candidate HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] is added to the block vector candidate list as follows: bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx] (8-916)

[0563] FIG. 22 is a block diagram of a video processing device 2200. The device 2200 may be used to implement one or more of the methods described herein. The device 2200 may be implemented in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 2200 may include one or more processors 2202, one or more memories 2204, and video processing hardware 2206. The one or more processors 2202 may be configured to implement one or more methods described herein. The one or more memories 2204 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 2206 may be used to implement the techniques described herein in a hardware circuit. The video processing hardware 2206 may be included partially or completely within the processor 2202 in the form of dedicated hardware, or a Graphical Processor Unit (GPU) or a dedicated signal processing block.

[0564] 23 is a flow chart illustrating an example of a video processing method 2300. The method 2300 includes, at operation 2302, determining to use a sub-block Intra Block Copy (sbIBC) coding mode. The method 2300 further includes, at operation 2304, performing a transform using the sbIBC coding mode.

[0565] Some embodiments can be described using the following item-based description.

[0566] The following items provide exemplary embodiments of the techniques discussed in item 1 of the previous section.

[0567] 1. A method of video processing, comprising: determining to use a sub-block Intra Block Copy (sbIBC) coding mode in a conversion between a current video block in the video domain and a bitstream representation of the current video block in which the current video block is divided into a number of sub-blocks and each sub-block is coded based on reference samples from the video domain, the size of the sub-blocks being based on a division rule; and performing the conversion using the sbIBC coding mode on the multiple sub-blocks.

[0568] 2. The method according to item 1, wherein the current image block is an M×N block, where M and N are integers, and the division rule specifies that each sub-block has the same size.

[0569] 3. The method according to item 1, wherein the bitstream representation includes a syntax element indicating a partitioning rule or a size of a subblock.

[0570] 4. The method according to any one of items 1 to 3, wherein the division rule specifies the size of the sub-blocks based on the color components of the current image block.

[0571] 5. The method according to any one of items 1 to 4, wherein a sub-block of a first color component can derive its motion vector information from a sub-block of a second color component that corresponds to the current video block.

[0572] The following items provide exemplary embodiments of the techniques described in items 2 and 6 of the previous section.

[0573] 1. A method of video processing, comprising: determining to use a sub-block Intra Block Copy (sbIBC) coding mode in a conversion between a current video block in the video domain and a bitstream representation of the current video block in which the current video block is divided into a plurality of sub-blocks and each sub-block is coded based on reference samples from the video domain; and performing the conversion using the sbIBC coding mode on the plurality of sub-blocks, wherein the conversion includes determining an initialized Motion Vector (initMV) for a given sub-block, identifying a reference block from the initMV, and deriving Motion Vector (MV) information for the given sub-block using the MV information for the reference block.

[0574] 2. The method of claim 1, wherein determining the initMV includes determining the initMV from one or more neighboring blocks of the given subblock.

[0575] 3. The method of item 2, checking one or more neighboring blocks in turn.

[0576] 4. The method of claim 1, wherein determining the initMV includes deriving the initMV from a motion candidate list.

[0577] 5. A method according to any one of items 1 to 4, wherein identifying the reference block includes converting the initMV to one pixel precision and identifying the reference block based on the converted initMV.

[0578] 6. A method according to any of items 1 to 4, wherein identifying the reference block includes applying initMV to an offset position within the given block, the offset position being represented as being offset by (offsetX, offsetY) from a given position of the given subblock.

[0579] 7. A method according to any of items 1 to 6, wherein deriving the MV for a given sub-block includes clipping MV information for a reference block.

[0580] 8. The method according to any one of items 1 to 7, wherein the reference block is a color component different from the color component of the current image block.

[0581] The following sections provide exemplary embodiments of the techniques described in Sections 3, 4, and 5 above.

[0582] 1. A method of video processing, comprising: determining to use a sub-block Intra Block Copy (sbIBC) coding mode in a conversion between a current video block in the video domain and a bitstream representation of the current video block in which the current video block is divided into a plurality of sub-blocks and each sub-block is coded based on reference samples from the video domain; and performing the conversion using the sbIBC coding mode on the plurality of sub-blocks, the conversion including generating sub-block IBC candidates.

[0583] 2. The method according to item 1, in which the sub-block IBC candidate is added to a candidate list containing alternative motion vector predictor candidates.

[0584] 3. The method according to item 1, in which the sub-block IBC candidates are added to a list containing affine merge candidates.

[0585] The following section presents exemplary embodiments of the techniques described in items 7, 8, 9, 10, 11, 12, and 13 of the previous section.

[0586] 1. A method of video processing, comprising performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a sub-block Intra Block Copy (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an intra coding mode.

[0587] 2. A method of video processing, comprising performing a conversion between a bitstream representation of a current video block and a current video block divided into a plurality of sub-blocks, the conversion comprising processing all sub-blocks of the plurality of sub-blocks using an intra-coding mode.

[0588] 3. A method of video processing, comprising performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion comprising processing all of the plurality of sub-blocks using a palette coding mode used to code each sub-block using a palette of representative pixel values.

[0589] 4. A method of video processing comprising performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a palette mode that encodes the first sub-block using representative pixel values, and processing a second sub-block of the plurality of sub-blocks using an intra block copy encoding mode.

[0590] 5. A method of video processing comprising performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a palette mode that encodes the first sub-block using a palette of representative pixel values, and processing a second sub-block of the plurality of sub-blocks using an intra-coding mode.

[0591] 6. A method of video processing, comprising performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a sub-block Intra Block Copy (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

[0592] 7. A method of video processing comprising performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, the conversion including processing a first sub-block of the plurality of sub-blocks using a sub-block intra-coding mode and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

[0593] The following sections present exemplary embodiments of the techniques discussed in item 14 of the previous section.

[0594] 8. The method of any of items 1 to 7, further comprising avoiding updating the IBC history-based motion vector predictor table after transforming the current video block.

[0595] The following section presents an exemplary embodiment of the technique described in item 15 of the previous section.

[0596] 9. The method of any of items 1 to 7, further comprising refraining from updating the non-IBC history based motion vector predictor table after transforming the current video block.

[0597] The following section presents an exemplary embodiment of the technique described in item 16 of the previous section.

[0598] 10. The method of any of items 1 to 7, wherein the transformation includes selective use of an in-loop filter based on the processing.

[0599] The following items provide exemplary embodiments of the techniques discussed in item 1 of the previous section.

[0600] 17. A method according to any of items 1 to 7, wherein performing the transformation includes performing the transformation by disabling a particular coding mode for the current video block by using the method, the particular coding mode including one or more of sub-block transform, affine motion prediction, intra prediction of multiple reference lines, matrix-based intra prediction, symmetric Motion Vector Difference (MVD) coding, merging with MVD decoder-side motion derivation / improvement, bidirectional optimal flow, contractive secondary transform, or a set of multiple transforms.

[0601] The following section presents exemplary embodiments of the techniques described in item 18 of the previous section.

[0602] 1. A method according to any of the preceding items, wherein the indicator in the bitstream representation contains information about how the method is to be applied to the current video block.

[0603] The following section presents an exemplary embodiment of the technique described in item 19 of the previous section.

[0604] 1. A method of video processing comprising: making a decision to use a method described in any of the preceding items to encode a current video block into a bitstream representation; and including, in the bitstream representation, information indicating the decision at a decoder parameter set level, a sequence parameter set level, or a video parameter set level, or a picture parameter set level, or a picture header level, or a slice header level, or a tile group header level, or a maximum coding unit level, or a coding unit level, or a maximum coding unit row level, or a group of LCU levels, or a transform unit level, or a prediction unit level, or a video coding unit level.

[0605] 2. A method of video processing, comprising: making a decision to use a method according to any of the preceding items to encode a current video block into a bitstream representation based on an encoding condition; and performing the encoding using a method according to any of the preceding items, wherein the condition is a position of a coding unit, a prediction unit, a transform unit, a current video block, or a video coding unit of the current video block;

[0606] the block dimensions of the current video block and / or its neighboring blocks,

[0607] the block shape of the current video block and / or its neighboring blocks;

[0608] the intra mode of the current video block and / or its neighboring blocks;

[0609] The motion / block vectors of the neighboring blocks of the current video block,

[0610] The color format of the current video block,

[0611] Coding tree structure,

[0612] The slice, tile group type, or picture type of the current video block,

[0613] The color components of the current video block,

[0614] The temporal layer ID of the current video block,

[0615] A method based on one or more of the profile, level, or standard used for the bitstream representation.

[0616] The following section presents exemplary embodiments of the techniques described in item 20 of the previous section.

[0617] 1. A method of video processing comprising: determining to use intra block copy modes and inter prediction modes for converting between a block in the video domain and a bitstream representation of the video domain; and performing the conversion using the intra block copy modes and the inter prediction modes for the block in the video domain.

[0618] 2. The method of claim 1, wherein the image region includes an image picture, an image slice, an image tile group, or an image tile.

[0619] 3. The method according to any one of items 1 to 2, wherein the inter prediction mode uses an Alternative Motion Vector Predictor (AMVP) coding mode.

[0620] 4. The method according to any one of items 1 to 3, wherein performing the transformation includes deriving merge candidates for intra block copy mode from neighboring blocks.

[0621] The following items present exemplary embodiments of the techniques described in item 21 of the previous section.

[0622] 1. A method of video processing comprising: performing a motion candidate list construction process and / or a table update process for updating a history-based motion vector predictor table based on an encoding condition during conversion between a current video block and a bitstream representation of the current video block; and performing a conversion based on the motion candidate list construction process and / or the table update process.

[0623] 2. The method according to item 1, in which different processing may be applied when an encoding condition is met and when it is not met.

[0624] 3. The method according to item 1, wherein if the encoding condition is met, the history-based motion vector predictor table update is not applied.

[0625] 4. The method according to item 1, in which if the coding condition is met, derivation of candidates from spatially nearby (adjacent or non-adjacent) blocks is skipped.

[0626] 5. The method according to item 1, wherein if the encoding condition is met, derivation of HMVP candidates is skipped.

[0627] 6. The method according to item 1, wherein the encoding condition is that block width x height is 16 or less, or 32 or less, or 64 or less.

[0628] 7. The method of claim 1, wherein the encoding conditions include that the block is encoded in IBC mode.

[0629] 8. The method described in item 1, wherein the encoding conditions are as described in section 21.bsiv of the previous chapter.

[0630] 13. The method of any preceding claim, wherein the transforming includes generating a bitstream representation from the current video block.

[0631] 13. The method of any preceding claim, wherein the transforming comprises generating samples of the current video block from a bitstream representation.

[0632] 23. A video processing device comprising a processor configured to implement the methods according to any one or more of the preceding items.

[0633] A computer readable medium having stored thereon code that, when executed, causes a processor to implement a method according to one or more of the preceding items.

[0634] FIG. 24 is a block diagram illustrating an example video processing system 2400 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 2400. System 2400 may include an input 2402 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8 or 10-bit multi-component pixel values, or may be received in a compressed or encoded format. Input 2402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0635] The system 2400 may include an encoding component 2404 that may implement various encoding or encoding methods described in this patent specification. The encoding component 2404 may reduce the average bit rate of the video from the input 2402 to the output of the encoding component 2404, generating an encoded representation of the video. Thus, this encoding technique may be referred to as a video compression or video transcoding technique. The output of the encoding component 2404 may be stored or transmitted via a connected communication, as represented by component 2406. The bitstream (or encoded) representation of the video received at the input 2402, stored or communicated, may be used by component 2408 to generate pixel values ​​or displayable video that are transmitted to the display interface 2410. The process of generating a user-viewable video from the bitstream representation may be referred to as video decompression. Additionally, although certain video processing operations are referred to as "encoding" operations or tools, it will be understood that the encoding tools or operations are performed by an encoder and corresponding decoding tools or operations that reverse the results of the decoding, performed by a decoder.

[0636] Examples of peripheral bus interface units or display interface units may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.

[0637] 25 is a flow chart illustrating a method 2500 of video processing according to the present technology. The method 2500 includes, at operation 2510, determining to divide a current block into a plurality of sub-blocks for conversion between the current block of video and a bitstream representation of the video. At least one of the plurality of blocks is encoded using a modified Intra-Block Copy (IBC) encoding technique that uses reference samples from one or more video regions from the current picture of the current block. The method 2500 includes, at operation 2520, performing the conversion based on the determination.

[0638] In some embodiments, the image region includes a current picture, slice, tile, brick, or tile group. In some embodiments, the current block dimensions are M×N, and where M and N are integers, the current block is divided into multiple sub-blocks. In some embodiments, the multiple sub-blocks have the same size of L×K, where L and K are integers. In some embodiments, L=K. In some embodiments, L=4 or K=4.

[0639] In some embodiments, the sub-blocks have different sizes. In some embodiments, the sub-blocks have a non-rectangular shape. In some embodiments, the sub-blocks have a triangular or wedge shape.

[0640] In some embodiments, the size of at least one of the plurality of subblocks is determined based on the size of a minimum coding unit, a minimum prediction unit, a minimum transform unit, or a minimum unit for storing motion information. In some embodiments, the size of the at least one subblock is expressed as (N1×minW)×(N2×minH), where minW×minH represents the size of a minimum coding unit, a prediction unit, a transform unit, or a unit for storing motion information, and N1 and N2 are positive integers. In some embodiments, the size of at least one of the plurality of subblocks is based on a coding mode in which the current block is coded in the bitstream representation. In some embodiments, the coding mode includes at least an Intra Block Copy (IBC) merge mode or one subblock temporal motion vector prediction mode. In some embodiments, the size of at least one of the plurality of subblocks is signaled in the bitstream representation.

[0641] In some embodiments, the method includes determining to divide a subsequent block of the image into a plurality of sub-blocks for transformation, where a size of a first sub-block in the current block is different from a size of a second sub-block in the subsequent block. In some embodiments, the size of the first sub-block is different according to a size of the second sub-block, a current block dimension, and a subsequent block dimension. In some embodiments, a size of at least one sub-block of the plurality of sub-blocks is based on a color format or color component of the image. In some embodiments, the first sub-block is associated with a first color component of the image, and the second sub-block is associated with a second color component of the image, and the first sub-block and the second sub-block have different dimensions. In some embodiments, when the color format of the image is 4:2:0, the first sub-block associated with the luma component has dimensions of 2L×2K, and the second sub-block associated with the chroma component has dimensions of L×K. In some embodiments, if the color format of the image is 4:2:2, the first sub-block associated with the luma component has dimensions of 2L×2K and the second sub-block associated with the chroma component has dimensions of 2L×K. In some embodiments, the first sub-block is associated with a first color component of the image and the second sub-block is associated with a second color component of the image, and the first and second sub-blocks have the same dimensions. In some embodiments, if the color format of the image is 4:2:0 or 4:4:4, the first sub-block associated with the luma component has dimensions of 2L×2K and the second sub-block associated with the chroma component has dimensions of 2L×2K.

[0642] In some embodiments, a motion vector of a first sub-block associated with a first color component of the image is determined based on one or more sub-blocks associated with a second color component of the image. In some embodiments, the motion vector of the first sub-block is an average of the motion vectors of the one or more sub-blocks associated with the second color component. In some embodiments, the current block is partitioned into multiple sub-blocks based on a single tree partitioning structure. In some embodiments, the current block is associated with a chroma component of the image. In some embodiments, the current block has a size of 4×4.

[0643] In some embodiments, the motion information of at least one subblock of the plurality of subblocks is determined based on identifying a reference block based on an initial motion vector and determining motion information of the subblock based on the reference block. In some embodiments, the reference block is located in a current picture. In some embodiments, the reference block is located in a reference picture of one or more reference pictures. In some embodiments, at least one reference picture of the one or more reference pictures is the current picture. In some embodiments, the reference block is located in a co-located reference picture that is co-located with one of the one or more reference pictures according to the temporal information. In some embodiments, the reference picture is determined based on motion information of the matched block or a neighboring block of the matched block. According to the temporal information, the co-located block is co-located with the current block. In some embodiments, the initial motion vector of the reference block is determined based on one or more neighboring blocks of the current block or one or more neighboring blocks of the subblock. In some embodiments, the one or more neighboring blocks include neighboring and / or non-neighboring blocks of the current block or subblock. In some embodiments, at least one of the one or more neighboring blocks and the reference block are located in the same picture. In some embodiments, at least one of the one or more neighboring blocks is located in a reference picture of the one or more reference pictures. In some embodiments, at least one of the one or more neighboring blocks is located in a co-located reference picture that is co-located with one of the one or more reference pictures according to the temporal information. In some embodiments, the reference picture is determined based on motion information of the matched block or a neighboring block of the matched block, and the matched block is co-located with the current block according to the temporal information.

[0644] In some embodiments, the initial motion vector is a motion vector stored in one or more neighboring blocks. In some embodiments, the initial motion vector is determined based on an order in which one or more neighboring blocks are examined for transformation. In some embodiments, the initial motion vector is a first identified motion vector associated with the current picture. In some embodiments, the first motion vector of the reference block is determined based on a list of motion candidates. In some embodiments, the list of motion candidates includes a list of Intra-Block Copy (IBC) candidates, a list of merge candidates, a list of sub-block temporal motion vector prediction candidates, or a list of history-based motion candidates determined based on past motion prediction results. In some embodiments, the initial motion vector is determined based on a candidate selected in the list. In some embodiments, the selected candidate is the first candidate in the list. In some embodiments, the list of candidates is constructed based on a process that uses spatially neighboring blocks that are different from spatially neighboring blocks in a conventional construction process.

[0645] In some embodiments, the initial motion vector of the reference block is determined based on a position of the current block in the current picture. In some embodiments, the initial motion vector of the reference block is determined based on a dimension of the current block. In some embodiments, the initial motion vector is set equal to a default value. In some embodiments, the initial motion vector is represented at a video unit level in the bitstream representation. In some embodiments, the video unit includes a tile, a slice, a picture, a brick, a coding tree unit (CTU) row, a CTU, a coding tree block (CTB), a prediction unit (PU), or a transform unit (TU). In some embodiments, the initial motion vector of a subblock is different from another initial motion block of a second subblock of the current block. In some embodiments, the initial motion vectors of multiple subblocks of the current block are determined differently by video unit. In some embodiments, the video unit includes a block, a tile, or a slice.

[0646] In some embodiments, before identifying the reference block, the initial motion vector is converted to integer precision of F pixels, where F is a positive integer equal to or greater than 1. In some embodiments, F is 1, 2, or 4. In some embodiments, the initial motion vector is represented as (vx, vy), and the converted motion vector (vx', vy') is represented as (vx×F, vy×F). In some embodiments, the top-left position of the subblock is represented as (x, y), the subblock has a size of L×K, where L is the width of the current subblock, and K is the height of the subblock. The reference block is identified as the region that contains (x+offsetX+vx', y+offsetY+vy'), where offsetX and offset are non-negative values. In some embodiments, offsetX is 0 and / or offsetY is 0. In some embodiments, offsetX is L / 2, L / 2+1, or L / 2-1. In some embodiments, offsetY is K / 2, K / 2+1, or K / 2-1. In some embodiments, offsetX and / or offsetY are clipped to within a range that includes a picture, slice, tile, brick, or intrablock copy reference region.

[0647] In some embodiments, the motion vector of the subblock is further determined based on the motion information of the reference block. In some embodiments, the motion vector of the subblock is the same as the motion vector of the reference block when the motion vector of the reference block is directed to the current picture. In some embodiments, the motion vector of the subblock is determined based on adding a first motion vector to the motion vector of the reference block when the motion vector of the reference block is directed to the current picture. In some embodiments, the motion vector of the subblock is clipped within a range such that the motion vector of the subblock is directed to the intra block copy reference area. In some embodiments, the motion vector of the subblock is a valid motion vector of the intra block copy candidate of the subblock.

[0648] In some embodiments, one or more intra block copy candidates of the subblock are determined to determine the motion information of the subblock. In some embodiments, the one or more intra block copy candidates are added to a list of motion candidates including one of the merge candidates of the subblock, the subblock temporal motion vector prediction candidates of the subblock, or the affine merge candidates of the subblock. In some embodiments, the one or more intra block copy candidates are located before the merge candidates of the subblock in the list. In some embodiments, the one or more intra block copy candidates are located after any subblock temporal motion vector prediction candidates of the subblock in the list. In some embodiments, the one or more intra block copy candidates are located after the inherited or constructed affine candidates of the subblock in the list. In some embodiments, whether to add the one or more intra block copy candidates to the list of motion candidates is determined based on the coding mode of the current block. In some embodiments, if the current block is coded in an Intra-Block Copy (IBC) subblock temporal motion vector prediction mode, the one or more intra block copy candidates are excluded from the motion candidate list.

[0649] In some embodiments, whether one or more intra block copy candidates are added to the motion candidate list is based on a partition structure of the current block. In some embodiments, one or more intra block copy candidates are added as merge candidates of sub-blocks in the list. In some embodiments, one or more intra block copy candidates are added to the motion candidate list based on different initial motion vectors. In some embodiments, in the bitstream representation, whether one or more intra block copy candidates are added to the list of motion candidates is indicated. In some embodiments, whether an index indicating a list of motion candidates is signaled in the bitstream representation is based on the coding mode of the current block. In some embodiments, if the current block is coded in an intra block copy merge mode, an index indicating a list of motion candidates including the intra block copy merge candidates is signaled in the bitstream representation. In some embodiments, if the current block is coded in an intra block copy sub-block temporal motion vector prediction mode, an index indicating a list of motion candidates including the intra block copy sub-block temporal motion vector prediction candidates is signaled in the bitstream representation. In some embodiments, a motion vector differential for the intra block copy sub-block temporal motion vector prediction mode is applied to multiple sub-blocks.

[0650] In some embodiments, the reference block and the sub-block are associated with the same color component of the video. In some embodiments, whether to split the current block into multiple sub-blocks is based on coding characteristics associated with the current block. In some embodiments, the coding characteristics include a decoder parameter set, a sequence parameter set, a video parameter set, a picture parameter set, an APS, a picture header, a slice header, a tile group header, a Largest Coding Unit (LCU), a Coding Unit (CU), a row of LCUs, a group of LCUs, a transform unit, a prediction unit, a prediction unit block, or a syntax flag in the bitstream representation of the video coding unit. In some embodiments, the coding characteristics include a position of the coding unit, the prediction unit, the transform unit, the block, or the video coding unit. In some embodiments, the coding characteristics include a dimension of the current block or a block in the vicinity of the current block. In some embodiments, the coding characteristics include a shape of the current block or a block in the vicinity of the current block. In some embodiments, the coding characteristics include an intra coding mode of the current block or a block in the vicinity of the current block. In some embodiments, the coding characteristics include a motion vector of a block in the vicinity of the current block. In some embodiments, the coding characteristics include an indication of a color format of the video. In some embodiments, the coding characteristics include a coding tree structure for the current block. In some embodiments, the coding characteristics include a slice type, a tile group type, or a picture type associated with the current block. In some embodiments, the coding characteristics include a color component associated with the current block. In some embodiments, the coding characteristics include a temporal layer identifier associated with the current block. In some embodiments, the coding characteristics include a profile, level, or hierarchy of a standard for bitstream representation.

[0651] 26 is a flow chart illustrating a method 2600 of video processing according to the present technology. The method 2600 includes, at operation 2610, determining to divide a current block into a plurality of sub-blocks for conversion between the current block of the video and a bitstream representation of the video. Each of the plurality of sub-blocks is encoded in the encoded representation using a corresponding encoding technique according to the pattern. The method further includes, at operation 2620, performing the conversion based on the determination.

[0652] In some embodiments, the pattern specifies encoding a first sub-block of the plurality of sub-blocks using a modified Intra-Block Copy (IBC) coding technique that uses reference samples from the video domain. In some embodiments, the pattern specifies encoding a second sub-block of the plurality of sub-blocks using an intra-prediction coding technique that uses samples from the same sub-block. In some embodiments, the pattern specifies encoding a second sub-block of the plurality of sub-blocks using a palette coding technique that uses a palette of representative pixel values. In some embodiments, the pattern specifies encoding a second sub-block of the plurality of sub-blocks using an inter coding technique that uses temporal information.

[0653] In some embodiments, the pattern specifies encoding a first sub-block of the plurality of sub-blocks using an intra-prediction coding technique that uses samples from the same sub-block. In some embodiments, the pattern specifies encoding a second sub-block of the plurality of sub-blocks using a palette coding technique that uses a palette of representative pixel values. In some embodiments, the pattern specifies encoding a second sub-block of the plurality of sub-blocks using an inter coding technique that uses temporal information.

[0654] In some embodiments, the pattern specifies that all of the multiple sub-blocks are coded using a single coding technique. In some embodiments, the single coding technique includes an intra-prediction coding technique that uses samples from the same sub-block to code the multiple sub-blocks. In some embodiments, the single coding technique includes a palette coding technique that uses a palette of representative pixel values ​​to code the multiple sub-blocks.

[0655] In some embodiments, when one or more patterns of encoding techniques are applied to a current block, the history-based table of motion candidates of the sub-block temporal motion vector prediction mode remains the same as the history-based table of motion candidates determined based on motion information in past transformations. In some embodiments, the history-based table is for an IBC encoding technique or a non-IBC encoding technique.

[0656] In some embodiments, if the pattern specifies that at least one subblock of the plurality of subblocks is coded using an IBC coding technique, the one or more motion vectors for the at least one subblock are used to update a history-based table of motion candidates for an IBC subblock temporal motion vector prediction mode, where the history-based table of motion candidates is determined based on motion information in a past transform. In some embodiments, if the pattern specifies that at least one subblock of the plurality of subblocks is coded using an IBC inter coding technique, the one or more motion vectors for the at least one subblock are used to update a history-based table of motion candidates for a non-IBC subblock temporal motion vector prediction mode, where the history-based table of motion candidates is determined based on motion information in a past transform.

[0657] In some embodiments, the use of the filtering process for filtering the boundaries of the sub-blocks is based on using at least one encoding technique according to a pattern. In some embodiments, when at least one encoding technique is applied, the filtering process for filtering the boundaries of the sub-blocks is applied. In some embodiments, when at least one encoding technique is applied, the filtering process for filtering the boundaries of the sub-blocks is omitted.

[0658] In some embodiments, the second encoding technique disables a transform following a pattern for the current block. In some embodiments, the second encoding technique includes at least one of a sub-block transform encoding technique, an affine motion prediction encoding technique, a multiple reference line intra prediction encoding technique, a matrix-based intra prediction encoding technique, a symmetric Motion Vector Differentiation (MVD) encoding technique, a merge with MVD decoder-side motion derivation or refinement encoding technique, a bidirectional optimal flow encoding technique, a secondary transform encoding technique with reduced dimensions based on the dimensions of the current block, or a multiple transform set encoding technique.

[0659] In some embodiments, the use of at least one encoding technique according to the pattern is signaled in the bitstream representation. In some embodiments, the use is signaled at a sequence level, a picture level, a slice level, a tile group level, a tile level, a brick level, a coding tree unit (CTU) level, a coding tree block (CTB) level, a coding unit (CU) level, a prediction unit (PU) level, a transform unit (TU), or another video unit level. In some embodiments, the at least one encoding technique includes a modified IBC encoding technique, and the modified IBC encoding technique is indicated in the bitstream representation based on an index value indicating a candidate in a motion candidate list. In some embodiments, a current block encoded with the modified IBC encoding technique is assigned a predefined value.

[0660] In some embodiments, the use of at least one encoding technique following a pattern is determined during the transformation. In some embodiments, the use of an Intra Block Copy (IBC) encoding technique for encoding a current block in which a reference sample from the current block is used is signaled in the bitstream. In some embodiments, the use of an Intra Block Copy (IBC) encoding technique for encoding a current block in which a reference sample from the current block is used is determined during the transformation.

[0661] In some embodiments, the motion information of the sub-blocks of the current block is used as a motion vector predictor for conversion between a subsequent block of the video and a bitstream representation. In some embodiments, the motion information of the sub-blocks of the current block is not allowed to be used for conversion between a subsequent block of the video and a bitstream representation. In some embodiments, the division of the current block into a plurality of sub-blocks coded with at least one coding technique is determined based on whether motion candidates are applicable to the block of the video or sub-blocks within the block.

[0662] 27 is a flow chart illustrating a method 2700 of video processing according to the present technology. The method 2700 includes, at operation 2710, determining an operation on a list of motion candidates for conversion between a current block of video and a bitstream representation of the video based on a condition associated with a characteristic of the current block. The list of motion candidates is constructed for an encoding technique or based on information from a previously processed block of video. The method 2700 further includes, at operation 2720, performing the conversion based on the determination.

[0663] In some embodiments, the encoding technique includes a merge encoding technique, an Intra Block Copy (IBC) sub-block temporal motion vector prediction encoding technique, a sub-block merge encoding technique, an IBC encoding technique, or a modified IBC encoding technique that uses reference samples from the video domain of the current block to encode at least one sub-block of the current block.

[0664] In some embodiments, the current block has dimensions W×H, where W and H are positive integers. The condition relates to the dimensions of the current block. In some embodiments, the condition relates to encoded information of the current block or encoded information of a neighboring block of the current block. In some embodiments, the condition relates to a merge sharing condition for sharing a list of motion candidates between the current block and another block.

[0665] In some embodiments, the operations include deriving spatial merge candidates for the motion candidate list using a merge encoding technique. In some embodiments, the operations include deriving motion candidates for the motion candidate list based on spatially neighboring blocks of the current block. In some embodiments, the spatially neighboring blocks include neighboring or non-adjacent blocks of the current block.

[0666] In some embodiments, the operations include deriving motion candidates for a motion candidate list constructed based on information of previously processed video blocks. In some embodiments, the operations include deriving pair-wise merge candidates for the motion candidate list. In some embodiments, the operations include one or more pruning operations to remove redundant entries in the list of motion candidates. In some embodiments, the one or more pruning operations are on spatial merge candidates in the list of motion candidates.

[0667] In some embodiments, the operation includes updating a list of motion candidates constructed based on information of previously processed video blocks after transformation. In some embodiments, the updating includes adding the derived candidate to the list of motion candidates without performing a pruning operation to remove redundancy in the list of motion candidates. In some embodiments, the operation includes adding a default motion candidate to the list of motion candidates. In some embodiments, the default motion candidate includes a zero motion candidate using an IBC sub-block temporal motion vector predictive coding technique. In some embodiments, if the condition is met, the operation is skipped.

[0668] In some embodiments, the operation includes checking the motion candidates in the motion candidate list in a predefined order. In some embodiments, the operation includes checking a predefined number of motion candidates in the motion candidate list. In some embodiments, the condition is met if W×H is greater than or equal to a threshold. In some embodiments, the condition is met if W×H is greater than or equal to a threshold and the current block is coded using an IBC sub-block temporal motion vector predictive coding technique or a merge coding technique. In some embodiments, the threshold is 1024.

[0669] In some embodiments, the condition is met if W and / or H are greater than or equal to a threshold. In some embodiments, the threshold is 32. In some embodiments, the condition is met if W×H is less than or equal to a threshold and the current block is coded using an IBC sub-block temporal motion vector predictive coding technique or a merge coding technique. In some embodiments, the threshold is 16. In some embodiments, the threshold is 32 or 64. In some embodiments, if the condition is met, an operation including inserting the candidate determined based on spatially neighboring blocks into a list of motion candidates is skipped.

[0670] In some embodiments, the condition is met if W is equal to T2, H is equal to T3, and a neighboring block above the current block is available and is coded with the same coding technique as the current block, where T2, T3 are positive integers. In some embodiments, the condition is met when the neighboring block and the current block are in the same coding tree unit.

[0671] In some embodiments, the condition is met if W equals T2, H equals T3, and the current block's upper neighbor is not available or is outside the current coding tree unit in which the current block is located, and T2 and T3 are positive integers. In some embodiments, T2 is 4 and T3 is 8. In some embodiments, the condition is met if W equals T4, H equals T5, and the current block's left neighbor is available and coded with the same coding technique as the current block, and T4, T5 are positive integers. In some embodiments, the condition is met if W equals T4, H equals T5, and the current block's left neighbor is unavailable, and T4 and T5 are positive integers. In some embodiments, T4 is 8 and T5 is 4.

[0672] In some embodiments, the condition is met if W×H is less than or equal to the threshold, the current block is coded using an IBC sub-block temporal motion vector predictive coding technique or a merge coding technique, and the first neighboring block above the current block and the second neighboring block to the left of the current block are both coded using the same coding technique. In some embodiments, the first and second neighboring blocks are available and coded using an IBC coding technique, and the second neighboring block is in the same coding tree unit as the current block. In some embodiments, the first neighboring block is unavailable and the second neighboring block is available and is in the same coding tree unit as the current block. In some embodiments, the first and second neighboring blocks are unavailable. In some embodiments, the first neighboring block is available and the second neighboring block is unavailable. In some embodiments, the first neighboring block is unavailable and the second neighboring block is outside the coding tree unit in which the current block is located. In some embodiments, the first neighboring block is available and the second neighboring block is outside the coding tree unit in which the current block is located. In some embodiments, the threshold is 32. In some embodiments, the first and second neighboring blocks are used to derive spatial merging candidates. In some embodiments, the top-left sample of the current block is located at (x,y) and the second neighboring block covers samples located at (x-1,y+H-1). In some embodiments, the top-left sample of the current block is located at (x,y) and the second neighboring block covers samples located at (x+W-1,y-1).

[0673] In some embodiments, the same encoding technique comprises an IBC encoding technique. In some embodiments, the same encoding technique comprises an inter-coding technique. In some embodiments, the neighboring blocks of the current block have dimensions equal to A×B. In some embodiments, the neighboring blocks of the current block have dimensions greater than A×B. In some embodiments, the neighboring blocks of the current block have dimensions smaller than A×B. In some embodiments, A×B is equal to 4×4. In some embodiments, the threshold is predefined. In some embodiments, the threshold is signaled in the bitstream representation. In some embodiments, the threshold is based on encoding characteristics of the current block, the encoding characteristics comprising an encoding mode for encoding the current block.

[0674] In some embodiments, the condition is met if the current block has a parent node that shares a list of motion candidates and the current block is encoded using an IBC sub-block temporal motion vector prediction encoding technique or a merge encoding technique. In some embodiments, the condition changes adaptively according to the encoding characteristics of the current block.

[0675] 28 is a flow chart illustrating a method 2800 of video processing according to the present technology. The method 2800 includes, at operation 2810, determining to divide a current block coded with an inter-coding technique into a plurality of sub-blocks based on temporal information for conversion between the current block of the video and a bitstream representation of the video. At least one of the plurality of blocks is coded using a modified Intra Block Copy (IBC) coding technique that uses reference samples from one or more video regions from the current picture that contains the current block. The method 2800 includes, at operation 2820, performing the conversion based on the determination.

[0676] In some embodiments, the image domain includes a current picture, a slice, a tile, a brick, or a tile group. In some embodiments, the inter-coding technique includes a sub-block temporal motion vector coding technique, and the bitstream representation includes one or more syntax elements indicating whether to code the current block based on both the current picture and a reference picture different from the current picture. In some embodiments, the one or more syntax elements indicate a reference picture used to code the current block if the current block is coded based on both the current picture and the reference picture. In some embodiments, the one or more syntax elements further represent motion information associated with the reference picture, and the motion information includes at least a motion vector prediction index, a motion vector differential, or a motion vector precision. In some embodiments, the first reference picture list includes only the current picture, and the second reference picture list includes only the reference pictures. In some embodiments, the inter-coding technique includes a temporal merge coding technique, and the motion information is determined based on neighboring blocks of the current block, and the motion information includes at least a motion vector or a reference picture. In some embodiments, when determining the neighboring blocks based only on the current picture, the motion information is only applicable to the current picture. In some embodiments, when determining neighboring blocks based on both the current picture and the reference picture, the motion information is applicable to both the current picture and the reference picture. In some embodiments, the motion information is applicable to the current picture only when determining neighboring blocks based on both the current picture and the reference picture. In some embodiments, when neighboring blocks are determined based only on the reference picture, they are discarded for determining merging candidates.

[0677] In some embodiments, the reference blocks from the current picture and the reference blocks from the reference pictures are assigned fixed weighting factors, hi some embodiments, the fixed weighting factors are signaled in the bitstream representation.

[0678] In some embodiments, performing the conversion includes generating a bitstream representation based on the blocks of the video. In some embodiments, performing the conversion includes generating the blocks of the video from the bitstream representation.

[0679] It will be appreciated that techniques are disclosed for video encoding or decoding that may be employed by a video encoder or decoder to use both intra block copying and sub-block based video processing to achieve better coding efficiency and performance.

[0680] Some embodiments of the disclosed technology include determining or determining to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, an encoder uses or implements the tool or mode in processing blocks of video, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, the conversion of blocks of video to a bitstream representation of video uses the video processing tool or mode when the video processing tool or mode is enabled based on the decision or determination. In another example, when a video processing tool or mode is enabled, a decoder processes the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion of the bitstream representation of video to blocks of video is performed using the video processing tool or mode enabled based on the decision or determination.

[0681] Some embodiments of the disclosed techniques include deciding or determining to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder does not use the tool or mode in converting blocks of a video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, a decoder processes the bitstream knowing that the bitstream has not been modified using a video processing tool or mode that was enabled based on the decision or determination.

[0682] Implementations of the disclosed and other solutions, examples, embodiments, modules, and functional operations described herein, including the structures disclosed herein and their structural equivalents, may be implemented in digital electronic circuitry, or computer software, firmware, or hardware, or in one or more combinations thereof. The disclosed and other embodiments may be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for implementation by or for controlling the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that provides a machine-readable propagated signal, or one or more combinations thereof. The term "data processing apparatus" includes all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors, or computers. In addition to hardware, the apparatus may include code that creates an environment for the execution of the computer program, e.g., code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, for example a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.

[0683] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), may be stored in a single file dedicated to the program, or may be stored in multiple coordinating files (e.g., a file that stores one or more modules, subprograms, or code portions). A computer program can be deployed to run on one computer located at one site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0684] The processing and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processing and logic flows may also be performed by, and an apparatus may be implemented as, special purpose logic circuitry, such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC).

[0685] Processors suitable for executing computer programs include, for example, both general purpose and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may include one or more mass storage devices, such as magnetic, magneto-optical, or optical disks, for storing data, or may be operatively coupled to receive data from or transfer data to these mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, EPROMs, EEPROMs, flash storage devices, magnetic disks, such as internal hard disks or removable disks, magneto-optical disks, and semiconductor storage devices such as CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0686] Although this patent specification contains many details, these should not be construed as limiting the scope of any subject matter or the scope of the claims, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent document in the context of separate embodiments may be implemented in combination in one example. Conversely, various features described in the context of one example may be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0687] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desired results, nor should the separation of various system components in the examples described in this patent specification be understood as requiring such separation in all embodiments.

[0688] Only certain implementations and examples have been described; other embodiments, extensions and variations are possible based on the content described and illustrated in this patent document.

Claims

1. 1. A method of image processing, comprising: determining, for a first conversion between a first video block of a current picture of a video and a bitstream of the video, that a first prediction mode applies to the first video block; constructing a block motion list for the first video block, the block motion list having only one or more motion candidates from a history-based motion candidate list; performing the first conversion based on the configuring; and having the history-based motion candidate list includes motion candidates derived according to information from previously processed blocks of the image; 11. The method of claim 10, wherein in the first prediction mode, reference samples from the current picture of the first video block are used to predict at least a portion of the first video block.

2. The method of claim 1 , wherein the one or more motion candidates from the history-based motion candidate list are added to the block motion list without pruning.

3. the first N entries from the history-based motion candidate list that are added to the block motion list are skipped; The method of claim 1 , wherein N is a non-negative integer.

4. The last M entries from the history-based motion candidate list are skipped to be added to the block motion list; The method of claim 1 , wherein M is a non-negative integer.

5. The method of claim 1 , wherein invalid block vectors from the history-based motion candidate list are skipped for addition to the block motion list.

6. A motion candidate from the history-based motion candidate list is determined to be added to the block motion list based on a dimension of the block corresponding to the motion candidate; The method of claim 1 , wherein the motion candidate from the history-based motion candidate list is determined to be added to the block motion list depending on whether the size of the block corresponding to the motion candidate satisfies a condition.

7. the block motion list only includes the one or more motion candidates from the history-based motion candidate list in response to a size of the block corresponding to the motion candidate satisfying a condition; the block motion list includes the one or more motion candidates from the history-based motion candidate list in response to W×H not being greater than a first threshold; W denotes a width of the first video block, H denotes a height of the first video block, The method of claim 1 , wherein the first threshold is equal to 16.

8. The method further comprises: constructing a motion candidate list for a second conversion between a second video block of the video and a bitstream of the video; determining motion information for the second video block based on the motion candidate list; and determining whether to update a history-based predictor table using the motion information of the second video block based on a predetermined condition, the predetermined condition related to a dimension of the second video block; performing the second transformation based on the motion information of the second video block; the history-based predictor table includes a set of motion candidates, each associated with corresponding motion information of a previous video block; The method of claim 1 , wherein the arrangement of the motion candidates in the history-based predictor table is based on an order of addition of the motion candidates to the table.

9. the motion candidate list and the history-based predictor table are applied in a second prediction mode, the motion candidate list being a block vector candidate list for the second video block, and the set of motion candidates in the history-based predictor table having block vector candidates; 9. The method of claim 8, wherein in the second prediction mode, predictive samples of the second video block are derived from blocks of sample values ​​of the same video region that contains the second video block.

10. the history-based predictor table is updated with the motion information of the second video block in response to the dimension of the second video block being greater than a second threshold; the dimension of the second image block that is greater than a second threshold is W1×H1 that is greater than 16; 10. The method of claim 9, wherein W1 is a width of the second image block and H1 is a height of the second image block.

11. 10. The method of claim 9, wherein in the second prediction mode, constructing the block vector candidate list comprises populating block vector candidates derived from spatially neighboring blocks of the second video block.

12. a block vector candidate derived from a spatial neighbor of the second video block is inserted into the block vector candidate list in response to the dimension of the second video block being greater than a third threshold; A block vector candidate derived from a spatially neighboring block of the second video block is inserted into the block vector candidate list in response to the prediction mode of the spatially neighboring block being the same as the second prediction mode; In the second prediction mode, one or more pruning operations are performed when inserting the block vector candidate into the block vector candidate list, and a maximum number of pruning operations is reduced or set to zero in response to the dimension of the second video block not being greater than a fourth threshold; or 12. The method of claim 11, wherein the block vector candidate list includes only motion candidates derived from the history-based predictor table and / or default candidates in response to the dimension of the second video block not being greater than a fifth threshold.

13. the motion candidate list and the history-based predictor table are applied in a third prediction mode; in the third prediction mode, prediction samples of a second video block are derived from blocks of sample values ​​of a decoded picture other than the picture that includes the second video block; the history-based predictor table is updated with the motion information of the second video block in response to satisfying a predetermined condition; the predetermined condition relates to a relationship between the second video block and a threshold value included in the bitstream; The method of claim 1 , wherein pruning operations among motion candidates from the history-based predictor table and other motion candidates that were present in the motion candidate list are reduced.

14. The method of claim 1 , wherein the first transformation comprises encoding the first video block into the bitstream.

15. The method of claim 1 , wherein the first transformation comprises decoding the first video block from the bitstream.

16. 1. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions, comprising: The instructions, when executed by the processor, cause the processor to: determining, for a first conversion between a first video block of a current picture of a video and a bitstream of the video, that a first prediction mode applies to the first video block; constructing a block motion list for the first video block, the block motion list having only one or more motion candidates from a history-based motion candidate list; performing the first conversion based on the configuring; and Run the command, the history-based motion candidate list includes motion candidates derived according to information from previously processed blocks of the image; 11. The apparatus, wherein in the first prediction mode, reference samples from the current picture of the first video block are used to predict at least a portion of the first video block.

17. the one or more motion candidates from the history-based motion candidate list are added to the block motion list without pruning; the first N entries from the history-based motion candidate list that are added to the block motion list are skipped, N being a non-negative integer; the last M entries from the history-based motion candidate list that are added to the block motion list are skipped, where M is a non-negative integer; The apparatus of claim 16 , wherein invalid block vectors from the history-based motion candidate list are skipped when added to the block motion list.

18. The processor: determining, for a first conversion between a first video block of a current picture of a video and a bitstream of the video, that a first prediction mode applies to the first video block; constructing a block motion list for the first video block, the block motion list having only one or more motion candidates from a history-based motion candidate list; performing the first conversion based on the configuring; and Run the command, the history-based motion candidate list includes motion candidates derived according to information from previously processed blocks of the image; 11. A non-transitory computer-readable storage medium storing instructions, wherein in the first prediction mode, reference samples from the current picture of the first video block are used to predict at least a portion of the first video block.

19. the one or more motion candidates from the history-based motion candidate list are added to the block motion list without pruning; the first N entries from the history-based motion candidate list that are added to the block motion list are skipped, N being a non-negative integer; the last M entries from the history-based motion candidate list that are added to the block motion list are skipped, where M is a non-negative integer; 20. The non-transitory computer-readable medium of claim 18, wherein invalid block vectors from the history-based motion candidate list that are added to the block motion list are skipped.

20. A method for storing a video bitstream, comprising the steps of: The method comprises: determining, for a first conversion between a first video block of a current picture of a video and a bitstream of the video, that a first prediction mode applies to the first video block; constructing a block motion list for the first video block, the block motion list having only one or more motion candidates from a history-based motion candidate list; performing the first conversion based on the configuring; and storing the bitstream on a non-transitory computer readable storage medium; having the history-based motion candidate list includes motion candidates derived according to information from previously processed blocks of the image; 11. The method of claim 10, wherein in the first prediction mode, reference samples from the current picture of the first video block are used to predict at least a portion of the first video block.

Images