Recursive partitioning of video coding blocks

Abstract

A method of video processing includes, for a conversion between a current block in a video region of a video and a coded representation of the video, determining that the current block is allowed to be divided into a plurality of transform units based on characteristics of the current block. Partitioned signaling notifications are omitted in encoding and decoding representations. The method also includes performing a conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a divisional application of the invention patent application with application date of September 9, 2020, application number 202080063234.4, and invention name “Recursive partitioning of video codec blocks”. Technical Field

[0003] This patent document relates to encoding and decoding of videos and images. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth required for digital video usage is expected to continue to grow. Summary of the invention

[0005] The invention relates to an apparatus, system and method related to digital video coding and decoding, in particular to an apparatus, system and method related to the coding and decoding of videos and images, wherein an intra-frame block copy tool is used for coding or decoding.

[0006] In one example aspect, a video processing method is disclosed. The method includes: for a conversion between a current block in a video region of a video and a codec representation of the video, determining that the current block is allowed to be divided into a plurality of transform units based on characteristics of the current block. Signaling of the division is omitted in the codec representation. The method also includes performing the conversion based on the determination.

[0007] In another example aspect, a video processing method is disclosed. The method includes: for a conversion between a current block in a video region of a video and a codec representation of the video, using an intra-block copy (IBC) model to determine prediction samples of the current block based on filtered reconstructed reference samples in the video region. The method also includes performing the conversion based on the determination.

[0008] In another example aspect, a video processing method is disclosed. The method includes: for a conversion between a current block of a video and a codec representation of the video, determining according to a rule whether a syntax element indicating a skip mode using an intra-block copy (IBC) codec model is included in the codec representation. The rule specifies that the signaling notification of the syntax element is based on the dimension of the current block and / or based on the maximum allowed dimension of a block encoded using the IBC codec model. The method also includes performing the conversion based on the determination.

[0009] In another example aspect, a video processing method is disclosed. The method includes: for the conversion between a current block of a video and the codec representation of the video, determining at least one context for encoding and decoding an index associated with a low-frequency non-separable transform (LFNST) codec model. The LFNST codec model includes: during encoding, applying a forward quadratic transform between a forward main transform and a quantization step; or during decoding, applying an inverse quadratic transform between a dequantization step and an inverse main transform. The dimensions of the forward quadratic transform and the inverse quadratic transform are smaller than the dimensions of the current block. Determining the at least one context based on the segmentation type of the current block, without considering the forward main transform or the inverse main transform. The method further includes performing the conversion according to the determination.

[0010] In another example aspect, a video processing method is disclosed. The method includes: for the conversion between a current block of a video region of a video and the codec representation of the video, determining whether to enable an intra block copy (IBC) codec model based on the maximum transform unit size applicable to the video region. The method further includes performing the conversion according to the determination.

[0011] In another example aspect, a video processing method is disclosed. The method includes: for the conversion between a current block of a video and the codec representation of the video, determining that the absolute value of a component of the motion vector of the current block is divided into two parts, where the motion vector is represented as (Vx, Vy), the component is represented as Vi, and Vi is Vx or Vy. The first of the two parts is equal to |Vi| - ((|Vi| >> N) << N), the second of the two parts is equal to |Vi| >> N, and N is a positive integer. The two parts are encoded and decoded separately in the codec representation. The method further includes performing the conversion according to the determination.

[0012] In another example aspect, a video processing method is disclosed. The method includes: for the conversion between a current block of a video and the codec representation of the video, determining information about the maximum dimension of the current block that allows sub-block transformation in the current block based on the maximum allowed dimension of the transform block. The method further includes performing the conversion according to the determination.

[0013] In another example aspect, a video processing method is disclosed. The method includes: for the conversion between the codec representation of a video block in a video region and the video block based on an intra block copy tool, determining that the video block is allowed to be divided into a plurality of transform units, where the determination is based on the codec conditions of the video block, and the codec representation omits signaling of the division; and performing the conversion based on the division.

[0014] In another example aspect, a video processing method is disclosed. The method includes: for conversion between a codec representation of a video block of a video region and the video block, based on a maximum transform size of the video region, determining whether an intra-block copy (IBC) tool is enabled for conversion of the video block; and performing the conversion based on the determination.

[0015] In another example aspect, a video processing method is disclosed. The method includes: for a conversion between a codec representation of a video block of a video region and the video block, determining whether signaling of an intra-block copy (IBC) tool for the conversion is included in the codec representation; and performing the conversion based on the determination, wherein the determination is based on a width and / or height of the video block and a maximum allowed IBC block size of the video region.

[0016] In another example aspect, a video processing method is disclosed. The method includes: for conversion between a codec representation of a video block of a video region and the video block using an intra-block copy (IBC) tool, determining that partitioning the video block into a plurality of transform units (TUs) is permitted for conversion; and performing the conversion based on the determination, wherein the conversion includes using separate motion information for the plurality of TUs.

[0017] In another example aspect, a video processing method is disclosed. The method includes: for conversion between a codec representation of a video block of a video region and the video block, determining that an intra block copy tool is enabled for the conversion; and performing the conversion using the intra block copy tool, wherein prediction of the video block is performed using filtered reconstructed samples of the video region.

[0018] In another example aspect, a video processing method is disclosed. The method includes performing conversion between a video including a plurality of video blocks and a codec representation of the video, wherein at least some of the video blocks are coded using motion vector information, and wherein the motion vector information is represented in the codec representation as a first portion based on a first less significant bit of an absolute value of the motion vector information, and the motion vector information is represented in the codec representation as a second portion based on remaining more significant bits that are more significant than the first less significant bit.

[0019] In another example aspect, a video processing method is disclosed. The method includes: for conversion between a codec representation of a video block of a video region and the video block, determining whether a sub-block transform tool is enabled for the conversion; and performing the conversion based on the determination, wherein the determination is based on a maximum allowed transform block size of the video region, and wherein signaling is conditionally included in the codec representation based on the maximum allowed transform block size.

[0020] In another example aspect, a video processing method is disclosed. The method includes: for a conversion between a codec representation of a video block of a video region and the video block, determining whether to use a low frequency non-separable transform (LFNST) during the conversion; and performing the conversion based on the determination, wherein the determination is based on a codec condition applied to the video block, and wherein the LFNST and a matrix index of the LFNST are encoded and decoded in the codec representation using two contexts.

[0021] In yet another representative aspect, the above method is implemented in the form of processor executable code and stored in a computer readable program medium.

[0022] In yet another representative aspect, an apparatus configured or operable to perform the above method is disclosed. The apparatus may include a processor programmed to implement the method.

[0023] In yet another representative aspect, a video decoder device may implement a method as described herein.

[0024] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, description, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 An example derivation process for Merge candidate list construction is shown.

[0026] Figure 2 Example locations of spatial merge candidates are shown.

[0027] Figure 3 Example candidate pairs considered for redundancy checking of spatial merge candidates are shown.

[0028] Figure 4 Example locations of the second PU for Nx2N and 2NxN partitions are shown.

[0029] Figure 5 is an illustration of motion vector scaling for temporal merge candidates.

[0030] Figure 6 Examples of candidate positions C0 and C1 of the time-domain Merge candidate are shown.

[0031] Figure 7 An example of combining bi-predictive Merge candidates is shown.

[0032] Figure 8 An example derivation process of motion vector prediction candidates is shown.

[0033] Fig. 9 is an illustration of motion vector scaling for spatial motion vector candidates.

[0034] Fig.10 Examples of candidate positions for the Affine Merge mode are shown.

[0035] Fig.11 The modified Merge list building process is shown.

[0036] Fig.12 An example of inter prediction based on triangular partitioning is shown.

[0037] Fig.13 An example of a CU to which the first weighting factor group is applied is shown.

[0038] Fig.14 An example of motion vector storage is shown.

[0039] Fig.15 is an example of a UMVE search process.

[0040] Fig.16 An example of a UMVE search point is shown.

[0041] Fig.17 is a diagrammatic illustration of the operation of the Intra Block Copy tool.

[0042] Fig.18 An example of a quadratic transform in JEM is shown.

[0043] Fig.19 An example of a Reduced Secondary Transform (RST) is shown.

[0044] Fig. 20 is a block diagram of an example video processing system in which the disclosed technology may be implemented.

[0045] Fig.21 is a block diagram of an example video processing device.

[0046] Fig. 22 is a flow chart of an example method of video processing.

[0047] Fig.23 is a flow chart representation of a video processing method according to the present technology.

[0048] Fig.24 is a flowchart representation of another video processing method according to the present technology.

[0049] Fig.25 is a flowchart representation of another video processing method according to the present technology.

[0050] Fig.26 is a flowchart representation of another video processing method according to the present technology.

[0051] Fig. 27 is a flowchart representation of another video processing method according to the present technology.

[0052] Fig.28 is a flowchart representation of another video processing method according to the present technology.

[0053] Fig.29 is a flowchart representation of yet another video processing method according to the present technology. DETAILED DESCRIPTION

[0054] Embodiments of the disclosed technology can be applied to existing video codec standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve the readability of the description and do not in any way limit the discussion or embodiments (and / or implementations) to only the corresponding section.

[0055] 1. Overview

[0056] The present application relates to video coding technology. Specifically, the present application relates to intra-block copy (IBC, also known as current picture reference (CPR)) coding. It can be applied to existing video coding standards such as HEVC, or to a pending standard (universal video codec). It may also be applicable to future video coding standards or video codecs.

[0057] 2. Background

[0058] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Vision, and the two organizations jointly produced the H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Codec (AVC), and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that uses temporal prediction plus transform codecs. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into reference software called the Joint Exploration Model (JEM). In April 2018, a Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard targeting a 50% bitrate reduction compared to HEVC.

[0059] 2.1 Inter-frame prediction in HEVC / H.265

[0060] For inter-coded codec units (CUs), they can be coded using one prediction unit (PU) or two PUs depending on the partitioning mode. Each inter-predicted PU has motion parameters for one or two reference picture lists. Motion parameters include motion vectors and reference picture indexes. The use of one of the two reference picture lists can also be signaled using inter_pred_idc. Motion vectors can be explicitly coded as deltas relative to the predictor.

[0061] When a CU is coded in skip mode, a PU is associated with the CU and there are no significant residual coefficients, no coded motion vector increments or reference picture indices. A Merge mode is specified, by which the motion parameters of the current PU can be obtained from neighboring PUs (including spatial and temporal candidates). Merge mode can be applied to any inter-predicted PU, not just skip mode. An alternative to Merge mode is the explicit transmission of motion parameters, where the motion vector (more precisely, the motion vector difference (MVD) compared to the motion vector predictor), the corresponding reference picture index of each reference picture list, and the use of the reference picture list are explicitly signaled per PU. In the present disclosure, such a mode is named Advanced Motion Vector Prediction (AMVP).

[0062] When signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of samples. This is called "unidirectional prediction". Unidirectional prediction is available for both P slices and B slices.

[0063] When signaling indicates that two reference picture lists are to be used, a PU is generated from two blocks of samples. This is called "bi-prediction". Bi-prediction is only available for B slices.

[0064] The following provides details about the inter prediction modes specified in HEVC. The description will start with the Merge mode.

[0065] 2.1.1 Reference Image List

[0066] In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. As in H.264 / AVC, a picture can be predicted from multiple reference pictures. Reference pictures used for inter prediction are organized in one or more reference picture lists. A reference index identifies which reference pictures in the list should be used to create the prediction signal.

[0067] A single reference picture list (list 0) is used for P slices, and two reference picture lists (list 0 and list 1) are used for B slices. Note that the reference pictures contained in lists 0 / 1 can be pictures from the past and the future in terms of capture / display order.

[0068] 2.1.2Merge Mode

[0069] 2.1.2.1 Derivation of Merge Mode Candidates

[0070] When Merge mode is used to predict a PU, an index pointing to an entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized in the following sequence of steps:

[0071] Step 1: Initial candidate derivation

[0072] oStep 1.1: Spatial Candidate Derivation

[0073] oStep 1.2: Redundancy check of spatial domain candidates

[0074] oStep 1.3: Time Domain Candidate Derivation

[0075] Step 2: Additional candidate insertions

[0076] oStep 2.1: Create bidirectional prediction candidates

[0077] oStep 2.2: Insert zero motion candidates

[0078] exist Figure 1 These steps are also schematically illustrated in , which shows an example derivation process for Merge candidate list reconstruction. For spatial Merge candidate derivation, at most four Merge candidates are selected from candidates located at five different positions. For temporal Merge candidate derivation, at most one Merge candidate is selected from two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates signaled in the slice header (maxNumMergeCand). Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of a 2N×2N prediction unit.

[0079] Hereinafter, operations associated with the aforementioned steps are described in detail.

[0080] 2.1.2.2 Spatial Candidate Derivation

[0081] In the derivation of spatial merge candidates, Figure 2 A maximum of four Merge candidates are selected from the candidates at the positions shown. The derivation order is A1, B1, B0, A0 and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is unavailable (for example, because it belongs to another slice or piece) or is intra-coded. After adding the candidate at position A1, a redundancy check is performed on the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce the complexity of the calculation, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only those with Figure 3 Only pairs linked by arrows in are considered, and candidates are added to the list only if the corresponding candidates for redundancy checking do not have the same motion information. Another source of duplicate motion information is the "second PU" associated with a different partition of 2Nx2N. For example, Figure 4 The second PU is described for the N×2N and 2N×N cases, respectively. When the current PU is partitioned into N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate may result in two prediction units with the same motion information, which is redundant for having only one PU in the codec unit. Similarly, position B1 is not considered when the current PU is partitioned into 2N×N.

[0082] 2.1.2.3 Time Domain Candidate Derivation

[0083] In this step, only one candidate is added to the list. In particular, in the derivation of this time-domain Merge candidate, the scaled motion vector is derived based on the co-located PU in the co-located picture. Figure 5 is a diagram of motion vector scaling for temporal Merge candidates. Figure 5 The scaled motion vector of the temporal Merge candidate is obtained as shown by the dashed line, which is scaled from the motion vector of the co-located PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC specification [1]. For the B slice, two motion vectors are obtained (one for reference picture list 0 and the other for reference picture list 1) and combined to make a bidirectional prediction Merge candidate.

[0084] 2.1.2.4 Co-located Pictures and Co-located PUs

[0085] When TMVP is enabled (eg, slice_temporal_mvp_enabled_flag is equal to 1), the variable ColPic representing the collocated picture is derived as follows:

[0086] – If the current slice is a B slice and the signaled collocated_from_l0_flag is equal to 0, ColPic is set equal to RefPicList1[collocated_ref_idx].

[0087] – Otherwise (slice_type is equal to B and collocated_from_l0_flag is equal to 1, or slice_type is equal to P), set ColPic equal to RefPicList0[collocated_ref_idx].

[0088] Here, collocated_ref_idx and collocated_from_l0_flag are two syntax elements, which may be signaled in the slice header.

[0089] In the co-located PU (Y) belonging to the reference frame, the position of the temporal candidate is selected between candidates C0 and C1, such as Figure 6 As shown. If the PU at position C0 is not available, is intra-coded, or is outside the current codec tree unit (CTU also known as LCU, the largest codec unit) row, position C1 is used. Otherwise, position C0 is used for the derivation of the time domain Merge candidate.

[0090] The relevant syntax elements are described as follows:

[0091] 7.3.6.1 Generic Stripe Segment Header Syntax

[0092]

[0093] 2.1.2.5 Derivation of MVs of TMVP Candidates

[0094] In some embodiments, the following steps are performed to derive TMVP candidates:

[0095] 1) Set reference picture list X = 0 and the target reference picture is the reference picture with index equal to 0 in list X (e.g., curr_ref). Call the derivation process of the concatenated motion vector to obtain the MV of list X pointing to curr_ref.

[0096] 2) If the current slice is a B slice, set reference picture list X = 1 and the target reference picture is the reference picture (eg, curr_ref) with index equal to 0 in list X. Call the derivation process of the concatenated motion vector to obtain the MV of list X pointing to curr_ref.

[0097] The derivation process of the collocated motion vector is described in the next subsection 2.1.2.5.1.

[0098] 2.1.2.5.1 Derivation process of concatenated motion vector

[0099] For the co-located blocks, unidirectional prediction or bidirectional prediction can be used for intra-frame or inter-frame coding and decoding. If it is intra-frame coding and decoding, the TMVP candidate is set to be unavailable.

[0100] If it is a uni-directional prediction from list A, the motion vector of list A is scaled to the target reference picture list X.

[0101] If it is bi-prediction and the target reference picture list is X, the motion vectors of list A are scaled to the target reference picture list X, and A is determined according to the following rule:

[0102] – If no reference picture has a larger POC value than the current picture, set A equal to X.

[0103] – Otherwise, set A equal to collocated_from_l0_flag.

[0104] 2.1.2.6 Additional Candidate Insertion

[0105] In addition to spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bi-prediction Merge candidates and zero Merge candidates. Combined bi-prediction Merge candidates are generated using spatial and temporal Merge candidates. Combined bi-prediction Merge candidates are only used for B slices. A combined bi-prediction candidate is generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another candidate. If the two tuples provide different motion hypotheses, they will form a new bi-prediction candidate. Figure 7 An example of combining bidirectional prediction Merge candidates is shown. As an example, Figure 7 The following scenario is shown where two candidates with mvL0 and refIdxL0 or with mvL1 and refIdxL1 in the original list (on the left) are used to create a combined bi-predictive Merge candidate that is added to the final list (on the right). There are many rules about the combinations that are considered to generate these additional Merge candidates.

[0106] Zero-motion candidates are inserted to fill the remaining entries in the Merge candidate list to the capacity of MaxNumMergeCand. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero-motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.

[0107] 2.1.3 Advanced Motion Vector Prediction (AMVP)

[0108] AMVP exploits the spatio-temporal correlation of motion vectors with neighboring PUs for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is first constructed by checking the availability of the temporally adjacent PU position to the upper left, removing redundant candidates, and adding zero vectors to make the candidate list length constant. The encoder can then select the best predictor from the candidate list and send the corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary. In this case, the maximum value to be encoded is 2 (see Figure 8 ). In the following sections, details about the derivation process of motion vector prediction candidates are provided.

[0109] 2.1.3.1 Derivation of AMVP Candidates

[0110] Figure 8 The derivation process of motion vector prediction candidates is summarized.

[0111] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, based on the Figure 2 The motion vectors of each PU at the five different positions shown ultimately lead to two motion vector candidates.

[0112] For the derivation of temporal motion vector candidates, one motion vector candidate is selected from two candidates that are derived based on two different co-located positions. After making the first list of spatio-temporal candidates, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates with reference picture index greater than 1 in the associated reference picture list are removed from the list. If the number of spatio-temporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0113] 2.1.3.2 Spatial Motion Vector Candidates

[0114] When deriving spatial motion vector candidates, at most two candidates are considered among five potential candidates, which are from the Figure 2The PUs at the positions shown are the same as the positions of the motion merge. The derivation order on the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order on the upper side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, there are four cases on each side that can be used as motion vector candidates, two of which do not require spatial scaling, and two use spatial scaling. The four different cases are summarized as follows:

[0115] No airspace scaling

[0116] –(1) Same reference picture list and same reference picture index (same POC)

[0117] – (2) Different reference picture lists, but same reference pictures (same POC)

[0118] Airspace scaling

[0119] –(3) Same reference picture list, but different reference pictures (different POC)

[0120] – (4) Different reference picture lists, and different reference pictures (different POC)

[0121] First check the case without spatial scaling, then check the case where spatial scaling is allowed. Spatial scaling is considered whenever the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, regardless of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, then scaling of the above motion vectors is allowed to help parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

[0122] In the spatial domain scaling process, the motion vectors of neighboring PUs are scaled in a similar way to the temporal domain scaling, such as Fig. 9 The main difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the time domain scaling process.

[0123] 2.1.3.3 Temporal Motion Vector Candidates

[0124] Except for the derivation of the reference picture index, all derivation processes of the temporal Merge candidate are the same as those of the spatial motion vector candidate (see Figure 6 ). The reference picture index is signaled to the decoder.

[0125] 2.2 Inter-frame prediction method in VVC

[0126] There are several new codec tools for improved inter-frame prediction, such as Adaptive Motion Vector Difference Resolution (AMVR) for signaling MVD, Motion Vector Difference Merge (MMVD), Triangular Prediction Mode (TPM), Combined Intra-Frame Inter-Frame Prediction (CIIP), Advanced TMVP (ATMVP, also known as SbTMVP), affine prediction mode, General Bidirectional Prediction (GBI), decoder-side motion vector refinement (DMVR), and Bidirectional Optical Flow (BIO, also known as BDOF).

[0127] VVC supports two different Merge list construction processes:

[0128] 1) Sub-block Merge candidate list: It includes ATMVP and affine Merge candidates. Both affine mode and ATMVP mode share a Merge list construction process. Here, ATMVP and affine Merge candidates can be added in sequence. The sub-block Merge list size is signaled in the stripe header, and the maximum value is 5.

[0129] 2) Regular Merge List: For inter-frame codec blocks, a Merge list construction process is shared. Here, spatial / temporal Merge candidates, HMVP, paired Merge candidates, and zero motion candidates can be inserted in sequence. The regular Merge list size is signaled in the slice header, and the maximum value is 6. MMVD, TPM, CIIP depend on the regular Merge list.

[0130] Similarly, three AMVP lists are supported in VVC:

[0131] 1) Affine AMVP candidate list

[0132] 2) Regular AMVP Candidate List

[0133] 2.2.1 Codec Block Structure in VVC

[0134] In VVC, a quadtree / binary tree / ternary tree (QT / BT / TT) structure is used to divide a picture into square or rectangular blocks.

[0135] In addition to QT / BT / TT, VVC also uses a separate tree (also known as a dual codec tree) for I frames. Using the separate tree, the codec block structure is signaled separately for the luminance and chrominance components.

[0136] In addition, CU is set equal to PU and TU, except for blocks encoded using a few specific coding methods (such as intra sub-partitioning prediction where PU is equal to TU but smaller than CU, and sub-block transform of inter-coded blocks where PU is equal to CU but TU is smaller than PU).

[0137] 2.2.2 MERGE of the entire block

[0138] 2.2.2.1 Translate the Merge List Construction of the Regular Merge Mode

[0139] 2.2.2.1.1 History-based Motion Vector Prediction (HMVP)

[0140] Different from the Merge list design, in VVC, a history-based motion vector prediction (HMVP) method is adopted.

[0141] In HMVP, the motion information of the previous codec is stored. The motion information of the previously coded block is defined as the HMVP candidate. Multiple HMVP candidates are stored in a table called HMVP table, and this table is maintained during the real-time encoding / decoding process. When a new slice / LCU row / strip is started to be encoded / decoded, the HMVP table is cleared. Whenever there is an inter-coded block and non-sub-block, non-TPM mode, the associated motion information is added as a new HMVP candidate to the last entry of the table. Fig.10 The overall encoding and decoding process is depicted in .

[0142] 2.2.2.1.2 Conventional Merge List Construction Process

[0143] The construction of a regular Merge list (for translational motion) can be summarized as the following sequence of steps:

[0144] Step 1: Derive spatial candidates

[0145] Step 2: Insert HMVP candidates

[0146] Step 3: Insert pairwise average candidates

[0147] Step 4: Default motion candidates

[0148] HMVP candidates can be used in both AMVP and Merge candidate list construction processes. Fig.11 A modified Merge candidate list construction process is depicted (highlighted in blue). When the Merge candidate list is not full after inserting the TMVP candidate, the HMVP candidates stored in the HMVP table can be used to fill the Merge candidate list. Considering that a block usually has a higher correlation with the nearest neighboring block in terms of motion information, the HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first, and the first entry is added last. Similarly, redundancy removal is applied to the HMVP candidates. Once the total number of available Merge candidates reaches the maximum number of Merge candidates allowed for signaling, the Merge candidate list construction process is terminated.

[0149] It should be noted that all spatial / temporal / HMVP candidates should be coded and decoded in non-IBC mode. Otherwise, they are not allowed to be added to the regular Merge candidate list.

[0150] The HMVP table contains up to 5 regular motion candidates, and each candidate is unique.

[0151] 2.2.2.1.2.1 Pruning

[0152] Candidates are added to the list only if the corresponding candidates for redundancy checking do not have the same motion information. This comparison process is called pruning.

[0153] The pruning process between spatial candidates depends on the usage of the TPM of the current block.

[0154] When the current block is not encoded in a TPM mode (eg, conventional Merge, MMVD, CIIP), the HEVC pruning process (eg, five prunings) for spatial Merge candidates is utilized.

[0155] 2.2.3 Triangular Prediction Model (TPM)

[0156] In VVC, inter prediction in triangular partitioning mode is supported. Triangular partitioning mode is only applicable to CUs that are 8x8 or larger and are encoded and decoded in Merge mode rather than MMVD or CIIP mode. For CUs that meet these conditions, a CU-level flag is signaled to indicate whether triangular partitioning mode is applied.

[0157] like Fig.11 As depicted, when this mode is used, the CU is evenly divided into two triangular partitions using diagonal or anti-diagonal partitions. Each triangular partition in the CU is inter-predicted using its own motion; each partition only allows unidirectional prediction, i.e., each partition has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure the same as regular bidirectional prediction: only two motion compensated predictions are required for each CU.

[0158] Fig.12 An example of inter prediction based on triangular partitioning is shown.

[0159] If the CU level flag indicates that the current CU is encoded using the triangle partitioning mode, a flag indicating the direction of the triangle partitioning (diagonal or anti-diagonal) and two Merge indices (one for each partition) are also signaled. After predicting each triangle partition, a hybrid process with adaptive weights is used to adjust the sample values ​​along the diagonal or anti-diagonal edges. This is the prediction signal for the entire CU, and like other prediction modes, transform and quantization processes are applied to the entire CU. Finally, the CU motion field predicted using the triangle partitioning mode is stored in 4x4 units.

[0160] The regular Merge candidate list is used again for triangular partition Merge prediction without additional motion vector pruning. For each Merge candidate in the regular Merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangular prediction. In addition, the order of L0 and L1 motion vectors is selected based on their Merge index parity. With this scheme, the regular Merge list can be used directly.

[0161] 2.2.3.1TPM Merge List Construction Process

[0162] Basically, the regular Merge list construction process can be added with some modifications.

[0163] Specifically, the following applies:

[0164] 1) How to perform pruning depends on the TPM usage of the current block

[0165] – If the current block is not encoded using TPM, call HEVC 5 pruning applied to the spatial merge candidate

[0166] – Otherwise (if the current block is encoded using TPM), full pruning is applied when adding new spatial merge candidates. That is, B1 is compared with A1; B0 is compared with A1 and B1; A0 is compared with A1, B1 and B0; B2 is compared with A1, B1, A0 and B0.

[0167] 2) The condition on whether to check the motion information from B2 depends on the TPM usage of the current block

[0168] – If the current block is not decoded using TPM, access and check B2 only if there are less than 4 spatial merge candidates before checking B2.

[0169] – Otherwise (if the current block is encoded using TPM), B2 is always accessed and checked before adding it, no matter how many available spatial merge candidates there are.

[0170] 2.2.3.2 Adaptive weighted processing

[0171] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangular prediction units to derive the final prediction for the entire CU. Two weighting factor groups are defined below:

[0172] The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luma and chroma samples respectively;

[0173] The second weighting factor group: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luma and chroma samples respectively.

[0174] The weighting factor set is selected based on comparing the motion vectors of the two triangular prediction units. The second weighting factor set is used when any of the following conditions is true:

[0175] – The reference pictures of the two triangular prediction units are different.

[0176] – The absolute value of the difference between the horizontal values ​​of two motion vectors is greater than 16 pixels.

[0177] – The absolute value of the difference between the vertical values ​​of two motion vectors is greater than 16 pixels.

[0178] Otherwise, the first set of weighting factors is used. Fig.13 An example is shown.

[0179] 2.2.3.3 Motion Vector Storage

[0180] The motion vector of the triangle prediction unit ( Fig.14 Mv1 and Mv2 in CU are stored in a 4×4 grid. For each 4×4 grid, a unidirectional prediction or bidirectional prediction motion vector is stored, depending on the position of the 4×4 grid in the CU. Fig.14 As shown, the unidirectional prediction motion vector (Mv1 or Mv2) is stored in a 4×4 grid located in the unweighted area (i.e., not located at the diagonal edge). On the other hand, the bidirectional prediction motion vector is stored in a 4×4 grid located in the weighted area. The bidirectional prediction motion vector is derived from Mv1 and Mv2 according to the following rules:

[0181] 1) In the case where Mv1 and Mv2 have motion vectors from different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector.

[0182] 2) In the case where both Mv1 and Mv2 come from the same L0 (or L1) direction,

[0183] – If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, Mv2 is scaled to that picture. Mv1 and scaled Mv2 are combined to form a bidirectional prediction motion vector.

[0184] – If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, then Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bidirectional prediction motion vector.

[0185] – Otherwise, store Mv1 only in the weighted region.

[0186] 2.2.4 Merge with Motion Vector Difference (MMVD)

[0187] In some embodiments, the final motion vector representation (UMVE, also called MMVD) and the proposed motion vector representation method are used in Skip or Merge mode.

[0188] UMVE reuses the same Merge candidates as those included in the regular Merge candidate list in VVC. Among the Merge candidates, a base candidate can be selected and further extended by the proposed motion vector expression method.

[0189] UMVE provides a new method for representing motion vector difference (MVD), in which the MVD is represented by the starting point, motion amplitude and motion direction.

[0190] Fig.15 is an example of a UMVE search process.

[0191] Fig.16 An example of a UMVE search point is shown.

[0192] This technique uses the Merge candidate list as is. However, only candidates of the default Merge type (MRG_TYPE_DEFAULT_N) are considered for UMVE expansion.

[0193] The base candidate index defines the starting point. The base candidate index indicates the best candidate among the candidates in the list, as shown below.

[0194] Table 4 Basic candidate IDX

[0195] Base candidate IDX 0 1 2 3 The Nth MVP The First MVP Second MVP The third MVP The fourth MVP

[0196] If the number of base candidates is equal to 1, the base candidate IDX is not signaled.

[0197] The distance index is the motion amplitude information. The distance index indicates the predefined distance from the start point information. The predefined distances are as follows:

[0198] Table 5 Distance IDX

[0199] Distance from IDX 0 1 2 3 4 5 6 7 Pixel distance 1 / 4 pixel 1 / 2 pixel 1 pixel 2 pixels 4 pixels 8 pixels 16 pixels 32 pixels

[0200] The direction index indicates the direction of the MVD relative to the starting point. The direction index can indicate four directions as shown below.

[0201] Table 6 Direction IDX

[0202] Directions to IDX 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0203] The UMVE flag is signaled immediately after the Skip flag or Merge flag is sent. If the Skip or Merge flag is true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. However, if not 1, the AFFINE flag is parsed. If the AFFINE flag is equal to 1, it is AFFINE mode, but if not 1, the Skip / Merge index of the Skip / Merge mode of the VTM is parsed.

[0204] Additional line buffers due to UMVE candidates are not needed, since software's Skip / Merge candidates are used directly as base candidates. By using the input UMVE index, the complement of the MV is determined directly before motion compensation. No long line buffers need to be kept for this.

[0205] Under the current common test conditions, the first or second Merge candidate in the Merge candidate list can be selected as the basic candidate.

[0206] UMVE is also called Merge with MV Difference (MMVD).

[0207] 2.2.5 MERGE based on sub-block technology

[0208] In some embodiments, all sub-block related motion candidates are put into a separate Merge list in addition to the regular Merge list of non-sub-block Merge candidates.

[0209] The sub-block related motion candidates are put into a separate Merge list named "Sub-block Merge Candidate List".

[0210] In one example, the sub-block Merge candidate list includes ATMVP candidates and affine Merge candidates.

[0211] Fill candidates in the sub-block Merge candidate list in the following order:

[0212] a. ATMVP candidate (may be available or not);

[0213] b. Affine Merge list (including inherited affine candidates and constructed affine candidates);

[0214] c. Filled with zeros MV 4-parameter affine model

[0215] 2.2.5.1 Advanced Temporal Motion Vector Predictor (ATMVP) (also known as Sub-Block Temporal Motion Vector Predictor, SbTMVP)

[0216] The basic idea of ​​ATMVP is to derive multiple sets of temporal motion vector predictors for a block. Each sub-block is assigned a set of motion information. When generating ATMVP Merge candidates, motion compensation is performed at the 8x8 level instead of the entire block level.

[0217] In the current design, ATMVP predicts the motion vector of a sub-CU in a CU in two steps, which are described in the following two subsections 2.2.5.1.1 and 2.2.5.1.2 respectively.

[0218] 2.2.5.1.1 Initialized motion vector derivation

[0219] The initialized motion vector is denoted by tempMv.When block A1 is available and is not intra-coded (eg coded in inter or IBC mode), the following applies to derive the initialized motion vector.

[0220] – Set tempMv equal to the motion vector of block A1 in List 1, denoted mvL1A1, if all of the following conditions are true:

[0221] – the reference picture index of List 1 is available (not equal to −1) and it has the same POC value as the collocated picture (e.g., DiffPicOrderCnt(ColPic, RefPicList[1][refIdxL1A1]) is equal to 0),

[0222] – all reference pictures do not have a larger POC than the current picture (e.g., for each picture aPic in each reference picture list of the current slice, DiffPicOrderCnt(aPic, currPic) is less than or equal to 0),

[0223] – The current stripe is equal to the B stripe,

[0224] –collocated_from_l0_flag is equal to 0.

[0225] – Otherwise, if all of the following conditions are true, then tempMv is set equal to the motion vector of block A1 in list 0, denoted as mvL0A1:

[0226] – the reference picture index of list 0 is available (not equal to -1),

[0227] – It has the same POC value as the collocated picture (eg, DiffPicOrderCnt(ColPic, RefPicList[0][refIdxL0A1]) equals 0).

[0228] – Otherwise, the zero motion vector is used as the initialized MV.

[0229] The corresponding block is identified in the collocated picture signaled at the slice header with the initialized motion vector (with the center position of the current block plus the rounded MV, which is clipped to a certain range if necessary).

[0230] If the block is inter-coded, go to step 2. Otherwise, set the ATMVP candidate to unavailable.

[0231] 2.2.5.1.2 Sub-CU motion derivation

[0232] The second step is to divide the current CU into sub-CUs and obtain motion information of each sub-CU from a block corresponding to each sub-CU in a collocated picture.

[0233] If the corresponding block of the sub-CU is coded in inter-frame mode, the motion information is used to derive the final motion information of the current sub-CU by calling the derivation process for the collocated MV, which is no different from the process for the conventional TMVP process. Basically, if the corresponding block is predicted from the target list X for unidirectional prediction or bidirectional prediction, the motion vector is used; otherwise, if the corresponding block is predicted from list Y (Y=1-X) for unidirectional prediction or bidirectional prediction, and NoBackwardPredFlag is equal to 1, the MV of list Y is used. Otherwise, no motion candidate can be found.

[0234] If the block in the collocated picture identified by the initialized MV and the position of the current sub-CU is intra or IBC coded, or no motion candidate can be found as described above, the following rules further apply:

[0235] Will be used to grab and place the image R col The motion vector of the motion field in is denoted as MV col To minimize the impact of MV scaling, the MVs used to derive the MVs are chosen in the following way. col MV in the spatial candidate list: If the reference picture of the candidate MV is a collocated picture, this MV is selected and used as the MV col , without any scaling. Otherwise, select the MV with the reference picture closest to the collocated picture to derive the MV with scaling col .

[0236] The relevant decoding process for collocated motion vector derivation processing in some embodiments is described as follows:

[0237] 8.5.2.12 Derivation of concatenated motion vectors

[0238] The input to this process is:

[0239] – The variable currCb specifies the current codec block,

[0240] – variable colCb specifies the collocated codec block within the collocated picture specified by ColPic,

[0241] – luma position (xColCb, yColCb), specifies the top left sample of the collocated luma codec block specified by colCb relative to the top left luma sample of the collocated picture specified by ColPic,

[0242] – Reference index refIdxLX, where X is 0 or 1,

[0243] – Flag sbFlag indicating the sub-block time domain merging candidate.

[0244] The output of this process is:

[0245] – Motion vector prediction mvLXCol with 1 / 16 fractional sample accuracy,

[0246] –Availability flag availableFlagLXCol.

[0247] The variable currPic specifies the current picture.

[0248] The arrays predFlagL0Col[x][y], mvL0Col[x][y], and refIdxL0Col[x][y] are set equal to PredFlagL0[x][y], MvDmvrL0[x][y], and RefIdxL0[x][y], respectively, of the collocated picture specified by ColPic, and the arrays predFlagL1Col[x][y], mvL1Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL1[x][y], MvDmvrL1[x][y], and RefIdxL1[x][y], respectively, of the collocated picture specified by ColPic.

[0249] The variables mvLXCol and availableFlagLXCol are derived as follows:

[0250] – If colCb is encoded in intra or IBC prediction mode, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

[0251] – Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows:

[0252] – If sbFlag is equal to 0, availableFlagLXCol is set equal to 1 and the following rules apply:

[0253] – If predFlagL0Col[xColCb][yColCb] is equal to 0, set mvCol, refIdxCol, and listCol equal to mvL1Col[xColCb][yColCb], refIdxL1Col[xColCb][yColCb], and L1, respectively.

[0254] – Otherwise, if predFlagL0Col[xColCb][yColCb] is equal to 1 and predFlagL1Col[xColCb][yColCb] is equal to 0, then set mvCol, refIdxCol, and listCol equal to mvL0Col[xColCb][yColCb], refIdxL0Col[xColCb][yColCb], and L0, respectively.

[0255] – Otherwise (predFlagL0Col[xColCb][yColCb] is equal to 1 and predFlagL1Col[xColCb][yColCb]

[0256] equal to 1), make the following allocations:

[0257] – If NoBackwardPredFlag is equal to 1, set mvCol, refIdxCol, and listCol equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb], and LX, respectively.

[0258] – Otherwise, set mvCol, refIdxCol, and listCol equal to mvLNCol[xColCb][yColCb],

[0259] refIdxLNCol[xColCb][yColCb] and LN, where N is the value of collocated_from_l0_flag.

[0260] – Otherwise (sbFlag is equal to 1), the following rules apply:

[0261] – If PredFlagLXCol[xColCb][yColCb] is equal to 1, then set mvCol, refIdxCol, and listCol equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb], and LX, respectively, and set availableFlagLXCol to 1.

[0262] – Otherwise (PredFlagLXCol[xColCb][yColCb] is equal to 0), the following rules apply:

[0263] – If DiffPicOrderCnt(aPic, currPic) for each picture aPic in each reference picture list of the current slice is less than or equal to 0, and PredFlagLYCol[xColCb][yColCb]

[0264] =1, then set mvCol, refIdxCol, and listCol equal to mvLYCol[xColCb][yColCb], refIdxLYCol[xColCb][yColCb], and LY, respectively, where Y equals !X, where X is the value of X used to invoke this process. Set availableFlagLXCol to 1.

[0265] – Set both components of mvLXCol to 0 and set availableFlagLXCol equal to 0.

[0266] – When availableFlagLXCol is equal to TRUE, mvLXCol and availableFlagLXCol are derived as follows:

[0267] – If LongTermRefPic(currPic, currCb, refIdxLX, LX) is not equal to LongTermRefPic

[0268] (ColPic, colCb, refIdxCol, listCol), then both components of mvLXCol are set to 0 and availableFlagLXCol is set to 0.

[0269] – Otherwise, set the variable availableFlagLXCol to 1 and set refPicList[listCol][refIdxCol]

[0270] Set to the picture with reference index refIdxCol in the reference picture list listCol of the slice containing the codec block colCb in the collocated picture specified by ColPic, and the following rules apply:

[0271] colPocDiff=DiffPicOrderCnt(ColPic,refPicList[listCol][refIdxCol])(8-402)

[0272] currPocDiff=DiffPicOrderCnt(currPic,RefPicList[X][refIdxLX])(8-403)

[0273] – Invoke the temporal motion buffer compression process for concatenated motion vectors as specified in clause 8.5.2.15 with mvCol as input and the modified mvCol as output.

[0274] – If RefPicList[X][refIdxLX] is a long-term reference picture, or colPocDiff is equal to currPocDiff, then mvLXCol is derived as follows:

[0275] mvLXCol=mvCol (8-404)

[0276] – Otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol as follows:

[0277] tx=(16384 + (Abs(td)>>1)) / td (8-405)

[0278] distScaleFactor=Clip3( -4096, 4095, ( tb * tx + 32 )>>6 ) (8-406)

[0279] mvLXCol=Clip3(-131072,131071,(distScaleFactor*mvCol+128-

[0280] (distScaleFactor * mvCol>=0))>>8)) (8-407)

[0281] Among them, td and tb are derived as follows:

[0282] td = Clip3( -128, 127, colPocDiff ) (8-408)

[0283] tb = Clip3( -128, 127, currPocDiff ) (8-409)

[0284] 2.3 Intra-frame Block Copy

[0285] Intra-block copy (IBC) (also known as current picture reference) is used in HEVC Screen Content Codec Extension (HEVC-SCC) and the current VVC Test Model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame codec to intra-frame codec. Fig.17 As shown, when IBC is applied, the current block is predicted by the reference block in the same picture. Before encoding or decoding the current block, the samples in the reference block must have been reconstructed. Although IBC is not effective enough for most camera-captured sequences, it shows significant codec gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. IBC can effectively remove the redundancy between these repeated patterns. In HEVC-SCC, if the current picture is selected as its reference picture, the inter-frame codec unit (CU) can apply IBC. In this case, MV is renamed as block vector (BV), and BV always has integer pixel precision. In order to be compatible with the main profile HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that, similarly, in the multi-view / 3D video codec standard, the inter-view reference picture is also marked as a "long-term" reference picture.

[0286] After the BV finds its reference block, a prediction is generated by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Transforms and quantization can then be applied like in other codec modes.

[0287] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to certain constraints, some or all pixel values ​​are undefined. Basically, there are two solutions to this problem. One is to prohibit this situation, for example in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0288] 2.3.1 Single BV List

[0289] In some embodiments, the BV predictors for Merge mode and AMVP mode in IBC will share a common predictor list, which includes the following elements:

[0290] (1) Two adjacent airspaces (such as Figure 2 A1, B1 in

[0291] (2) 5 HMVP entries

[0292] (3) Default zero vector

[0293] The number of candidates in the list is controlled by a variable derived from the stripe header. For Merge mode, at most the first 6 entries of this list will be used; for AMVP mode, the first 2 entries of this list will be used. Also, the list complies with the shared Merge list region requirement (sharing the same list within an SMR).

[0294] In addition to the above BV predictor candidate list, the pruning operation between the HMVP candidate and the existing Merge candidate (A1, B1) can be simplified. In the simplified process, there will be at most 2 pruning operations because it only compares the first HMVP candidate with the spatial Merge candidate.

[0295] For IBC AMVP mode, the mv difference compared to the selected MVP in the list is also transmitted. For IBC Merge mode, the selected MVP is directly used as the MV of the current block.

[0296] 2.3.2 Size Limits of IBC

[0297] In the latest VVC and VTM5, it is proposed to explicitly use syntax constraints to disable the 128x128 IBC mode in previous VTM and VVC versions based on the current bitstream constraints, which makes the presence of the IBC flag dependent on the CU size <128x128.

[0298] 2.3.3IBC Shared Merge List

[0299] In order to reduce the complexity of the decoder and support parallel encoding, in some embodiments, all leaf codec units (CUs) that are an ancestor node in the CU partition tree share the same Merging candidate list to enable parallel processing of small skip / Merge coded CUs. The ancestor node is named Merge shared node. A shared merging candidate list is generated at the Merge shared node, assuming that the Merge shared node is a leaf CU.

[0300] More specifically, the following can be applied:

[0301] - If the block has no more than 32 luma samples (e.g., 4x8 or 8x4) and is divided into 2 4x4 child blocks, a shared Merge list between very small blocks (e.g., two adjacent 4x4 blocks) is used.

[0302] - If a block has more than 32 luma samples, but, after partitioning, at least one child block is smaller than a threshold (32), then all child blocks of the partition share the same Merge list (e.g., 16x4 or 4x16 for triple partitioning or 8x8 for quad partitioning).

[0303] This limitation applies only to IBC Merge mode.

[0304] 2.3.4 Syntax Table

[0305] 7.3.8.5 Codec unit syntax

[0306]

[0307]

[0308] 7.3.8.8 Motion vector difference syntax

[0309]

[0310] 2.4 Maximum transform block size

[0311] The maximum luma transform size is 64 or 32, enabled only by a flag at the SPS level. The maximum chroma transform size is derived from the chroma sampling ratio relative to the maximum luma transform size.

[0312] When the CU / CB size is larger than the maximum luma transform size, tiles from smaller TUs to larger CUs may be invoked.

[0313] The maximum brightness transform size is represented by MaxTbSizeY.

[0314] 7.3.2.3 Sequence parameter set RBSP syntax

[0315]

[0316]

[0317] sps_max_luma_transform_size_64_flag equal to 1 specifies that the maximum transform size of luma samples is equal to 64. sps_max_luma_transform_size_64_flag equal to 0 specifies that the maximum transform size of luma samples is equal to 32.

[0318] When CtbSizeY is less than 64, the value of sps_max_luma_transform_size_64_flag shall be equal to 0.

[0319] The variables MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, and MaxTbSizeY are derived as follows:

[0320] MinTbLog2SizeY = 2 (7-27)

[0321] MaxTbLog2SizeY = sps_max_luma_transform_size_64_flag? 6:5 (7-28)

[0322] MinTbSizeY=1< <MinTbLog2SizeY (7-29)

[0323] MaxTbSizeY=1< <MaxTbLog2SizeY (7-30)

[0324] sps_sbt_max_size_64_flag equal to 0 specifies that the maximum CU width and height allowed for sub-block transform is 32 luma samples. sps_sbt_max_size_64_flag equal to 1 specifies that the maximum CU width and height allowed for sub-block transform is 64 luma samples.

[0325] MaxSbtSize= Min(MaxTbSizeY, sps_sbt_max_size_64_flag? 64:32) (7-32)

[0326] 2.4.1 Tools that depend on the maximum transform block size

[0327] The partitioning of binary and ternary trees depends on MaxTbSizeY.

[0328] 7.3.8.5 Codec unit syntax

[0329]

[0330]

[0331]

[0332]

[0333]

[0334] 2.4.2 Decoding of Intra-frame and Inter-frame Codec Blocks

[0335] When the size of the current intra-coded CU is larger than the maximum transform block size, the CU is recursively divided into smaller blocks (PUs) until both the width and height are not larger than the maximum transform block size. All PUs share the same intra prediction mode, but the reference samples used for intra prediction are different. At the same time, TU is set equal to PU. Note that the current block should not be coded in intra sub-partitioning (ISP) mode.

[0336] When the size of the CU of the current inter-frame codec is larger than the maximum transform block size, the CU is recursively divided into smaller blocks (TUs) until both the width and height are not larger than the maximum transform block size. All TUs share the same motion information, but the reference samples used for intra prediction are different.

[0337] 8.4.5 Decoding of Intra-frame Blocks

[0338] 8.4.5.1 General decoding process for intra-frame blocks

[0339] The input to this process is:

[0340] – Sample point position (xTb0, yTb0), specifies the upper left sample point of the current transform block relative to the upper left sample point of the current picture,

[0341] – The variable nTbW specifies the width of the current transform block,

[0342] – The variable nTbH specifies the height of the current transform block,

[0343] – variable predModeIntra, specifies the intra prediction mode,

[0344] – The variable cIdx specifies the color component of the current block.

[0345] The output of this process is the modified reconstructed picture before loop filtering.

[0346] The width maxTbWidth and height maxTbHeight of the maximum transform block are derived as follows:

[0347] maxTbWidth=(cIdx==0)? MaxTbSizeY : MaxTbSizeY / SubWidthC(8-41)

[0348] maxTbHeight=(cIdx==0)? MaxTbSizeY : MaxTbSizeY / SubHeightC(8-42)

[0349] The luminance sample locations are derived as follows:

[0350] (xTbY,yTbY)=(cIdx==0)? (xTb0,yTb0):(xTb0*SubWidthC,yTb0*SubHeightC)(8-43)

[0351] Depending on maxTbSize, the following rules apply:

[0352] – If Inter-SubpartitionsSplitType is equal to ISP_NO_SPLIT and nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the following sequential steps apply.

[0353] 1. Derived variables newTbW and newTbH as follows:

[0354] newTbW = (nTbW>maxTbWidth)? ( nTbW / 2 ) : nTbW (8-44)

[0355] newTbH = (nTbH>maxTbHeight)? ( nTbH / 2 ) : nTbH (8-45)

[0356] 2. Invoke the general decoding process for intra blocks specified in this clause with position (xTb0, yTb0), transform block width nTbW set equal to newTbW, transform block height nTbH set equal to newTbH, intra prediction mode predModeIntra, and variable cIdx as input, and the output is the modified reconstructed picture before loop filtering.

[0357] 3. If nTbW is greater than maxTbWidth, the position (xTb0, yTb0) is set equal to (xTb0+newTbW, yTb0), the transform block width nTbW is set equal to newTbW, and the transform block height nTbH is set equal to

[0358] Set equal to newTbH, the intra prediction mode predModeIntra and the variable cIdx as input, the general decoding process for intra blocks specified in this clause is invoked, and the output is the modified reconstructed picture before loop filtering.

[0359] 4. If nTbH is greater than maxTbHeight, then the general decoding process for intra blocks specified in this clause is invoked with position (xTb0, yTb0) set equal to (xTb0, yTb0+newTbH), transform block width nTbW set equal to newTbW, transform block height nTbH set equal to newTbH, intra prediction mode predModeIntra, and variable cIdx as input, and the output is the modified reconstructed picture before loop filtering.

[0360] 5. If nTbW is greater than maxTbWidth and nTbH is greater than maxTbHeight, the position (xTb0, yTb0) is set equal to (xTb0+newTbW, yTb0+newTbH), the transform block width nTbW is set equal to newTbW, the transform block height nTbH is set equal to newTbH, and the intra prediction mode predModeIntra is

[0361] and the variable cIdx as input, the general decoding process for the intra block specified in this clause is invoked, and the output is the modified reconstructed picture before loop filtering.

[0362] – Otherwise, the following sequence of steps applies (standard intra prediction processing):

[0363] –

[0364] …

[0365] 8.5.8 Decoding of residual signals of codec blocks coded and decoded in inter-frame prediction mode

[0366] The input to this process is:

[0367] – Sample point position (xTb0, yTb0), specifies the upper left sample point of the current transform block relative to the upper left sample point of the current picture,

[0368] – The variable nTbW specifies the width of the current transform block,

[0369] – The variable nTbH specifies the height of the current transform block,

[0370] – The variable cIdx specifies the color component of the current block.

[0371] The output of this process is the (nTbW)×(nTbH) array resSamples.

[0372] The width maxTbWidth and height maxTbHeight of the maximum transform block are derived as follows:

[0373] maxTbWidth=(cIdx==0)? MaxTbSizeY : MaxTbSizeY / SubWidthC(8-883)

[0374] maxTbHeight=(cIdx==0)? MaxTbSizeY : MaxTbSizeY / SubHeightC(8-884)

[0375] The luminance sample locations are derived as follows:

[0376] (xTbY,yTbY)=(cIdx==0)? (xTb0,yTb0):(xTb0*SubWidthC,yTb0*SubHeightC)(8-885)

[0377] Depending on maxTbSize, the following rules apply:

[0378] – If nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the following sequential steps apply.

[0379] 1. Derived variables newTbW and newTbH as follows:

[0380] newTbW = (nTbW>maxTbWidth)? ( nTbW / 2 ) : nTbW (8-886)

[0381] newTbH = (nTbH>maxTbHeight)? ( nTbH / 2 ) :nTbH (8-887)

[0382] 2. Call the decoding process of the residual signal of the codec unit encoded and decoded in inter-frame prediction mode specified in this clause with position (xTb0, yTb0), transform block width nTbW set to be equal to newTbW, transform block height nTbH set to be equal to newTbH and variable cIdx as input, and the output is the modified reconstructed picture before loop filtering.

[0383] 3. When nTbW is greater than maxTbWidth, the decoding process of the residual signal of the codec unit encoded and decoded in inter-frame prediction mode specified in this clause is called with the position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0), the transform block width nTbW set equal to newTbW, the transform block height nTbH set equal to newTbH and the variable cIdx as input, and the output is the modified reconstructed picture.

[0384] 4. When nTbH is greater than maxTbHeight, the decoding process of the residual signal of the codec unit encoded and decoded in inter-frame prediction mode specified in this clause is called with the position (xTb0, yTb0) set equal to (xTb0, yTb0+newTbH), the transform block width nTbW set equal to newTbW, the transform block height nTbH set equal to newTbH and the variable cIdx as input, and the output is the modified reconstructed picture before loop filtering.

[0385] 5. When nTbW is greater than maxTbWidth and nTbH is greater than maxTbHeight, the decoding process of the residual signal of the codec unit encoded and decoded in inter-frame prediction mode specified in this clause is called with the position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0+newTbH), the transform block width nTbW set equal to newTbW, the transform block height nTbH set equal to newTbH and the variable cIdx as input, and the output is the modified reconstructed picture before loop filtering.

[0386] – Otherwise, if cu_sbt_flag is equal to 1, the following rules apply:

[0387] – Derived variables sbtMinNumFourths, wPartIdx and hPartIdx as follows:

[0388] sbtMinNumFourths = cu_sbt_quad_flag? 1:2 (8-888)

[0389] wPartIdx = cu_sbt_horizontal_flag? 4: sbtMinNumFourths (8-889)

[0390] hPartIdx = ! cu_sbt_horizontal_flag? 4: sbtMinNumFourths (8-890)

[0391] – Derived the variables xPartIdx and yPartIdx as follows:

[0392] – If cu_sbt_pos_flag is equal to 0, set xPartIdx and yPartIdx equal to 0.

[0393] – Otherwise (cu_sbt_pos_flag is equal to 1), derive the variables xPartIdx and yPartIdx as follows:

[0394] xPartIdx = cu_sbt_horizontal_flag? 0 : ( 4 - sbtMinNumFourths ) (8-891)

[0395] yPartIdx = ! cu_sbt_horizontal_flag? 0 : ( 4 - sbtMinNumFourths )(8-892)

[0396] – The variables xTbYSub, yTbYSub, xTb0Sub, yTb0Sub, nTbWSub and nTbHSub are derived as follows:

[0397] xTbYSub=xTbY+((nTbW*((cIdx==0)?1:SubWidthC)*xPartIdx / 4)

[0398] (8-893)

[0399] yTbYSub=yTbY+((nTbH*((cIdx==0)?1:SubHeightC)*yPartIdx / 4)

[0400] (8-894)

[0401] xTb0Sub = xTb0 + ( nTbW * xPartIdx / 4 ) (8-895)

[0402] yTb0Sub = yTb0 + ( nTbH * yPartIdx / 4 ) (8-896)

[0403] nTbWSub = nTbW * wPartIdx / 4 (8-897)

[0404] nTbHSub = nTbH * hPartIdx / 4 (8-898)

[0405] – With the luma position (xTbYSub, yTbYSub), variables cIdx, nTbWSub and nTbHSub as input, invoke the scaling and transform process specified in clause 8.7.2, and the output is (nTbWSub)×(nTbHSub)

[0406] Array resSamplesTb.

[0407] – Set the residual samples resSamples[x][y] with x=0..nTbW–1, y=0..nTbH–1 to 0.

[0408] – Derived as follows with x = xTb0Sub..xTb0Sub+nTbWSub–1,

[0409] Residual sample points resSamples[x][y] of y=yTb0Sub..yTb0Sub+nTbHSub–1:

[0410] resSamples[ x ][ y ] = resSamplesTb[ x – xTb0Sub ][ y – yTb0Sub ](8-899)

[0411] – Otherwise, the scaling and transform process specified in clause 8.7.2 is invoked with the luma position (xTbY, yTbY), the variable cIdx, the transform width nTbW and the transform height nTbH as input, and the output is the (nTbW)×(nTbH) array resSamples.

[0412] 2.5 Low-Frequency Non-Separable Transform (LFNST) (also known as Reduced Quadratic Transform (RST) / Non-Separable Quadratic Transform (NSST))

[0413] In JEM, a secondary transform is applied between the forward primary transform and quantization (at the encoder) and between dequantization and the inverse primary transform (at the decoder side). Fig.18 As shown, a 4×4 (or 8×8) secondary transform is performed depending on the block size. For example, for each 8×8 block, an 8×8 secondary transform is applied to a larger block (e.g., minimum (width, height)>4), and a 4×4 secondary transform is applied to a smaller block (e.g., minimum (width, height)<8).

[0414] The secondary transform uses a non-separable transform, so it is also called a non-separable secondary transform (NSST). There are 35 transform sets in total, and each transform set uses 3 non-separable transform matrices (kernels, each kernel has a 16×16 matrix).

[0415] Reduced secondary transform (RST, also called low frequency non-separable transform (LFNST)) is introduced, and 4 transform sets (instead of 35 transform sets) mapping is introduced according to the intra prediction direction. In this paper, 16×48 and 16×16 matrices are used for 8×8 and 4×4 blocks respectively. For the convenience of notation, the 16×48 transform is denoted as RST8×8 and the 16×16 transform is denoted as RST4×4. VVC has recently adopted this approach.

[0416] Fig.19 An example of a reduced secondary transform (RST) is shown.

[0417] The secondary forward transform and the secondary inverse transform are independent processing steps from the main transform.

[0418] For the encoder, the main forward transform is performed first, followed by the secondary forward transform and quantization, and CABAC bit encoding. For the decoder, CABAC bit decoding, inverse quantization, and secondary inverse transform are performed first, followed by the main inverse transform. RST is only applied to TUs encoded and decoded within the frame.

[0419] 3. Example technical problems solved by the technical solution disclosed in this article

[0420] The current VVC design has the following problems in IBC mode and transformation design:

[0421] (1) The signaling of the IBC mode flag depends on the block size limit, but the IBC skip mode flag does not, which leaves the worst case unchanged (e.g., a larger block, such as N×128 or 128×N, can still select IBC mode).

[0422] (2) In current VVC, it is assumed that the transform block size is always equal to 64×64, and the transform block size is always set to the codec block size. It is unknown how to handle the case where the IBC block size is larger than the transform block size.

[0423] (3) IBC uses unfiltered constructed samples in the same video unit (e.g., slice). However, compared with the unfiltered samples, the filtered samples (e.g., via deblocking filter / SAO / ALF) can have less distortion. Using filtered samples can bring additional codec gain.

[0424] (4) The context modeling of the LFNST matrix depends on the primary transform and the type of segmentation (single tree or dual tree). However, from our analysis, there is no clear dependency between the selected RST matrix and the primary transform.

[0425] 4. Examples of Techniques and Embodiments

[0426] In this application document, intra-block copy (IBC) is not limited to current IBC techniques, but can be interpreted as a technique that uses reference samples within the current slice / slice / brick / sub-picture / picture / other video units (e.g., CTU rows), without including traditional intra-prediction methods. In one example, the reference samples are those reconstructed samples where no loop filtering process (e.g., deblocking filter, SAO, ALF) is invoked.

[0427] The VPDU size can be expressed as vSizeX*vSizeY. In one example, vSizeX=vSizeY=min(ctbSizeY, 64), where ctbSizeY is the width / height of the CTB. The VPDU is a vSize*vSize block with an upper left position (m*vSize, n*vSize) relative to the upper left corner of the picture, where m and n are integers. Alternatively, vSizeX / vSizeY is a fixed number, such as 64.

[0428] The following detailed inventions should be considered as examples to explain the general concept. These inventions should not be interpreted in a narrow way. In addition, these inventions can be combined in any way.

[0429] About IBC Signaling notify and use

[0430] Assume that the current block size is denoted as W curr ×H curr , the maximum allowed IBC block size is denoted as W IBCMax ×H IBCMax .

[0431] 1. An IBC codec block can be divided into multiple TBs / TUs without signaling the division information. The current IBC code block size is represented as W curr ×H curr .

[0432] a. In one example, W curr Greater than the first threshold and / or H curr If the TB size is greater than the second threshold, a split can be called. Therefore, the TB size is smaller than the current block size.

[0433] i. In one example, the first and / or second threshold is the maximum transform size, expressed as

[0434] MaxTbSizeY.

[0435] b. In one example, recursive partitioning may be invoked until the width or height of a TU / TB is not greater than a threshold.

[0436] i. In one example, the recursive partitioning can be the same as the partitioning used for residual blocks of inter-frame coding, for example, in each case, if the width is greater than MaxTbSizeY, the width is divided by half, and if the height is greater than MaxTbSizeY, the height is divided by half.

[0437] ii. Alternatively, recursive partitioning may be invoked until the partition width and height of the TU / TB are each no greater than a threshold.

[0438] c. Alternatively, in addition, all or some TBs / TUs may share the same motion information (eg, BV).

[0439] 2. Whether to enable or disable IBC may depend on the maximum transform size (eg, MaxTbSizeY in the specification).

[0440] a. In one example, IBC may be disabled when the current luma block dimension (width and / or height) or (when the current block is a chroma block) the corresponding luma block dimension (width and / or height) is larger than the maximum transform size in luma samples.

[0441] i. In one example, when both the width and height of a block are larger than the maximum transform size in luma samples, IBC may be disabled.

[0442] ii. In one example, when the width or height of a block is larger than the maximum transform size in luma samples, IBC may be disabled.

[0443] b. In one example, whether and / or how to signal the indication of using IBC may depend on the dimensions (width and / or height) of the block and / or the maximum transform size.

[0444] i. In one example, the indication of using IBC may include IBC in I slice / strip / tile / sub-picture

[0445] Skip flag (for example, cu_skip_flag).

[0446] 1) In one example, when W curr Greater than MaxTbSizeY and / or H curr When it is greater than MaxTbSizeY, cu_skip_flag may not be signaled.

[0447] 2) Alternatively, when W curr No larger than MaxTbSizeY and / or H curr When it is not greater than MaxTbSizeY, cu_skip_flag may be signaled.

[0448] 3) In one example, when W curr Greater than MaxTbSizeY and / or H curr When larger than MaxTbSizeY, cu_skip_flag may be signaled but shall be equal to 0 in conforming bitstreams.

[0449] 4) For example, in addition, when the current slice / slice / tile / sub-picture is an I slice / slice / tile / sub-picture, the above method can be called.

[0450] ii. In one example, the indication of using IBC may include an IBC mode flag (eg, pred_mode_ibc_flag).

[0451] 1) In one example, when W curr Greater than MaxTbSizeY and / or H curr When it is greater than MaxTbSizeY, pred_mode_ibc_flag may not be signaled.

[0452] 2) Alternatively, when W curr No larger than MaxTbSizeY and / or H curr When it is not greater than MaxTbSizeY, pred_mode_ibc_flag may be signaled.

[0453] In one example, when W curr Greater than MaxTbSizeY and / or H curr When it is greater than MaxTbSizeY, pred_mode_ibc_flag may be signaled but shall be equal to 0 in the conformant bitstream.

[0454] iii. In one example, when certain zero-forcing conditions are met, the residual block of the IBC codec block can be forced to be all zero.

[0455] 1) In one example, when W curr Greater than MaxTbSizeY and / or H curr When it is greater than MaxTbSizeY, the residual block of the IBC codec block can be forced to be all zero.

[0456] 2) In one example, when W curr is greater than a first threshold (e.g., 64 / vSizeX) and / or H curr When it is greater than a second threshold (eg, 64 / vSizeY), the residual block of the IBC codec block may be forced to be all zero.

[0457] 3) Alternatively, in addition to the above situation, the codec block flag can be skipped (e.g.

[0458] Signaling notification of cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma).

[0459] 4) Alternatively, in addition, the codec block flag (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr,

[0460] The signaling notification of tu_cbf_luma) can remain unchanged, however, the conforming bitstream shall satisfy this flag equal to 0.

[0461] iv. In one example, whether to signal codec block flags (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may depend on the use of IBC and / or a threshold on the allowed IBC size.

[0462] 1) In one example, the current block is in IBC mode and W curr Greater than MaxTbSizeY

[0463] and / or H curr When it is greater than MaxTbSizeY, the codec block flag (for example, cu_cbf,

[0464] Signaling notification of tu_cbf_cb, tu_cbf_cr, tu_cbf_luma).

[0465] 2) In one example, the current block is in IBC mode and W curr Greater than the first threshold

[0466] (e.g., 64 / vSizeX) and / or H curr When it is greater than a second threshold (e.g., 64 / vSizeY), the signaling notification of the codec block flag (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may be skipped.

[0467] 3. Whether to signal an indication of using IBC skip mode may depend on the current block dimensions (width and / or height) and the maximum allowed IBC block size (eg, vSizeX / vSizeY).

[0468] a. In one example, when W curr Greater than W IBCMax and / or H curr Greater than H IBCMax cu_skip_flag may not be signaled.

[0469] i. Alternatively, when W curr No greater than W IBCMax and / or H curr No greater than H IBCMax cu_skip_flag can be signaled when

[0470] b. Alternatively, furthermore, when the current slice / slice / tile / sub-picture is an I slice / slice / tile / sub-picture, the above method may be called.

[0471] c. In one example, W IBCMax and H IBCMax Both equal 64.

[0472] d. In one example, W IBCMax Set to vSizeX,H IBCMax Set to vSizeY.

[0473] e. In one example, W IBCMax Set to the maximum transform block width, H IBCMax Set equal to the height of the maximum transform block size.

[0474] About recursive partitioning of IBC / inter-frame codec blocks

[0475] 4. When an IBC / inter-frame coding block (CU) is divided into multiple TBs / TUs and the division information is not signaled (for example, the block size is larger than the maximum transform block size), but the motion information is signaled once for the entire CU, multiple signaling or derivation of motion information is proposed.

[0476] a. In one example, all TBs / TUs may share the same AMVP / Merge flag, which may be signaled once.

[0477] b. In one example, the motion candidate list may be constructed once for the entire CU, however, different candidates may be assigned to different TBs / TUs in the list.

[0478] i. In one example, the index of the assigned candidate may be encoded in the bitstream.

[0479] c. In one example, the motion candidate list may not be constructed using the motion information of neighboring TBs / TUs in the current CU.

[0480] i. In one example, the motion candidate list may be constructed by accessing the motion information of spatially adjacent blocks (adjacent or non-adjacent) relative to the current CU.

[0481] 5. When an IBC / inter-frame coding block (CU) is divided into multiple TB / TUs and the division information is not signaled (for example, the block size is larger than the maximum transform block size), the first TU / TB can be predicted by at least one reconstructed or predicted sample in the second TU / TB.

[0482] a. BV verification check can be performed on each TU / TB individually.

[0483] 6. When an IBC / inter-frame coding block (CU) is divided into multiple TBs / TUs and the division information is not signaled (for example, the block size is larger than the maximum transform block size), the first TU / TB cannot be predicted by any reconstructed or predicted samples in the second TU / TB.

[0484] a. BV verification check can be performed on the entire codec block.

[0485] About the prediction block generation of IBC codec block

[0486] 7. It is proposed to use the filtered reconstructed reference samples for IBC prediction block generation.

[0487] a. In one example, the filtered reconstructed reference samples may be generated by applying a deblocking filter / bilateral filter / SAO / ALF.

[0488] b. In one example, for an IBC codec block, at least one reference sample is filtered, and

[0489] At least one reference sample is before filtering.

[0490] 8. Both filtered and unfiltered reconstructed reference samples can be used for IBC prediction block generation.

[0491] a. In one example, filtered reconstructed reference samples are generated by applying a deblocking filter / bilateral filter / SAO / ALF.

[0492] b. In one example, for the prediction of an IBC block, a portion of the prediction may come from filtered samples, and another portion of the prediction may come from unfiltered samples.

[0493] 9. For IBC prediction block generation, whether to use the filtered or unfiltered values ​​of the reconstructed reference samples depends on the location of the reconstructed samples.

[0494] a. In one example, the decision may depend on the relative position of the reconstructed sample with respect to the current CTU and / or the CTU covering the reference sample.

[0495] b. In one example, if the reference sample is outside the current CTU / CTB, or in a different CTU / CTB

[0496] If the sample point is large and far away from the CTU boundary (for example, 4 pixels), the corresponding filtered sample point can be used.

[0497] c. In one example, if the reference samples are within the current CTU / CTB, the unfiltered reconstructed samples may be used to generate the prediction block.

[0498] d. In one example, if the reference sample is outside the current VPDU, the corresponding filtered sample can be used.

[0499] e. In one example, if the reference sample is within the current VPDU, the unfiltered reconstructed sample can be used to generate the prediction block.

[0500] About MVD Codec

[0501] The motion vector (or block vector), or the motion vector difference or block vector difference, is represented by (Vx, Vy).

[0502] 10. It is proposed to divide the absolute value of the motion vector component (such as Vx or Vy) represented by AbsV into two parts, where the first part is equal to AbsV – ((AbsV >> N) << N) (for example, AbsV & (1 << N), the least significant N bits), and the other part is equal to (AbsV >> N) (for example, the remaining most significant bits). Each part can be encoded and decoded separately.

[0503] a. In one example, each component of a motion vector or motion vector difference can be encoded and decoded separately.

[0504] i. In one example, the first part is encoded and decoded using fixed-length encoding, such as N bits.

[0505] 1) Alternatively, a first flag can be encoded and decoded to indicate whether the first part is equal to 0.

[0506] a. Alternatively, in addition, if not, the value of the first part minus 1 is encoded and decoded.

[0507] ii. In one example, the second part with MV / MVD flag information can be encoded and decoded using the current MVD encoding and decoding method.

[0508] b. In one example, the first part of each component of a motion vector or motion vector difference can be jointly encoded and decoded, while the second part can be encoded and decoded separately.

[0509] i. In one example, the first parts of Vx and Vy can be formed into a new positive value with 2N bits (for example, (Vx << N) + Vy).

[0510] 1) In one example, a first flag can be encoded and decoded to indicate whether the new positive value is equal to 0.

[0511] a. Alternatively, in addition, if not, the new positive value minus 1 is encoded and decoded.

[0512] 2) Alternatively, fixed-length encoding or exp-golomb encoding can be used (for example,

[0513] EG-0 th ) to encode new positive values.

[0514] ii. In one example, the current MVD encoding and decoding method may be used to encode and decode the second part of each component of a motion vector having MV / MVD flag information.

[0515] c. In one example, the first part of each component of a motion vector may be jointly coded and decoded, and the second part may also be jointly coded and decoded.

[0516] d. In one example, N is a positive value, such as 1 or 2.

[0517] e. In one example, N may depend on the MV precision used for motion vector storage.

[0518] i. In one example, if the MV precision used for motion vector storage is 1 / 2 M pixels, then N is set to M.

[0519] 1) In one example, if the MV precision for motion vector storage is 1 / 16 pixel, then N is equal to 4.

[0520] 2) In one example, if the MV precision for motion vector storage is 1 / 8 pixel, then N is equal to 3.

[0521] 3) In one example, if the MV precision used for motion vector storage is 1 / 4 pixel, then N is equal to 2.

[0522] f. In one example, N may depend on the MV of the current motion vector (eg, block vector in IBC)

[0523] Accuracy.

[0524] i. In one example, if BV is 1 pixel precision, N can be set to 1.

[0525] About SBT

[0526] 11. Whether to signal an indication of the maximum CU width and / or height for which sub-block transforms are allowed may depend on the maximum allowed transform block size.

[0527] a. In one example, whether the maximum CU width and height allowed for sub-block transforms is 64 or 32 luma samples (eg, sps_sbt_max_size_64_flag) may depend on the value of the maximum allowed transform block size (eg, sps_max_luma_transform_size_64_flag).

[0528] i. In one example, only when sps_sbt_enabled_flag and

[0529] sps_sbt_max_size_64_flag is signaled only when both sps_max_luma_transform_size_64_flag are true.

[0530] About LFNST matrix index

[0531] 12. It is proposed to use 2 contexts to encode and decode the use of LFNST / LFNST matrix index (e.g., lfnst_idx), where the selection is completely based on the use of the partition tree.

[0532] g. In one example, one context is used for single-tree splits and another context is used for dual-trees, regardless of dependencies on the main transform.

[0533] 13. Propose to encode and decode the usage of LFNST / LFNST matrix index (eg, lfnst_idx) based on the usage of dual tree and / or slice type and / or color component.

[0534] 5. Additional Example Embodiments

[0535] by Bold Underline Italic Use [[ ]] brackets to mark deleted text.

[0536] 5.1 Example #1

[0537] Table 9-82 Assigning ctxInc to syntax elements using context codec containers

[0538]

[0539] 5.2 Example #2

[0540] 7.3.8.5 Codec unit syntax

[0541]

[0542] Fig. 202000 . 2002 may be a block diagram illustrating an example video processing system 2000 in which various techniques disclosed herein may be implemented. Various embodiments may include some or all of the components of system 2000. System 2000 may include an input 2002 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8-bit or 10-bit multi-component pixel values, or may be received in a compressed or encoded format. Input 2002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0543] System 2000 may include a codec component 2004, which may implement various codecs or coding methods described in the present application document. Codec component 2004 may reduce the average bit rate of the video from input 2002 to the output of codec component 2004 to generate the codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 2004 may be stored, or may also be transmitted by a connected communication, as shown in component 2006. The storage or communication bitstream (or codec) representation of the video received at input 2002 may be used by component 2008 to generate pixel values ​​or send to a displayable video of display interface 2010. The process of generating a user-viewable video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at encoders, and the corresponding decoding tools or operations of the reverse codec results will be performed by decoders.

[0544] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Display Port, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and / or video display.

[0545] Fig.212100. The device 2100 may be used to implement one or more methods described herein. The device 2100 may be implemented in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 2100 may include one or more processors 2102, one or more memories 2104, and video processing hardware 2106. The processor 2102 may be configured to implement one or more methods described in the present application document. The memory (memory) 2104 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 2106 may be used to implement some of the techniques described in the present application document in hardware circuits.

[0546] In some embodiments, the following solutions may be implemented as preferred solutions.

[0547] The following solution can be implemented with the additional techniques described in the items listed in the previous section (eg, item 1).

[0548] 1. A video processing method (eg, Fig. 22 The method 2200 depicted in the claims includes: for conversion between a codec representation of a video block in a video region and the video block based on an intra-block copy tool, determining (2202) that the video block is allowed to be partitioned into a plurality of transform units, wherein the determination is based on a codec condition of the video block and the codec representation omits signaling notification of the partitioning; and performing (2204) the conversion based on the partitioning.

[0549] 2. The method as described in solution 1, wherein the encoding and decoding conditions correspond to the dimensions of the video blocks.

[0550] 3. The method as described in solution 2, wherein the partition is determined to be allowed because the width of the video block is greater than a first threshold, or the partition is determined to be allowed because the height of the video block is greater than a second threshold.

[0551] 4. The method as described in solution 3, wherein the first threshold and / or the second threshold is equal to the maximum transformation size of the video area.

[0552] 5. A method as described in any of solutions 1-4, wherein the partitioning is recursively applied to the video blocks.

[0553] The following solution can be implemented with the additional techniques described in the items listed in the previous section (e.g., item 2).

[0554] 6. A video processing method, comprising: for conversion between a codec representation of a video block of a video region and the video block, determining whether an intra-block copy (IBC) tool is enabled for conversion of the video block based on a maximum transform size of the video region; and performing the conversion based on the determination.

[0555] 7. The method of solution 6, wherein the video block is a chroma block, and wherein the maximum transform size corresponds to a size of a corresponding luma block, wherein IBC is disabled due to the maximum transform size being greater than a threshold.

[0556] 8. The method as described in any of solutions 6-7, wherein a syntax element in the codec representation indicates whether the IBC tool is enabled because the dimensions or the maximum transform size of the video block meet the criteria.

[0557] 9. The method as described in any one of solutions 6-8, wherein, when it is determined to enable the IBC tool, the conversion further includes forcing the residual block of the video block to be all zero based on a zero forcing condition.

[0558] The following solution can be implemented with the additional techniques described in the items listed in the previous section (e.g., item 3).

[0559] 10. A video processing method, comprising: for conversion between a codec representation of a video block of a video region and the video block, determining whether signaling notification of an intra-frame block copy (IBC) tool for conversion is included in the codec representation; and performing the conversion based on the determination, wherein the determination is based on a width and / or height of the video block and a maximum allowed IBC block size of the video region.

[0560] 11. The method of solution 10, wherein the signaling notification is not included because the width is greater than a first threshold or the height is greater than a second threshold.

[0561] The following solution can be implemented with the additional techniques described in the items listed in the previous section (e.g., item 4).

[0562] 12. A video processing method, comprising: for conversion between a codec representation of a video block of a video region and the video block using an intra-block copy (IBC) tool, determining that the video block is allowed to be partitioned into a plurality of transform units (TUs) for conversion; and performing the conversion based on the determination, wherein the conversion includes using separate motion information for the plurality of TUs.

[0563] 13. The method of solution 12, wherein multiple TUs are constrained to share the same advanced motion vector predictor or merge mode flag.

[0564] 14. The method of solution 12, wherein, during conversion, motion information of one of the plurality of TUs is determined without using motion information of another of the plurality of TUs.

[0565] The following solution can be implemented with the additional techniques described in the items listed in the previous section (eg, item 5).

[0566] 15. A method as described in any of solutions 12-14, wherein a first TU among multiple TUs is predicted from a second TU among multiple TUs.

[0567] The following solution can be implemented with the additional techniques described in the items listed in the previous section (eg, item 6).

[0568] 16. A method as described in any of solutions 12-15, wherein predicting a first TU among multiple TUs from a second TU among multiple TUs is disabled.

[0569] The following solution can be implemented with the additional techniques described in the items listed in the previous section (e.g., item 7).

[0570] 17. A method of video processing, comprising: for conversion between a codec representation of a video block of a video region and the video block, determining that an intra block copy tool is enabled for the conversion; and using the intra block copy tool to perform the conversion, wherein prediction of the video block is performed using filtered reconstructed samples of the video region.

[0571] 18. The method of solution 17, wherein the filtered reconstructed samples include samples generated by applying a loop filter to the reconstructed samples, wherein the loop filter is a deblocking filter or a bilateral filter or a sample adaptive offset or adaptive loop filter.

[0572] The following solution can be implemented with the additional techniques described in the items listed in the previous section (e.g., item 8).

[0573] 19. A method as described in any of solutions 17-18, wherein, based on rules, the prediction also selectively uses unfiltered reconstructed samples of the video area.

[0574] The following solution can be implemented with the additional techniques described in the items listed in the previous section (eg, item 9).

[0575] 20. The method of solution 19, wherein the rule depends on the position of the reconstruction sample point.

[0576] 21. The method of solution 20, wherein the position of the reconstructed sample for the regularity is the relative position of the reconstructed sample with respect to the codec tree unit of the video block.

[0577] The following solution can be implemented with the additional techniques described in the items listed in the previous section (eg, item 10).

[0578] 22. A video processing method, comprising: performing a conversion between a video including a plurality of video blocks and an encoded / decoded representation of the video, wherein at least some of the video blocks are encoded / decoded using motion vector information, and wherein the motion vector information is represented as a first part in the encoded / decoded representation based on a first least significant bit of an absolute value of the motion vector information, and the motion vector information is represented as a second part in the encoded / decoded representation based on remaining higher significant bits more significant than the first least significant bit.

[0579] 23. The method according to solution 22, wherein an absolute value of a motion vector V represented as AbsV is divided into a first part and a second part, the first part being equal to AbsV – ((AbsV >> N) << N) or AbsV & (1 << N), the second part being equal to (AbsV >> N), and N being an integer.

[0580] 24. The method according to any one of solutions 22-23, wherein the first part and the second part are separately encoded / decoded in the encoded / decoded representation.

[0581] 25. The method according to any one of solutions 22-24, wherein the first part of the x motion vector and the first part of the y motion vector are jointly encoded / decoded at the same position.

[0582] 26. The method according to solution 25, wherein the second part of the x motion vector and the second part of the y motion vector are separately encoded / decoded.

[0583] 27. The method according to any one of solutions 22-26, wherein N is a function of a precision for storing the motion vector.

[0584] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., item 11).

[0585] 28. A video processing method, comprising: for a conversion between an encoded / decoded representation of a video block of a video region and the video block, determining whether a sub-block transform tool is enabled for the conversion; and performing the conversion based on the determination, wherein the determination is based on a maximum allowable transform block size of the video region, and wherein signaling is conditionally included in the encoded / decoded representation based on the maximum allowable transform block size.

[0586] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., items 12, 13).

[0587] 29. A video processing method, comprising: for conversion between a codec representation of a video block of a video region and the video block, determining whether a low frequency non-separable transform (LFNST) is used during the conversion; and performing the conversion based on the determination, wherein the determination is based on a codec condition applied to the video block, and wherein the LFNST and a matrix index of the LFNST are encoded and decoded in the codec representation using two contexts.

[0588] 30. The method of solution 29, wherein the encoding and decoding conditions include the partition tree used, the slice type of the video block, or the color component identifier of the video block.

[0589] 31. A method as described in any of solutions 1-30, wherein the video region includes a codec unit, and wherein the video block corresponds to a luma or chroma component of the video region.

[0590] 32. A method as described in any of solutions 1-31, wherein converting includes generating a bitstream representation from the current video block.

[0591] 33. A method as described in any of solutions 1-31, wherein the conversion includes generating samples of the current video block from a bitstream representation.

[0592] 34. A video processing device, comprising a processor configured to implement any one or more of the methods described in solutions 1-33.

[0593] 35. A computer-readable medium having codes stored thereon, which, when executed, cause a processor to implement the method described in any one or more of solutions 1-33.

[0594] 36. A method, system or apparatus as herein described.

[0595] Fig.23 23 is a flowchart representation of a video processing method 2300 according to the present technology. The method 2300 includes: at operation 2310, for conversion between a current block in a video region of a video and a codec representation of the video, determining that it is allowed to divide the current block into a plurality of transform units based on characteristics of the current block. Signaling notification of the division is omitted in the codec representation. The method 2300 includes: at operation 2320, performing conversion based on the determination.

[0596] In some embodiments, the characteristic of the current block includes a dimension of the current block. In some embodiments, in a case where the width of the current block is greater than a first threshold, the partitioning is called for conversion. In some embodiments, in a case where the height of the current block is greater than a second threshold, the partitioning is called for conversion. In some embodiments, the partitioning of the current block includes: recursively partitioning the current block until the dimension of a transform unit in a plurality of transform units is equal to or less than a threshold. In some embodiments, the width of the current block is recursively divided by half until the width of the current block is less than or equal to the first threshold. In some embodiments, the height of the current block is recursively divided by half until the height of the current block is less than or equal to the second threshold. In some embodiments, the current block is recursively partitioned until the width of the current block is less than or equal to the first threshold and the height of the current block is less than or equal to the second threshold. In some embodiments, the first threshold is equal to the maximum transform size of the video area. In some embodiments, the second threshold is equal to the maximum transform size of the video area.

[0597] In some embodiments, at least a subset of the plurality of transform units share the same motion information. In some embodiments, the same motion information includes a syntax flag for an advanced motion vector prediction mode or a merge mode, and the syntax flag is signaled once in the codec representation of the current block. In some embodiments, the motion information of the current block is determined multiple times in the codec representation. In some embodiments, a motion candidate list is constructed once for the current block, and different candidates are assigned to the plurality of transform units in the motion candidate list. In some embodiments, the indexes of the different candidates are signaled in the codec representation. In some embodiments, for a transform unit of the plurality of transform units, a motion candidate list is constructed without using motion information of neighboring transform units within the current block. In some embodiments, the motion candidate list is constructed using motion information of spatially neighboring blocks of the current block. In some embodiments, a first transform unit is reconstructed based on at least one reconstructed sample in a second transform unit. In some embodiments, a validation check is performed separately for each transform unit in the plurality of transform units. In some embodiments, the first transform unit is predicted without using any reconstructed samples in other transform units. In some embodiments, a validation check is performed on the block after the block is reconstructed.

[0598] Fig.24 24 is a flowchart representation of a video processing method 2400 according to the present technology. The method 2400 includes: at operation 2410, for conversion between a current block in a video region of a video and a codec representation of the video, using an intra-block copy (IBC) model based on filtered reconstructed reference samples in the video region, determining prediction samples for the current block. The method 2400 includes: at operation 2420, performing conversion based on the determination.

[0599] In some embodiments, the filtered reconstructed reference samples are generated based on a filtering operation, the filtering operation comprising at least one of the following: a deblocking filtering operation, a bilateral filtering operation, a sample adaptive offset (SAO) filtering operation, or an adaptive loop filtering operation. In some embodiments, the prediction samples of the current block are determined using the IBC model further based on unfiltered reconstructed reference samples in the video region. In some embodiments, the prediction samples in the first part of the current block are determined based on the filtered reconstructed reference samples, and the prediction samples in the second part of the current block are determined based on the unfiltered reconstructed reference samples. In some embodiments, whether the filtered reconstructed reference samples or the unfiltered reconstructed reference samples are used to determine the prediction samples of the block is based on the position of the prediction samples relative to the current codec tree unit or to the codec tree unit covering the samples. In some embodiments, when the prediction samples are located outside the current codec tree unit, the filtered reconstructed reference samples are used to reconstruct the prediction samples. In some embodiments, when the prediction samples are located inside the codec tree unit and the prediction samples have a distance D equal to or greater than 4 pixels from one or more boundaries of the codec tree unit, the filtered reconstructed reference samples are used to reconstruct the prediction samples. In some embodiments, when the prediction sample is located inside the current codec tree unit, the prediction sample is reconstructed using the unfiltered reconstructed reference sample. In some embodiments, when the prediction sample is located outside the current virtual pipeline data unit, the prediction sample is reconstructed using the filtered reconstructed reference sample. In some embodiments, when the prediction sample is located inside the current virtual pipeline data unit, the prediction sample is reconstructed using the unfiltered reconstructed reference sample.

[0600] Fig.25 25 is a flowchart representation of a video processing method 2500 according to the present technology. The method 2500 includes: at operation 2510, for a conversion between a current block of a video and a codec representation of the video, determining according to a rule whether a syntax element indicating a skip mode using an intra-block copy (IBC) codec model is included in the codec representation. The rule specifies that the signaling notification of the syntax element is based on the dimension of the current block and / or based on the maximum allowed dimension of a block encoded using the IBC codec model. The method 2500 also includes: at operation 2520, performing the conversion based on the determination.

[0601] In some embodiments, the rules provide that syntax elements are omitted from the codec representation if the width of the current block is greater than the maximum allowed width. In some embodiments, the rules provide that syntax elements are omitted from the codec representation if the height of the current block is greater than the maximum allowed height. In some embodiments, the rules provide that syntax elements are included in the codec representation if the width of the current block is less than or equal to the maximum allowed width. In some embodiments, the rules provide that syntax elements are included in the codec representation if the height of the current block is less than or equal to the maximum allowed height.

[0602] In some embodiments, the current block is in a video region of the video, and wherein the rule applies to the case where the video region includes an I-slice, an I-strip, an I-tile, or an I-sub-picture. In some embodiments, the maximum allowed width or the maximum allowed height is 64. In some embodiments, the maximum allowed width or the maximum allowed height is equal to the dimension of the virtual pipeline data unit. In some embodiments, the maximum allowed width or the maximum allowed height is equal to the maximum dimension of the transform unit.

[0603] Fig.26 26 is a flowchart representation of a video processing method 2600 according to the present technology. The method 2600 includes: at operation 2610, for a conversion between a current block of a video and a codec representation of the video, determining at least one context for encoding an index associated with a low-frequency non-separable transform (LFNST) codec model. The LFNST codec model includes: during encoding, applying a positive quadratic transform between a positive main transform and a quantization step; or during decoding, applying an inverse quadratic transform between a dequantization step and an inverse main transform. The size of the positive quadratic transform and the inverse quadratic transform is smaller than the size of the current block. At least one context is determined based on the partition type of the current block without considering the positive main transform or the inverse main transform. The method 2600 also includes: at operation 2620, performing the conversion based on the determination.

[0604] In some embodiments, when the partition type is single-tree partition, only one context is used for codec indexing. In some embodiments, when the partition type is dual-tree partition, two contexts are used for codec indexing.

[0605] In some embodiments, an index indicating the use of a low frequency non-separable transform (LFNST) codec model is included in the codec representation based on characteristics associated with the block, including a partition type, a slice type, or a color component associated with the current block.

[0606] Fig. 2727 is a flowchart representation of a video processing method 2700 according to the present technology. The method 2700 includes: at operation 2710, for conversion between a current block of a video region of a video and a codec representation of the video, determining whether to enable an intra-block copy (IBC) codec model based on a maximum transform unit size applicable to the video region. The method 2700 also includes: at operation 2720, performing the conversion based on the determination.

[0607] In some embodiments, the IBC codec model is disabled when the dimension of the current block or the dimension of the luma block corresponding to the current block is greater than the maximum transform unit size. In some embodiments, the IBC codec model is disabled when the width of the current block or the width of the luma block corresponding to the current block is greater than the maximum transform unit width. In some embodiments, the IBC codec model is disabled when the height of the current block or the height of the luma block corresponding to the current block is greater than the maximum transform height. In some embodiments, the manner in which the IBC codec model is signaled in the codec representation is based on the dimension of the current block and the maximum transform unit size. In some embodiments, the signaling of the use of the IBC codec model includes a syntax element indicating a skip mode of the IBC codec model. In some embodiments, the syntax element is included in an I slice, I stripe, I tile, or I sub-picture in the codec representation.

[0608] In some embodiments, signaling the use of the IBC codec model includes a syntax element indicating the IBC mode. In some embodiments, the syntax element is omitted from the codec representation when the width of the current block is greater than the maximum transform unit width or the height of the current block is greater than the maximum transform unit height. In some embodiments, the syntax element is included in the codec representation when the width of the current block is less than or equal to the maximum transform unit width or the height of the current block is less than or equal to the maximum transform unit height. In some embodiments, the syntax element is included in the codec representation and is set to 0 when the width of the current block is greater than the maximum transform unit width or the height of the current block is greater than the maximum transform unit height.

[0609] In some embodiments, when the IBC codec model is enabled for the current block, according to the rules, the samples in the residual block corresponding to the current block are set to 0. In some embodiments, the rules stipulate that when the width of the current block is greater than the maximum transform unit width or the height of the current block is greater than the maximum transform unit height, the samples are set to 0. In some embodiments, the rules stipulate that when the width of the current block is greater than a first threshold or the height of the current block is greater than a second threshold, the samples are set to 0. In some embodiments, the first threshold is 64 / vSizeX, where vSizeX is the width of the virtual pipeline data unit. In some embodiments, the second threshold is 64 / vSizeY, where vSizeY is the height of the virtual pipeline data unit. In some embodiments, the syntax flag of the codec block is omitted in the codec representation. In some embodiments, the syntax flag of the codec block is set to 0 in the codec representation.

[0610] In some embodiments, signaling in the codec representation that the syntax flag for the codec block is based on the use of the IBC codec model. In some embodiments, when the IBC codec model is enabled for the current block and the dimensions of the block are greater than the maximum transform unit size, the syntax flag is omitted in the codec representation. In some embodiments, when the IBC codec model is enabled for the current block and the dimensions of the block are greater than the threshold associated with the virtual pipeline data unit, the syntax flag is omitted in the codec representation.

[0611] Fig.28 is a flowchart representation of a video processing method 2800 according to the present technology. Method 2800 includes, at operation 2810, for the conversion between the current block of a video and the codec representation of the video, determining that the absolute value of the component of the motion vector of the current block is divided into two parts. The motion vector is represented as (Vx, Vy), the component is represented as Vi, and Vi is either Vx or Vy. The first part of the two parts is equal to |Vi| - ((|Vi| >> N) << N), and the second part of the two parts is equal to |Vi| >> N, where N is a positive integer. The two parts are separately coded in the codec representation. Method 2800 further includes: at operation 2820, performing the conversion based on the determination.

[0612] In some embodiments, two components Vx and Vy are signaled separately in the codec representation. In some embodiments, the first part is codec-encoded with a fixed length of N bits. In some embodiments, the first parts of component Vx and component Vy are jointly codec-encoded, and the second parts of component Vx and component Vy are separately codec-encoded. In some embodiments, the first parts of component Vx and component Vy are codec-encoded into a value with a length of 2N bits. In some embodiments, the value is equal to (Vx << N) + Vy. In some embodiments, a fixed-length codec process or an exp-golomb codec process is used to codec-encode the value.

[0613] In some embodiments, the first parts of component Vx and component Vy are jointly codec-encoded, and the second parts of component Vx and component Vy are jointly codec-encoded. In some embodiments, a syntax flag is included in the codec representation to indicate whether the first part is equal to 0. In some embodiments, the first part has a value K, where K ≠ 0, and the value (K - 1) is codec-encoded in the codec representation. In some embodiments, a motion vector difference codec process is used to codec-encode the second part of each component with flag information. In some embodiments, N is 1 or 2. In some embodiments, N is determined based on the motion vector precision for motion vector data storage. In some embodiments, N is 4 when the motion vector precision is 1 / 16 pixel. In some embodiments, N is 3 when the motion vector precision is 1 / 8 pixel. In some embodiments, N is 2 when the motion vector precision is 1 / 4 pixel. In some embodiments, N is determined based on the motion vector precision of the current motion vector. In some embodiments, N is 1 when the motion vector precision is 1 pixel.

[0614] Fig.29 is a flowchart representation of a video processing method 2900 according to the present technology. Method 2900 includes, at operation 2910, for the conversion between the current block of a video and the codec representation of the video, determining information about the maximum dimension of the current block that allows sub-block transformation in the current block based on the maximum allowed dimension of the transform block. Method 2900 further includes: at operation 2920, performing the conversion based on the determination.

[0615] In some embodiments, the maximum dimension of the current block that allows sub-block transformation in the current block corresponds to the maximum allowed dimension of the transform block. In some embodiments, the maximum dimension of the current block is 64 or 32. In some embodiments, a syntax flag indicating the maximum dimension of the current block is included in the codec representation when a first syntax flag in the slice parameter set indicates enabling sub-block transformation and a second syntax flag in the slice parameter set indicates that the maximum allowed dimension of the transform block is 64.

[0616] In some embodiments, converting comprises generating a codec representation from the current block. In some embodiments, converting comprises generating samples of the current block from the codec representation.

[0617] In the above solutions, performing the conversion includes using the results of previous decision steps (eg, using or not using certain encoding or decoding steps) during the encoding or decoding operation to obtain the conversion result.

[0618] Some embodiments of the disclosed technology include making a decision or determining to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a block of a video, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is started based on a decision or determination, the conversion from a block of a video to a bitstream representation of a video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, a decoder will process the bitstream using knowledge that the bitstream has been modified based on the video processing tool or mode. That is, the video processing tool or mode enabled based on the decision or determination will be used to perform a conversion from a bitstream representation of a video to a block of a video.

[0619] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when the video processing tool or mode is disabled, the encoder does not use the tool or mode in converting between blocks of video to a bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream with the knowledge that the bitstream was not modified using the video processing tool or mode that was enabled based on the decision or determination.

[0620] The disclosure and other solutions, examples, embodiments, modules and functional operations described in this application document can be implemented in digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this application document and their structural equivalents, or a combination of one or more thereof. The embodiments disclosed in this application document and other embodiments can be implemented as one or more computer program products, such as one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium, for data processing equipment to execute or control the operation of the data processing equipment. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing equipment" includes all devices, devices and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0621] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0622] The processing and logic flows described in this application document can be performed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The processing and logic flows can also be performed by special-purpose logic circuits, and the device can also be implemented as special-purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits).

[0623] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory can be supplemented by dedicated logic circuits, or merged into dedicated logic circuits.

[0624] Although this patent document contains many details, they should not be construed as limitations on the scope of any invention or the claims, but rather as descriptions of features of particular embodiments of particular inventions. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially claimed to be so, in some cases, one or more features in the claim combination may be removed from the combination, and the claim combination may be directed to subcombinations or variations of subcombinations.

[0625] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0626] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: for a first conversion between a current transform block of a video and a bitstream of the video, determining, according to a first rule, whether a first syntax element is included in the bitstream; as well as performing the first conversion based at least on the determination, The first syntax element indicates whether a prediction mode is applied to the current video block. wherein, in the prediction mode, the prediction samples are derived from a block of sample values ​​of the same video area determined by a block vector, and The first rule specifies that when the width of the current video block is greater than a first threshold and the height of the current image block is greater than a second threshold, the first syntax element is not included in the bitstream.

2. The method according to claim 1, further comprising: determining, according to a second rule, whether a second syntax element indicating whether to apply a skip mode to the current video block is included in the bitstream, The second rule stipulates that when the current video block is not a chroma block of a tree type with a dual tree, a sequence enable flag indicates that the prediction mode is enabled, the width of the current video block is less than or equal to the first threshold, and the height of the current video block is less than or equal to the second threshold, the second syntax element is included in the bitstream.

3. The method according to claim 2, wherein: The first threshold and the second threshold are 64, or when the size of the coding tree block is greater than 64, the first threshold and the second threshold are equal to the size of the virtual unit.

4. The method according to claim 1, further comprising: determining whether the bitstream includes a third syntax element for the current video block, The third syntax element specifies whether to use a low frequency non-separable transform (LFNST) kernel in the selected transform set, and which one of the low frequency non-separable transform (LFNST) kernels in the selected transform set is used, wherein a single context selected from the contexts with index increments of 0 or 1 is used for the first binary bit of the third syntax element, and The index increment is determined only based on whether the tree type of the current video block is a single tree.

5. The method according to claim 4, wherein: In response to the tree type of the current video block being a single tree, the index increment is 0, and in response to the tree type of the current video block being not a single tree, the index increment is 1.

6. The method according to claim 4, wherein: The third syntax element is associated with the LFNST codec model, The LFNST codec model includes applying a positive quadratic transform between a positive primary transform and a quantization step during encoding, or applying an inverse quadratic transform between a dequantization step and an inverse primary transform during decoding, and Wherein, the index increment is determined without reference to the positive primary transform or the inverse primary transform.

7. The method according to claim 1, further comprising: For a second conversion between a current transform block of a video and a bitstream of the video, determining whether a width of the current transform block is greater than a first threshold or whether a height of the current transform block is greater than a second threshold; When the width of the current transformation block is greater than the first threshold or the height of the current transformation block is greater than the second threshold, calling a split operation of the current transformation block; as well as performing the second conversion based on at least two or more transform blocks, The signaling notification of the call and the signaling notification of the segmentation operation are omitted in the bit stream.

8. The method according to claim 7, wherein: The first threshold and the second threshold are equal to a first value specifying a maximum transform size in units of luma samples, Optionally, in the splitting operation, when the width of the current transform block is greater than the first value, calling to split the width of the current transform block by half, and Wherein, when the height of the current transform block is greater than the first value, a call is made to divide the height of the current transform block in half.

9. The method according to claim 8, wherein: In the partitioning operation, whether to divide the width of the current transform block is also determined based on the height of the current video block, and whether to divide the height of the current transform block is also determined based on the width of the current transform block.

10. The method according to claim 9, wherein: In the partitioning operation, if the width of the current transform block is greater than the first value and greater than the height of the current video block, the width of the current transform block is divided by half.

11. The method according to claim 10, wherein: In the partitioning operation, if the width of the current transform block is greater than the first value and greater than the height of the current video block, the height of the current transform block remains unchanged.

12. The method according to claim 9, wherein: In the splitting operation, when the height of the current transform block is greater than the first value and the width of the current transform block is less than or equal to at least one of (1) the first value or (2) the height of the current video block, the height of the current transform block is divided by half.

13. The method according to claim 12, wherein: In the splitting operation, when the height of the current transform block is greater than the first value and the width of the current transform block is less than or equal to at least one of (1) the first value or (2) the height of the current video block, the width of the current transform block remains unchanged.

14. The method according to claim 7, wherein: The segmentation operation is recursively called until the width of the current transform block is less than or equal to the first threshold, and the height is less than or equal to the second threshold.

15. The method according to claim 8, wherein: The first value is equal to 32, wherein the splitting operation is called in case the current transform block is derived based on a codec block coded and decoded in a first prediction mode, in which prediction samples are derived from blocks of sample values ​​of the same video area determined by a block vector, and The width of the codec block and the height of the codec block are less than or equal to 64 and greater than 32.

16. The method according to claim 1, wherein: The first conversion includes encoding the video into the bitstream.

17. The method according to claim 1, wherein: The first conversion includes decoding the video from the bitstream.

18. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to: for a first conversion between a current transform block of a video and a bitstream of the video, determining, according to a first rule, whether a first syntax element is included in the bitstream; as well as performing the first conversion based at least on the determination, The first syntax element indicates whether a prediction mode is applied to the current video block. wherein, in the prediction mode, the prediction samples are derived from a block of sample values ​​of the same video area determined by a block vector, and The first rule specifies that when the width of the current video block is greater than a first threshold and the height of the current image block is greater than a second threshold, the first syntax element is not included in the bitstream.

19. A non-transitory computer-readable storage medium having stored therein instructions that cause a processor to: For a first conversion between a current transform block of a video and a bitstream of the video, determining whether a first syntax element is included in the bitstream according to a first rule; and performing the first conversion based at least on the determination, in, The first syntax element indicates whether a prediction mode is applied to the current video block, wherein, in the prediction mode, the prediction samples are derived from a block of sample values ​​of the same video area determined by a block vector, and The first rule specifies that when the width of the current video block is greater than a first threshold and the height of the current image block is greater than a second threshold, the first syntax element is not included in the bitstream.

20. A non-transitory computer-readable recording medium storing a bit stream generated by a method performed by a video processing device, wherein: The method comprises: For a first conversion between a current transform block of a video and a bitstream of the video, determining whether a first syntax element is included in the bitstream according to a first rule; and performing the first conversion based at least on the determination, The first syntax element indicates whether a prediction mode is applied to the current video block. wherein, in the prediction mode, the prediction samples are derived from a block of sample values ​​of the same video area determined by a block vector, and The first rule specifies that when the width of the current video block is greater than a first threshold and the height of the current image block is greater than a second threshold, the first syntax element is not included in the bitstream.

21. A method for storing a bit stream of a video, comprising: for a first conversion between a current transform block of a video and a bitstream of the video, determining, according to a first rule, whether a first syntax element is included in the bitstream; as well as generating the bitstream based at least on the determining; as well as storing the bitstream in a non-transitory computer-readable storage medium, The first syntax element indicates whether a prediction mode is applied to the current video block. wherein, in the prediction mode, the prediction samples are derived from a block of sample values ​​of the same video area determined by a block vector, and The first rule specifies that when the width of the current video block is greater than a first threshold and the height of the current image block is greater than a second threshold, the first syntax element is not included in the bitstream.

Images