Coding and Decoding Intra Block Copy

By dividing video blocks into transformation units and optimizing IBC signaling based on block dimensions and transform sizes, the method addresses inefficiencies in current video coding standards, enhancing compression performance and reducing complexity.

JP7596465B2Active Publication Date: 2024-12-09DOUYIN VISION CO LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
JP2023122019
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-09-09
Filing Date
2023-07-26
Publication Date
2024-12-09
Estimated Expiration
2040-09-09

AI Technical Summary

Technical Problem

Current video coding standards face challenges in efficiently handling large block sizes and intra block copy (IBC) modes, particularly in terms of signaling and transform block sizes, which can lead to inefficiencies and increased computational complexity.

Method used

The proposed method involves dividing current blocks into multiple transformation units based on their characteristics, using filtered reconstructed reference samples for IBC prediction, and applying transformations based on determined conditions, including recursive partitioning and separate motion information for sub-blocks, while signaling is optimized based on block dimensions and maximum transform sizes.

Benefits of technology

This approach enhances video coding efficiency by reducing signaling overhead and computational complexity, improving compression performance, especially for large blocks and IBC modes, thereby optimizing bandwidth usage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007596465000016
    Figure 0007596465000016
  • Figure 0007596465000017
    Figure 0007596465000017
  • Figure 0007596465000018
    Figure 0007596465000018
Patent Text Reader

Abstract

To provide a video processing method applicable to existing video coding standards and future video coding standards or video codec.SOLUTION: A video processing method includes determining, for a conversion between a current block of a video and a coded representation of the video, whether a syntax element indicating usage of a skip mode for an intra-block copy (IBC) coding model is included in the coded representation according to a rule that specifies that signaling of the syntax element is based on a dimension of the current block and / or a maximum allowed dimension for a block that is coded using the IBC coding model. The method also includes performing the conversion on the basis of the determination.SELECTED DRAWING: Figure 25
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Books The wish is, This is a divisional application based on Japanese application No. 2022-515128, which is based on international patent application No. PCT / CN2020 / 114232 filed on September 9, 2020, and this international patent application is Priority and benefit of international patent application PCT / CN2019 / 104869, filed September 9, 2019 Mainly Stretch All patent applications are By reference The whole To be incorporated.

[0002] This patent document relates to video and image coding and decoding. [Background technology]

[0003] Despite advances in video compression, digital video still accounts for the largest bandwidth usage in the Internet and other digital communications networks. The bandwidth demands for digital video use are expected to continue to grow as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention

[0004] The present invention relates to digital video coding, and in particular to devices, systems and methods for video and image coding and decoding that use intra block copy tools for the coding or decoding.

[0005] In one exemplary embodiment, a method of video processing is disclosed. The method includes determining, based on a characteristic of the current block, that partitioning of the current block into multiple transform units is permitted for conversion between a current block in a video domain of the video and a coding representation of the video, in which signaling of the partitioning is omitted. The method also includes performing the conversion based on the determination.

[0006] In another exemplary aspect, a method of video processing is disclosed, the method including determining a prediction sample of a current block using an intra block copy (IBC) mode based on filtered reconstructed reference samples in the video domain for conversion between a current block of a video domain of a video and a coded representation of the video, the method also including performing the conversion based on the determination.

[0007] In another exemplary aspect, a method of video processing is disclosed. The method includes determining whether a syntax element indicating use of a skip mode of an intra block copy (IBC) coding model is included in a coding representation of a current block of a video according to a rule for conversion between the current block and the coding representation of the video. The rule specifies that the signaling of the syntax element is based on a dimension of the current block and / or a maximum allowed dimension for a block coded using the IBC coding model. The method also includes performing the conversion based on the determination.

[0008] In another exemplary aspect, a method of video processing is disclosed. The method includes determining at least one context for coding an index associated with a low-frequency non-separable transform (LFNST) coding model for transforming a current block of video and a coding representation of the video. The LFNST coding model includes applying a forward secondary transform between a forward primary transform and a quantization step during encoding, or applying an inverse secondary transform between an inverse quantization step and an inverse primary transform during decoding. A size of the forward secondary transform and the inverse secondary transform is smaller than a size of the current block. The at least one context is determined based on a partition type of the current block without considering the forward primary transform or the inverse primary transform. The method also includes performing the transform according to the determination.

[0009] In another exemplary aspect, a method of video processing is disclosed. The method includes determining whether an intra block copy (IBC) coding model is enabled based on a maximum transform unit size applicable to a video region for conversion of a current block of the video region and a coding representation of the video, and performing the conversion according to the determination.

[0010] In another exemplary aspect, a method of video processing is disclosed. The method includes determining to divide an absolute value of a component of a motion vector for a current block into two parts for conversion of the current block, the video, and a coding representation of the video, representing the motion vector as (Vx, Vy), representing the component as Vi, where Vi is either Vx or Vy, a first part of the two parts being equal to |Vi| - ((|Vi| >> N) << N), a second part of the two parts being equal to |Vi| >> N, N being a positive integer, the two parts being separately coded in the coding representation, and performing the conversion according to the determination.

[0011] In another exemplary aspect, a method of video processing is disclosed. The method includes determining information regarding a maximum size of a current block that enables sub-block conversion in the current block based on a maximum allowable size of a conversion block for conversion of the current block and the video, and performing the conversion according to the determination.

[0012] In another exemplary aspect, a method of video processing is disclosed. The method includes determining to permit dividing a video block into a plurality of transform units for conversion of a coding representation of the video block and the video block in a video region based on an intra block copy tool, the determination being made based on coding conditions of the video block, the coding representation omitting signaling of the division, and performing the conversion based on the division.

[0013] In another example aspect, a method of video processing is disclosed that includes determining, for conversion between a coded representation of a video block in a video domain and the video block based on a maximum conversion size of the video domain, whether an intra block copy (IBC) tool is enabled for conversion of the video block, and performing the conversion based on the determination. In another exemplary aspect, a method of video processing is disclosed that includes determining whether a coding representation includes an intra block copy (IBC) tool signaling for conversion between a coding representation of a video block of a video region and the video block, and performing the conversion based on the determination, the determination being based on a width and / or height of the video block and a maximum allowed IBC block size for the video region.

[0014] In another exemplary aspect, a method of video processing is disclosed that includes determining, using an intra block copy (IBC) tool, for conversion between a coding representation of the video block in a video domain and the video block to allow for conversion to divide the video block into a plurality of transform units (TUs), and performing the conversion based on the determination, the conversion including using separate motion information for the plurality of TUs.

[0015] In another exemplary aspect, a method of video processing is disclosed that includes determining that an intra block copy tool is enabled for conversion between a coded representation of a video block in a video domain and the video block, and performing the conversion using the intra block copy tool, and performing a prediction of the video block using filtered reconstructed samples in the video domain.

[0016] In another exemplary aspect, a method of video processing is disclosed that includes performing a conversion of a video including a plurality of video blocks and a coding representation of the video, where at least some of the video blocks are coded using motion vector information, the motion vector information being represented in the coding representation as a first portion based on first least significant bits of an absolute value of the motion vector information and as a second portion based on remaining more significant bits that are more significant than the first least significant bits.

[0017] In another exemplary aspect, a method of video processing is disclosed that includes determining whether a sub-block transform tool is enabled for transforming between a coded representation of a video block in a video domain and the video block, and performing the transform based on the determination, the determination being based on a maximum allowed transform block size for the video domain, and conditionally including a signaling in the coded representation based on the maximum allowed transform block size.

[0018] In another exemplary aspect, a method of video processing is disclosed, the method including: determining whether a low frequency non-separable transform (LFNST) is used during conversion for converting between a coding representation of a video block in a video domain and the video block, and performing the conversion based on the determination, the determination being based on a coding condition applied to the video block, and the LFNST and a matrix index for the LFNST being coded in the coding representation using two contexts.

[0019] In yet another exemplary embodiment, the method is embodied in a processor executable code and stored on a computer readable program medium.

[0020] In yet another exemplary aspect, a device configured or operable to perform the above-described method is disclosed. The device may include a processing device programmed to implement the method.

[0021] In yet another exemplary aspect, a video decoder device may implement the methods as described herein.

[0022] These and other aspects and features of the disclosed technology are explained in more detail in the drawings, description and claims. [Brief description of the drawings]

[0023] [Figure 1] 13 illustrates an example of a derivation process for building a merge candidate list. [Diagram 2] 1 shows examples of spatial merge candidate locations. [Diagram 3] 1 shows examples of candidate pairs considered for redundancy check of spatial merge candidates. [Figure 4] 13 shows example locations for the second PU of N×2N and 2N×N divisions. [Diagram 5] FIG. 13 is an illustration of motion vector scaling for temporal merge candidates. [Figure 6] 1 shows examples of candidate positions for temporal merge candidates, C0 and C1. [Figure 7] 1 illustrates an example of a joint bi-predictive merge candidate. [Figure 8] 13 shows an example of a process for deriving motion vector prediction candidates. [Figure 9] 1 illustrates a description of motion vector scaling for spatial motion vector candidates. [Figure 10] 13 shows examples of candidate positions for the affine merge mode. [Figure 11] 1 shows a modified merge list construction process. [Figure 12] 1 illustrates an example of inter prediction based on triangulation. [Figure 13] 1 illustrates an example of a central processing unit applying a first group of weighting factors. [Figure 14] 2 shows an example of a motion vector storage device. [Figure 15] An example of UMVE search processing is shown below. [Figure 16] An example of UMVE search points is shown. [Figure 17] FIG. 13 illustrates the operation of an intra block copy tool. [Figure 18] Here is an example of a secondary transformation in JEM. [Figure 19] An example of the Reduced Quadratic Transform (RST) is shown below. [Figure 20] FIG. 1 is a block diagram illustrating an example video processing system in which the disclosed techniques can be implemented. [Figure 21] FIG. 1 is a block diagram illustrating an example of a video processing device. [Figure 22] 1 is a flowchart illustrating an example of a video processing method. [Diagram 23] 1 is a flowchart illustrating a video processing method according to the present technology. [Figure 24] 10 is a flowchart illustrating another video processing method according to the present technology. [Diagram 25] 10 is a flowchart illustrating another video processing method according to the present technology. [Figure 26] 10 is a flowchart illustrating another video processing method according to the present technology. [Figure 27] 10 is a flowchart illustrating another video processing method according to the present technology. [Figure 28] 10 is a flowchart illustrating another video processing method according to the present technology. [Figure 29] 11 is a flowchart illustrating yet another video processing method according to the present technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used herein to improve readability of the description and are not intended to limit the description or embodiments (and / or implementations) to only the respective section.

[0025] 1. Summary of the Invention

[0026] This specification relates to video coding technology. In particular, it relates to Intra Block Copy (IBC, also known as Current Picture Reference, CPR) coding. It may be applied to existing video coding standards such as HEVC, or to finalize standards (Versatile Video Coding). The present invention is also applicable to future video coding standards or video codecs.

[0027] 2. Background technology

[0028] Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, ISO / IEC created MPEG-1 and MPEG-4 Visual, and the two organizations jointly created H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC standards. Since H.262, video coding standards have been based on hybrid video coding structures where temporal prediction and transform coding are utilized. In 2015, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) to explore future video coding technologies beyond HEVC. Since then, many new methods have been adopted by JVET and incorporated into reference software called JEM (Joint Exploration Mode). In April 2018, the Joint Video Expert Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG), and is working on formulating the VVC standard with the goal of reducing the bitrate by 50% compared to HEVC.

[0029] 2.1 Inter Prediction in HEVC / H.265

[0030] For a coding unit (CU) in inter-coding, it may be coded with one prediction unit (PU), two PUs according to the partition mode. Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists may be signaled using inter_pred_idc. The motion vector may be explicitly coded as a delta to the predictor.

[0031] If one CU is coded in skip mode, one PU is associated with this CU, there are no significant residual coefficients, and there are no coded motion vector differentials or reference picture indexes. A merge mode is specified, which obtains motion parameters for the current PU from neighboring PUs, including spatial and temporal candidates. The merge mode can be applied to any inter-predicted PU, not just for skip mode. As an alternative to the merge mode, there is an explicit transmission of motion parameters, where the motion vectors (more precisely, the motion vector differentials (MVDs) compared to the motion vector predictors), the corresponding reference picture indexes of each reference picture list, and the usage status of the reference picture lists are explicitly signaled to each PU. Such a mode is referred to as Advanced Motion Vector Prediction (AMVP) in this disclosure.

[0032] If the signaling indicates to use one of the two reference picture lists, the PU is generated from a block of one sample. This is called "uni-prediction." Uni-prediction is available for both P and B slices.

[0033] If the signaling indicates to use both reference picture lists, generate the PU from a block of two samples. This is called "bi-prediction." Bi-prediction is only available for B slices.

[0034] The inter prediction modes defined in HEVC will be described in detail below. First, the merge mode will be described.

[0035] 2.1.1 Reference Picture List

[0036] In HEVC, the term inter prediction is used to indicate a prediction derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. Similar to H.264 / AVC, an image can be predicted from multiple reference pictures. The reference pictures used for inter prediction are organized into one or more reference picture lists. A reference index identifies which reference picture in the list to use to generate the prediction signal.

[0037] One reference picture list, List0, is used for P slices, and two reference picture lists, List0 and List1, are used for B slices, where the reference pictures included in List0 / 1 may be from past and future pictures in terms of capture / display order.

[0038] 2.2.1 Merge Mode

[0039] 2.1.2.1. Deriving Merge Mode Candidates

[0040] When predicting a PU using merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to look up the motion information. The construction of this list is specified in the HEVC standard and can be summarized based on the following sequence of steps: ● Step 1: Derive initial candidates o Step 1.1: Spatial candidate derivation o Step 1.2: Redundancy check of spatial candidates o Step 1.3: Temporal candidate derivation Step 2: Insert additional candidates o Step 2.1: Creating bidirectional prediction candidates o Step 2.2: Insertion of zero motion candidates

[0041] These steps are also shown diagrammatically in FIG. 1, which shows an exemplary derivation process for merging candidate list construction. For spatial merging candidate derivation, select up to four merging candidates among candidates at five different positions. For temporal merging candidate derivation, select up to one merging candidate among two candidates. Since the decoder side expects a constant number of candidates per PU, if the number of candidates obtained in step 1 does not reach the maximum number of merging candidates (MaxNumMergeCand) signaled in the slice header, generate additional candidates. Since the number of candidates is constant, a truncated unary binarization (TU) is used to code the index of the best merging candidate. If the size of a CU is equal to 8, all PUs of the current CU share one merging candidate list, which is the same as the merging candidate list of 2N×2N prediction units.

[0042] The operations associated with the above steps are described in detail below.

[0043] 2.1.2.2 Spatial candidate derivation

[0044] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any of the PUs at positions A1, B1, B0, A0 are not available (e.g., because they belong to another slice or tile) or are intra-coded. After adding the candidate at position A1, the remaining candidates are subjected to a redundancy check, which ensures that candidates with the same motion information are removed from the list, improving coding efficiency. To reduce computational complexity, the aforementioned redundancy check does not consider all possible candidate pairs. Instead, only the pairs linked by arrows in FIG. 3 are considered, and a candidate is added to the list only if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a partition different from 2N×2N. As an example, FIG. 4 shows the second PU for N×2N and 2N×N cases, respectively. When splitting the current PU into N×2N, the candidate at position A1 is not considered in the list construction. In fact, adding this candidate would lead to two prediction units with the same motion information, which is redundant for having only one PU in one coding unit. Similarly, when splitting the current PU into 2N×N, position B1 is not considered.

[0045] 2.1.2.3 Temporal candidate derivation

[0046] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the co-located PU in the co-located picture. Figure 5 is an explanatory diagram of the scaling of motion vectors for a temporal merge candidate. As shown by the dotted line in Figure 5, a scaled motion vector of the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located PU using the POC distances tb and td. tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located PU and the co-located picture. Set the reference picture index of the temporal merge candidate equal to zero. The practical realization of this scaling process is described in the HEVC specification [1]. For B slices, two motion vectors, one for reference picture list 0 and the other for reference picture list 1, are obtained and combined to form a bi-predictive merge candidate.

[0047] 2.1.2.4 Co-located Pictures and Co-located PUs

[0048] When TMVP is enabled (eg, slice_temporal_mvp_enabled_flag is 1), the variable ColPic, which represents a col-located picture, is derived as follows:

[0049] - If the current slice is a B slice and the signal collocated_from_l0_flag is 0, ColPic is set equal to RefPicList1[collocated_ref_idx].

[0050] - Otherwise (slice_type is B and collocated_from_l0_flag is 1, or slice_type is P), set ColPic to RefPicList0[collocated_ref_idx].

[0051] Here, collocated_ref_idx and collocated_from_l0_flag are two syntax elements that can be signaled in the slice header.

[0052] For the co-located PU(Y) belonging to the reference frame, select a temporal candidate position between candidates C0 and C1 as shown in Figure 6. If the PU at position C0 is not available, is intra-coded, or is outside the current coding tree unit (CTU, a.k.a. LCU maximum coding unit) row, position C1 is used. Otherwise, position C0 is used to derive the temporal merge candidate.

[0053] The relevant syntax elements are described below.

[0054] [Table 1]

[0055] 2.1.2.5 Deriving MVs for TMVP Candidates

[0056] In some embodiments, to derive TMVP candidates, the following operations are performed.

[0057] 1) In list X, set reference picture list X=0 and set the target reference picture to the reference picture with index 0 (e.g., curr_ref). Invoke the derivation process of the co-located motion vector to obtain the MV of list X that points to curr_ref.

[0058] 2) If the current slice is a B slice, set reference picture list X=1, and the target reference picture is the reference picture with index 0 in list X (e.g., curr_ref). Invoke the co-located motion vector derivation process to get the MV of list X that points to curr_ref.

[0059] The process of derivation of co-located motion vectors is described in the next subsection, 2.1.2.5.1.

[0060] 2.1.2.5.1 Derivation of motion vectors located at the same position

[0061] For a co-located block, it may be intra-coded with uni- or bi-prediction, or it may be inter-coded. If it is intra-coded, the TMVP candidates are equally set to unavailable.

[0062] If it is uni-prediction from list A, scale the motion vectors of list A to reference picture list X.

[0063] If it is bi-predictive and the target reference picture list is X, scale the motion vectors of list A to the target reference picture list X, where A is determined according to the following rules:

[0064] - If none of the reference pictures has a larger POC value compared to the current picture, A is set equal to X.

[0065] - Otherwise, A is set equal to collocated_from_l0_flag.

[0066] 2.1.2.6 Additional Candidate Inserts

[0067] Besides the spatial-temporal merge candidates, there are two additional types of merge candidates: joint bi-predictive merge candidates and zero merge candidates. The spatial-temporal merge candidates are utilized to generate joint bi-predictive merge candidates. The joint bi-predictive merge candidates are used only for B slices. The joint bi-predictive candidate is generated by combining the first reference picture list motion parameters of the first candidate with the second reference picture list motion parameters of another candidate. If these two tuples provide different motion hypotheses, these tuples form a new bi-predictive candidate. Figure 7 shows an example of joint bi-predictive merge candidates. As an example, Figure 7 shows the case where two candidates with mvL0 and refIdxL0 or mvL1 and refIdxL1 in the original list (left side) are used to generate a joint bi-predictive merge candidate that is added to the final list (right side). There are various rules for the combinations considered to generate these additional merge candidates.

[0068] By inserting zero motion candidates and filling the remaining entries in the merge candidate list, the MaxNumMergeCand capacity is hit. These candidates have a spatial displacement of zero and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.

[0069] 2.1.3 Advanced Motion Vector Prediction (AMVP)

[0070] AMVP exploits the spatial-temporal correlation between motion vectors and neighboring PUs and uses it for explicit transmission of motion parameters. For each reference picture list, we first build a motion vector candidate list by checking the availability of the left and top temporal neighboring PU positions, removing redundant candidates, and adding zero vectors to keep the length of the candidate list constant. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the merge index signaling, the index of the best motion vector candidate is encoded using a shortened unary term. The maximum value to be coded in this case is 2 (see Figure 8). The following sections will explain the details of the motion vector prediction candidate derivation process.

[0071] 2.1.3.1 Derivation of AMVP Candidates

[0072] FIG. 8 summarizes the process of deriving motion vector predictor candidates.

[0073] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. To derive spatial motion vector candidates, as shown in Figure 2, we finally derive two motion vector candidates based on the motion vectors of each PU at five different positions.

[0074] To derive a temporal motion vector candidate, select one motion vector candidate from two candidates derived based on two different co-located positions. After creating an initial list of spatial-temporal candidates, remove duplicate motion vector candidates in the list. If the number of candidates is greater than two, remove from the list the motion vector candidates whose reference picture index in the associated reference picture list is greater than 1. If the number of spatial-temporal motion vector candidates is less than two, add an additional zero motion vector candidate to the list.

[0075] 2.1.3.2 Spatial Motion Vector Candidates

[0076] In deriving spatial motion vector candidates, we consider up to two candidates among five possible candidates derived from PUs located as shown in Fig. 2, which are in the same position as the motion merge. The order of derivation for the left side of the current PU is specified as A0, A1, scaled A0, scaled A1. The order of derivation for the top side of the current PU is specified as B0, B1, B2, scaled B0, scaled B1, scaled B2. So, for each edge, there are four possible cases for motion vector candidates: two cases that do not require spatial scaling and two cases that use spatial scaling. The four different cases can be summarized as follows:

[0077] ● No spatial scaling -(1) The same reference picture list and the same reference picture index (same POC) - (2) Different reference picture lists, but the same reference picture (same POC) ● Spatial scaling - (3) Same reference picture list, but different reference pictures (different POC) (4) Different reference picture lists and different reference pictures (different POCs)

[0078] First check the non-spatial scaling case, then do spatial scaling. Regardless of the reference picture list, if the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, consider spatial scaling. If all PUs of the left candidate are not available or are intra-coded, scaling of the top motion vector is useful for parallel derivation of the left and top MV candidates. Otherwise, spatial scaling is not allowed for the top motion vector.

[0079] In the spatial scaling process, the motion vectors of neighboring PUs are scaled in the same manner as in temporal scaling, as shown in Figure 9. The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as in temporal scaling.

[0080] 2.1.3.3 Temporal Motion Vector Candidates

[0081] Except for deriving the reference picture index, all the processes for deriving the temporal merge candidates are the same as those for deriving the spatial motion vector candidates (see FIG. 6). The reference picture index is signaled to the decoder.

[0082] 2.2 Inter Prediction Method in VVC

[0083] New coding tools to improve inter prediction include Adaptive Motion Vector Difference Resolution (AMVR) for signaled MVD, Merged with Motion Vector Difference (MMVD), Triangular Prediction Mode (TPM), Combined Intra-Inter Prediction (CIIP), Advanced TMVP (ATMVP, aka SbTMVP), Affine Prediction Mode, Generalized Bidirectional Prediction (GBI), Decoder-side Motion Vector Refinement (DMVR), and Bidirectional Optical Flow (BIO, aka BDOF).

[0084] There are two different merge list building processes supported by VVC.

[0085] (1) Sub-block merge candidate list: includes ATMVP and affine merge candidates. One merge list construction process is shared for both affine and ATMVP modes. Note that ATMVP and affine merge candidates may be added in sequence. The size of the sub-block merge list is signaled in the slice header, and its maximum value is 5.

[0086] (2) Regular merge list: For inter-coding blocks, we share one merge list construction process. Here, spatial / temporal merge candidates, HMVPs, pairwise combined merge candidates, and zero motion candidates may be inserted in order. The size of the regular merge list is signaled in the slice header, and its maximum value is 6. MMVD, TPM, and CIIP rely on the regular merge list.

[0087] Similarly, there are three AMVP lists supported by VVC: (1) Affine AMVP Candidate List (2) Regular AMVP candidate list

[0088] 2.2.1 Coding Block Structure in VVC

[0089] In VVC, a quadtree / binarytree / ternarytree (QT / BT / TT) structure is adopted to divide a picture into square or rectangular blocks.

[0090] Besides QT / BT / TT, a separate tree (also called a dual coding tree) is also adopted in VVC for I-frames. Using separate trees, the coding block structure is signaled separately for luma and chroma components.

[0091] In addition, the CU is set to PU and TU except for blocks coded using two specific coding methods (e.g., intra sub-partition prediction, where PU is equal to TU but smaller than CU, and sub-block transformation of inter-coding blocks, where PU is equal to CU but TU is smaller than PU).

[0092] 2.2.2 Merging Whole Blocks

[0093] 2.2.2.1 Merge list construction in translational normal merge mode

[0094] 2.2.2.1.1 History-Based Motion Vector Prediction (HMVP)

[0095] Unlike the merge list design, VVC employs a history-based motion vector prediction (HMVP) method.

[0096] In HMVP, the previously coded motion information is stored. The motion information of the previously coded block is defined as an HMVP candidate. Multiple HMVP candidates are stored in a table denoted as HMVP table, which is maintained on-the-fly during the encoding / decoding process. When starting to encode / decode a new tile / LCU row / slice, the HMVP table is emptied. Whenever there are inter-coded blocks and non-sub-blocks in non-TPM mode, the associated motion information is added to the last entry of the table as a new HMVP candidate. The overall coding flow is shown in Figure 10.

[0097] 2.2.2.1.2 Normal merge list construction process

[0098] Building a typical merge list (for translation) can be summarized by following the sequence of steps:

[0099] Step 1: Derive spatial candidates Step 2: Inserting HMVP candidates Step 3: Inserting pairwise average candidates Step 4: Default movement candidates

[0100] HMVP candidates can be used for both AMVP and merge candidate list construction process. Figure 11 shows the modified merge candidate list construction process (highlighted in blue). After inserting the TMVP candidates, if the merge candidate list is not full, the HMVP candidates stored in the HMVP table can be utilized to fill the merge candidate list. Considering that a block usually has a high correlation with its nearest neighbors in terms of motion information, the HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first, and the first entry at the end. Similarly, redundancy elimination is applied to the HMVP candidates. When the total number of available merge candidates reaches the maximum number of mergeable merge candidates that can be signaled, the merge candidate list construction process is terminated.

[0101] Note that all spatial / temporal / HMVP candidates are coded in non-IBC mode, otherwise they are not allowed to be added to the regular merge candidate list.

[0102] The HMVP table contains up to five canonical move candidates, each unique.

[0103] 2.2.2.1.2.1 Pruning Process

[0104] A candidate is added to the list only if the corresponding candidate used in the redundancy check does not have the same motion information. Such a comparison process is called a pruning process.

[0105] The pruning process among spatial candidates depends on the use of the TPM of the current block.

[0106] If the current block is not coded in a TPM mode (e.g., regular merging, MMVD, CIIP), the HEVC pruning process for spatial merging candidates (e.g., 5 pruning) is utilized.

[0107] 2.2.3 Triangle Prediction Mode (TPM)

[0108] In VVC, triangle partition mode is supported for inter prediction. Triangulation mode applies only to CUs that are 8x8 or larger and coded in merge mode, and does not apply in MMVD or CIIP modes. For CUs that meet these conditions, a CU level flag is signaled to indicate whether triangulation mode applies.

[0109] When using this mode, a CU is divided into two equal triangular partitions using either diagonal or anti-diagonal partitioning, as shown in Figure 11. Each triangular partition in a CU is inter-predicted using its own motion, and only uni-prediction is allowed for each partition, i.e., each partition has one motion vector and one reference index. In order to require only two motion-compensated predictions per CU, similar to traditional bi-prediction, a uni-prediction motion constraint is applied.

[0110] FIG. 12 shows an example of inter prediction based on triangulation.

[0111] If the CU level flag indicates that the current CU is coded in triangular partition mode, we further signal a flag indicating the direction of the triangular partition (diagonal or anti-diagonal) and two merge indices (one for each partition). After predicting each of the triangular partitions, we use an adaptive weighted blending process to adjust the sample values ​​along the diagonal or anti-diagonal edges. This is the prediction signal for the entire CU, and we perform transform and quantization processes on the entire CU, similar to other prediction modes. Finally, we store the motion field of the CU predicted using the triangular partition mode in 4x4 units.

[0112] The normal merge candidate list is reused for triangulation merge prediction without extra pruning of motion vectors. For each merge candidate in the normal merge candidate list, only one of its L0 or L1 motion vectors is used for triangulation prediction. And the order of selecting L0 vs. L1 motion vectors is based on their merge index parity. According to this scheme, the normal merge list can be used directly.

[0113] 2.2.3.1 TPM Merge List Construction Process

[0114] Essentially, the usual merge list construction process can be slightly modified. In particular, the following applies:

[0115] (1) How the pruning process is performed depends on the TPM usage of the current block.

[0116] - If the current block is not TPM coded, HEVC5 pruning applied to spatial merging candidates is invoked.

[0117] - Otherwise (if the current block is TPM coded), full pruning is applied when adding new spatial merge candidates, i.e., compare B1 with A1, compare B0 with A1,B1, compare A0 with A1,B1, compare B0,B2 with A1,B1,A0,and B0.

[0118] (2) The condition for checking the motion information from B2 depends on the TPM usage for the current block.

[0119] - If the current block is not TPM coded, access B2 and check B2 only if there are less than four spatial merge candidates before checking B2.

[0120] - Otherwise (if the current block is TPM coded), always access and check B2 before adding it, regardless of the number of available spatial merge candidates.

[0121] 2.2.3.2 Adaptive Weighting

[0122] After each triangular prediction unit is predicted, an adaptive weighting process is applied to the diagonal edge between two triangular prediction units to derive the final prediction for the entire CU. Two weighting coefficient groups are defined as follows:

[0123] For luma and chroma samples, the first weighting factor groups of {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used, respectively.

[0124] The second weighting factor groups of {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for the luma and chroma samples, respectively.

[0125] A weighting factor group is selected based on a comparison of the motion vectors of the two triangular prediction units. The second weighting factor group is used if any one of the following conditions is true:

[0126] - The reference pictures of the two triangular prediction units are different from each other. - The absolute value of the difference between the horizontal values ​​of the two motion vectors is greater than 16 pixels. - The absolute value of the difference between the vertical values ​​of the two motion vectors is greater than 16 pixels.

[0127] If not, the first weighting factor group is used. An example is shown in FIG.

[0128] 2.2.3.3 Motion Vector Storage

[0129] The motion vectors of the triangular prediction unit (Mv1, Mv2 in FIG. 14) are stored in 4×4 grids. For each 4×4 grid, a uni-predictive or bi-predictive motion vector is stored based on the location of the 4×4 grid in the CU. As shown in FIG. 14, for a 4×4 grid located in a non-weighted area (i.e., not located on a diagonal edge), a uni-predictive motion vector of either Mv1 or Mv2 is stored. Meanwhile, for a 4×4 grid located in a weighted area, a bi-predictive motion vector is stored. A bi-predictive motion vector is derived from Mv1 and Mv2 according to the following rules:

[0130] (1) If Mv1 and Mv2 have motion vectors in different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional predictive motion vector.

[0131] (2) If both Mv1 and Mv2 are coming from the same L0 (or L1) direction, then: - If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, Mv2 is scaled to that picture. Mv1 and the scaled Mv2 are combined to form a bi-predictive motion vector. - If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bi-predictive motion vector. - Otherwise, only Mv1 is stored for the weighting region.

[0132] 2.2.4 Merging with Motion Vector Difference (MMVD)

[0133] In some embodiments, for the proposed motion vector representation method, the ultimate motion vector representation (UMVE, also known as MMVD) is used for either skip or merge mode.

[0134] UMVE reuses merge candidates similar to those in the regular merge candidate list in VVC. Among the merge candidates, a base candidate can be selected, which is further extended by the proposed motion vector representation method.

[0135] UMVE provides a new motion vector difference (MVD) representation method, where a starting point, a motion magnitude, and a motion direction are used to represent one MVD.

[0136] FIG. 15 shows an example of a UMVE search process.

[0137] FIG. 16 shows an example of UMVE search points.

[0138] Our technique uses the merge candidate list as is. However, to extend UMVE, we only consider candidates with the default merge type (MRG_TYPE_DEFAULT_N).

[0139] The base candidate index defines a starting point. The base candidate index indicates the best candidate among the candidates in the list as follows:

[0140] [Table 2]

[0141] If the number of base candidates is 1, the base candidate IDX is not signaled.

[0142] The distance index is information on the magnitude of the movement. The distance index indicates a predefined distance from the starting point information. The predefined distances are as follows:

[0143] [Table 3]

[0144] The direction index represents the direction of the MVD relative to the starting point. The direction index can represent four directions, as shown below.

[0145] [Table 4]

[0146] Immediately after sending the skip or merge flag, signal the UMVE flag. If the skip or merge flag is true, parse the UMVE flag. If the UMVE flag is 1, UMVE syntax is parsed. But if it is not 1, parse the AFFINE flag. If the AFFINE flag is 1, i.e. AFFINE mode, but not 1, parse the skip / merge index for VTM skip / merge mode.

[0147] No additional line buffers due to UMVE candidates are required, since we use the software skip / merge candidates directly as the base candidates. The input UMVE index is used to determine the MV completion just before motion compensation. There is no need to maintain a long line buffer for this.

[0148] Under the current common test conditions, either the first or second merge candidate in the merge candidate list may be selected as the base candidate.

[0149] UMVE is also known as Merge with MV Difference (MMVD).

[0150] 2.2.5 Merging for Sub-Block-Based Techniques

[0151] In some embodiments, all sub-block related motion candidates are put into a separate merge list in addition to the regular merge list of non-sub-block merge candidates.

[0152] Sub-block related motion candidates are put into a separate merge list, called the "sub-block merge candidate list."

[0153] In one example, the sub-block merging candidate list includes ATMVP candidates and affine merging candidates.

[0154] The subblock merging candidate list is filled with candidates in the following order: a. ATMVP candidates (possibly available or unavailable). b. An affine merge list (containing inheritance affine candidates and construction affine candidates). c.0Padding as a MV4 parameter affine model

[0155] 2.2.5.1 Advanced Temporal Motion Vector Prediction Module (ATMVP) (also known as Sub-block Temporal Motion Vector Predictor, SbTMVP)

[0156] The basic idea of ​​ATMVP is to derive multiple temporal motion vector predictors for one block. Each subblock is assigned one set of motion information. When generating ATMVP merge candidates, we perform motion compensation at the 8x8 level instead of the whole block level.

[0157] In the current design, ATMVP predicts motion vectors for sub-CUs within a CU in two steps, described below in subsections 2.2.5.1.1 and 2.2.5.1.2, respectively.

[0158] 2.2.5.1.1 Deriving the Initial Motion Vector

[0159] Here, let the initialized motion vector be tempMv. If block A1 is available and is not intra-coded (e.g., coded in inter or IBC mode), the following applies to derive the initialized motion vector: - If all of the following conditions are true, tempMv is set equal to the motion vector of block A1 from list 1, represented by mvL1A1: - a reference picture index in list 1 is available (is not -1) and has the same POC value as the co-located picture (e.g. DiffPicOrderCnt(ColPic,RefPicList[1][refIdxL1A1)] is equal to 0), - All reference pictures have no greater POC than the current picture (e.g., DiffPicOrderCnt(aPic, currPic) is less than or equal to 0 for all pictures aPic in all reference picture lists of the current slice). - The current slice is the B slice. - collocated_from_l0_flag is equal to 0. - Otherwise, if all of the following conditions are true, then tempMv is set equal to the motion vector of block A1 from list 0, represented by mvL0A1: - The reference picture index in list 0 is available (it is not -1). - It has the same POC value as the co-located picture (e.g. DiffPicOrderCnt(ColPic,RefPicList[0][refIdxL0A1]) equals 0). - Otherwise, the zero motion vector is used as the initialized MV.

[0160] Identify the corresponding block (add the rounded MV to the center of the current block, clipped to range if necessary) in the co-located picture where the initialized motion vector is signaled in the slice header.

[0161] If the block is inter-coded, proceed to the second step. Otherwise, set the ATMVP candidate to unavailable.

[0162] 2.2.5.1.2 Sub-CU motion derivation

[0163] The second step is to divide the current CU into sub-CUs, and obtain the motion information of each sub-CU from its corresponding block in the co-located picture.

[0164] If the corresponding block for one sub-CU is coded in inter mode, this motion information is used to derive the final motion information of the current sub-CU by invoking the derivation process for the co-located MV, which is not different from the case of conventional TMVP processing. Basically, if the corresponding block is predicted from target list X for uni- or bi-prediction, the motion vector is utilized; otherwise, if it is predicted from list Y (Y=1-X) for uni- or bi-prediction and NoBackwardPredFlag is 1, the MV for list Y is utilized. Otherwise, no motion candidate can be found.

[0165] If the co-located block in the picture identified by the initialized MV and the position of the current sub-CU is intra-coded or IBC coded, or if no motion candidate can be found as described above, the following further applies.

[0166] Co-located Picture R col The motion vectors used to extract the motion regions in MV col In order to minimize the effect of MV scaling, MV col For an MV in the spatial candidate list for deriving ,,if the reference picture of the candidate MV is a co-located picture, this MV is selected, and the MV is derived without scaling. col Otherwise, select the MV with the reference picture closest to the co-located picture and use it as the MV in scaling. col Derive:

[0167] The decoding process for the co-located motion vector derivation process in the embodiment of the present invention will now be described.

[0168] 8.5.2.12 Co-located motion vector derivation process

[0169] The inputs to this process are: - the variable currCb that defines the current coding block, - a variable colCb specifying a coding block arranged at the same position in the picture arranged at the same position specified by ColPic; A luminance location (xColCb, yColCb) specifying the top-left sample of a co-located luma coding block defined by colCb relative to the top-left luma sample of the co-located picture defined by ColPic; - a reference index refIdxLX, where X is 0 or 1; - Flag indicating sub-block temporal merging candidate sbFlag.

[0170] The output of this process is: - Motion vector prediction mvLXCol with 1 / 16 fractional sample accuracy. -Availability flag availableFlagLXCol.

[0171] The variable currPic specifies the current picture. The arrays predFlagL0Col[x][y], mlvL0Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL0[x][y], MvDmvrL0[x][y], and RefIdxL0[x][y], respectively, of the co-located picture specified by ColPic, and the arrays predFlagL1Col[x][y], mvL1Col[x][y], and refIdxL1Col[x][y] are set equal to PredFlagL1[x][y], MvDmvrL1[x][y], and RefIdxL1[x][y], respectively, of the co-located picture specified by ColPic.

[0172] The variables mvLXCol and availableFlagLXCol are derived as follows. - If colCb is coded in intra or IBC prediction mode, both modules of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0. - Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows: If -sbFlag is 0, availableFlagLXCol is set equal to 1 and the following applies: -If predFlagL0Col[xColCb][yColCb] is 0, then mvCol, refIdxCol and listCol are set equal to mvL1Col[xColCb][yColCb], refIdxL1Col[xColCb][yColCb] and L1, respectively. - Otherwise, if predFlagL0Col[xColCb][yColCb] is equal to 1 and predFlagL1Col[xColCb][yColCb] is 0, then mvCol, refxIdCol and listCol are set equal to mvL0Col[xColCb][yColCb], refIdxL0Col[xColCb][yColCb] and L0, respectively. - Otherwise (predFlagL0Col[xColCb][yColCb] is equal to 1 and predFlagL1Col[xColCb][yColCb] is 1), make the following assignments: -If NoBackwardPredFlag is 1, then mvCol, refIdxCol and listCol are set equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb], and LX, respectively. - Otherwise, set mvCol, refIdxCol, and listCol equal to mvLNCol[xColCb][yColCb], refIdxLNCol[xColCb][yColCb] and LN, respectively, where N is the value of collocated_from_l0_flag. - Otherwise (sbFlag is 1), the following applies: - if PredFlagLXCol[xColCb][yColCb] is 1, then mvCol, refxIdCol and listCol are set equal to mvLXCol[xColCb][yColCb], refIdxLXCol[xColCb][yColCb] and LX, respectively, and availableFlagLXCol is set equal to 1. - Otherwise (PredFlagLXCol[xColCb][yColCb] is 0), the following applies: -If DiffPicOrderCnt(aPic, currPic) is less than or equal to 0 in all reference picture lists aPic of the current slice and PredFlagLYCol[xColCb][yColCb] is 1, then mvCol, refxIdCol, and listCol are set equal to mvLYCol[xColCb][yColCb], refIdxLYCol[xColCb][yColCb], and LY, respectively, where Y is !X, where X is the value of X for which this process is called, and availableFlagLXCol is set equal to 1. Both modules of -mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0. If -availableFlagLXCol is TRUE, then mvLXCol and availableFlagLXCol are derived as follows: - if LongTermRefPic(currPic, currCb, refIdxLX, LX) is LongTermRefPic(ColPic, colCb, refIdxCol, listCol), the modules of mvLXCol are both set to 0 and availableFlagLXCol is set equal to 0. - Otherwise, set the variable availableFlagLXCol to 1, set refPicList[listCol][refIdxCol] to the picture with reference index refIdxCol in the reference picture list listCol of the slice containing the coding block colb in the co-located picture specified by ColPic, and apply: colPocDiff=DiffPicOrderCnt(ColPic,refPicList[listCol][refIdxCol]) (8-402) currPocDiff=DiffPicOrderCnt(currPic,RefPicList[X][refIdxLX]) (8-403) - The co-located motion vector temporal motion buffer compression process as specified in subclause 8.5.2.15 is invoked with mvCol as input and modified mvCol as output. - If RefPicList[X][refIdxLX] is a long-term reference picture or if colPocDiff is currPocDiff, then mvLXCol is derived as follows: mvLXCol=mvCol (8-404) Otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol as follows: tx=(16384+(Abs(td)>>1)) / td (8-405) distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6) (8-406) mvLXCol=Clip3(-131072,131071,(distScaleFactor*mvCol+128-(distScaleFactor*mvCol>=0))>>8)) (8-407) Here, td and tb are derived as follows: td=Clip3(-128,127,colPocDiff) (8-408) tb=Clip3(-128,127,currPocDiff) (8-409)

[0173] 2.3 Intra-block copy

[0174] Intra block copy (IBC), also known as current picture reference, is adopted in the HEVC Screen Content Coding Extension (HEVC-SCC) and the current VVC Test Model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in Figure 17, Figure 17 shows the operation of intra block copy, where the current block is predicted by one reference block in the same picture when IBC is applied. Before encoding or decoding the current block, the samples in the reference block must already be reconstructed. IBC is not very efficient for most camera-captured sequences, but it shows significant coding gains for screen content. The reason is that there are many repeating patterns, such as icons, characters, etc., in screen content pictures. IBC can effectively remove the redundancy between these repeating patterns. In HEVC-SCC, an inter-coding unit (CU) can apply IBC if it selects the current picture as its reference picture. In this case, the MV is renamed as block vector (BV), and the BV always has integer pixel accuracy. To conform to the Main Profile HEVC, the current picture is marked as a "long-term" reference picture in the Decoded Picture Buffer (DPB), just as in multiple view / 3D video coding standards, inter-view reference pictures are also marked as "long-term" reference pictures.

[0175] After the BV finds its reference block, it can generate a prediction by copying this reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Then, transforms and quantization can be applied as in other coding modes.

[0176] However, if the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed region, or is outside the valid region limited by some constraint, some or all of the pixel values ​​are undefined. Basically, there are two solutions to deal with such problems. One is to not allow such situations, e.g. bitstream conformance. The other is to apply padding to these undefined pixel values. The following sub-sections will explain the solutions in detail.

[0177] 2.3.1 Single BV List

[0178] In some embodiments, the BV predictors for merge mode and AMVP mode in IBC share a common predictor list that includes the following elements:

[0179] (1) Two spatially adjacent locations (A1 and B1 as in Figure 2) (2) Five HMVP entries (3) Default zero vector

[0180] The number of candidates in the list is controlled by a variable derived from the slice header. In merge mode, up to the first six entries of this list are used, and in AMVP mode, the first two entries of this list are used. The list also complies with the requirements for a shared merge list area (sharing the same list within an SMR).

[0181] In addition to the BV predictor candidate list mentioned above, the pruning operation between HMVP candidates and existing merge candidates (A1, B1) can be simplified by only comparing the first HMVP candidate with the spatial merge candidates, so that the pruning operation is limited to a maximum of two times.

[0182] In IBC AMVP mode, send the mv difference with the selected MVP in the list as well. In IBC merge mode, use the selected MVP as it is as the MV for the current block.

[0183] 2.3.2 Size Restrictions for IBCs

[0184] In recent VVC and VTM5, in previous VTM and VVC versions, it was proposed to explicitly use a syntax constraint to disable IBC mode at 128x128 in addition to the current bitstream constraints, so that the presence of the IBC flag depends on CU size < 128x128.

[0185] 2.3.3 Shared merge list for IBC

[0186] To reduce decoder complexity and support parallel coding, in some embodiments, the same merge candidate list is shared for all leaf coding units (CUs) of one ancestor node in a CU partition tree to enable parallel processing of small skip / merge coded CUs. The ancestor node is called a merge share node. A shared merge candidate list is generated at the merge share node, making it appear as if the merge share node is a leaf CU.

[0187] Specifically, the following may apply:

[0188] - Use merge lists between very small blocks (e.g., two adjacent 4x4 blocks) if the block has 32 luma samples or less (e.g., 4x8 or 8x4) and is split into two 4x4 child blocks.

[0189] - However, if the luma samples of this block are greater than 32, then after one split, at least one child block is smaller than the threshold (32), so all children of this split will share the same merge list (e.g. 16x4 or 4x16 split into 3, or 8x8 split into 4).

[0190] Such restrictions apply only to the IBC merge mode.

[0191] 2.3.4 Syntax Tables

[0192] [Table 5] [Table 6]

[0193] [Table 7]

[0194] 2.4 Maximum Transform Block Size

[0195] Only the maximum luma transform size, which is either 64 or 32, is enabled by a flag at the SPS level. The maximum chroma transform size is derived from the chroma sampling ratio to the maximum luma transform size.

[0196] If the CU / CB size is larger than the maximum luma transform size, tiling of large CUs with smaller TUs can be invoked.

[0197] Let MaxTbSizeY represent the size of the maximum luminance transform.

[0198] [Table 8]

[0199] sps_max_luma_transform_size_64_flag equal to 1 specifies that the maximum transform size for luma samples is equal to 64. sps_max_luma_transformation_size_64_flag equal to 0 specifies that the maximum transform size for luma samples is equal to 32. If CtbSizeY is less than 64, the value of sps_max_luma_transform_size_64_flag is equal to 0. The variables MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, and MaxTbSizeY are derived as follows: MinTbLog2SizeY=2 (7-27) MaxTbLog2SizeY=sps_max_luma_transform_size_64_flag?6:5 (7-28) MinTbSizeY=1< <MinTbLog2SizeY (7-29) MaxTbSizeY=1< <MaxTbLog2SizeY (7-30) sps_sbt_max_size_64_flag=0 specifies that the maximum CU width and height to allow for sub-block transforms is 32 luma samples. sps_sbt_max_size_64_flag=1 specifies that the maximum CU width and height to allow for sub-block transforms is 64 luma samples. MaxSbtSize=Min(MaxTbSizeY,sps_sbt_max_size_64_flag ? 64:32) (7-32)

[0200] 2.4.1 Tools that depend on maximum conversion block size

[0201] The splitting of binary and ternary trees depends on MaxTbSizeY.

[0202] [Table 9] [Table 10] [Table 11] [Table 12]

[0203] 2.4.2 Decoding Process of Intra- and Inter-Coding Blocks

[0204] If the size of the current intra-coding CU is larger than the maximum transform block size, the CU is recursively divided into smaller blocks (PUs) until both the width and height are not larger than the maximum transform block size. All PUs share the same intra-prediction mode, but the reference samples for intra-prediction are different. Meanwhile, the TU is set equal to the PU. Note that the current block must not be coded in intra-sub-partition (ISP) mode.

[0205] If the size of the current inter-coding CU is larger than the maximum transform block size, the CU is recursively divided into smaller blocks (TUs) until both the width and height are not larger than the maximum transform block size. All TUs share the same motion information but have different reference samples for intra prediction.

[0206] 8.4.5 Intrablock Decoding Process

[0207] 8.4.5.1 General Decoding Process for Intra Blocks

[0208] The inputs to this process are: a sample position (xTb0, yTb0) that defines the top-left sample of the current transform block relative to the top-left sample of the current picture; - the variable nTbW, which specifies the width of the current transformation block; - the variable nTbH, which specifies the height of the current transformation block; - a variable predModeIntra that specifies the intra prediction mode; - A variable cIdx that defines the color components of the current block.

[0209] The output of this processing is the modified reconstructed image before in-loop filtering. The maximum transform block width, maxTbWidth, and height, maxTbHeight, are derived as follows: maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubWidthC (8-41) maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubHeightC (8-42) The luma sample positions are derived as follows: (xTbY,yTbY)=(cIdx==0)?(xTb0,yTb0):(xTb0*SubWidthC,yTb0*SubHeightC) (8-43) Based on maxTbSize the following applies: If IntraSubPartionsSplitType is equal to ISP_NO_SPLIT and nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the following ordered steps are applied:

[0210] 1. The variables newTbW and newTbH are derived as follows: newTbW=(nTbW>maxTbWidth)?(nTbW / 2):nTbW (8-44) newTbH=(nTbH>maxTbHeight)?(nTbH / 2):nTbH (8-45)

[0211] 2. The general decoding process for an intra block specified in this section is called with inputs the position (xTb0, yTb0), the transform block width nTbW set equal to newTbW and height nTbH set equal to newTbH, the intra prediction mode predModeIntra, and the variable cIdx, and the output is the modified reconstructed picture before in-loop filtering.

[0212] 3. If nTbW is greater than maxTbWidth, the general decoding process for intra blocks specified in this section is called with inputs: position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0), transform block width nTbW set equal to newTbW and height nTbH set equal to newTbH, intra prediction mode predModeIntra, and variable cIdx, and the output is the modified reconstructed picture before in-loop filtering.

[0213] 4. If nTbH is greater than maxTbHeight, the general decoding process for intra blocks specified in this section is called with inputs position (xTb0, yTb0) set equal to (xTb0, yTb0 + newTbH), transform block width nTbW set equal to newTbW and height nTbH set equal to newTbH, intra prediction mode predModeIntra, and variable cIdx, and the output is the modified reconstructed picture before in-loop filtering.

[0214] 5. If nTbW is greater than maxTbWidth and nTbH is greater than maxTbHeight, the general decoding process for intra blocks specified in this section is called with inputs: position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0+newTbH), transform block width nTbW set equal to newTbW and height nTbH set equal to newTbH, intra prediction mode predModeIntra, and variable cIdx, and the output is the modified reconstructed picture before in-loop filtering. Alternatively, the following ordered steps are applied (normal intra prediction process): - …

[0215] 8.5.8 Decoding process for residual signal of coding block coded in inter prediction mode

[0216] The inputs to this process are: a sample position (xTb0, yTb0) that defines the top-left sample of the current transform block relative to the top-left sample of the current picture; - the variable nTbW, which specifies the width of the current transformation block; - the variable nTbH, which specifies the height of the current transformation block; - A variable cIdx that defines the color components of the current block.

[0217] The output of this process is a (nTbW) x (nTbH) array resSamples. The maximum transform block width, maxTbWidth, and height, maxTbHeight, are derived as follows: maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubWidthC (8-883) maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubHeightC (8-884) The luma sample positions are derived as follows: (xTbY,yTbY)=(cIdx==0)?(xTb0,yTb0):(xTb0*SubWidthC,yTb0*SubHeightC) (8-885) Depending on maxTbSize the following applies:

[0218] - If nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the following ordered steps apply:

[0219] 1. The variables newTbW and newTbH are derived as follows: newTbW=(nTbW>maxTbWidth)?(nTbW / 2):nTbW (8-886) newTbH=(nTbH>maxTbHeight)?(nTbH / 2):nTbH (8-887)

[0220] 2. Invoke a decoding process on the residual signal of a coding unit coded in inter prediction mode as specified in this section with inputs the position (xTb0, yTb0), the transform block width nTbW set equal to newTbW, the height nTbH set equal to newTbH, and the variable cIdx, and the output is the modified reconstructed picture before in-loop filtering.

[0221] 3. If nTbW is greater than maxTbWidth, call a decoding process for the residual signal of a coding unit coded in inter prediction mode as specified in this section using the position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0), the transform block width nTbW set equal to newTbW, the height nTbH set equal to newTbH, and the variable cIdx as input, and the output is a modified reconstructed picture.

[0222] 4. If nTbH is greater than maxTbHeight, call a process for decoding the residual signal of a coding unit coded in inter prediction mode as specified in this section using the position (xTb0, yTb0) set equal to (xTb0, yTb0 + newTbH), the transform block width nTbW set equal to newTbW, the height nTbH set equal to newTbH, and the variable cIdx as input, and the output is a modified reconstructed picture before in-loop filtering.

[0223] 5. If nWTbW is greater than maxTbWidth and nTbH is greater than maxTbHeight, call a process for decoding the residual signal of a coding unit coded in inter prediction mode as specified in this section using the position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0+newTbH), the transform block width nTbW set equal to newTbW, the height nTbH set equal to newTbH, and the variable cIdx as input, and the output is a modified reconstructed picture before in-loop filtering.

[0224] - Or, if cu_sbt_flag is equal to 1, the following applies: The variables sbtMinNumFourth, wPartIdx, and hPartIdx are derived as follows: sbtMinNumFourths=cu_sbt_quad_flag?1:2 (8-888) wPartIdx=cu_sbt_horizontal_flag?4:sbtMinNumFourths (8-889) hPartIdx=!cu_sbt_horizontal_flag?4:sbtMinNumFourths (8-890) The variables xPartIdx and yPartIdx are derived as follows: - If cu_sbt_pos_flag is equal to 0, xPartIdx and yPartIdx are set equal to 0. - Otherwise (cu_sbt_pos_flag is equal to 1), the variables xPartIdx and yPartIdx are derived as follows: xPartIdx=cu_sbt_horizontal_flag?0:(4-sbtMinNumFourths) (8-891) yPartIdx=!cu_sbt_horizontal_flag?0:(4-sbtMinNumFourths) (8-892) The variables xTbYSub, yTbYSub, xTb0Sub, yTb0Sub, nTbWSub and nTbHSub are derived as follows: xTbYSub=xTbY+((nTbW*((cIdx==0)?1:SubWidthC)*xPartIdx / 4) (8-893) yTbYSub=yTbY+((nTbH*((cIdx==0)?1:SubHeightC)*yPartIdx / 4) (8-894) xTb0Sub=xTb0+(nTbW*xPartIdx / 4) (8-895) yTb0Sub=yTb0+(nTbH*yPartIdx / 4) (8-896) nTbWSub=nTbW*wPartIdx / 4 (8-897) nTbHSub=nTbH*hPartIdx / 4 (8-898) - Invoke the scaling and transformation process as specified in 8.7.2 taking as input the luminance position (xTbYSub, yTbYSub), the variables cIdx, nTbWSub, and nTbHSub, and the output is the (nTbWSub) by (nTbHSub) array resSamplesTb. - Set the residual samples resSamples[x][y](x=0..nTbW-1,y=0..nTbH-1) equal to 0. - The residual samples resSamples[x][y] (x=xTb0Sub..xTb0Sub+nTbWSub-1, y=yTb0Sub..yTb0Sub+nTbHSub-1) are derived as follows: resSamples[x][y]=resSamplesTb[x-xTb0Sub][y-yTb0Sub] (8-899) - Alternatively, invoke the scaling and transformation process as specified in 8.7.2 using the luminance location (xTbY, yTbY), the variable cIdx, the transformation width nTbW, and the transformation height nTbH as input, and the output is the (nTbW) x (nTbH) array resample.

[0225] 2.5 Low Frequency Nonseparable Transform (LFNST) (also known as Reduced Quadratic Transform (RST) / Nonseparable Quadratic Transform (NSST))

[0226] In JEM, the secondary transform is between the forward primary transform and the quantization (at the encoder). and is applied between the inverse quantization and the inverse primary transform (at the decoder side). As shown in Figure 18, 4x4 (or 8x8) secondary transforms are performed depending on the block size. For example, a 4x4 secondary transform is applied to small blocks (e.g., min(width, height)<8) and an 8x8 secondary transform is applied to larger blocks (e.g., min(width, height)>4) per 8x8 block.

[0227] Since a non-separable transformation is applied to the secondary transformation, it is also called the Non-Separable Secondary Transform (NSST). There are 35 transformation sets in total, and three non-separable transformation matrices (kernels, each with a 16x16 matrix) are used per transformation set.

[0228] According to the intra prediction direction, we introduce the Reduced Quadratic Transform (RST), also known as the Low Frequency Non-separable Transform (LFNST), and introduce a mapping of 4 transform sets (instead of 35 transform sets). In this contribution, 16×48 and 16×16 matrices are used for 8×8 and 4×4 blocks, respectively. For notational convenience, we denote the 16×48 transform as RST8×8 and the 16×16 transform as RST4×4. Such methods have been adopted by VVC in recent years.

[0229] FIG. 19 shows an example of a Reduced Quadratic Transform (RST).

[0230] The secondary forward and inverse transforms are separate processing steps from that of the primary transform.

[0231] For the encoder, we first perform a primary forward transform, then a secondary forward transform and quantization, and CABAC bit encoding. For the decoder, CABAC bit decoding and inverse quantization, we first perform a secondary inverse transform, then a primary inverse transform. RST is only applied to intra-coded TUs.

[0232] 3. Exemplary technical problems solved by the technical solutions disclosed herein

[0233] The current VVC design has the following problems from the perspective of IBC mode and conversion design:

[0234] (1) The signaling of the IBC mode flag depends on the block size limitation, however the IBC skip mode flag is not worst-case immutable (e.g., large blocks such as Nx128 or 128xN can still select IBC mode).

[0235] (2) In the current VVC, the size of a transform block is always equal to 64 × 64, and the size of a transform block is always set to the size of a coding block. It is unclear how to handle the case where the IBC block size is larger than the transform block size.

[0236] (3) IBC uses unfiltered construction samples within the same video unit (e.g., slice). However, filtered samples (e.g., via deblocking filter / SAO / ALF) may have less distortion compared to unfiltered samples. Using filtered samples may result in additional coding gain.

[0237] (4) The context modeling of the LFNST matrix depends on the linear transformation and the partition type (single or dual tree). However, from our analysis, there is no clear dependency that the chosen RST matrix is ​​correlated with the linear transformation.

[0238] 4. Examples of Techniques and Embodiments

[0239] In this specification, intra block copy (IBC) is not limited to the current IBC technique, but may be interpreted as a technique that uses reference samples within a current slice / tile / brick / subpicture / picture / other video unit (e.g., CTU row) except for conventional intra prediction methods. In one example, the reference samples are reconstructed samples without in-loop filtering processes (e.g., deblocking filter, SAO, ALF) being invoked.

[0240] The size of a VPDU can be expressed as vSizeX*vSizeY. In one example, let vSizeX=vSizeY=min(ctbSizeY,64), where ctbSizeY is the width / height of the CTB. A VPDU is a vSize*vSize block whose top-left position is (m*vSize,n*vSize) relative to the top-left of the picture, where m, n are integers. Alternatively, vSizeX / vSizeY is a fixed number, such as 64.

[0241] The following detailed inventions should be considered as examples to illustrate the general concept. These inventions should not be construed in a narrow sense. Moreover, these inventions can be combined in any manner.

[0242] IBC Signaling and Usage

[0243] Let the size of the current block be W curr ×H curr and the maximum allowable IBC block size is W IBCMax ×H IBCMax Let us express it as:

[0244] 1. An IBC coding block may be split into multiple TBs / TUs without signaling the split information. Let the current IBC code block size be W curr ×H curr It is expressed as: In one example, W curr is greater than the first threshold, and / or H currSplitting can be invoked if, ... i. In one example, the first and / or second threshold is a maximum transform size, represented by MaxTbSizeY. b. In one example, recursive splitting may be invoked until either the width or height of the TU / TB is below a threshold. i. In one example, the recursive partitioning may be the same as that used for inter-coded residual blocks, e.g., each time the width is greater than MaxTbSizeY, the width is halved, and each time the height is greater than MaxTbSizeY, the height is halved. ii. Alternatively, recursive partitioning may be invoked until the partition width and height of TU / TB are below a threshold, respectively. c. Alternatively, all or some of the multiple TBs / TUs may share the same motion information (eg, BVs).

[0245] 2. Whether IBC is enabled or disabled depends on the maximum transform size (eg, MaxTbSizeY herein). a. In one example, IBC may be disabled if the dimensions (width and / or height) of the current luma block or the dimensions (width and / or height) of the corresponding luma block (if the current block is a chroma block) are larger than the maximum transform size in luma samples. i. In one example, if both the width and height of a block are larger than the maximum transform size in luma samples, IBC may be disabled. ii. In one example, if either the width or height of a block is larger than the maximum transform size in luma samples, IBC may be disabled. b. In one example, whether and / or how to signal an indication of IBC usage may depend on the block dimensions (width and / or height) and / or maximum transform size. i. In one example, an indication of IBC usage may include an IBC skip flag (e.g., cu_skip_flag) in the tile / slice / brick / subpicture. 1) In one example, cu_skip_flag is curr is greater than MaxTbSizeY, and / or H curr If is greater than MaxTbSizeY, it may not be signaled. 2) Alternatively, cu_skip_flag can be set to W curr is less than or equal to MaxTbSizeY, and / or H curr It may be signaled if is less than or equal to MaxTbSizeY. 3) In one example, cu_skip_flag is curr is greater than MaxTbSizeY, and / or H curr If is greater than MaxTbSizeY, it may be signaled, but must be 0 in the conformance bitstream. 4) Additionally, for example, the above method may be called when the current tile / slice / brick / subpicture is a tile / slice / brick / subpicture. ii. In one example, the indication of IBC usage may include an IBC mode flag (e.g., pred_mode_ibc_flag). 1) In one example, pred_mode_ibc_flag is set to W curr is greater than MaxTbSizeY, and / or H curr If is greater than MaxTbSizeY, it may not be signaled. 2) Alternatively, pred_mode_ibc_flag can be set to W curr is less than or equal to MaxTbSizeY, and / or H curr It may be signaled if is less than or equal to MaxTbSizeY. In one example, pred_mode_ibc_flag is curr is greater than MaxTbSizeY, and / or H currIf is greater than MaxTbSizeY, it may be signaled, but must be 0 in the conformance bitstream. iii. In one example, residual blocks of an IBC coding block may be forced to all zeros if certain zero-forcing conditions are met. 1) In one example, W curr is greater than MaxTbSizeY and / or H curr If is greater than MaxTbSizeY, the residual blocks of the IBC coding block may be all-zeroed. 2) In one example, W curr is greater than a first threshold (e.g., 64 / vSizeX) and / or H curr If vSizeY is greater than a second threshold (eg, 64 / vSizeY), then the residual block of the IBC coding block may be forced to all zeros. 3) Alternatively, in the above case, signaling of coding block flags (e.g. cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may be skipped. 4) Alternatively, the signaling of coding block flags (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may be kept unchanged, but a conforming bitstream shall satisfy this flag being 0. iv. In one example, whether to signal coding block flags (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may depend on IBC usage and / or a threshold of allowed IBC size. 1) In one example, the current block is in IBC mode and W curr is greater than MaxTbSizeY and / or H curr If is greater than MaxTbSizeY, the coding block flags (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may be skipped. 2) In one example, the current block is in IBC mode and W curr If H is greater than a first threshold (e.g., 64 / vSizeX) and / or H is greater than a second threshold (e.g., 64 / vSizeY), then signaling of coding block flags (e.g., cu_cbf, tu_cbf_cb, tu_cbf_cr, tu_cbf_luma) may be skipped.

[0246] 3. Whether to signal an indication to use IBC skip mode may depend on the dimensions (width and / or height) of the current block and the maximum allowed IBC block size (eg, vSizeX / vSizeY). In one example, cu_skip_flag is curr W IBCMax If greater than, and / or H curr H IBCMax If it is greater, it may not be signaled. i. Alternatively, W curr W IBCMax is less than or equal to and / or H curr H IBCMax If it is less than or equal, then cu_skip_flag may not be signaled. b. Alternatively, the above method may be called when the current tile / slice / brick / subpicture is a tile / slice / brick / subpicture. c. In one example, W IBCMax and H IBCMax are both equal to 64. d. In one example, W IBCMax Set to vSizeX and H IBCMax Set vSizeY equal to vSizeY. e. In one example, W IBCMax is set to the width of the largest transform block, and H IBCMax is set equal to the height of the largest transform block size.

[0247] Recursive division of IBC / Inter-coding blocks

[0248] 4. When an IBC / inter-coding block (CU) is partitioned into multiple TBs / TUs without signaling partition information (e.g., the block size is larger than the maximum transform block size), it is proposed to signal or derive motion information multiple times rather than signaling motion information once for the entire CU. In one example, all TBs / TUs may share the same AMVP / Merge flag that may be signaled once. b. In one example, the motion candidate list may be built once for the entire CU, but different TBs / TUs may be assigned different candidates in the list. i. In one example, the index of the assigned candidate may be coded in the bitstream. c. In one example, the motion candidate list may be constructed without using motion information of neighboring TBs / TUs in the current CU. i. In one example, the motion candidate list may be constructed by accessing motion information of spatially neighboring blocks (adjacent or non-adjacent) to the current CU.

[0249] 5. When an IBC / inter-coding block (CU) is partitioned into multiple TBs / TUs without signaling partitioning information (e.g., the block size is larger than the maximum transform block size), the first TU / TB may be predicted by at least one reconstructed or predicted sample in the second TU / TB. a.BV validation may be performed separately for each TU / TB.

[0250] 6. When an IBC / inter-coding block (CU) is partitioned into multiple TBs / TUs without signaling partitioning information (e.g., the block size is larger than the maximum transform block size), the first TU / TB cannot be predicted by any reconstructed or predicted samples in the second TU / TB. a.BV validation may be performed on the entire coding block.

[0251] On generating predicted blocks for IBC coding blocks

[0252] 7. It is proposed to use filtered reconstructed reference samples for IBC prediction block generation. In one example, the filtered reconstructed reference samples may be generated by applying a deblocking filter / bilateral filter / SAO / ALF. b. In one example, for an IBC coding block, at least one reference sample is filtered and at least one reference sample is unfiltered.

[0253] 8. Both filtered and unfiltered reconstructed reference samples may be used for IBC prediction block generation. In one example, the filtered reconstructed reference samples may be generated by applying a deblocking filter / bilateral filter / SAO / ALF. b. In one example, for prediction for an IBC block, a portion of the prediction may be from filtered samples and another portion may be from unfiltered samples.

[0254] 9. Whether to use filtered or unfiltered values ​​of reconstructed reference samples for IBC predicted block generation may depend on the position of the reconstructed samples. In one example, this determination may depend on the relative position of the reconstructed sample with respect to the current CTU and / or the CTU that spans the reference sample. b. In one example, if the reference sample is outside the current CTU / CTB or within a different CTU / CTB and has a large distance (e.g., 4 pixels) to the CTU boundary, the corresponding filtered sample may be used. c. In one example, if the reference sample is within the current CTU / CTB, a prediction block may be generated using the unfiltered reconstructed sample. d. In one example, if the reference sample is outside the current VPDU, the corresponding filtered sample may be used. e. In one example, if the reference sample is within the current VPDU, a prediction block may be generated using the unfiltered reconstructed sample.

[0255] About MVD coding

[0256] The motion vector (or block vector), or the difference of motion vectors or block vectors, is represented by (Vx, Vy).

[0257] 10. It is proposed to split the absolute value of the component of the motion vector (e.g., Vx or Vy) represented by AbsV into two parts, where the first part is equal to AbsV - ((AbsV >> N) << N) (e.g., AbsV & (1 << N), the least significant N bits), and the other part is (AbsV >> N) (e.g., the remaining most significant bits). Each part may be coded individually. a. In one example, each component of one motion vector or the difference of motion vectors may be coded separately. i. In one example, the first part is coded with a fixed - length coding, e.g., N bits. 1) Alternatively, a first flag may be coded to indicate whether the first part is 0. a. Further, alternatively, if not, the value obtained by subtracting 1 from the value of the first part is coded. ii. In one example, the second part having the sign information of the MV / MVD may be coded with the current MVD coding method. b. In one example, the first parts of each component of one motion vector or the difference of motion vectors may be coded together, and the second parts may be coded separately. i. In one example, the first parts of Vx and Vy may be formed to be a new positive value in 2N bits (e.g., (Vx << N) + Vy). 1) In one example, the first flag may be coded to indicate whether the new positive value is equal to 0. a. Further alternatively, if not, code a value obtained by subtracting 1 from the new positive value. 2) Alternatively, this new positive value may be coded by fixed-length coding or exponential Golomb coding (e.g., EG-0th). ii. In one example, the second parts of each component of one motion vector having MV / MVD sign information may be coded using the current MVD coding method. c. In one example, the first parts of each component of one motion vector may be coded together, and the second parts may also be coded together. d. In one example, N is a positive value such as 1 or 2. e. In one example, N may depend on the accuracy of the MV used for storing the motion vector. i. In one example, if the MV accuracy used for storing the motion vector is 1 / 2M pixels, set N to M. 1) In one example, if the MV accuracy used for storing the motion vector is 1 / 16 pixels, N is equal to 4. 2) In one example, if the MV accuracy used for storing the motion vector is 1 / 8 pixels, N is equal to 3. 3) In one example, if the MV accuracy used for storing the motion vector is 1 / 4 pixels, N is equal to 2. f. In one example, N may depend on the MV accuracy of the current motion vector (e.g., the block vector in IBC). i. In one example, if the BV is of 1-pixel accuracy, N may be set to 1.

[0258] About SBT

[0259] 11. Whether to signal an indication of maximum CU width and / or height to allow sub-block transforms may depend on the maximum allowed transform block size. a. In one example, whether the maximum CU width and maximum CU height for allowing sub-block transforms is 64 or 32 luma samples (e.g., sps_sbt_max_size_64_flag) may depend on the value of the maximum allowed transform block size (e.g., sps_max_luma_transform_size_64_flag). i. In one example, sps_sbt_max_size_64_flag is signaled only if both sps_sbt_enabled_flag and sps_max_luma_transform_size_64_flag are true.

[0260] About LFNST matrix indexing

[0261] 12. It is proposed to use two contexts to code the use of LFNST / LFNST matrix indices (e.g., lfnst_idx), where the choice is purely based on the use of the partition tree. g. In one example, in the case of a single tree partition, one context is utilized, and in the case of a dual tree, a different context is utilized regardless of the dependency on the primary transformation.

[0262] 13. It is proposed to code the use of LFNST / LFNST matrix index (e.g., lfnst_idx) based on the use of dual trees and / or slice type and / or color components.

[0263] 5. Additional Exemplary Embodiments

[0264] [ka]

[0265] 5.1 Embodiment #1

[0266] [Table 13]

[0267] 5.2 Embodiment #2

[0268] [Table 14]

[0269] FIG. 20 is a block diagram illustrating an example video processing system 2000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the modules of the system 2000. The system 2000 may include an input unit 2002 for receiving video content. The video content may be received in a raw or uncompressed format, for example, 8 or 10 bit multi-module pixel values, or may be received in a compressed or coded format. The input unit 2002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0270] The system 2000 may include a coding module 2004 capable of implementing various coding or encoding methods described herein. The coding module 2004 may reduce the average bit rate of the video from the input unit 2002 to the output of the coding module 2004 to generate a coded representation of the video. Thus, this coding technique may be referred to as a video compression or video transcoding technique. The output of the coding module 2004 may be stored or transmitted via a connected communication, as represented by module 2006. The bitstream (or coded) representation of the video received at the input unit 2002, stored or communicated may be used by module 2008 to generate pixel values ​​or a displayable video that is transmitted to the display interface unit 2010. The process of generating a user-viewable video from the bitstream representation may be referred to as video decompression. Furthermore, although certain video processing operations are referred to as "coding" operations or tools, it will be understood that the coding tools or operations are used in an encoder and that corresponding decoding tools or operations that reverse the results of the coding are performed by a decoder.

[0271] Examples of peripheral bus interface units or display interface units may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.

[0272] FIG. 21 is a block diagram of a video processing device 2100. The device 2100 may be used to implement one or more of the methods described herein. The device 2100 may be implemented in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 2100 may include one or more processing units 2102, one or more memories 2104, and video processing hardware 2106. The one or more processors 2102 may be configured to implement one or more methods described herein. The memory(s) 2104 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 2106 may be used to implement the techniques described herein in a hardware circuit.

[0273] In the embodiments of the present invention, the following solutions can be implemented as preferred solutions.

[0274] The following solutions may be implemented in conjunction with the additional techniques described in the items listed in the previous chapter (e.g., item 1).

[0275] 1. A video processing method (e.g., method 2200 shown in FIG. 22), comprising: determining (2202) to allow partitioning of a video block into multiple transform units for conversion between a coding representation of the video block in the video domain and the video block based on an intra block copy tool, the determination being made based on coding conditions of the video block, the coding representation omitting signaling of the partition; and performing conversion based on the partition (2204).

[0276] 2. The method according to solution 1, wherein the coding condition corresponds to a dimension of the video block.

[0277] 3. The method of solution 2, wherein the splitting is determined to be allowed because the width of the video block is greater than a first threshold, or the splitting is determined to be allowed because the height of the video block is greater than a second threshold.

[0278] 4. The method according to solution 3, wherein the first threshold and / or the second threshold is a maximum transformation size for the image area.

[0279] 5. The method according to any of solutions 1 to 4, wherein the partitioning is applied recursively to the video blocks.

[0280] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 2).

[0281] 6. A video processing method, comprising: determining whether an intra block copy (IBC) tool is enabled for converting a video block between a coding representation of the video block in a video domain and the video block based on a maximum transform size for the video domain; and performing the conversion based on the determination.

[0282] 7. The method according to Solution 6, wherein the image block is a chroma block, a maximum transform size corresponds to that of a corresponding luma block, and IBC is disabled due to the maximum transform size being greater than a threshold.

[0283] 8. The method according to solution 6 or 7, wherein a syntax element in the coding representation indicates whether the IBC tool is enabled according to a maximum transform size that meets the dimensions of the video block or a criterion.

[0284] 9. The method of any of Solutions 6-8, wherein if it is determined that the IBC tool is enabled, the transformation further includes forcing the residual block for the video block to be all zero based on a zero-forcing condition.

[0285] The following solutions may be implemented in conjunction with the additional techniques described in the items listed in the previous chapter (e.g., item 3).

[0286] 10. A video processing method comprising: determining whether a signaling of an intra block copy (IBC) tool for conversion between a coding representation of a video block of a video region and the video block is included in the coding representation; and performing the conversion based on the determination, the determination being based on a width and / or height of the video block and a maximum allowable IBC block size for the video region.

[0287] 11. The method of solution 10, wherein no signal notification is included due to the width being greater than a first threshold or the height being greater than a second threshold.

[0288] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 4).

[0289] 12. A video processing method, comprising: determining, using an intra block copy (IBC) tool, that division of a video block into a plurality of transform units (TUs) is permitted for conversion between a coding representation of the video block in the video domain and the video block for conversion; and performing the conversion based on the determination, wherein the conversion includes using separate motion information for the plurality of TUs.

[0290] 13. The method of solution 12, wherein multiple TUs are constrained to share the same advanced motion vector predictor or merge mode flag.

[0291] 14. The method according to solution 12, wherein during the transformation, motion information for one of the plurality of TUs is determined without using motion information of another of the plurality of TUs.

[0292] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 5).

[0293] 15. The method according to any of solutions 12 to 14, wherein a first TU of the plurality of TUs is predicted from a second TU of the plurality of TUs.

[0294] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 6).

[0295] 16. The method according to any of Solutions 12 to 15, wherein prediction of a first TU of the plurality of TUs from a second TU of the plurality of TUs is disabled.

[0296] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 7).

[0297] 17. A video processing method, comprising: for converting between a coding representation of a video block in the video domain and the video block, determining that an intra block copy tool is enabled for the conversion; and performing the conversion using the intra block copy tool; and performing a prediction of the video block using filtered reconstructed samples of the video domain.

[0298] 18. The method of solution 17, wherein the filtered reconstructed samples include samples generated by applying an in-loop filter to the reconstructed samples, the in-loop filter being a deblocking filter, a bilateral filter, or a sample adaptive offset or adaptive loop filter.

[0299] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 8).

[0300] 19. Based on the rules, the prediction further uses selectively the non-filtered reconstructed samples of the video region, according to the method described in any of solutions 17 to 18.

[0301] The following solutions may be implemented together with the additional techniques described in the items (e.g., item 9) described in the previous chapter.

[0302] 20. The method according to solution 19, wherein the rules depend on the position of the reconstructed samples.

[0303] 21. The method according to solution 20, wherein the position of the reconstructed samples used in the rules is the relative position of the reconstructed samples with respect to the coding tree unit of the video block.

[0304] The following solutions may be implemented together with the additional techniques described in the items (e.g., item 10) described in the previous chapter.

[0305] 22. A video processing method, including performing a conversion between a video including a plurality of video blocks and a coding representation of the video, wherein at least some of the video blocks are coded using motion vector information, and the motion vector information is represented in the coding representation as a first part based on the first least significant bit of the absolute value of the motion vector information, and also as a second part based on the remaining more significant bits that are higher than the first least significant bit.

[0306] 23. Divide the absolute value of the motion vector V represented as AbsV into a first part equal to AbsV - ((AbsV >> N) << N) or AbsV & (1 << N), and the second part is equal to (AbsV >> N), where N is an integer, according to the method described in solution 22.

[0307] 24. The method according to any of solutions 22 to 23, wherein the first part and the second part are coded separately in the coding representation.

[0308] 25. The method according to any of solutions 22 to 24, wherein a first part of the x motion vector is jointly coded at the same position as a first part of the y motion vector.

[0309] 26. The method according to solution 25, wherein the second part of the x motion vector and the second part of the y motion vector are coded separately.

[0310] 27. The method according to any of solutions 22 to 26, wherein N is a function of the precision used to store the motion vectors.

[0311] The following solutions may be implemented in conjunction with additional techniques described in the items listed in the previous chapter (e.g., item 11).

[0312] 28. A method of video processing comprising: determining whether a sub-block transform tool is enabled for transforming between a coding representation of a video block in a video domain and a video block; and performing the transform based on the determination, the determination being based on a maximum allowable transform block size for the video domain; and conditionally including a signal in the coding representation based on the maximum allowable transform block size.

[0313] The following solutions may be implemented in conjunction with the additional techniques described in the items listed in the previous chapter (e.g., items 12 and 13).

[0314] 29. A video processing method, comprising: for transforming between a coding representation of a video block in a video domain and the video block, determining whether a low frequency non-separable transform (LFNST) is used during the transform; and performing the transform based on the determination, the determination being based on a coding condition applied to the video block, and the LFNST and a matrix index for the LFNST are coded in the coding representation using two contexts.

[0315] 30. The method of solution 29, wherein the coding conditions include the partition tree used, or the slice type of the video block, or the color component identity of the video block.

[0316] 31. A method according to any of solutions 1 to 30, wherein the image region includes a coding unit and the image block corresponds to a luminance component or a chroma component of the image region.

[0317] 32. A method according to any one of solutions 1 to 31, wherein the conversion includes generating the bitstream representation from the current video block.

[0318] 33. A method according to any of solutions 1 to 31, wherein the conversion includes generating samples of the current video block from the bitstream representation.

[0319] 34. A video processing device comprising a processing device configured to implement the methods described in any one or more of solutions 1 to 33.

[0320] 35. A computer-readable medium having stored thereon code that, when executed, causes a processor to implement a method according to any one or more of solutions 1 to 33.

[0321] 36. A method, system or apparatus as described herein.

[0322] 23 is a flow chart illustrating a video processing method 2300 according to the present technology. The method 2300 includes, at step 2310, determining that a current block is allowed to be divided into multiple transform units based on characteristics of the current block for conversion between a current block in an image domain of the image and a coding representation of the image, where signaling of the division is omitted. The method 2300 includes, at operation 2320, performing the conversion based on the determination.

[0323] In some embodiments, the characteristics of the current block include dimensions of the current block. In some embodiments, if the width of the current block is greater than a first threshold, splitting is invoked for the transformation. In some embodiments, if the height of the current block is greater than a second threshold, said splitting is invoked for the transformation. In some embodiments, splitting the current block includes recursively splitting the current block until a dimension of one of the plurality of transform units is less than or equal to a threshold. In some embodiments, the width of the current block is recursively divided in half until the width of the current block is less than or equal to the first threshold. In some embodiments, the height of the current block is recursively divided in half until the height of the current block is less than or equal to a second threshold. In some embodiments, recursively splitting the current block until the width of the current block is less than or equal to the first threshold and the height of the current block is less than or equal to the second threshold. In some embodiments, the first threshold is equal to a maximum transform size for the image domain. In some embodiments, the second threshold is equal to a maximum transform size for the image domain.

[0324] In some embodiments, at least a subset of the plurality of transform units share the same motion information. In some embodiments, the same motion information includes a syntax flag for an advanced motion vector prediction mode or a merge mode that is signaled once in the coding representation for the current block. In some embodiments, the motion information is determined multiple times for the current block in the coding representation. In some embodiments, a motion candidate list is constructed once for the current block, and multiple transform units are assigned different candidates in the motion candidate list. In some embodiments, indices of the different candidates are signaled in the coding representation. In some embodiments, for a transform unit of a multiple transform unit, a motion candidate list is constructed without using motion information of neighboring transform units in the current block. In some embodiments, the motion candidate list is constructed utilizing motion information of spatially neighboring blocks of the current block. In some embodiments, the first transform unit is reconstructed based on at least one reconstructed sample in the second transform unit. In some embodiments, a verification check is performed separately for each of the plurality of transform units. In some embodiments, the first transform unit is predicted without using reconstructed samples in other transform units. In some embodiments, after reconstructing the block, a validity check is performed on the block.

[0325] 24 is a flow chart illustrating a method 2400 of video processing according to the present technology. The method 2400 includes, at step 2410, determining a prediction sample of the current block based on a filtered reconstructed reference sample in the video domain using an intra block copy (IBC) mode for conversion between the current block in the video domain of the video and a coded representation of the video. The method 2400 includes, at operation 2420, performing the conversion based on the determination.

[0326] In some embodiments, the filtered reconstructed reference sample is generated based on a filtering operation including at least one of a deblocking filtering operation, a bilateral filtering operation, a sample adaptive offset (SAO) filtering operation, or an adaptive loop filtering operation. In some embodiments, determining the prediction sample of the current block utilizing the IBC model is further based on an unfiltered reference sample in the video domain. In some implementations, determining the prediction sample in a first portion of the current block based on the filtered reconstructed reference sample, and determining the prediction sample in a second portion of the current block based on the unfiltered reconstructed reference sample. In some embodiments, whether the filtered reconstructed reference sample or the unfiltered reconstructed reference sample is used to determine the prediction sample of the block is based on a position of the prediction sample relative to the current coding tree unit or a coding tree unit that spans the sample. In some embodiments, the filtered reconstructed reference sample is used to reconstruct the prediction sample when the prediction sample is located outside the current coding tree unit. In some embodiments, the filtered reconstructed reference sample is used to reconstruct the prediction sample when the prediction sample is located within a coding tree unit and there is a distance D that is 4 pixels or more from a boundary of the coding tree unit. In some embodiments, an unfiltered reconstructed reference sample is used to reconstruct a prediction sample if the prediction sample is located within the current coding tree unit. In some embodiments, a filtered reconstructed reference sample is used to reconstruct a prediction sample if the prediction sample is located outside the current virtual pipeline data unit. In some embodiments, an unfiltered reconstructed reference sample is used to reconstruct a prediction sample if the prediction sample is located within the current virtual pipeline data unit.

[0327] 25 is a flow chart illustrating a video processing method 2500 in accordance with the present technology. The method 2500 includes, at operation 2510, determining whether a syntax element indicating use of a skip mode for an intra block copy (IBC) coding model is included in the coding representation according to a rule to convert between a current block of the video and the coding representation of the video. The rule specifies that the signaling of the syntax element is based on a dimension of the current block and / or a maximum allowed dimension for a block coded using the IBC coding model. The method 2500 also includes, at operation 2520, performing the conversion based on the determination.

[0328] In some embodiments, the rules provide for omitting the syntax element in the coding representation if a width of a current block is greater than a maximum allowed width. In some embodiments, the rules provide for omitting the syntax element in the coding representation if a height of a current block is greater than a maximum allowed height. In some embodiments, the rules provide for the syntax element to be included in the coding representation when a width of a current block is less than or equal to a maximum allowed width. In some embodiments, the rules provide for the syntax element to be included in the coding representation when a height of a current block is less than or equal to a maximum allowed height.

[0329] In some embodiments, the current block is in an image region of an image, and this rule is applicable when the image region includes an I-tile, an I-slice, an I-brick, or an I-subpicture. In some embodiments, the maximum allowed width or height is 64. In some embodiments, the maximum allowed width or height is equal to a dimension of a virtual pipeline data unit. In some embodiments, the maximum allowed width or height is equal to a maximum dimension of a transform unit.

[0330] FIG. 26 is a flow chart illustrating a video processing method 2600 according to the present technology. The method 2600 includes, at step 2610, determining at least one context for coding an index associated with a low-frequency non-separable transform (LFNST) coding model for transforming between a current block of video and a coding representation of the video. The LFNST coding model includes applying a forward secondary transform between a forward primary transform and a quantization step during encoding, or applying an inverse secondary transform between an inverse quantization step and an inverse primary transform during decoding. The size of the forward secondary transform and the inverse secondary transform is smaller than the size of the current block. The at least one context is determined based on a partition type of the current block without considering the forward primary transform or the inverse primary transform. The method 2600 also includes, at operation 2620, performing a transform according to the determination.

[0331] In some embodiments, when the partition type is a single tree partition, only one context is used to code the index. In some embodiments, when the partition type is a dual tree partition, two contexts are used to code the index.

[0332] In some embodiments, an index indicating the use of a low-frequency non-separable transform (LFNST) coding model is included in the coding representation based on block-related features, including partition type, slice type, or color components associated with the current block.

[0333] 27 is a flow chart illustrating a video processing method 2700 according to the present technology. The method 2700 includes, at operation 2710, determining whether an intra block copy (IBC) coding model is enabled for transforming between a current block of a video domain of a video and a coded representation of the video based on a maximum transform unit size applicable to the video domain. The method 2700 also includes, at operation 2720, performing the transform according to the determination.

[0334] In some embodiments, the IBC coding model is disabled if the dimensions of the current block or the dimensions of the luma block corresponding to the current block are greater than the size of the largest transform unit. In some embodiments, the IBC coding model is disabled if the width of the current block or the width of the luma block corresponding to the current block are greater than the width of the largest transform unit. In some embodiments, the IBC coding model is disabled if the height of the current block or the height of the luma block corresponding to the current block are greater than the largest transform height. In some embodiments, the manner of signaling the use of the IBC coding model in the coded representation is based on the dimensions of the current block and the size of the largest transform unit. In some embodiments, the signaling the use of the IBC coding model includes a syntax element indicating a skip mode of the IBC coding model. In some embodiments, the syntax element is included in an I-tile, an I-slice, an I-brick, or an I-subpicture in the coded representation.

[0335] In some examples, signaling the use of the IBC coding model includes a syntax element indicating the IBC mode. In some embodiments, if the width of the current block is greater than the width of the largest transform unit or the height of the current block is greater than the height of the largest transform unit, the syntax element is omitted in the coding representation. In some embodiments, if the width of the current block is less than or equal to the width of the largest transform unit or the height of the current block is less than or equal to the height of the largest transform unit, the syntax element is included in the coding representation and set to 0. In some embodiments, if the width of the current block is greater than the width of the largest transform unit or the height of the current block is greater than the height of the largest transform unit, the syntax element is included in the coding representation and set to 0.

[0336] In some embodiments, when enabling an IBC coding model for a current block, a sample in a residual block corresponding to the current block is set to zero according to a rule. In some embodiments, the rule specifies setting the sample to zero if the width of the current block is greater than the width of the largest transform unit or the height of the current block is greater than the height of the largest transform unit. In some embodiments, the rule specifies setting the sample to zero if the width of the current block is greater than a first threshold or the height of the current block is greater than a second threshold. In some embodiments, the first threshold is 64 / vSizeX, where vSizeX is the width of a virtual pipeline data unit. In some embodiments, the second threshold is 64 / vSizeY, where vSizeY is the height of a virtual pipeline data unit. In some embodiments, a syntax flag for a coding block is omitted in the coding representation. In some embodiments, a syntax flag for a coding block is set to zero in the coding representation.

[0337] In some embodiments, signaling of a syntax flag for a coding block in a coding representation is based on the use of an IBC coding model. In some embodiments, if the IBC coding model is enabled for a current block and the size of the block is larger than the size of the maximum transform unit, the syntax flag can be omitted in the coding representation. In some embodiments, if the IBC coding model is enabled for a current block and the size of the block is larger than a threshold associated with a virtual pipeline data unit, the syntax flag is omitted in the coding representation.

[0338] FIG. 28 is a flowchart illustrating a video processing method 2800 according to the present technology. Method 2800 includes, in operation 2810, determining to split an absolute value of a component of a motion vector for a current block into two parts to convert a current block of video and a coding representation of the video. Represent the motion vector as (Vx, Vy), represent its components as Vi, where Vi is either Vx or Vy. The first of the two parts is equal to |Vi| - ((|Vi| >> N) << N), the second of the two parts is equal to |Vi| >> N, and N is a positive integer. The two parts are coded separately in the coding representation. Method 2800 also includes, in operation 2820, performing the conversion according to the determination.

[0339] In some embodiments, the two components Vx and Vy are signaled separately in the coding representation. In some embodiments, the first part is coded with a fixed length of N bits. In some embodiments, the first part of component Vx and the first part of component Vy are coded together, and the second part of component Vx and the second part of component Vy are coded separately. In some embodiments, the first part of component Vx and the first part of component Vy are coded as a value having a length of 2N bits. In some embodiments, the value is equal to (Vx << N) + Vy). In some embodiments, this value is coded using a fixed-length coding process or an exponential Golomb coding process.

[0340] In some embodiments, the first part of component Vx and the first part of component Vy are coded together, and the second part of component Vx and the second part of component Vy are coded together. In some embodiments, the coding representation includes a syntax flag to indicate whether the first part is equal to 0. In some embodiments, the first part has a value of K, where K ≠ 0, and the value of (K - 1) is coded in the coding representation. In some embodiments, the second part of each component having sign information is coded using a motion vector differential coding process. In some embodiments, N is 1 or 2. In some embodiments, N is determined by the accuracy of the motion vector used for storing the motion vector data. In some embodiments, when the accuracy of the motion vector is 1 / 16 pixel, N is 4. In some embodiments, when the accuracy of the motion vector is 1 / 8 pixel, N is 3. In some embodiments, when the accuracy of the motion vector is 1 / 4 pixel, N is 2. In some embodiments, N is determined based on the motion vector accuracy of the current motion vector. In some embodiments, when the accuracy of the motion vector is 1 pixel, N is 1.

[0341] 29 is a flow chart illustrating a method 2900 for processing video in accordance with the present technology. The method 2900 includes, at operation 2910, determining information regarding a maximum dimension of the current block that enables a sub-block transformation on the current block based on a maximum allowed dimension of a transform block for transforming between the current block of video and a coded representation of the video. The method 2900 also includes, at operation 2920, performing the transformation in accordance with the determination.

[0342] In some embodiments, a maximum dimension of the current block that enables sub-block transformation on the current block corresponds to a maximum allowed dimension of a transform block. In some embodiments, the maximum dimension of the current block is 64 or 32. In some implementations, if a first syntax flag in a slice parameter set indicates that sub-block transformation is enabled and a second syntax flag in the slice parameter set indicates that the maximum allowed dimension of a transform block is 64, a syntax flag indicating the maximum dimension of the current block is included in the coding representation.

[0343] In some embodiments, the transforming includes generating a coding representation from the current block.In some embodiments, the transforming includes generating samples of the current block from the coding representation.

[0344] In the above solution, performing the transformation involves using the results of a previous decision step during the encoding or decoding operation (e.g., to use or not use a particular encoding or decoding step) to arrive at a transformation result.

[0345] Some embodiments of the disclosed technology include determining or determining to enable a video processing tool or mode. In one example, if a video processing tool or mode is enabled, the encoder uses or implements the tool or mode when processing one video block, but does not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, the conversion from a block of video to a bitstream representation of video uses the video processing tool or mode if the video processing tool or mode is enabled based on the decision or determination. In another example, if the video processing tool or mode is enabled, the decoder processes the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of video to a block of video is performed using the video processing tool or mode enabled based on the decision or determination.

[0346] Some embodiments of the disclosed techniques include deciding or determining to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder does not use the tool or mode when converting blocks of a video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, a decoder processes the bitstream with the knowledge that the bitstream has not been modified using a video processing tool or mode that was enabled based on the decision or determination.

[0347] Implementations of the disclosed and other solutions, examples, embodiments, modules, and functional operations described herein, including the structures disclosed herein and their structural equivalents, may be implemented in digital electronic circuitry, or computer software, firmware, or hardware, or in one or more combinations thereof. The disclosed and other embodiments may be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for implementation by or for controlling the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that provides a machine-readable propagated signal, or one or more combinations thereof. The term "data processing apparatus" includes all apparatus, devices, and machines for processing data, including, for example, a programmable processing apparatus, a computer, or multiple processing apparatuses, or computers. In addition to hardware, the apparatus may include code that creates an environment for the execution of the computer program, e.g., code that constitutes a processing apparatus firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, for example a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.

[0348] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), may be stored in a single file dedicated to the program, or may be stored in multiple coordinating files (e.g., a file that stores one or more modules, subprograms, or code portions). A computer program can be deployed to run on one computer located at one site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0349] The processes and logic flows described herein may be performed by one or more programmable processing devices executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0350] Processors suitable for executing a computer program include, for example, both general purpose and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may include one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or may be operatively coupled to receive data from or transfer data to these mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor storage devices such as EPROMs, EEPROMs, flash storage devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0351] Although this patent specification contains many details, these should not be construed as limiting the scope of any subject matter or the scope of the claims, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent document in the context of separate embodiments may be implemented in combination in one example. Conversely, various features described in the context of one example may be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0352] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desired results. It should not be understood that all embodiments require such separation.

[0353] Only certain implementations and examples have been described; other embodiments, extensions and variations are possible based on the content described and illustrated in this patent document.

Claims

1. 1. A method for processing video data, comprising the steps of: For converting a current transform block of an image to a bitstream of the image, determining whether a width of the current transform block is greater than a first threshold or a height of the current transform block is greater than a second threshold; invoking a split operation on the current transform block if the width of the current transform block is greater than a first threshold or the height of the current transform block is greater than the second threshold; performing the transformation based on at least two or more transformation blocks; the signaling of the invocation of the split operation and the signaling of the split operation are omitted in the bitstream; the first threshold and the second threshold are equal to a first value; the first value is equal to 32; calling the splitting operation if the current transformed block is derived based on a coding block coded with a first prediction mode in which prediction samples are derived from blocks of sample values ​​of the same image region as determined by a block vector; the width of the coding block and the height of the coding block are less than or equal to 64 and greater than 32; method.

2. the first value defines a maximum transform size in luma samples. The method of claim 1.

3. In the dividing operation, if the width of the current transform block is greater than the first value, calling dividing the width of the current transform block in half; if the height of the current transformation block is greater than the first value, then invoking dividing the height of the current transformation block in half; The method of claim 2.

4. recursively invoking the split operation until the width of the current transformation block is less than or equal to a first threshold and the height of the current transformation block is less than or equal to a second threshold.

4. The method according to any one of claims 1 to 3.

5. the converting includes encoding the video into a bitstream; 5. The method according to any one of claims 1 to 4.

6. the converting includes decoding the video from the bitstream.

5. The method according to any one of claims 1 to 4.

7. 1. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions stored thereon, the apparatus comprising: The instructions, when executed by the processing device, cause the processing device to: For converting a current transform block of an image to a bitstream of the image, determining whether a width of the current transform block is greater than a first threshold or a height of the current transform block is greater than a second threshold; invoking a split operation on the current transform block if the width of the current transform block is greater than a first threshold or the height of the current transform block is greater than the second threshold; performing the transformation based on at least two or more transformation blocks; the signaling of the invocation of the split operation and the signaling of the split operation are omitted in the bitstream; the first threshold and the second threshold are equal to a first value; the first value is equal to 32; calling the splitting operation if the current transformed block is derived based on a coding block coded with a first prediction mode in which prediction samples are derived from blocks of sample values ​​of the same image region as determined by a block vector; the width of the coding block and the height of the coding block are less than or equal to 64 and greater than 32; Device.

8. A non-transitory computer-readable storage medium storing instructions, comprising: The instructions may be for causing a processor to: For converting a current transform block of an image to a bitstream of the image, determining whether a width of the current transform block is greater than a first threshold or a height of the current transform block is greater than a second threshold; invoking a split operation on the current transform block if the width of the current transform block is greater than a first threshold or the height of the current transform block is greater than the second threshold; performing the transformation based on at least two or more transformation blocks; the signaling of the invocation of the split operation and the signaling of the split operation are omitted in the bitstream; the first threshold and the second threshold are equal to a first value; the first value is equal to 32; calling the splitting operation if the current transformed block is derived based on a coding block coded with a first prediction mode in which prediction samples are derived from blocks of sample values ​​of the same image region as determined by a block vector; the width of the coding block and the height of the coding block are less than or equal to 64 and greater than 32; A non-transitory computer-readable storage medium.

9. A method for storing a video bitstream, comprising the steps of: determining for a current transformation block of the image whether a width of the current transformation block is greater than a first threshold or a height of the current transformation block is greater than a second threshold; invoking a split operation on the current transform block if the width of the current transform block is greater than a first threshold or the height of the current transform block is greater than the second threshold; generating the bitstream based on at least two or more transformation blocks; storing the bitstream on a non-transitory computer-readable recording medium; the signaling of the invocation of the split operation and the signaling of the split operation are omitted in the bitstream; the first threshold and the second threshold are equal to a first value; the first value is equal to 32; calling the splitting operation if the current transformed block is derived based on a coding block coded with a first prediction mode in which prediction samples are derived from blocks of sample values ​​of the same image region as determined by a block vector; the width of the coding block and the height of the coding block are less than or equal to 64 and greater than 32; method.

Citation Information

Cited By

  • Intra block copy coding and decoding

    JP2025037967A