Combined inter-intra prediction based on transform unit

Abstract

A method of visual media processing, comprising: determining, for a current video block, a motion vector used in a sub-block based motion vector prediction (sbTMVP) process to locate a corresponding block in a co-located picture for conversion between the current video block and a bitstream representation of the current video block, wherein the motion vector used in the sbTMVP process is calculated according to a scaling operation; and performing the conversion between the current video block and the bitstream representation of the visual media data based on using the motion vector.

Description

[0001] Cross-references to related applications

[0002] Under applicable patent law and / or the rules of the Paris Convention, this application promptly claims priority and benefit to International Patent Application No. PCT / CN2019 / 107159, filed September 22, 2019. For all purposes under U.S. patent law, the entire disclosure of the foregoing application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to image and video encoding and decoding technologies, devices, and systems. Background Technology

[0004] Digital video consumes the largest amount of bandwidth on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] The techniques disclosed herein can be used by video decoder or encoder embodiments during video decoding or encoding using alternative temporal motion vector prediction.

[0006] In one exemplary aspect, a video processing method is disclosed. The method includes, for a current video block, determining motion vectors used in a sub-block-based motion vector prediction (sbTMVP) process to locate corresponding blocks in a co-bit image used for conversion between the current video block and a bitstream representation of the current video block, wherein the motion vectors used in the sbTMVP process are calculated based on a scaling operation; and performing a conversion between the current video block and a bitstream representation of the visual media data based on the use of the motion vectors.

[0007] In another exemplary aspect, another video processing method is disclosed. The method includes, for a current video block, determining the motion vectors used in a sub-block-based temporal motion vector prediction (sbTMVP) process to locate corresponding blocks in a co-bit image used for the conversion between the current video block and a bitstream representation of the current video block, wherein the motion vectors used in the sbTMVP process are calculated with respect to the center point of the current video block; modifying the center point of the current video block by applying one or more operations; and performing the conversion between the current video block and the bitstream representation of the visual media data based on the center point modified by applying one or more operations.

[0008] In another exemplary aspect, a different video processing method is disclosed. The method includes, for a current video block, determining motion vectors used in a sub-block-based temporal motion vector prediction (sbTMVP) process to locate corresponding blocks in a co-bitmap image used for the conversion between the current video block and a bitstream representation of the current video block, wherein motion vectors used in the sbTMVP process are calculated with respect to points in the corresponding blocks in the co-bitmap image; modifying the points in the corresponding blocks in the co-bitmap image by applying one or more operations; and performing the conversion between the current video block and a bitstream representation of the visual media data based on the points in the corresponding blocks in the co-bitmap image modified by applying one or more operations.

[0009] In another aspect, a different method for video processing is disclosed. The method includes: a conversion between a video picture contained in visual media data and a bitstream representation of the visual media data using subpictures; determining that rules relating to one or more subpictures are satisfied by the conversion; and performing the conversion according to constrained rules, wherein the rules specify that the size of the subpicture in the video picture is an integer multiple of the codec tree unit size associated with the video picture.

[0010] In yet another exemplary aspect, a method for video processing is disclosed. The method includes: converting a video picture contained in visual media data using subpicks to a bitstream representation of the visual media data; determining that rules relating to one or more subpicks are satisfied by the conversion; and performing the conversion according to constrained rules, wherein the rules specify that all subpicks in the video picture do not overlap, and that all subpicks in the video picture together cover the entire video picture.

[0011] In yet another exemplary aspect, a method for video processing is disclosed. The method includes: converting between video units of visual media and a bitstream representation of visual media data; determining whether to use a reference picture resampling (RPR) technique during the conversion; and performing the conversion based on the determination, wherein a flag corresponding to the determination is included in the bitstream representation at the sequence parameter set level.

[0012] In yet another exemplary aspect, a method for video processing is disclosed. The method includes selecting an interpolation filter during a motion compensation process to derive a predicted block of a current block of visual media data based on satisfying conditions, wherein the conditions are at least partially based on determining that the resolution of a reference image is different from the resolution of the current image and / or the dimension of a window associated with the reference image is different from the dimension of a window associated with the current image; and performing a conversion between the current block of the visual media data and a bitstream representation of the current block.

[0013] In yet another exemplary aspect, a method for video processing is disclosed. The method includes: a conversion between a current block of visual media data and a bitstream representation of the visual media data; determining that the current block is a combined inter-frame intra-prediction (CIIP) block, wherein the intra-prediction blocks of the CIIP block are generated using the size of a transform unit (TU), wherein, in the combined inter-frame intra-prediction (CIIP) block, the final prediction of the current block is based on a weighted sum of the inter-frame predictions and intra-frame predictions of the current block; and performing the conversion based on the determination.

[0014] In yet another exemplary aspect, a video encoder device is disclosed. The video encoder device includes a processor configured to implement the methods described herein.

[0015] In yet another exemplary aspect, a video decoder device is disclosed. The video decoder device includes a processor configured to implement the methods described herein.

[0016] In another aspect, a computer-readable medium is disclosed having code stored thereon. When the code is executed by a processor, the processor implements the methods described in this document.

[0017] These and other aspects are described in this document. Attached Figure Description

[0018] Figure 1 This is an example of the derivation process for constructing the Merge candidate list.

[0019] Figure 2 An exemplary location of the airspace merge candidate is shown.

[0020] Figure 3 An example of candidate pairs considered for redundancy checks of spatial merge candidates is shown.

[0021] Figures 4A-4B Exemplary locations of the second PU, divided into N×2N and 2N×N segments, are shown.

[0022] Figure 5 This is an exemplary illustration of motion vector scaling for temporal Merge candidates.

[0023] Figure 6 Exemplary candidate positions, C0 and C1, are shown for the temporal Merge candidate.

[0024] Figure 7 An example of combined bidirectional prediction of Merge candidates is shown.

[0025] Figure 8 An exemplary derivation process for motion vector prediction candidates is shown.

[0026] Figure 9 This is an exemplary illustration of motion vector scaling for spatial motion vector candidates.

[0027] Figure 10 An example of motion prediction using the alternative temporal motion vector prediction (ATMVP) method for CU is shown.

[0028] Figure 11 An example of a CU with four sub-blocks (AD) and its adjacent blocks (a–d) is shown.

[0029] Figure 12 Flowcharts showing examples of encoding with different MV precision

[0030] Figures 13A-13B The diagram illustrates a 135-degree segmentation type (divided from the top left corner to the bottom right corner) and a 45-degree segmentation pattern. It also shows an illustration of dividing the CU into two triangular prediction units (two segmentation patterns).

[0031] Figure 14 An example of the location of adjacent blocks is shown.

[0032] Figure 15 Examples of the top and left blocks of a video block are shown.

[0033] Figures 16A-16B Examples of 2-control-point motion vector (CPMV) and 3-CPMV are shown.

[0034] Figure 17 An example of an affine MVF for each sub-block is shown.

[0035] Figures 18A-18B Examples of affine models with 4 and 6 parameters are shown.

[0036] Figure 19 It is the MVP that inherits from the affine candidate AF_INTER.

[0037] Figure 20 An example of constructing an affine motion predictor in AF_INTER is shown.

[0038] Figures 21A-21B An example of control point motion vectors in affine encoding / decoding is shown in AF_MERGE.

[0039] Figure 22 An example of candidate positions for the affine Merge pattern is shown.

[0040] Figure 23 An example of an intra-frame image block copying operation is shown.

[0041] Figure 24 An example of a valid corresponding region in a co-location image is shown.

[0042] Figure 25 An exemplary flowchart for historical motion vector prediction is shown.

[0043] Figure 26 The modified Merge list construction process is shown.

[0044] Figure 27 An exemplary embodiment of proposing a valid region is shown when the current block is within the base region.

[0045] Figure 28 An exemplary embodiment of the effective region is shown when the current block is not within the base region.

[0046] Figure 29 This is a block diagram of an example video processing device.

[0047] Figure 30 This is a flowchart of an example of a video processing method.

[0048] Figure 31A This shows an example of where the default motion information identifier is located in the current standard.

[0049] Figure 31B An example of the location of the identifier for the default motion information in the proposed standard is shown.

[0050] Figure 32 This is a block diagram of an exemplary video processing system that can implement publicly available technologies.

[0051] Figure 33 This is a flowchart illustrating an example of a visual media processing method.

[0052] Figure 34 This is a flowchart illustrating an example of a visual media processing method.

[0053] Figure 35 This is a flowchart illustrating an example of a visual media processing method.

[0054] Figure 36 This is a flowchart illustrating an example of a visual media processing method.

[0055] Figure 37 This is a flowchart illustrating an example of a visual media processing method.

[0056] Figure 38 This is a flowchart illustrating an example of a visual media processing method.

[0057] Figure 39 This is a flowchart illustrating an example of a visual media processing method.

[0058] Figure 40 This is a flowchart illustrating an example of a visual media processing method.

[0059] Figure 41 This is a block diagram illustrating an exemplary video encoding and decoding system.

[0060] Figure 42 This is a block diagram illustrating an encoder according to some embodiments of the present disclosure.

[0061] Figure 43 This is a block diagram illustrating a decoder according to some embodiments of the present disclosure. Detailed Implementation

[0062] This document provides a variety of techniques that can be used by decoders of video bitstreams to improve the quality of decompressed or decoded digital video or images. Additionally, video encoders can implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0063] Chapter headings are used in this document for ease of understanding and do not limit the embodiments and techniques to the corresponding chapters. Therefore, embodiments from one chapter can be combined with embodiments from other chapters.

[0064] 1. Overview

[0065] This patent document relates to video codec technology. Specifically, it relates to motion vector encoding and decoding in video codecs. It can be applied to existing video codec standards like HEVC or standards yet to be finalized (general video codecs). It can also be applied to future video codec standards or video encoders.

[0066] 2. Introduction

[0067] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed the H.261 and H.263 standards, while ISO / IEC developed the MPEG-1 and MPEG-4 Visual standards. The two organizations jointly developed the H.262 / MPEG-2 video standard, the H.264 / MPEG-4 Advanced Video Coding (AVC) standard, and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was established to work on the VVC standard, which has a 50% lower bitrate compared to HEVC.

[0068] The latest version of the VVC draft, namely, Multi-Functional Video Codec (Draft 3), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 12_Macao / wg11 / JVET-L1001-v2.zip.

[0069] The latest reference software for VVC, named VTM, can be found at: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-3.0rc1

[0070] 2.1 Inter-frame prediction in HEVC / H.265

[0071] Each inter-frame prediction unit (PU) has motion parameters for one or two lists of reference images. The motion parameters consist of motion vectors and reference image indices. The use of one of the two reference image lists can also be notified via inter_pred_idc signaling. The motion vectors can be explicitly encoded as increments relative to the predictor.

[0072] When encoding and decoding a CU using the skip mode, a PU is associated with that CU, and there are no significant residual coefficients, no encoding / decoding motion vector increments, or reference picture indices. A merge mode is specified, thereby obtaining motion parameters for the current PU from adjacent PUs—including spatial and temporal candidates. The merge mode can be applied to any PU for inter-frame prediction, not just the skip mode. An alternative to the merge mode is explicit transmission of motion parameters, where the motion vectors (more precisely, the motion vector difference (MVD) relative to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the reference picture list usage are explicitly signaled to each PU. Such a mode is referred to in this disclosure as Advanced Motion Vector Prediction.

[0073] When signaling indicates that one of two lists of reference images should be used, a PU is generated from a sample block. This is often referred to as "one-way prediction." One-way prediction can be used for both P and B strips.

[0074] When signaling indicates that two lists of reference images should be used, the PU is generated from two sample blocks. This is called "bidirectional prediction". Bidirectional prediction can only be used for B-strips.

[0075] The following section provides details on the inter-frame prediction modes specified in HEVC. The description will begin with Merge mode.

[0076] 2.1.1 List of Reference Images

[0077] In HEVC, the term inter-frame prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference images other than the currently decoded image. As in H.264 / AVC, images can be predicted from multiple reference images. The reference images used for inter-frame prediction are organized into one or more reference image lists. A reference index identifies which reference images in the list should be used to create the predicted signal.

[0078] A single list of reference images—List 0—is used for the P-strip, and two lists of reference images—List 0 and List 1—are used for the B-strip. The reference images contained in List 0 / 1 can be from the past and future, depending on the capture / display order.

[0079] 2.1.2 Merge Mode

[0080] 2.1.2.1 Derivation of Merge Pattern Candidates

[0081] When predicting a PU using the Merge mode, indices pointing to entries in the Merge candidate list are parsed from the bitstream, and these indices are used to retrieve motion information. The construction of this list is explicit in the HEVC standard and can be summarized in the following steps:

[0082] Step 1: Initial Candidate Derivation

[0083] Step 1.1: Spatial Candidate Derivation

[0084] Step 1.2: Redundancy check of airspace candidates

[0085] Step 1.3: Time-domain candidate derivation

[0086] Step 2: Add candidate insertions

[0087] Step 2.1: Create bidirectional prediction candidates

[0088] Step 2.2: Insert zero-motion candidates

[0089] exist Figure 1 These steps are also schematically depicted. For spatial merge candidate derivation, up to four merge candidates are selected from candidates located at five different positions. For temporal merge candidate derivation, up to one merge candidate is selected from two candidates. Since the number of candidates per PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the stripe header. Because the number of candidates is constant, a binary unary truncation (TU) is used to encode the index of the best merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single merge candidate list, which is the same as the merge candidate list of the 2N×2N prediction unit.

[0090] The operations associated with the foregoing steps are described in detail below.

[0091] 2.1.2.2 Derivation of Airspace Candidates

[0092] In the derivation of the spatial Merge candidate, the candidate located in... Figure 2The diagram shows a maximum of four merged candidates for each position. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another stripe or slice) or if it is an intra-frame codec. After the candidate at position A1 is added, the remaining candidates undergo a redundancy check, which ensures that candidates with the same motion information are excluded from the list, thereby improving codec efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only those with... Figure 3 Pairs linked by arrows are considered, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a segmentation different from 2Nx2N. As an example, Figures 4A-4B The diagrams illustrate the second PU in the N×2N and 2N×N cases, respectively. When the current PU is segmented into N×2N, the candidate at position A1 is not considered during list construction. In fact, adding this candidate would result in two prediction units with the same motion information, which is redundant for a PU with only one encoding / decoding unit. Similarly, position B1 is not considered when the current PU is segmented into 2N×N.

[0093] 2.1.2.3 Time-domain candidate derivation

[0094] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located PU, which belongs to the image with the smallest POC difference relative to the current image within a given list of reference images. The list of reference images used for the derivation of the co-located PU is explicitly signaled in the strip header. Figure 5 As shown by the dashed line, the derivation of the motion vector used for scaling temporal merge candidates is obtained. This motion vector is scaled from the motion vector of the co-located PU using POC distances tb and td, where tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the co-located image and the co-located image. The reference image index of the temporal merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For B-strips, two motion vectors are obtained and combined to produce bidirectional prediction merge candidates, one of which is used for reference image list 0 (list 0) and the other for reference image list 1 (list 1).

[0095] Figure 5 This is a diagram illustrating the scaling of motion vectors for time-domain Merge candidates.

[0096] In the co-occurrence PU(Y) of the reference frame, the position of the temporal candidate is selected between candidate C0 and C1, such as... Figure 6 As shown. If the PU at position C0 is unavailable, is intra-coded, or is outside the current codec tree unit (CTU, also known as LCU, maximum codec unit) row, then position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0097] Figure 6 Examples of candidate positions for the temporal Merge candidate are shown, C0 and C1.

[0098] 2.1.2.4 Additional Candidate Insertion

[0099] In addition to spatial and temporal merge candidates, two additional types of merge candidates exist: combined bidirectional prediction merge candidates and zero merge candidates. Combined bidirectional prediction merge candidates are generated by utilizing spatial and temporal merge candidates. These combined bidirectional prediction merge candidates are only used for B-strips. They are generated by combining the motion parameters of the first reference image list of the initial candidate with the motion parameter matrix of the second reference image list of another candidate. If these two tuples provide different motion hypotheses, they form a new bidirectional prediction candidate. As an example, Figure 7 The following scenario illustrates a process where two candidates from the original list (on the left) with either mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create combined bidirectional predictive Merge candidates, which are then added to the final list (on the right). There are numerous rules governing the combinations considered to generate these additional Merge candidates.

[0100] Zero-motion candidates are inserted to populate the remaining entries in the Merge candidate list, thus reaching the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference image index, which starts at zero and increases each time a new zero-motion candidate is added to the list.

[0101] More specifically, perform the following steps in sequence until the Merge list is full:

[0102] 1. Set the variable numRef to the number of reference images associated with list 0 of the P strip, or the minimum number of reference images in the two lists of the B strip;

[0103] 2. Add non-repetitive zero motion candidates:

[0104] For variable i = 0…numRef-1, add a default motion candidate to list 0 (if P strip) or to both lists (if B strip), where MV is set to (0, 0) and the reference image index is set to i.

[0105] 3. Add repeating zero motion candidates, where MV is set to (0, 0), the reference image index of list 0 is set to 0 (if P stripe) and the reference image indices of both lists are set to 0 (if B stripe).

[0106] Ultimately, redundancy checks are not performed on these candidates.

[0107] 2.1.3 AMVP

[0108] AMVP utilizes the spatial-temporal correlation between motion vectors and adjacent PUs, which is used for the explicit transmission of motion parameters. For each list of reference images, a candidate list of motion vectors is constructed by first checking the availability of temporally adjacent PU locations to the left and top, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the Merge index signaling notification, a unary truncation is used to encode the index of the best motion vector candidate. The maximum value to be encoded in this case is 2 (see [link to documentation]). Figure 8 The following sections provide details of the derivation process for the motion vector prediction candidates.

[0109] 2.1.3.1 Derivation of AMVP Candidates

[0110] Figure 8 The derivation process of motion vector prediction candidates is summarized.

[0111] In motion vector prediction, two types of motion vector candidates are considered: spatial domain motion vector candidates and temporal domain motion vector candidates. For example... Figure 2 As shown, for the derivation of spatial motion vector candidates, two motion vector candidates are ultimately derived based on the motion vector of each PU located at five different positions.

[0112] For temporal motion vector candidate derivation, a motion vector candidate is selected from two candidates derived based on two distinct co-located positions. After creating an initial list of spatial-temporal candidates, duplicate motion vector candidates are removed from the list. If the number of potential candidates is greater than two, motion vector candidates whose reference image index is greater than 1 in the associated reference image list are removed from the list. If the number of spatial-temporal motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

[0113] 2.1.3.2 Candidate Spatial Motion Vectors

[0114] In the derivation of the spatial motion vector candidates, at most two candidates are considered from five potential candidates. The five potential candidates are selected from those located at... Figure 2 The positions shown are derived from the PUs, which are the same as the positions where motion is merged. The derivation order to the left of the current PU is defined as A0, A1 and scaled A0, scaled A1. The derivation order to the upper side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling and two of which do. The four different cases are summarized below.

[0115] • No spatial scaling

[0116] –(1) A list of identical reference images, and the same reference image index (same POC)

[0117] –(2) Different lists of reference images, but with the same reference image (same POC)

[0118] • Spatial scaling

[0119] –(3) Same list of reference images, but different reference images (different POCs)

[0120] –(4) A list of different reference images, and different reference images (different POCs)

[0121] First, we check for the case without spatial scaling, then we consider spatial scaling. Spatial scaling is considered when the reference image of the POC differs from the reference image of the current PU, regardless of the list of reference images. If all PUs in the left-hand candidate pool are unavailable or intra-frame encoded / decoded, scaling of the upper motion vector is allowed to aid in the parallel derivation of the left-hand and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

[0122] During spatial scaling, the motion vectors of adjacent PUs are scaled in a manner similar to temporal scaling, such as... Figure 9 As shown. The main difference is that the current PU's reference image list and index are given as input; the actual scaling process is the same as temporal scaling.

[0123] 2.1.3.3 Candidate Motion Vectors in the Time Domain

[0124] Apart from the derivation of the reference image index, the entire process of deriving the temporal merge candidate and the derivation of the spatial motion vector candidate (see...) Figure 6 (Same as above.) The reference image index is signaled to the decoder.

[0125] 2.2 Sub-CU-based motion vector prediction method in JEM

[0126] In a JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. Two sub-CU level motion vector prediction methods are considered in the encoder by dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU. The Alternative Temporal Motion Vector Prediction (ATMVP) method allows each CU to retrieve multiple sets of motion information from multiple blocks smaller than the current CU in a co-located reference image. In the Spatial-Temporal Motion Vector Prediction (STMVP) method, the motion vectors of sub-CUs are recursively derived using a temporal motion vector predictor and spatially adjacent motion vectors.

[0127] To preserve a more accurate motion field for sub-CU motion prediction, motion compression of the reference frame is currently disabled.

[0128] Figure 10 An example of CU's ATMVP motion prediction is shown.

[0129] 2.2.1 Alternative Time-Domain Motion Vector Prediction

[0130] In the Alternate Temporal Motion Vector Prediction (ATMVP) method, the temporal motion vector prediction (TMVP) is modified by retrieving multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. The sub-CU is a square N×N block (N is set to 4 by default).

[0131] ATMVP predicts motion vectors for sub-CUs within a CU in two steps. The first step is to identify corresponding blocks of a reference image with so-called temporal vectors. The reference image is called the motion source image. The second step is to divide the current CU into sub-CUs and obtain the motion vectors and reference indices for each sub-CU from the blocks corresponding to each sub-CU.

[0132] In the first step, reference images and corresponding blocks are determined using motion information of the spatially adjacent blocks of the current CU. To avoid repeated scanning of adjacent blocks, the first merge candidate in the current CU's merge candidate list is used. The first available motion vector and its associated reference index are set as a temporal vector and an index to the motion source image. In this way, in ATMVP, corresponding blocks can be identified more accurately compared to TMVP, where the corresponding block (sometimes called a co-located block) is always located at the bottom right or center position relative to the current CU.

[0133] In the second step, the corresponding block of the sub-CU is identified from the temporal vector in the motion source image by adding a temporal vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample point) is used to derive the sub-CU's motion information. After identifying the motion information of the corresponding N×N block, it is converted into the motion vector and reference index of the current sub-CU in the same manner as the TMVP of HEVC, where motion scaling and other processes are applied. For example, the decoder checks whether the low-latency condition (i.e., the POC of all reference images of the current image is less than the POC of the current image) is met, and may use the motion vector MV for each sub-CU. x (Motion vector corresponding to reference image list X) to predicted motion vector MV y (where X equals 0 or 1, and Y equals 1-X).

[0134] 2.2.2 Spacetime Motion Vector Prediction (STMVP)

[0135] In this method, the motion vector of the sub-CU is recursively derived following the grid scanning sequence. Figure 11 The concept is illustrated. For example, Figure 11 An 8×8 CU is shown, which has four 4×4 sub-CUs A, B, C, and D. Adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

[0136] Motion derivation for sub-CU A begins with identifying its two spatial neighborhoods. The first neighborhood is the N×N block (block c) above sub-CU A. If block c is unavailable or intra-coded, the other N×N blocks above sub-CU A are checked (from left to right, starting with block c). The second neighborhood is the block to the left of sub-CU A (block b). If block b is unavailable or intra-coded, the other blocks to the left of sub-CU A are checked (from top to bottom, starting with block b). Motion information obtained from adjacent blocks for each list is scaled to the first reference frame for the given list. Next, the temporal motion vector predictor (TMVP) for sub-block A is derived using the same procedure as the TMVP derivation specified in HEVC. Motion information for the co-occurring block at position D is retrieved and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The averaged motion vector is assigned as the motion vector for the current sub-CU.

[0137] 2.2.3 Sub-CU Motion Prediction Mode Signaling Notification

[0138] Sub-CU modes are enabled as additional Merge candidates, and no additional syntax elements are required for signaling notification modes. Two additional Merge candidates are added to the Merge candidate list for each CU to represent ATMVP and STMVP modes. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven Merge candidates are used. The encoding and decoding logic for additional Merge candidates is the same as the encoding logic for Merge candidates in HM, meaning that for each CU in a P or B stripe, two additional Merge candidates require two more RD checks.

[0139] In JEM, the entire binary number of the Merge index is encoded and decoded using the CABAC context. In HEVC, however, only the first binary number is encoded and decoded using the context, and the remaining binary numbers are encoded and decoded using the context bypass.

[0140] 2.3 Inter-frame prediction methods in VVC

[0141] Several new encoding and decoding tools exist for improving inter-frame prediction, such as Adaptive Motion Vector Differential Resolution (AMVR) for signaling notification MVD, Affine Prediction Mode, Triangle Prediction Mode (TPM), ATMVP, Normalized Bidirectional Prediction (GBI), and Bidirectional Optical Flow (BIO).

[0142] 2.3.1 Adaptive motion vector difference resolution

[0143] In HEVC, when `use_integer_mv_flag` is equal to 0 in the strip header, the motion vector difference (MVD) (between the motion vector and the predicted motion vector of the PU) is signaled in quarter-luminance samples. In VVC, Local Adaptive Motion Vector Resolution (LAMVR) is introduced. In VVC, MVD can be in quarter-luminance samples, integer luminance samples, or four luminance samples (i.e., 1 Encoding and decoding are performed in units of 4-pixels, 1-pixels, and 4-pixels. MVD resolution is controlled at the codec unit (CU) level, and an MVD resolution flag is conditionally signaled for each CU having at least one non-zero MVD component.

[0144] For a CU with at least one non-zero MVD component, signaling informs a first flag to indicate whether quarter-luminance sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter-luminance sample MV precision is not used, signaling informs another flag to indicate whether integer luminance sample MV precision or four-luminance sample MV precision is used.

[0145] When the first MVD resolution flag of the CU is zero, or when the CU is not coded or decoded (meaning all MVDs in the CU are zero), a quarter-luma sample MV resolution is used for the CU. When the CU uses integer-luma sample MV precision or 4-luma sample MV precision, the MVP in the AMVP candidate list of the CU is rounded to the corresponding precision.

[0146] In the encoder, CU-level RD checks are used to determine which MVD resolution is to be used for the CU. That is, three CU-level RD checks are performed for each MVD resolution. To speed up the encoder, the following coding scheme is applied in JEM.

[0147] · During the RD check of a CU with normal quarter-luma sample MVD resolution, the motion information (integer-luma sample accuracy) of the current CU is stored. The stored motion information (after rounding) is used as the starting point for a small-range motion vector refinement during the RD check of the same CU with integer-luma sample and 4-luma sample MVD resolutions, so that the time-consuming motion estimation process is not repeated three times.

[0148] · The RD check of a CU with 4-luma sample MVD resolution is conditionally invoked. For a CU, when the RD cost of integer-luma sample MVD resolution is much greater than that of quarter-luma sample MVD resolution, the RD check of the 4-luma sample MVD resolution of the CU is skipped.

[0149] The encoding process is shown in Figure 12 First, the 1 / 4 pixel MV is tested, and the RD cost is calculated and designated as RDCost0. Then the integer MV is tested, and the RD cost is designated as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive value), then the 4 pixel MV is tested; otherwise, the 4 pixel MV is skipped. Basically, when checking the integer or 4 pixel MV, the motion information and RD cost, etc. for the 1 / 4 pixel MV are known, which can be reused to accelerate the encoding process of the integer or 4 pixel MV.

[0150] 2.3.2 Triangle Prediction Mode

[0151] The concept of the triangle prediction mode (TPM) is to introduce a new triangle partition for motion compensation prediction. As Figures 13A-13BAs shown, the CU is divided into two triangular prediction units along either the diagonal or anti-diagonal direction. Each triangular prediction unit in the CU is inter-frame predicted using its own unidirectional predicted motion vector and a reference frame index derived from a single unidirectional prediction candidate list. An adaptive weighting process is applied to the diagonal edges after predicting the triangular prediction units. Then, a transform and quantization process is applied to the entire CU. This mode is only applied to the Merge mode (skip mode is considered a special Merge mode).

[0152] Figures 13A-13B This is a diagram illustrating the division of the CU into two triangular prediction units (two partitioning modes). For example, Figure 13A An example of a 135-degree segmentation type (divided from the top left corner to the bottom right corner) is shown, and Figure 13B An example of a 45-degree division pattern is shown.

[0153] 2.3.2.1 One-way prediction candidate list for TPM

[0154] The one-way prediction candidate list (named the TPM motion candidate list) consists of five one-way prediction motion vector candidates. It is derived from seven adjacent blocks, including five spatial adjacent blocks (1 to 5) and two temporal co-located blocks (6 to 7), as follows: Figure 14 As shown, motion vectors from seven adjacent blocks are collected and placed into a unidirectional prediction candidate list in the following order: unidirectional prediction motion vector, L0 motion vector of bidirectional prediction motion vector, L1 motion vector of bidirectional prediction motion vector, and the average motion vector of L0 and L1 motion vectors of bidirectional prediction motion vector. If the number of candidates is less than five, a zero motion vector is added to the list. Motion candidates added to this list for TPM are called TPM candidates, and motion information derived from spatial / temporal blocks is called regular motion candidates.

[0155] More specifically, the following steps are involved:

[0156] 1) When adding regular motion candidates from spatial adjacent blocks, use the full pruning operation. From A1, B1, B0, A0, B2, Col, and Col2 (corresponding to...) Figure 14 Obtain regular motion candidates from blocks 1-7.

[0157] 2) Set the variable numCurrMergeCand = 0

[0158] 3) For each regular motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, if it is not pruned and numCurrMergeCand is less than 5, and if the regular motion candidate is a one-way prediction (from list 0 or list 1), it is directly added to the Merge list as a TPM candidate, and numCurrMergeCand is incremented by 1. Such a TPM candidate is named 'Original One-Way Prediction Candidate'.

[0159] Apply full pruning .

[0160] 4) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, if no pruning is performed and numCurrMergeCand is less than 5, and if the regular motion candidate is a bidirectional prediction, the motion information from list 0 is added to the TPM Merge list (i.e., modified to a unidirectional prediction from list 0) as a new TPM candidate and numCurrMergeCand is incremented by 1. Such a TPM candidate is named 'Truncation List0 Prediction Candidate'.

[0161] Apply full pruning.

[0162] 5) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, if it is not pruned and numCurrMergeCand is less than 5, and if the regular motion candidate is a bidirectional prediction, the motion information from List 1 is added to the TPM Merge list (i.e., modified to a unidirectional prediction from List 1), and numCurrMergeCand is incremented by 1. Such a TPM candidate is named 'truncated List 1 prediction candidate'.

[0163] Apply full pruning.

[0164] 6) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, if no pruning is performed and numCurrMergeCand is less than 5, and if the regular motion candidate is a bidirectional prediction,

[0165] – If the strip QP of the reference image in list 0 is less than the strip QP of the reference image in list 1, then the motion information of list 1 is first scaled to the reference image in list 0, and the average of the two MVs (one from the original list 0 and the other from the scaled MV in list 1) is added to the TPM Merge list. Such a candidate is called the average one-way prediction from the motion candidate in list 0, and numCurrMergeCand is increased by 1.

[0166] - Otherwise, the motion information from list 0 is first scaled to the reference image from list 1, and the average of the two MVs (one from the original list 1 and the other from the scaled MV from list 0) is added to the TPM Merge list. Such TPM candidates are called the average one-way predictions from the motion candidates in list 1, and numCurrMergeCand is increased by 1.

[0167] Apply full pruning.

[0168] 7) If numCurrMergeCand is less than 5, add a zero motion vector candidate.

[0169] When a candidate is inserted into a list, if it must be compared with all previously added candidates to find out if it is the same as one of them, such a process is called full pruning.

[0170] 2.3.2.2 Adaptive Weighting Process

[0171] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangular prediction units to derive the final prediction for the entire CU. The two weighting factor arrays are defined as follows:

[0172] • First weighted factor array: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8}

[0173] They were used for luminance and chrominance samples, respectively.

[0174] • The second weighted factor set: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8,

[0175] 4 / 8 and 2 / 8 were used for luminance and chrominance samples, respectively.

[0176] The weighted factor array is selected based on the comparison of the motion vectors of two triangular prediction cells. A second weighted factor array is used when the reference images of the two triangular prediction cells are different from each other or when the difference in their motion vectors is greater than 16 pixels. Otherwise, a first weighted factor array is used.

[0177] 2.3.2.3 Signaling notification for Triangle Prediction Mode (TPM)

[0178] First, a signaling notification can be sent indicating whether to use a one-bit flag of TPM. Then, two partitioning modes (such as...) Figures 13A-13B As shown in the diagram, the selected Merge index is further signaled to each of the two partitions.

[0179] 2.3.2.3.1 Signaling notification of TPM flag

[0180] If the width and height of the luma block are represented by W and H respectively, then the triangle prediction mode is disabled for W*H<64.

[0181] When a block is encoded or decoded using affine mode, triangle prediction mode is also disabled.

[0182] When a block is encoded or decoded in Merge mode, a signaling flag can be sent to indicate whether triangle prediction mode is enabled or disabled for that block.

[0183] The flag is encoded and decoded using 3 contexts, based on the following equation:

[0184] Ctxindex=((left blockLavailable&&L is coded with TPM?)1:0)

[0185] +((Above block A available&&Ais coded with TPM?)1:0);

[0186] Figure 15 Examples of adjacent blocks (A and L) used for context selection in TPM flag encoding and decoding are shown.

[0187] 2.3.2.3.2 Signaling notifications indicating the two partitioning modes (as shown in Figure 13), and the selected Merge index for each of the two partitions.

[0188] Note that the partitioning pattern and the merge indexes of the two partitions are jointly encoded and decoded. Furthermore, the merge indexes are restricted so that the two partitions cannot use the same reference index. Therefore, there are 2(partitioning patterns) * N (maximum number of merge candidates) * (N-1) possibilities, where N is set to 5. An instruction is encoded and decoded, and the mapping between partitioning patterns, the two merge indices, and the encoding / decoding instruction are derived from an array defined below:

[0189] const uint8_t g_TriangleCombination[TRIANGLE_MAX_NUM_CANDS][3]={

[0190] {0,1,0},{1,0,1},{1,0,2},{0,0,1},{0,2,0},

[0191] {1,0,3},{1,0,4},{1,1,0},{0,3,0},{0,4,0}

[0192] {0,0,2},{0,1,2},{1,1,2},{0,0,4},{0,0,3},

[0193] {0,1,3},{0,1,4},{1,1,4},{1,1,3},{1,2,1},

[0194] {1,2,0},{0,2,1},{0,4,3},{1,3,0},{1,3,2},

[0195] {1,3,4},{1,4,0},{1,3,1},{1,2,3},{1,4,1},

[0196] {0,4,1},{0,2,3},{1,4,2},{0,3,2},{1,4,3},

[0197] {0,3,1},{0,2,4},{1,2,4},{0,4,2},{0,3,4};

[0198] Division mode (45 degrees or 135 degrees) = g_TriangleCombination[signaled indication][0];

[0199] Merge index of candidate A = g_TriangleCombination[signaled indication][1];

[0200] Merge index of candidate B = g_TriangleCombination[signaled indication][2];

[0201] Once two motion candidates A and B are derived, the motion information for the two segments (PU1 and PU2) can be set from either A or B. Whether PU1 uses the motion information from merged candidate A or B depends on the prediction directions of the two motion candidates. Table 1 shows the relationship between the two motion candidates A and B derived using the two segments.

[0202] Table 1: Derivation of motion information from the segmentation of the two Merge candidates (A, B)

[0203]

[0204] 2.3.2.3.3 indicates the entropy encoding / decoding (referred to by merge_triangle_idx).

[0205] merge_triangle_idxWithin the range [0, 39] (inclusive). The K-order Exponential Golomb (EG) encoding / decoding is used for binarization of merge_triangle_idx, where K is set to 1.

[0206] K times EG

[0207] To encode larger numbers in fewer bits (at the cost of using more bits to encode smaller numbers), this can be normalized using a non-negative integer parameter k. For encoding a non-negative integer x in k-th power exponent Columbus encoding / decoding:

[0208] 1. Use the 0th power Columbus codec described above to encode x / 2 k ,but

[0209] 2. Encode x modulo 2 in binary. k

[0210] Table 2: Examples of Exponential Columbus k-Encoding / Decoding

[0211]

[0212] 2.3.3 Affine Motion Compensation Prediction

[0213] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, in the real world, many types of motion exist, such as scaling, rotation, perspective motion, and other unconventional motions. In VVC, simplified affine transformation motion compensation prediction is applied using 4-parameter and 6-parameter affine models. Figures 16A-16B As shown, the block affine motion field for a 4-parameter affine model ( Figure 16A ) is described by two control point motion vectors (CPMV) and for a 6-parameter affine model ( Figure 16B It is described by 3 CPMVs.

[0214] The motion vector field (MVF) of the block is described by the following equations, using the 4-parameter affine model in equation (1) (where the 4 parameters are defined as variables a, b, e, and f) and the 6-parameter affine model in equation (2) (where the 4 parameters are defined as variables a, b, c, d, e, and f):

[0215]

[0216] Where (mv h 0,mv h 0) is the motion vector of the left top corner control point, and (mv h 1, MV h 1) is the motion vector of the right top corner control point, and (mv h 2, MV h2) is the motion vector of the left bottom corner control point. All three motion vectors are called the control point motion vector (CPMV). (x, y) represents the coordinates of the representative point relative to the left top sample point within the current block, and (mv h (x,y),mv v (x, y) is the motion vector derived from the sample point located at (x, y). The CP motion vector can be signaled (as in affine AMVP mode) or derived in real time (as in affine Merge mode). w and h are the width and height of the current block. In practice, division is implemented by a right shift with a rounding operation. In VTM, the representative point is defined as the center position of the sub-block; for example, when the coordinates of the left top corner of the sub-block relative to the left top sample point within the current block are (xs, ys), the coordinates of the representative point are located at (xs+2, ys+2). For each sub-block (i.e., 4x4 in VTM), the representative point is used to derive the motion vector for the entire sub-block.

[0217] To further simplify motion compensation prediction, a sub-block-based affine transformation prediction is applied. To derive the motion vector for each M×N (where M and N are set to 4 in the current VVC) sub-block, as follows... Figure 17 The motion vector of the center sample point of each sub-block shown is calculated according to equations (1) and (2) and rounded to 1 / 16 fractional accuracy. Then, a 1 / 16 pixel motion-compensated interpolation filter is applied to generate a prediction for each sub-block using the derived motion vector. The 1 / 16 pixel interpolation filter is introduced by an affine pattern.

[0218] After MCP, the high-accuracy motion vector of each sub-block is rounded and saved with the same accuracy as the normal motion vector.

[0219] 2.3.3.1 Signaling notification for affine prediction

[0220] Similar to the translational motion model, there are also two modes that signal auxiliary information for affine prediction. These are the AFFINE_INTER and AFFINE_MERGE modes.

[0221] 2.3.3.2 AF_INTER mode

[0222] For CUs with both width and height greater than 8, the AF_INTER mode can be applied. The affine flags in the CU level are signaled in the bitstream to indicate whether the AF_INTER mode is used.

[0223] In this mode, for each list of reference images (list 0 or list 1), an affine AMVP candidate list is constructed using three types of affine motion predictors in the following order, where each candidate contains an estimated CPMV for the current block. The best CPMV found on the encoder side (such as...) Figure 20 The differences between mv0, mv1, and mv2 in the CPMV and the estimated CPMV are signaled. Furthermore, the index of the affine AMVP candidate derived from the estimated CPMV is also signaled.

[0224] 1) Inherited affine motion predictor

[0225] The checking order is similar to the order of the spatial MVPs in the HEVC AMVP list construction. First, a left-inherited affine motion predictor is derived from the first block in {A1, A0}, which is affine-coded and has the same reference picture as the current block. Second, an upper-inherited affine motion predictor is derived from the first block in {B1, B0, B2}, which is affine-coded and has the same reference picture as the current block. Figure 19 The image shows five blocks: A1, A0, B1, B0, and B2.

[0226] Once an adjacent block is found to be encoded in affine mode, the CPMV of the codec unit covering the adjacent block is used to derive the predictor of the CPMV of the current block. For example, if A1 is encoded in non-affine mode and A0 is encoded in 4-parameter affine mode, the left-inherited affine MV predictor will be derived from A0. In this case, the CPMV of the CU covering A0 (in...) Figure 21B In the middle, the left-top CPMV refers to... And the right-top CPMV refers to The values ​​) are used to derive the estimated CPMV of the current block, where the left top (with coordinates (x0, y0)), right top (with coordinates (x1, y1)), and right bottom (with coordinates (x2, y2)) of the current block are represented as

[0227] 2) Constructed affine motion predictor

[0228] The constructed affine motion predictor consists of control point motion vectors (CPMVs) derived from adjacent inter-frame codec blocks with the same reference image, such as... Figure 20 As shown. If the current affine motion model is a 4-parameter affine, then the number of CPMVs is 2; otherwise, if the current affine motion model is a 6-parameter affine, then the number of CPMVs is 3. Left-top CPMV Based on the MV derivation at the first block in group {A, B, C}, the first block in group {A, B, C} is inter-frame encoded and has the same reference image as the current block. (Right top CPMV) Based on the MV derivation at the first block in group {D, E}, the first block in group {D, E} is inter-frame encoded and has the same reference image as the current block. (Left bottom CPMV) Based on the MV derivation at the first block in group {F, G}, the first block in {F, G} is inter-frame encoded and has the same reference picture as the current block.

[0229] –If the current affine motion model is a 4-parameter affine, then only if and All were found (i.e., and The affine motion predictor is only inserted into the candidate list when it is used as the estimated CPMV of the left top (with coordinates (x0, y0)) and right top (with coordinates (x1, y1)) position of the current block.

[0230] –If the current affine motion model is a 6-parameter affine, then only if and All were found (i.e., and The affine motion predictor constructed is only inserted into the candidate list when it is used as an estimated CPMV of the left top (with the coordinates (x0, y0) of the current block), right top (with the coordinates (x1, y1)), and right bottom (with the coordinates (x2, y2)).

[0231] When the constructed affine motion predictor is inserted into the candidate list, no pruning process is applied.

[0232] 3) Standard AMVP motion predictor

[0233] Apply the following until the number of affine motion predictors reaches its maximum value.

[0234] 1) By setting all CPMV to equal (If available) Derive the affine motion predictor.

[0235] 2) By setting all CPMV to equal (If available) Derive the affine motion predictor.

[0236] 3) By setting all CPMV to equal (If available) Derive the affine motion predictor.

[0237] 4) Derive the affine motion predictor by setting all CPMVs to equal HEVC TMVP (if available).

[0238] 5) Derive the affine motion predictor by setting all CPMVs to zero MV.

[0239] It is noted that It has been derived in the constructed affine motion predictor.

[0240] Figure 18AAn example of a 4-parameter affine model is shown. Figure 18B An example of a 6-parameter affine model is shown.

[0241] Figure 20 An example of an MVP for the constructed affine candidate AF_INTER is shown.

[0242] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required, and therefore 2 / 3 MVDs need to be encoded and decoded for these control points, as shown in Figure 18. JVET-K0337 proposes the following derivation of MV: predicting mvd1 and mvd2 from mvd0.

[0243]

[0244] in mvd i mv1 and mv1 are the predicted motion vector, motion vector difference, and motion vector of the left top pixel (i=0), right top pixel (i=1), or left bottom pixel (i=2), respectively. Figure 18B As shown. Note that adding two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) equals the sum of two separate components, i.e., newMV = mvA + mvB, and the two components of newMV are set as (xA + xB) and (yA + yB) respectively.

[0245] 2.3.3.3 AF_MERGE mode

[0246] When CU is applied in AF_MERGE mode, it obtains the first block encoded and decoded using affine mode from the reconstructed blocks of the effective neighborhood. Furthermore, the selection order of candidate blocks is from left, top, top right, bottom left to top left, as follows: Figure 21A As shown (referred to as A, B, C, D, E in order). For example, if the neighboring left bottom block is encoded and decoded in affine mode (in... Figure 21B If the middle finger is represented as A0, then retrieve the motion vector mv0 of the control point (CP) of the adjacent CU / PU containing block A, specifically the left top corner, right top corner, and left bottom corner. N mv1 N and or mv2 N And the motion vector mv0 of the current CU / PU's left top corner / right top corner / left bottom corner. C mv1 C and mv2 C (It is only used for 6-parameter affine models) and is based on mv0. N mv1 N and mv2 NCalculation. In VTM-2.0, if the current block is affine-coded, the sub-block at the top left corner (e.g., a 4×4 block in VTM) stores mv0, and the sub-block at the top right corner stores mv1. If the current block is affine-coded with a 6-parameter model, the sub-block at the bottom left corner stores mv2; otherwise (with a 4-parameter affine model), LB stores mv2'. Other sub-blocks store the MV used for MC.

[0247] Derivation of the current CU's CPMV mv0 C mv1 C and mv2 C Then, the MVF of the current CU is generated according to the simplified affine motion model equations (1) and (2). In order to identify whether the current CU is encoded and decoded in AF_MERGE mode, when there is at least one neighboring block encoded and decoded in affine mode, the affine flag is signaled in the bit stream.

[0248] In JVET-L0142 and JVET-L0632, the affine Merge candidate list is constructed using the following steps:

[0249] 1) Inserting affine inheritance candidate

[0250] Inherited affine candidates are candidates derived from the affine motion models of their effective neighboring affine codec blocks. At most two inherited affine candidates are derived from the affine motion models of adjacent blocks and inserted into the candidate list. For the left predictor, the scan order is {A0, A1}; for the top predictor, the scan order is {B0, B1, B2}.

[0251] 2) Insertion of constructed affine candidates

[0252] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (e.g., 5), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate is one that is built by combining the neighborhood motion information of each control point.

[0253] a) First from Figure 22 The diagram shows the derivation of motion information for control points within specified spatial and temporal neighborhoods. CPk (k = 1, 2, 3, 4) represents the k-th control point. A0, A1, A2, B0, B1, B2, and B3 are used to predict the spatial position of CPk (k = 1, 2, 3); T is used to predict the temporal position of CP4.

[0254] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H) respectively, where W and H are the width and height of the current block.

[0255] Motion information for each control point is obtained according to the following priority order:

[0256] – For CP1, the check priority is B2->B3->A2. B2 is used if it is available. Otherwise, if B2 is available, B3 is used. If neither B2 nor B3 is available, A2 is used. If all three candidates are unavailable, motion information for CP1 cannot be obtained.

[0257] – For CP2, the check priority is B1->B0.

[0258] – For CP3, the check priority is A1->A0.

[0259] – For CP4, use T.

[0260] b) Secondly, combinations of control points are used to construct affine Merge candidates.

[0261] I. Motion information from three control points is needed to construct a 6-parameter affine candidate. The three control points can be selected from one of the following four combinations: ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combinations {CP1, CP2, CP3}, {CP2, CP3, CP4}, and {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the left top, right top, and left bottom control points.

[0262] II. Motion information from two control points is needed to construct a 4-parameter affine candidate. Two control points can be selected from one of two combinations: ({CP1, CP2}, {CP1, CP3}). The two combinations will be transformed into a 4-parameter motion model represented by the left and right top control points.

[0263] III. The combinations of constructed affine candidates are inserted into the candidate list in the following order:

[0264] {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}

[0265] i. For each combination, check the reference index of list X for each CP. If they are all the same, the combination has a valid CPMV for list X. If the combination does not have a valid CPMV for both list 0 and list 1, the combination is marked as invalid. Otherwise, it is valid, and the CPMV is placed in the sub-block Merge list.

[0266] 3) Fill with zero motion vector

[0267] If the number of candidates in the affine Merge candidate list is less than 5, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.

[0268] More specifically, for the sub-block Merge candidate list, there are 4-parameter Merge candidates with an MV set to (0,0) and a prediction direction set to unidirectional prediction (for P-strips) and bidirectional prediction (for B-strips) from list 0.

[0269] 2.3.4 Current Image Reference

[0270] Intra-frame block copying (also known as IBC, or intra-frame picture block compensation) (also called current picture reference (CPR)) is used in HEVC Screen Content Codec Extensions (SCC). This tool is highly efficient for encoding and decoding screen content video because recurring patterns in rich text and graphic content occur frequently within the same picture. Using previously reconstructed blocks with equal or similar patterns as predictors can effectively reduce prediction errors and thus improve encoding and decoding efficiency. An example of intra-frame block compensation is shown in... Figure 23 The diagram in the middle is shown.

[0271] Similar to the CRP design in HEVC SCC, in VVC, the use of IBC mode is announced at both the sequence and picture levels. When IBC mode is enabled in the Sequence Parameter Set (SPS), it can also be enabled at the picture level. When IBC mode is enabled at the picture level, the currently reconstructed picture is treated as the reference picture. Therefore, based on existing VVC inter-frame modes, no block-level syntax changes are required to announce the use of IBC mode via signaling.

[0272] Main features:

[0273] – It is considered a normal inter-frame mode. Therefore, Merge mode and Skip mode are also applicable to IBC.

[0274] The mode is available. The merge candidate list is constructed uniformly, containing merge candidates from adjacent positions encoded and decoded in IBC mode or HEVC inter-frame mode. Depending on the selected merge index, the current block in merge mode or skip mode can be merged into the neighborhood encoded and decoded in IBC mode or the neighborhood encoded and decoded in other normal inter-frame modes, with a different picture as the reference picture.

[0275] – The block vector prediction and encoding / decoding scheme in IBC mode reuses the scheme used for motion vector prediction and the encoding / decoding scheme in HEVC inter-frame mode (AMVP and MVD encoding / decoding).

[0276] Motion vectors (also known as block vectors) in IBC mode are encoded and decoded with integer pixel precision, but stored in memory with 1 / 16 pixel precision after decoding because quarter-pixel precision is used in the interpolation and deblocking stages. When used in motion vector prediction for IBC mode, the stored vector predictor is shifted right by 4.

[0277] – Search scope: It is limited to the current CTU.

[0278] – CPR is disabled when affine mode / triangle mode / GBI / weighted prediction is enabled.

[0279] 2.3.5 Merge List Design in VVC

[0280] There are three different Merge list construction processes supported in VVC:

[0281] 1) Sub-block Merge Candidate List: This list contains ATMVP and affine Merge candidates. The process for building a Merge list is shared for both affine and ATMVP modes. Here, ATMVP and affine Merge candidates can be added sequentially. The size of the sub-block Merge list is signaled in the stripe header and has a maximum value of 5.

[0282] 2) Unidirectional Prediction TPM Merge List: For the triangular prediction pattern, the process of constructing a merge list for two segments is shared, even though the two segments can choose their own merge candidate indices. When constructing this merge list, the spatial neighboring blocks and two temporal blocks of the block are checked. The motion information derived from the spatial neighboring blocks and temporal blocks is called regular motion candidates in our IDF. These regular motion candidates are further used to derive multiple TPM candidates. Note that transformations are performed at the block level, even though the two segments can use different motion vectors to generate their own prediction blocks.

[0283] The size of the one-way predictive TPM Merge list is fixed at 5.

[0284] 3) Regular Merge List: A Merge list construction process is shared for the remaining codec blocks. Here, spatial / temporal / HMVP, pairwise combined bidirectional prediction Merge candidates, and zero-motion candidates can be inserted sequentially. The size of the regular Merge list is signaled in the stripe header and has a maximum value of 6.

[0285] 2.3.5.1 Sub-block Merge Candidate List

[0286] It is recommended to place all sub-blocks related to motion candidates into a separate Merge list, in addition to the regular Merge list used for non-sub-block Merge candidates.

[0287] Motion candidates related to sub-blocks are placed into a separate Merge list, named 'Sub-block Merge Candidate List'.

[0288] In one example, the sub-block Merge candidate list contains affine Merge candidates, ATMVP candidates, and / or sub-block-based STMVP candidates.

[0289] 2.3.5.1.1 JVET-L0278

[0290] In this submission, ATMVP Merge candidates from the regular Merge list are moved to the first position in the affine Merge list. This ensures that all Merge candidates in the new list (i.e., the sub-block-based Merge candidate list) are based on the sub-block encoding / decoding tool.

[0291] 2.3.5.1.2 ATMVP in VTM-3.0

[0292] In VTM-3.0, in addition to the regular merge candidate list, a special merge candidate list, known as the sub-block merge candidate list (and also known as the affine merge candidate list), is added. The sub-block merge candidate list is populated with candidates in the following order:

[0293] a. ATMVP candidate (may be available or not);

[0294] b. Inheriting affine candidates;

[0295] c. Construct affine candidates;

[0296] d. As a filler for a zero MV 4-parameter affine model

[0297] The derivation of the maximum number of candidates (referred to as ML) in the sub-block Merge candidate list is as follows:

[0298] 1) If ATMVP is enabled (equal to 1) using a flag (e.g., the flag could be named "sps_sbtmvp_enabled_flag"), but affine is disabled (equal to 0) using a flag (e.g., the flag could be named "sps_affine_enabled_flag"), then ML is set to equal to 1.

[0299] 2) If ATMVP is off using the flag (equal to 0), and affine is off using the flag (equal to 0), then ML is set to 0. In this case, the sub-block Merge candidate list is not used.

[0300] 3) Otherwise (affine uses the flag on (equal to 1), ATMVP uses the flag on or off), signal ML from the encoder to the decoder. Valid ML is 0 <= ML <= 5.

[0301] When constructing the sub-block merge candidate list, the ATMVP candidate is checked first. If any of the following conditions are true, the ATMVP candidate is skipped and not added to the sub-block merge candidate list.

[0302] 1) ATMVP is turned off using a flag;

[0303] 2) Any TMVP using a flag (e.g., when notifying at the stripe level signaling level, the flag can be named "slice_temporal_mvp_enabled_flag") is turned off;

[0304] 3) The reference image with reference index 0 in reference list 0 is identical to the current image (which is CPR).

[0305] ATMVP in VTM-3.0 is much simpler than in JEM. When generating ATMVP merge candidates, the following procedure is applied:

[0306] a. Check adjacent blocks A1, B1, B0, A0 in sequence, such as Figure 22 As shown, the block that is the first inter-frame codec but not CPR codec is identified and referred to as block X;

[0307] b. Initialize TMV = (0,0). If there exists an MV (referring to MV') for block X, referencing the common bit reference image (such as the signaling notification in the strip header), then TMV is set to be equal to MV'.

[0308] c. Assuming the center point of the current block is (x0, y0), locate the corresponding position of (x0, y0) in the co-location image as M = (x0 + MV'x, y0 + MV'y). Find the block Z that covers M.

[0309] i. If Z is an intra-frame codec, then ATMVP is not available;

[0310] ii. If Z is inter-frame encoded, then the two lists of block Z, MVZ_0 and MVZ_1, are scaled to (Ref list 0 index 0) and (Ref list 1 index 0) as MVdefault0 and MVdefault1, and stored.

[0311] d. For each 8x8 sub-block, assuming its center point is (x0S, y0S), locate the corresponding position of (x0S, y0S) in the co-location image as MS = (x0S + MV'x, y0S + MV'y). Find the block ZS that covers MS.

[0312] i. If ZS is intra-frame encoded, then MVdefault0 and MVdefault1 are assigned to sub-blocks;

[0313] ii. If ZS is inter-frame encoded, then MVZS_0 and MVZS_1 of the two lists of block ZS are scaled to (Ref list 0 index 0) and (Ref list 1 index 0) and assigned to sub-blocks;

[0314] MV cropping and masking in ATMVP:

[0315] When locating a corresponding position such as M or MS in a co-location image, it is cropped within a predetermined area. The CTU size is S×S, where S = 128 in VTM-3.0. Assuming the top left position of the co-location CTU is (xCTU, yCTU), then the corresponding position M or MS at (xN, yN) will be cropped to the effective area xCTU <= xN. <xCTU+S+4;yCTU<=yN<yCTU+S。

[0316] In addition to clipping, (xN, yN) is also masked as xN = xN & MASK and yN = yN & MASK, where MASK is equal to ~(2 N The integers are given by N = 3, and the lowest 3 bits are set to 0. Therefore, xN and yN are numbers that are multiples of 8 ("~" indicates the bitwise compensation operator).

[0317] Figure 24 An example of a valid corresponding region in a co-location image is shown.

[0318] 2.3.5.1.3 Syntax Design in the Header

[0319]

[0320]

[0321] 2.3.5.2 Regular Merge List

[0322] Unlike the Merge list design, VVC uses the History-Based Motion Vector Prediction (HMVP) method.

[0323] The HMVP stores motion information from previously encoded / decoded blocks. Motion information from previously encoded / decoded blocks is defined as HMVP candidates. Multiple HMVP candidates are stored in a table named HMVP, and this table is maintained in real-time during the encoding / decoding process. The HMVP table is cleared when encoding / decoding a new stripe begins. As long as an inter-frame encoded / decoded block exists, the associated motion information is added to the last entry of the table as a new HMVP candidate. The overall encoding / decoding flow is as follows: Figure 25 As shown in the image.

[0324] HMVP candidates can be used in both the AMVP and Merge candidate list construction processes. Figure 26 The diagram illustrates the process of constructing the modified Merge candidate list (using...). Figure 26 (Highlighted boxes are shown in the image). When the Merge Candidate List is not full after the insertion of TMVP candidates, HMVP candidates stored in the HMVP table can be used to populate the Merge Candidate List. Considering that a block is generally highly correlated with its nearest neighboring block in terms of motion information, HMVP candidates in the table are inserted in descending order of their indexes. The last entry in the table is added to the list first, and the first entry is added last. Similarly, redundancy removal is applied to HMVP candidates. Once the total number of available Merge Candidates reaches the maximum number of Merge Candidates allowed for signaling notification, the Merge Candidate List construction process is terminated.

[0325] 2.4 MV Rounding

[0326] In VVC, when MV is right-shifted, it is required to be rounded toward zero. Formulaically, for MV(MVx, MVy) to be right-shifted by N bits, the result MV'(MVx', MVy') can be derived as:

[0327] MVx'=(MVx+((1<<N)> >1)-(MVx>=0?1:0))>>N;

[0328] MVy'=(MVy+((1<<N)> >1)-(MVy>=0?1:0))>>N;

[0329] 2.5 RPR in JVET-O2001-v14

[0330] ARC, also known as RPR (Reference Image Resampling), is integrated into JVET-O2001-v14.

[0331] In the case of RPR in JVET-O2001-v14, TMVP is disabled if the co-bit image has a different resolution than the current image. Furthermore, BDOF and DMVR are disabled when the reference image has a different resolution than the current image.

[0332] To handle regular MC when the reference image has a different resolution than the current image, the interpolation segment is defined as follows:

[0333] 8.5.6.3 Fractional Sample Interpolation Process

[0334] 8.5.6.3.1 Overall

[0335] The input to this process is:

[0336] – Luminance position (xSb, ySb), specifies the left top sample of the current codec sub-block relative to the left top luminance sample of the current image.

[0337] – The variable sbWidth specifies the width of the current codec sub-block.

[0338] – The variable sbHeight specifies the height of the current encoding / decoding sub-block.

[0339] – Motion vector offset mvOffset

[0340] –Refine the motion vector refMvLX,

[0341] –Selected reference image sample array refPicLX,

[0342] – Half-sample interpolation filter index hpelIfIdx,

[0343] – Bidirectional optical flow flag bdofFlag

[0344] – The variable cIdx specifies the color component index of the current block.

[0345] The output of this process is:

[0346] – The array predSamplesLX represents the predicted sample values ​​(sbWidth+brdExtSize)x(sbHeight+brdExtSize).

[0347] The predicted block bound extension size brdExtSize is derived as follows:

[0348] brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))? 2:0(8-752)

[0349] The variable fRefWidth is set to be equal to the PicOutputWidthL of the reference image in the brightness sample.

[0350] The variable fRefHeight is set to be equal to the PicOutputHeightL of the reference image in the brightness sample.

[0351] The motion vector mvLX is set to (refMvLX-mvOffset).

[0352] – If cIdx equals 0, then the following applies:

[0353] – Scaling factors and their fixed-point representations are defined as

[0354] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0355] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0356] – Let (xIntL, yIntL) be the luminance position given in whole sample units, and (xFracL, yFracL) be the offset given in 1 / 16 sample units. These variables are used only in this clause to specify the fractional sample positions within the reference sample array refPicLX.

[0357] – The left top coordinate of the boundary block filled with reference samples (xSbInt) L ySbInt L ) is set to (xSb+(mvLX[0]>>4), ySb+(mvLX[1]>>4)).

[0358] – For each luminance sample location (x) within the predicted luminance sample array predSamplesLX L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), corresponding to the predicted brightness sample value predSamplesLX[x L ][y L The derivation is as follows:

[0359] –make (refxSb) L ,refySb L ) and (refx L refy L) represents the brightness position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in 1 / 16 sample units. The variable refxSb L refx L ,refySb L and refy L The derivation is as follows:

[0360] refxSb L =((xSb<<4)+refMvLX[0])*hori_scale_fp(8-755)

[0361] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6(8-756)

[0362] refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp(8-757)

[0363] refyL=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6(8-758)

[0364] – Variable xInt L yInt L xFrac L and yFrac L The derivation is as follows:

[0365] xInt L =refx L >>4 (8-759)

[0366] yInt L =refy L >>4 (8-760)

[0367] xFrac L =refx L &15 (8-761)

[0368] yFrac L =refy L &15 (8-762)

[0369] – If bdofFlag equals true or (sps_affine_prof_enabled_flag equals true and inter_affine_flag[xSb][ySb] equals true), and one or more of the following conditions are true, then the luminance integer sample retrieval procedure as specified in Clause 8.5.6.3.3 is invoked to retrieve (xInt) the luminance integer sample. L +(xFrac L >>3)-1), yInt L +(yFrac L >>3)-1) and refPicLX are used as inputs to derive the predicted luminance sample values ​​predSamplesLX[x L ][y L ].

[0370] 1.x L It equals 0.

[0371] 2.x L It equals sbWidth+1.

[0372] 3.y L It equals 0.

[0373] 4.y L It equals sbHeight + 1.

[0374] – Otherwise, by invoking the luminance sample 8-tap interpolation filtering procedure specified in Clause 8.5.6.3.2, the result is (xIntL-(brdExtSize>0?1:0), yIntL-(brdExtSize>0?1:0)), (xFracL, yFracL), (xSbInt) L ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are used as inputs to derive the predicted luminance sample values ​​predSamplesLX[xL][yL].

[0375] Otherwise (cIdx is not equal to 0), the following applies:

[0376] – Let (xIntC, yIntC) be the chromaticity position given in whole sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify the overall fractional sample position within the reference sample array refPicLX.

[0377] – The left top coordinates (xSbIntC, ySbIntC) of the boundary block filled by the reference sample points are set to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0378] – For each chromaticity sample position (xC = 0..sbWidth-1, yC = 0..sbHeight-1) within the predicted chromaticity sample array predSamplesLX, the corresponding predicted chromaticity sample value predSamplesLX[xC][yC] is derived as follows:

[0379] –make (refxSb) C ,refySb C ) and (refx C refy C ) represents the chromaticity position pointed to by the motion vector (mvLX[0], mvLX[1]) given in 1 / 32 sample units. The variable refxSb C ,refySb C refx C and refy C The derivation is as follows:

[0380] refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp

[0381] (8-763)

[0382] refx C =((Sign(refxSb) C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5(8-764)

[0383] refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp

[0384] (8-765)

[0385] refy C =((Sign(refySb) C )*((Abs(refySb C)+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>>5(8-766)

[0386] – Variable xInt C yInt C xFrac C and yFrac C The derivation is as follows:

[0387] xInt C =refx C >>5 (8-767)

[0388] yInt C =refy C >>5 (8-768)

[0389] xFrac C =refy C &31 (8-769)

[0390] yFrac C =refy C &31 (8-770)

[0391] – By calling the procedure specified in Clause 8.5.6.3.4, the predicted sample values ​​predSamplesLX[xC][yC] are derived with (xIntC, yIntC), (xFracC, yFracC), (xSbIntC, ySbIntC), sbWidth, sbHeight, and refPicLX as input.

[0392] 8.5.6.3.2 Brightness Sample Interpolation and Filtering Process

[0393] The input to this process is:

[0394] – Brightness position in whole sample units (xInt) L yInt L ),

[0395] – Brightness position in fractional sample units (xFrac) L yFrac L ),

[0396] – Brightness position in whole sample units (xSbInt) L ySbInt L ), specifies the left top sample of the boundary block filled with the reference sample relative to the left top brightness sample of the reference image.

[0397] –Luminance reference sample array refPicLXL ,

[0398] – Half-sample interpolation filter index hpelIfIdx,

[0399] – The variable sbWidth specifies the width of the current child block.

[0400] – The variable sbHeight specifies the height of the current child block.

[0401] – Luminance position (xSb, ySb), specifies the left top sample of the current sub-block relative to the left top luminance sample of the current image.

[0402] The output of this process is the predicted luminance sample value, predSampleLX. L

[0403] The derivation of variables shift1, shift2, and shift3 is as follows:

[0404] – The variable shift1 is set to equal Min(4, BitDepth). Y -8), variable shift2 is set to 6, and variable shift3 is set to Max(2, 14-BitDepth). Y ).

[0405] – The variable picW is set to equal pic_width_in_luma_samples, and the variable picH is set to equal pic_height_in_luma_samples.

[0406] equals xFrac L or yFrac L The brightness interpolation filter coefficient f at each 1 / 16 fractional sample location p L [p]The derivation is as follows:

[0407] – If MotionModelIdc[xSb][ySb] is greater than 0, and sbWidth and sbHeight are both equal to 4, then the luminance interpolation filter coefficient f L [p] is specified in Table 8-12.

[0408] Otherwise, the brightness interpolation filter coefficient f L [p] depends on hpelIfIdx and is specified in Table 8-11.

[0409] Brightness position in whole sample units (xInt) i yInt i For i = 0..7, the derivation is as follows:

[0410] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0411] xInt i =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i-3)(8-771)

[0412] yInt i =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i-3)(8-772)

[0413] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0414] xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[0415] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY, picW, xInt L +i-3): (8-773)

[0416] xInt L +i-3)

[0417] yInt i =Clip3(0, picH-1, yInt) L +i-3) (8-774)

[0418] The brightness position in integer units for i = 0..7 is further modified as follows:

[0419] xInt i =Clip3(xSbInt) L -3, xSbInt L +sbWidth+4,xInt i (8-775)

[0420] yInt i =Clip3(ySbInt) L -3,ySbInt L+sbHeight+4,yInt i (8-776)

[0421] Predicted brightness sample value predSampleLX L The derivation is as follows:

[0422] –If xFrac L and yFrac L Both are equal to 0, predSampleLX L The value is derived as follows: predSampleLX L =refPicLX L [xInt3][yInt3]< <shift3 (8-777)

[0423] Otherwise, if xFrac L Not equal to 0 and yFrac L If it equals 0, then predSampleLX L The value is derived as follows:

[0424]

[0425]

[0426] Otherwise, if xFrac L Equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The value is derived as follows:

[0427]

[0428] Otherwise, if xFrac L Not equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The value is derived as follows:

[0429] The sample array temp[n], where n = 0..7, is derived as follows:

[0430]

[0431] –Predicted luminance sample value predSampleLX L The derivation is as follows:

[0432]

[0433] Table 8-11 – Brightness interpolation filter coefficients f for each 1 / 16 fractional sample location p LThe specification of [p].

[0434]

[0435]

[0436] Table 8-12 – Luminance interpolation filter coefficients f for each 1 / 16 fractional sample location p of the affine motion pattern L The specification of [p].

[0437]

[0438] 8.5.6.3.3 Luminance Integer Sample Retrieval Process

[0439] The input to this process is:

[0440] – Brightness position in whole sample units (xInt) L yInt L ),

[0441] – Luminance reference sample array refPicLX L ,

[0442] The output of this process is the predicted luminance sample value, predSampleLX. L

[0443] The variable offset is set to equal Max(2, 14-BitDepth). Y ).

[0444] The variable picW is set to equal pic_width_in_luma_samples, and the variable picH is set to equal pic_height_in_luma_samples.

[0445] The brightness position (xInt, yInt) in integer sample units is derived as follows:

[0446] xInt=Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[0447] (8-782)

[0448] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY, picW, xInt L ): xInt L )

[0449] yInt = Clip3( 0, picH- 1, yInt L (8-783)

[0450] Predicted brightness sample value predSampleLX L The derivation is as follows:

[0451] predSampleLX L = refPicLX L [ xInt ][ yInt ] << shift3 (8-784)

[0452] 8.5.6.3.4 Chromaticity Sample Interpolation Process

[0453] The input to this process is:

[0454] - Chromaticity position in whole sample point units (xInt) C yInt C ),

[0455] – Chromaticity position in 1 / 32 fractional sample units (xFrac) C yFrac C ),

[0456] – Chromaticity position in whole point units (xSbIntC, ySbIntC), specifies the left top point of the reference point filled boundary block relative to the left top chromaticity point of the reference image.

[0457] – The variable sbWidth specifies the width of the current child block.

[0458] – The variable sbHeight specifies the height of the current child block.

[0459] –Color reference sample array refPicLX C .

[0460] The output of this process is the predicted chromaticity sample value, predSampleLX. C

[0461] The derivation of variables shift1, shift2, and shift3 is as follows:

[0462] – The variable shift1 is set to equal Min(4, BitDepth). C -8), variable shift2 is set to 6, and variable shift3 is set to Max(2, 14-BitDepth). C ).

[0463] – picW CIt is set to be equal to pic_width_in_luma_samples / SubWidthC, and the variable picH C It is set to be equal to pic_height_in_luma_samples / SubHeightC.

[0464] equals xFrac C or yFrac C The chromaticity interpolation filter coefficients f at each 1 / 32 fractional sample location p C [p] is specified in Table 8-13.

[0465] The variable xOffset is set to (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0466] Chromaticity position in whole sample point units (xInt) i yInt i The derivation for i = 0..3 is as follows:

[0467] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0468] xInt i =Clip3(SubPicLeftBoundaryPos / SubWidthC, SubPicRightBoundaryPos / SubWidthC, xInt L +i) (8-785)

[0469] yInt i =Clip3(SubPicTopBoundaryPos / SubHeightC, SubPicBotBoundaryPos / SubHeightC, yInt L +i) (8-786)

[0470] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0471] xInt i =Clip3(0, picW C -1, sps_ref_wraparound_enabled_flag? ClipH(xOffset,picW C xInt C+i-1): (8-787)

[0472] xInt C +i-1)

[0473] yInt i =Clip3(0, picH C -1, yInt C +i-1) (8-788)

[0474] Chromaticity position in whole sample point units (xInt) i yInt i For i = 0..3, the following further modifications are made:

[0475] xInt i =Clip3(xSbIntC-1,xSbIntC+sbWidth+2,xInt i (8-789)

[0476] yInt i =Clip3(ySbIntC-1,ySbIntC+sbHeight+2,yInt i (8-790)

[0477] Predicted chromaticity sample value predSampleLX C The derivation is as follows:

[0478] –If xFrac C and yFrac C If both are equal to 0, then predSampleLX C The value is derived as follows:

[0479] predSampleLX C =refPicLX C [xInt1][yInt1]< <shift3 (8-791)

[0480] Otherwise, if xFrac C Not equal to 0 and yFrac C If it equals 0, then predSampleLX C The value is derived as follows:

[0481]

[0482] Otherwise, if xFrac C Equal to 0 and yFrac C If it is not equal to 0, then predSampleLX C The value is derived as follows:

[0483]

[0484] Otherwise, if xFrac C Not equal to 0 and yFrac C If it is not equal to 0, then predSampleLX C The value is derived as follows:

[0485] The sample array temp[n], where n = 0..3, is derived as follows:

[0486]

[0487] –Predicted chromaticity sample value predSampleLX C The derivation is as follows:

[0488]

[0489]

[0490] Table 8-13 – Chromaticity interpolation filter coefficients f for each 1 / 32 fractional sample location p C The specification of [p].

[0491]

[0492]

[0493] 2.6 Sub-images in JVET-O2001-v14

[0494] Given the current syntax design of subpictures in JVET-O2001-vE, the position and dimensions of subpictures are derived as follows:

[0495]

[0496] A subpics_present_flag value of 1 indicates that the subpics parameter currently exists in the SPS RBSP syntax. A subpics_present_flag value of 0 indicates that the subpics parameter currently does not exist in the SPS RBSP syntax.

[0497] When the bitstream is the result of a sub-bitstream extraction process and contains only subgroups of subpics of the input bitstream of the sub-bitstream extraction process, it may be required to set the value of subpics_present_flag to 1 in the RBSP of the SPS.

[0498] Incrementing `max_subpics_minus1` by 1 specifies the maximum number of subpicks that can exist in CVS. `max_subpics_minus1` should be in the range of 0 to 254. A value of 255 is reserved for future use by ITU-T|ISO / IEC.

[0499] `subpic_grid_col_width_minus1` plus 1 specifies the width of each element of the subpicture identifier grid, in 4 samples. The length of the syntax element is `Ceil(Log2(pic_width_max_in_luma_samples / 4))` bits.

[0500] The variable NumSubPicGridCols is derived as follows:

[0501] NumSubPicGridCols=(pic_width_max_in_luma_samples+subpic_grid_col_width_minus1*4+3) /

[0502] (subpic_grid_col_width_minus1*4+4) (7-5)

[0503] `subpic_grid_row_height_minus1` plus 1 specifies the height of each element of the subpic identifier grid in 4 samples. The length of the syntax element is Ceil(Log2(pic_height_max_in_luma_samples / 4)) bits.

[0504] The variable NumSubPicGridRows is derived as follows:

[0505] NumSubPicGridRows=(pic_height_max_in_luma_samples+subpic_grid_row_height_minus1*4+3) /

[0506] (subpic_grid_row_height_minus1*4+4) (7-6)

[0507] `subpic_grid_idx[i][j]` specifies the subpick index at grid position (i, j). The length of the syntax element is Ceil(Log2(max_subpics_minus1+1)) bits.

[0508] The derivation of variables SubPicTop[subpic_grid_idx[i][j]], SubPicLeft[subpic_grid_idx[i][j]], SubPicWidth[subpic_grid_idx[i][j]], SubPicHeight[subpic_grid_idx[i][j]], and NumSubPics is as follows:

[0509]

[0510]

[0511] A subpic_treated_as_pic_flag[i] equal to 1 indicates that the i-th subpic of each codec image in CVS is considered as a picture in the decoding process excluding in-loop filtering operations. A subpic_treated_as_pic_flag[i] equal to 0 indicates that the i-th subpic of each codec image in CVS is not considered as a picture in the decoding process excluding in-loop filtering operations. When it does not exist, the value of subpic_treated_as_pic_flag[i] is inferred to be equal to 0.

[0512] 2.7 CIIP in JVET-O2001-v14

[0513] Combined Inter-Frame Intra-Frame Prediction (CIIP) is used in VVC as a special Merge candidate. It can only be enabled for WxH blocks (W<=64 and H<=64).

[0514] 3. Examples of problems solved by the disclosed technology and embodiments

[0515] In the current design of VVC, ATMVP has the following problems:

[0516] 1) Whether ATMVP is applied at the stripe level and the CU level is mismatched;

[0517] 2) In the stripe header, ATMVP can be enabled even if TMVP is disabled. At the same time, the ATMVP flag is signaled before the TMVP flag.

[0518] 3) Always masking, regardless of whether the MV is compressed;

[0519] 4) The effective corresponding area may be too large;

[0520] 5) The derivation of TMV is too complex;

[0521] 6) ATMVP may be unavailable in some cases, and a better default MV is expected.

[0522] 7) The MV scaling method in ATMVP may not be efficient;

[0523] 8) ATMVP should consider CPR (Continuous Respiration) scenarios;

[0524] 9) Even when affine prediction is disabled, the default zero affine Merge candidate can still be added to the list.

[0525] 10) The current image is considered a long-term reference image, while other images are considered short-term reference images. For both ATMVP and TMVP candidates, motion information from temporal blocks in co-located images is scaled to reference images with fixed reference indices (i.e., each reference image list is 0 in the current design). However, when CPR mode is enabled, the current image is also considered a reference image, and the current image can be added to reference image list 0 (RefPicList0) with index 0.

[0526] a. For TMVP, if the temporal block is encoded and decoded in CPR mode and the reference picture of RefPicList0 is a short-term reference picture, then the TMVP candidate is set to unavailable.

[0527] b. If the reference image of RefPicList0 with index 0 is the current image and the current image is an Intra-Random Access Point (IRAP) image, then the ATMVP candidate is set to unavailable.

[0528] c. For ATMVP sub-blocks within a block, when deriving motion information of a sub-block from a temporal block, if the temporal block is encoded and decoded in CPR mode, then by default, ATMVP candidates (derived from a temporal block identified by the starting TMV and the center position of the current block) are used to fill in the motion information of the sub-block.

[0529] 11) MV is right-shifted to integer precision, but does not follow the rounding rules in VVC.

[0530] 12) In ATMVP, the MV(MVx, MVy) used to locate corresponding blocks in different images (e.g., TMV in 2.3.5.1.2) is used directly because it points to a co-located image. This is based on the assumption that all images have the same resolution. However, when RPR is enabled, different image resolutions may be used. A similar issue exists regarding the identification of corresponding blocks in co-located images for deriving sub-block motion information.

[0531] 13) If a block width or height is greater than 32 and the maximum transform unit size is 32, then for CIIP codec blocks, intra-frame prediction signaling notifications are generated at the codec unit (CU) size; while inter-frame prediction signaling notifications are generated at the transform unit (TU) size (recursively dividing the current block into multiple 32x32 blocks). Using CUs to derive intra-frame prediction signaling notifications results in lower efficiency.

[0532] The current design has some issues. First, if the reference image with index 0 of RefPicList0 is the current image and the current image is not an IRAP image, the ATMVP procedure is still called, but the ATMVP procedure cannot locate any available ATMVP candidates because no temporal motion vector can be scaled to the current image.

[0533] 4. Examples of Implementation Methods and Technologies

[0534] The following list of techniques and embodiments should be considered as examples to illustrate the overall concepts. These techniques should not be interpreted in a narrow sense. Furthermore, these techniques can be combined in any way in encoder or decoder embodiments.

[0535] 1. Whether to allow TMVP and / or whether to use CPR should be considered to determine / parse the maximum number of candidates in the sub-block merge candidate list and / or to determine whether an ATMVP candidate should be added to the candidate list. The maximum number of candidates in the sub-block merge candidate list is referred to as ML. Although some exemplary embodiments are discussed below using Alternative Temporal Motion Vector Prediction (ATMVP), in alternative embodiments, the techniques disclosed in this document are applicable to sub-block-based temporal motion vector prediction (sbTMVP).

[0536] a) In one example, when determining or resolving the maximum number of candidates in the Merge candidate list for sub-blocks, ATMVP is inferred to be inapplicable if ATMVP is turned off with a flag (equal to 0) or TMVP is disabled.

[0537] i. In one example, ATMVP is enabled with the flag (equal to 1) and TMVP is disabled, so ATMVP candidates are not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0538] ii. In one example, ATMVP is enabled with a flag (equal to 1) and TMVP is disabled, and affine is disabled with a flag (equal to 0), then ML is set to 0, meaning that sub-block Merge is not applicable.

[0539] iii. In one example, ATMVP is enabled with the flag on (equal to 1) and TMVP is enabled, while affine is disabled with the flag on (equal to 0), then ML is set to equal to 1.

[0540] b) In one example, when determining or resolving the maximum number of candidates in the Merge candidate list for a sub-block, ATMVP is inferred to be inapplicable if the ATMVP is turned off with a flag (equal to 0) or the co-bit reference picture of the current picture is the current picture itself.

[0541] i. In one example, when ATMVP is enabled with the flag (equal to 1) and the co-bit reference image of the current image is the current image itself, the ATMVP candidate is not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0542] ii. In one example, ATMVP is enabled with the flag (equal to 1), and the co-bit reference image of the current image is the current image itself, and affine is disabled with the flag (equal to 0), then ML is set to 0, meaning that sub-block Merge is not applicable.

[0543] iii. In one example, ATMVP is enabled with the flag (equal to 1), and the co-bit reference image of the current image is not the current image itself, and affine is disabled with the flag (equal to 0), then ML is set to equal to 1.

[0544] c) In one example, when determining or resolving the maximum number of candidates in the Merge candidate list for a sub-block, ATMVP is inferred to be inapplicable if ATMVP is turned off with a flag (equal to 0), or if the reference picture with reference picture index 0 in reference list 0 is the current picture itself.

[0545] i. In one example, ATMVP is opened with a flag (equal to 1) and the reference image with reference image index 0 in reference list 0 is the current image itself. ATMVP candidates are not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0546] ii. In one example, ATMVP is turned on with the flag (equal to 1), and the reference picture with reference picture index 0 in reference list 0 is the current picture itself, and affine is turned off with the flag (equal to 0), then ML is set to equal to 0, meaning that sub-block Merge is not applicable.

[0547] iii. In one example, ATMVP is turned on with the flag (equal to 1) and the reference image with reference image index 0 in reference list 0 is not the current image itself, and affine is turned off with the flag (equal to 0), then ML is set to equal to 1.

[0548] d) In one example, when determining or resolving the maximum number of candidates in the Merge candidate list for a sub-block, ATMVP is inferred to be inapplicable if ATMVP is turned off with a flag (equal to 0), or if the reference picture with reference picture index 0 in reference list 1 is the current picture itself.

[0549] i. In one example, ATMVP is opened with a flag (equal to 1) and the reference image with reference image index 0 in reference list 1 is the current image itself. ATMVP candidates are not added to the sub-block Merge candidate list or the ATMVP candidate list.

[0550] ii. In one example, ATMVP is turned on with the flag (equal to 1), and the reference picture with reference picture index 0 in reference list 1 is the current picture itself, and affine is turned off with the flag (equal to 0), then ML is set to equal to 0, meaning that sub-block Merge is not applicable.

[0551] iii. In one example, ATMVP is turned on with the flag (equal to 1) and the reference image with reference image index 0 in reference list 1 is not the current image itself, and affine is turned off with the flag (equal to 0), then ML is set to equal to 1.

[0552] 2. It is proposed that if TMVP is disabled at the strip / slice / image level, ATMVP is implicitly disabled and no signaling is sent to the ATMVP flag.

[0553] a) In one example, the ATMVP flag is signaled in the strip header / preview / PPS after the TMVP flag.

[0554] b) In one example, the ATMVP or / and TMVP flags may not be signaled in the strip header / preview / PPS and may only be signaled in the SPS header.

[0555] 3. Whether and how the corresponding position in ATMVP is masked depends on whether and how the MV is compressed. Assume (xN, yN) is the corresponding position calculated using the coordinates of the current block / sub-block and the starting motion vector (e.g., TMV) in the co-location image.

[0556] a) In one example, if the MV is not required to be compressed (e.g., the signaling notification sps_disable_motioncompression in SPS is 1), then (xN, yN) is not masked; otherwise, (the MV can be compressed) (xN, yN) is masked as xN = xN&MASK, yN = yN&MASK, where MASK is equal to ~(2 M M is an integer of (-1), and M can be an integer such as 3 or 4.

[0557] b) Assume the MV compression method used for MV storage results in each 2 K x 2 K The blocks share the same motion information, and the mask in the ATMVP process is defined as ~(2). M -1). It is proposed that K may not be equal to M, for example, M = K + 1.

[0558] c) The masks used in ATMVP and TMVP can be the same, or they can be different.

[0559] 4. In one example, the MV compression method can be flexible.

[0560] a) In one example, the MV compression method can be chosen between no compression, 8×8 compression (M=3 in point 3.a), or 16×16 compression (M=4 in point 3.a).

[0561] b) In one example, the MV compression method can be signaled in the VPS / SPS / PPS / strip header / fragment header.

[0562] c) In one example, the MV compression method can be set differently in different standard configuration files / levels / hierarchies.

[0563] 5. The effective corresponding region in ATMVP can be adaptive;

[0564] a) For example, the effective corresponding region can depend on the width and height of the current block;

[0565] b) For example, the effective corresponding region may depend on the MV compression method;

[0566] i. In one example, if the MV compression method is not used, the effective corresponding area is smaller; if the MV compression method is used, the effective corresponding area is larger.

[0567] 6. In ATMVP, a valid corresponding region can be based on a base region with a size M×N smaller than the CTU region. For example, the CTU size in VTM-3.0 is 128×128, and the base region size can be 64×64. Assume the width and height of the current block are W and H, respectively.

[0568] a) In one example, if W <= M and H <= N, it means that the current block is within the base region, and the valid corresponding region in ATMVP is the co-bit base region and the extension in the co-bit picture. Figure 27 An example is shown.

[0569] i. For example, suppose the left top position of the co-located basic region is (xBR, yBR), then the corresponding position at (xN, yN) will be clipped to the effective region xBR<=xN. <xBR+M+4;yBR<=yN<yBR+N。

[0570] Figure 27 An exemplary embodiment of proposing a valid region when the current block is within the base region (BR) is shown.

[0571] Figure 28 An exemplary embodiment of the effective region is shown when the current block is not within the base region.

[0572] b) In one example, if W>M and H>N, meaning the current block is not within the base region, the current block is divided into several parts. Each part has a separate valid corresponding region in ATMVP. For position A in the current block, its corresponding position B in the co-located block should be within the valid corresponding region of the part where position A is located.

[0573] i. For example, the current block is divided into non-overlapping basic regions. A valid corresponding region for a basic region is its co-located basic region and its extension in the co-located image. Figure 28 An example is shown.

[0574] 1. For example, suppose position A in the current block is within a basic region R. The co-located basic region of R in the co-located image is designated as CR. The corresponding position of A in the co-located block is position B. The top left position of CR is (xCR, yCR), then position B at (xN, yN) will be pruned to the effective region xCR <= xN. <xCR+M+4;yCR<=yN<yCR+N。

[0575] 7. The motion vectors used to locate corresponding blocks in different images in ATMVP (e.g., TMV in 2.3.5.1.2) can be derived as follows:

[0576] a) In one example, TMV is always set to be equal to the default MV, such as (0, 0).

[0577] i. In one example, the default MV is signaled in VPS / SPS / PPS / strip header / fragment header / CTU / CU.

[0578] b) In one example, TMV is set to be a MV stored in the HMVP table using the following method;

[0579] i. If the HMVP list is empty, then the TMV is set to equal the default MV, such as (0, 0).

[0580] ii. Otherwise (HMVP list is not empty),

[0581] 1. TMV can be set to be equal to the first element stored in the HMVP table;

[0582] 2. Alternatively, TMV can be set to be equal to the last element stored in the HMVP table;

[0583] 3. Alternatively, TMV can be set to be equal to a specific MV stored in the HMVP table;

[0584] a. In one example, a specific MV refers to reference list 0.

[0585] b. In one example, a specific MV refers to Reference List 1.

[0586] c. In one example, a specific MV refers to a specific reference image in reference list 0, such as a reference image with index 0.

[0587] d. In one example, a specific MV references a specific reference image in reference list 1, such as the reference image with index 0.

[0588] e. In one example, a specific MV references a co-bit image.

[0589] 4. Alternatively, if a specific MV stored in the HMVP table (e.g., mentioned in point 3.) cannot be found, the TMV can be set to be equal to the default MV;

[0590] a. In one example, only the first element stored in the HMVP table is searched to find a specific MV.

[0591] b. In one example, only the last element stored in the HMVP table is searched to find a specific MV.

[0592] c. In one example, some or all of the elements in the HMVP table are stored to find a specific MV.

[0593] 5. Alternatively, the TMV obtained from HMVP cannot reference the current image itself.

[0594] 6. Alternatively, the TMV table obtained from HMVP can be scaled to the co-bit image if it does not reference the co-bit image.

[0595] c) In one example, the TMV is set as a MV of a specific adjacent block. Other adjacent blocks are not referenced.

[0596] i. A specific adjacent block can be Figure 22 Blocks A0, A1, B0, B1, and B2 in the dataset.

[0597] ii. If the following are true, then TMV can be set to equal the default MV:

[0598] 1. The specific adjacent block does not exist;

[0599] 2. Certain adjacent blocks are not inter-frame encoded or decoded;

[0600] iii. TMV can be set to be equal to a specific MV stored in a specific adjacent block;

[0601] 1. In one example, a specific MV refers to reference list 0.

[0602] 2. In one example, a specific MV refers to reference list 1.

[0603] 3. In one example, a specific MV references a specific reference image in reference list 0, such as the reference image with index 0.

[0604] 4. In one example, a specific MV references a specific reference image in reference list 1, such as the reference image with index 0.

[0605] 5. In one example, a specific MV references a co-located image.

[0606] 6. If a specific MV stored in a particular adjacent block cannot be found, the TMV can be set to be equal to the default MV;

[0607] iv. TMVs obtained from specific adjacent blocks can be scaled to the co-bit image if no co-bit image is referenced.

[0608] v. A TMV obtained from a specific adjacent block cannot reference the current image itself.

[0609] 8. As disclosed in 2.3.5.1.2, the MVdefault0 and MVdefault1 used in ATMVP can be derived as follows:

[0610] a) In one example, MVdefault0 and MVdefault1 are set to (0,0);

[0611] b) In one example, derive MVdefaultX (X = 0 or 1) from HMVP.

[0612] i. If the HMVP list is empty, MVdefaultX is set to equal the predefined default MV, such as (0, 0).

[0613] 1. The default MV can be signaled in VPS / SPS / PPS / strip header / group header / CTU / CU.

[0614] ii. Otherwise (HMVP list is not empty),

[0615] 1. MVdefaultX can be set to equal the first element stored in the HMVP table;

[0616] 2. MVdefaultX can be set to equal the last element stored in the HMVP table;

[0617] 3. MVdefaultX can be set to be equal to a specific MV stored in the HMVP table;

[0618] a. In one example, a specific MV refers to reference list X.

[0619] b. In one example, a specific MV refers to a specific reference image in a reference list X, such as a reference image with index 0.

[0620] 4. If a specific MV stored in the HMVP table cannot be found, MVdefaultX can be set to equal the predefined default MV;

[0621] a. In one example, only the first element stored in the HMVP table is searched.

[0622] b. In one example, only the last element stored in the HMVP table is searched.

[0623] c. In one example, search for some or all of the elements stored in the HMVP table.

[0624] 5. The MVdefaultX obtained from the HMVP table can be scaled to the co-bit image if it does not reference the co-bit image.

[0625] 6. The MVdefaultX obtained from HMVP cannot reference the current image itself.

[0626] c) In one example, MVdefaultX (X = 0 or 1) is derived from the adjacent block.

[0627] i. Adjacent blocks can contain blocks. Figure 22 A0, A1, B0, B1, B2.

[0628] 1. For example, only one of these blocks is used to derive MVdefaultX.

[0629] 2. Alternatively, some or all of these blocks are used to derive MVdefaultX.

[0630] a. Check these blocks sequentially until a valid MVdefaultX is found.

[0631] 3. If no valid MVdefaultX is found from the selected one or more adjacent blocks, then the default MV is set to be equal to the predefined default MV, such as (0, 0).

[0632] a. The default MV can be scheduled in the signaling notification of VPS / SPS / PPS / strip header / group header / CTU / CU.

[0633] ii. A valid MVdefaultX cannot be found from a specific adjacent block if:

[0634] 1. The specific adjacent block does not exist;

[0635] 2. A specific adjacent block is not an inter-frame codec;

[0636] iii.MVdefaultX can be set to be equal to a specific MV stored in a specific adjacent block;

[0637] 1. In one example, a specific MV refers to reference list X.

[0638] 2. In one example, a specific MV refers to a specific reference image in a reference list X, such as the reference image with index 0.

[0639] iv. MVdefaultX obtained from a specific adjacent block can be scaled to a specific reference image, such as a reference image with index 0 in reference list X.

[0640] v. The MVdefaultX obtained from a specific adjacent block cannot reference the current image itself.

[0641] 9. For sub-block or non-sub-block ATMVP candidates, if the temporal block of the sub-block / entire block in the co-bit image is encoded and decoded in CPR mode, the default motion candidate can be used instead.

[0642] a) In one example, the default motion candidate may be defined as the motion candidate associated with the center position of the current block (e.g., MVdefault0 and / or MVdefault1 used in ATMVP as disclosed in 2.3.5.1.2).

[0643] b) In one example, the default motion candidate can be defined as a (0, 0) motion vector, and for two lists of reference images, the reference image index is equal to 0 (if available).

[0644] 10. The proposed default motion information for the ATMVP process (e.g., MVdefault0 and / or MVdefault1 used in the ATMVP as disclosed in 2.3.5.1.2) can be derived based on the position used in the sub-block motion information derivation process. Using the proposed method, no further motion information derivation is required for the sub-block, as the default motion information will be directly assigned.

[0645] a) In one example, instead of using the center position of the current block, the center position of a child block within the current block (e.g., the center child block) can be used.

[0646] b) such as Figure 31B An example is shown. Figure 31A An example is shown of where the default motion information is identified in the current standard.

[0647] 11. The following method is always available for proposing ATMVP candidates:

[0648] a) Assuming the center point of the current block is (x0, y0), the corresponding position of (x0, y0) in the co-location image is M = (x0 + MV'x, y0 + MV'y). Find the block Z that covers M. If Z is intra-frame encoded, MVdefault0 and MVdefault1 are derived using some methods proposed in Project 6.

[0649] b) Instead of locating block Z to obtain motion information, some of the methods proposed in Project 8 are directly applied to obtain MVdefault0 and MVdefault1.

[0650] c) Alternatively, the default motion candidate used in the ATMVP process is always available. If it is set to be unavailable based on the current design (e.g., the temporal block is intra-coded), other motion vectors can be used instead of the default motion candidate.

[0651] i. In one example, the scheme in international application PCT / CN2018 / 124639 may be applied, which is incorporated herein by reference.

[0652] d) Alternatively, whether ATMVP candidates are always available depends on other high-level syntactic information.

[0653] i. In one example, an ATMVP candidate can always be set to available only if the ATMVP enable flag at the strip / slice / picture header or other video unit is set to true, inferred to be true.

[0654] ii. In one example, the above method may be applied only if the ATMVP enable flag in the strip header / picture header or other video unit is set to true and the current picture is not an IRAP picture and the current picture is not inserted into RefPicList0 with a reference index of 0.

[0655] e) Fixed indices or fixed index groups are assigned to ATMVP candidates. When ATMVP candidates are always unavailable, fixed indexes / groups of indices can be inferred as other types of motion candidates (such as affine candidates).

[0656] 12. Whether a zero-motion affine merge candidate is included in the sub-block merge candidate list should depend on whether affine prediction is enabled.

[0657] a) For example, if the affine use flag is off (sps_affine_enabled_flag equals 0), then zero-motion affine merge candidates are not placed in the sub-block merge candidate list.

[0658] b) Alternatively, in addition, a default motion vector candidate is added as a non-affine candidate.

[0659] 13. Propose that non-affine filling candidates can be added to the sub-block Merge candidate list.

[0660] a) If the sub-block Merge candidate list is not satisfied, then zero-motion non-affine filling candidates can be filled.

[0661] b) When such a filler candidate is selected, the affine_flag of the current block should be set to 0.

[0662] c) Alternatively, if the sub-block Merge candidate list is not satisfied and the affine use flag is off, the zero-motion non-affine fill candidate is placed in the sub-block Merge candidate list.

[0663] 14. Suppose MV0 and MV1 represent the MVs in reference lists 0 and 1 that cover the corresponding block (e.g., MV0 and MV1 could be MVZ_0 and MVZ_1 or MVZS_0 and MVZS_1 as described in section 2.3.5.1.2). MV0' and MV1' represent the MVs in reference lists 0 and 1 to be deduced for the current block or sub-block. Then MV0' and MV1' should be deduced by scaling.

[0664] a) MV0, if the image is in reference list 1;

[0665] b) MV1, if the co-bit image is in reference list 0.

[0666] 15. When the current image is considered a reference image with an index set to M (e.g., 0) in a reference image list X (PicRefListX, e.g., X=0), the ATMVP and / or TMVP enable / disable flags can be inferred as pseudo for strips / slices or other types of video units. Here, M can be equal to the target reference image index, whose temporal block motion information should be scaled to PicRefListX during the ATMVP / TMVP process.

[0667] a) Alternatively, the above method applies only if the current image is an Intra-Random Access Point (IRAP) image.

[0668] b) In one example, the ATMVP and / or TMVP enable / disable flags can be inferred to be false when the current image is considered a reference image with an index set to M (e.g., 0) in PicRefListX and / or the current image is considered a reference image with an index set to N (e.g., 0) in PicRefListY. Variables M and N represent the target reference image index used in the TMVP or ATMVP process.

[0669] c) For the ATMVP process, the following rule should be followed to restrict the acknowledgment bitstream: the co-bit image from which the motion information of the current block is derived should not be the current image.

[0670] d) Alternatively, the ATMVP or TMVP procedure is not invoked when the above conditions are true.

[0671] 16. Propose that if the reference image in the current block's reference image list X (PicRefListX, e.g., X=0) with an index set to M (e.g., 0) is the current image, then ATMVP can still be enabled for that block.

[0672] a) In one example, the motion information for all sub-blocks points to the current image.

[0673] b) In one example, when obtaining motion information of a sub-block from a temporal block, the temporal block applies at least one reference image encoding / decoding pointing to the current image of the temporal block.

[0674] c) In one example, when obtaining motion information of a sub-block from a temporal block, no scaling operation is applied.

[0675] 17. The encoding and decoding methods for sub-block merge indexes are consistent, regardless of whether ATMVP is used.

[0676] a) In one example, the first L binary numbers are context-encoded. The remaining binary numbers are bypass-encoded. In one example, L is set to 1.

[0677] b) Alternatively, for all binary numbers, they are encoded and decoded using context.

[0678] 18. In ATMVP, the MV(MVx, MVy) used to locate corresponding blocks of different images (e.g., TMV in 2.3.5.1.2) can be right-shifted to integer precision (referring to (MVx', MVy') using the same rounding method as in the MV scaling process.

[0679] a) Alternatively, the MV used by ATMVP to locate corresponding blocks in different images (e.g., TMV in 2.3.5.1.2) can be right-shifted to integer precision using the same rounding method as in the MV averaging process.

[0680] b) Alternatively, the MV used by ATMVP to locate corresponding blocks in different images (e.g., TMV in 2.3.5.1.2) can be right-shifted to integer precision using the same rounding method as in the Adaptive MV Resolution (AMVR) process.

[0681] 19. The MV(MVx, MVy) used in ATMVP to locate corresponding blocks in different images (e.g., TMV in 2.3.5.1.2) can be right-shifted to integer precision (referring to (MVx', MVy') by rounding towards zero.

[0682] a) For example, MVx' = (MVx + ((1 <<N)> >1)-(MVx>=0?1:0))>>N; N is an integer representing the MV resolution, for example, N=4.

[0683] i. For example, MVx' = (MVx + (MVx >= 0? 7: 8)) >> 4.

[0684] b) For example, MVy' = (MVy + ((1 <<N)> >1)-(MVy>=0?1:0))>>N; N is an integer representing the MV resolution, for example, N=4.

[0685] i. For example, MVy' = (MVy + (MVy >= 0? 7: 8)) >> 4.

[0686] 20. In one example, MV(MVx, MVy) from points 18 and 19 is used to locate the corresponding block to derive the default motion information used in ATMVP, such as using the center position and shifted MV of the child block, or using the top left position and shifted MV of the current block.

[0687] a) In one example, MV(MVx, MVy) is used to locate the corresponding block during the ATMVP process to derive motion information of the sub-blocks in the current block, such as using the center position of the sub-block and the shifted MV.

[0688] 21. The methods proposed in points 18, 19, and 20 can be applied to other encoding and decoding tools associated with different images or reference blocks in the current image that have motion vectors.

[0689] 22. In ATMVP, MV(MVx, MVy) used to locate corresponding blocks in different images (e.g., TMV in 2.3.5.1.2) can be scaled even if they point to co-located images.

[0690] a) In one example, if the width and / or height (or the consistency window therein) of the co-bit image differs from that of the current image (or the consistency window therein), then the MV can be scaled.

[0691] (b) Assume the width and height of the consistency window of the co-located image are designated as W1 and H1, respectively. Then the width and height of the consistency window of the current image are designated as W2 and H2, respectively. MV(MVx, MVy) can then be scaled to MVx' = MVx * W1 / W2 and MVy' = MVy * H1 / H2.

[0692] 23. The center point of the current block used to derive motion information during the ATMVP process (such as the position (x0, y0) in 2.3.5.1.2) can be further modified by scaling and / or adding offsets.

[0693] a) In one example, if the width and / or height of the co-position image (or its consistency window) differs from the current image (or its consistency window), the center point can be further modified.

[0694] (b) Assume the top left position of the consistency window in the co-located image is designated as X1 and Y1. The top left position of the constrained consistency window in the current image is designated as X2 and Y2. The width and height of the consistency window in the co-located image are designated as W1 and H1, respectively. The width and height of the consistency window in the current image are designated as W2 and H2, respectively. Then (x0, y0) can be modified to x0' = (x0 - X2) * W1 / W2 + X1 and y0' = (y0 - Y2) * H1 / H2 + Y1.

[0695] i. Alternatively, x0' = x0*W1 / W2, y0' = y0*H1 / H2.

[0696] 24. The corresponding positions used to derive motion information during the ATMVP process (such as position M in 2.3.5.1.2) can be further modified by scaling and / or adding offsets.

[0697] a) In one example, if the width and / or height of the co-position image (or its consistency window) differs from the current image (or its consistency window), the corresponding position can be further modified.

[0698] (b) Assume the top left position of the consistency window in the co-located image is designated as X1 and Y1. The top left position of the constrained consistency window in the current image is designated as X2 and Y2. The width and height of the consistency window in the co-located image are designated as W1 and H1, respectively. The width and height of the consistency window in the current image are designated as W2 and H2, respectively. Then M(x, y) can be modified to x' = (x - X2) * W1 / W2 + X1 and y' = (y - Y2) * H1 / H2 + Y1.

[0699] i. Alternatively, x' = x*W1 / W2, y' = y*H1 / H2.

[0700] Sub-image related

[0701] 25. In one example, if positions (i, j) and (i, j-1) belong to different sub-pictures, the width of the sub-picture S ending at column (j-1) can be set to be equal to j minus the leftmost column of sub-picture S.

[0702] a) Embodiments based on JVET-O2001-vE are emphasized in bold italics below.

[0703]

[0704]

[0705] 26. In one example, the height of a sub-image S ending at (NumSubPicGridRows-1) rows can be set to (NumSubPicGridRows–1) minus the top row of sub-image S plus one.

[0706] a) Implementations based on JVET-O2001-vE are highlighted below.

[0707]

[0708]

[0709] 27. In one example, the width of a sub-picture S ending at (NumSubPicGridColumns-1) columns can be set to (NumSubPicGridColumns–1) minus the leftmost column of sub-picture S plus 1.

[0710] a) Implementations based on JVET-O2001-vE are highlighted below.

[0711]

[0712]

[0713] 28. Sub-image grids can be integer multiples of the CTU size.

[0714] a) Implementations based on JVET-O2001-vE are emphasized below.

[0715] `subpic_grid_col_width_minus1` plus 1 specifies the width of each element in the subpicture identifier grid, in units of `CtbSizeY`. The length of the syntax element is `Ceil(Log2(pic_width_max_in_luma_samples / CtbSizeY))` bits.

[0716] The variable NumSubPicGridCols is derived as follows:

[0717] NumSubPicGridCols=(pic_width_max_in_luma_samples+subpic_grid_col_width_minus1*CtbSi zeY+CtbSizeY-1) /

[0718] (subpic_grid_col_width_minus1*CtbSizeY+CtbSizeY)(7-5)

[0719] `subpic_grid_row_height_minus1` plus 1 specifies the height of each element in the subpicture identifier grid, which is in units of 4 samples. The length of the syntax element is Ceil(Log2(pic_height_max_in_luma_samples / CtbSizeY)) bits.

[0720] The variable NumSubPicGridRows is derived as follows:

[0721] NumSubPicGridRows=(pic_height_max_in_luma_samples+subpic_grid_row_height_minus1*Ctb SizeY+CtbSizeY-1) /

[0722] (subpic_grid_row_height_minus1*CtbSizeY+CtbSizeY) (7-6)

[0723] 29. Add consistency constraints to ensure that sub-images do not overlap each other and that all sub-images can cover the entire image.

[0724] a) The embodiment based on JVET-O2001-vE is indicated as follows.

[0725] If all of the following conditions are met, then any subpic_grid_idx[i][j] must be equal to idx:

[0726] i>=SubPicTop[idx]and i <SubPicTop[idx]+SubPicHeight[idx]。

[0727] j>=SubPicLeft[idx]and j <SubPicLeft[idx]+SubPicWidth[idx]。

[0728] If none of the following conditions are met, then any subpic_grid_idx[i][j] must be different from idx:

[0729] i>=SubPicTop[idx]and i <SubPicTop[idx]+SubPicHeight[idx]。

[0730] j>=SubPicLeft[idx]and j <SubPicLeft[idx]+SubPicWidth[idx]。

[0731] RPR related

[0732] 30. Signaling notification refers to syntax elements (such as flags) referred to as RPR_flag to indicate whether RPR can be used in a video unit (e.g., a sequence). RPR_flag can be used for signaling notification in SPS, VPS, or DPS.

[0733] a) In one example, if the signaling notification RPR is not used (e.g., RPR_flag is 0), the width / height of all signaling notifications in the Picture Parameter Set (PPS) can be the same as the maximum width / maximum height of the signaling notifications in the Sequence Parameter Set (SPS).

[0734] b) In one example, if the signaling notification RPR is not used (e.g., RPR_flag is 0), then the full width / height is not signaled in the PPS and is inferred to be the maximum width / maximum height signaled in the SPS.

[0735] c) In one example, if the signaling notification RPR is not used (e.g., RPR_flag is 0), then the consistency window information is not used during decoding. Otherwise (if the signaling notification requires the use of RPR), the consistency window information can be used during decoding.

[0736] 31. It is proposed that the interpolation filter used to derive the prediction block of the current block during motion compensation can be selected based on whether the resolution of the reference image is different from that of the current image, or whether the width and / or height of the reference image is greater than that of the current image.

[0737] a. In one example, an interpolation filter with fewer taps can be applied when condition A is met, where condition A depends on the dimensions of the current image and / or the reference image.

[0738] i. In one example, condition A is that the resolution of the reference image is different from that of the current image.

[0739] ii. In one example, condition A is that the width and / or height of the reference image is greater than that of the current image.

[0740] iii. In one example, condition A is W1>a*W2 and / or H1>b*H2, where (W1, H1) represents the width and height of the reference image, and (W2, H2) represents the width and height of the current image, and a and b are two factors, such as a = b = 1.5.

[0741] iv. In one example, condition A may also depend on whether bidirectional forecasting is used.

[0742] 1) Condition A is satisfied only if bidirectional prediction is used in the current block.

[0743] v. In one example, condition A can depend on M and N, where M and N represent the width and height of the current block.

[0744] 1) For example, condition A is satisfied only if M*N<=T, where T is an integer such as 64.

[0745] 2) For example, condition A is satisfied only if M <= T1 or N <= T2, where T1 and T2 are integers, such as T1 = T2 = 4.

[0746] 3) For example, condition A is satisfied only if M <= T1 and N <= T2, where T1 and T2 are integers, such as T1 = T2 = 4.

[0747] 4) For example, condition A is satisfied only if M*N<=T, or M<=T1 or N<=T2, where T, T1 and T2 are integers, such as T=64, T1=T2=4.

[0748] 5) In one example, the less than condition in the above sub-point can be replaced with greater than.

[0749] vi. In one example, a 1-tap filter is applied. In other words, unfiltered integer pixels are output as the interpolation result.

[0750] vii. In one example, bilinear filtering is applied when the resolution of the reference image differs from that of the current image.

[0751] viii. In one example, when the resolution of the reference image is different from that of the current image, or when the width and / or height of the reference image is greater than that of the current image, apply a 4-tap filter or a 6-tap filter.

[0752] 1) 6-tap filtering can also be used for affine motion compensation.

[0753] 2) 4-tap filtering can also be used for interpolation of chromaticity samples.

[0754] b. Whether and / or how the methods disclosed in point 31 are applied may depend on the color components.

[0755] i. For example, the method is applied only to the luminance component.

[0756] c. Whether and / or how the methods disclosed in point 31 are applied can depend on the direction of the interpolation filter.

[0757] i. For example, the method is only applied to horizontal filtering.

[0758] ii. For example, the method is only applied to vertical filtering.

[0759] CIIP related

[0760] 32. Intraframe prediction signaling notifications used in CIIP can be performed at the TU level instead of the CU level (e.g., using reference samples outside the TU instead of the CU).

[0761] a) In one example, if the CU width or height is greater than the maximum transform unit size, the CU can be divided into multiple TUs, and intra-frame / inter-frame predictions can be generated for each TU, for example, using reference samples outside the TU.

[0762] b) In one example, if the maximum transform size K is less than 64 (such as K = 32), the intra-prediction used in CIIP is performed recursively, as in a normal intra-code / decode block.

[0763] c) For example, a KM×KN CIIP codec block (where M and N are integers) is divided into MN K×K blocks, and intra-frame prediction is performed on each K×K block. The intra-frame prediction of the later encoded / decoded K×K blocks can depend on the reconstructed samples of the previously encoded / decoded K×K blocks.

[0764] 5. Additional Exemplary Embodiments

[0765] 5.1 Example #1: Example of syntax design in SPS / PPS / strip header / fragment header

[0766] The changes compared to the VTM3.0.1rc1 reference software are highlighted in large bold as follows:

[0767]

[0768]

[0769] 5.2 Example #2: Example of syntax design in SPS / PPS / strip header / fragment header

[0770] 7.3.2.1 Sequence Parameter Set (RBSP) Syntax

[0771]

[0772]

[0773] `sps_sbtmvp_enabled_flag` equal to 1 indicates that the sub-block-based temporal motion vector predictor can be used in CVS for decoding all slices where `slice_type` is not equal to `I`. `sps_sbtmvp_enabled_flag` equal to 0 indicates that the sub-block-based temporal motion vector predictor is not used in CVS. When `sps_sbtmvp_enabled_flag` does not exist, it is inferred to be equal to 0.

[0774] `five_minus_max_num_subblock_merge_cand` specifies 5 minus the maximum number of subblock-based merged motion vector prediction (MVP) candidates supported in the stripe. When `five_minus_max_num_subblock_merge_cand` is not present, it is inferred to be equal to 5 - `sps_sbtmvp_enabled_flag`. The maximum number of subblock-based merged MVP candidates, `MaxNumSubblockMergeCand`, is derived as follows:

[0775] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand(7-45)

[0776] The value of MaxNumSubblockMergeCand should be in the range of 0 to 5 (inclusive).

[0777] 8.3.4.2 Derivation of motion vectors and reference indices in sub-block Merge mode

[0778] The input to this process is:

[0779] ...[No changes have been made to the current VVC specification draft].

[0780] The output of this process is:

[0781] ...[No changes have been made to the current VVC specification draft].

[0782] The variables numSbX, numSbY, and the candidate list of subblocks for merging ...

[0783] The following applies when sps_sbtmvp_enabled_flag equals 1 and (the current image is IRAP and index 0 of reference image list 0 is the current image) is not true:

[0784] Invoke the procedure for deriving Merge candidates from adjacent codec units as specified in Clause 8.3.2.3, taking the luma codec block position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight, and luma codec block width as input, and outputting availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1, and availableFlagA1. lagB2, reference indices refIdxLXA0, refIdxLXA1, refIdxLXB0, refIdxLXB1 and refIdxLXB2, prediction list using flags predFlagLXA0, predFlagLXA1, predFlagLXB0, predFlagLXB1 and predFlagLXB2, and motion vectors mvLXA0, mvLXA1, mvLXB0, mvLXB1 and mvLXB2, where X is 0 or 1.

[0785] Invoke the derivation procedure for the sub-block-based temporal merge candidate as specified in Clause 8.3.4.3, with the luminance position (xCb, yCb), luminance codec block width cbWidth, luminance codec block height cbHeight, availability flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1, reference indexes refIdxLXA0, refIdxLXA1, refIdxLXB0, refIdxLXB1, prediction list usage flags predFlagLXA0, predFlagLXA1, predFlagLXB0, pr The edFlagLXB1 and motion vectors mvLXA0, mvLXA1, mvLXB0, and mvLXB1 are taken as input, and the output is the availability flag availableFlagSbCol, the number of luminance codec subblocks in the horizontal direction numSbX and the number of luminance codec subblocks in the vertical direction numSbY, the reference index refIdxLXSbCol, the luminance motion vectors mvLXSbCol[xSbIdx][ySbIdx], and the prediction list usage flag predFlagLXSbCol[xSbIdx][ySbIdx], where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1, and X is 0 or 1.

[0786] When sps_affine_enabled_flag equals 1, the sample locations (xNbA0, yNbA0), (xNbA1, yNbA1), (xNbA2, yNbA2), (xNbB0, yNbB0), (xNbB1, yNbB1), (xNbB2, yNbB2), (xNbB3, yNbB3) and the variables numSbX and numSbY are derived as follows:

[0787] [No changes have been made to the current VVC specification draft].

[0788] 5.3 Example of MV rounding in Implementation #3

[0789] The syntax change is based on JVET-O2001-vE.

[0790] 8.5.5.3 Derivation of Sub-Block-Based Temporal Merge Candidates

[0791] …

[0792] The positions (xColSb, yColSb) of the colocation subblocks within –ColPic are derived as follows.

[0793] 1. The following applies:

[0794] yColSb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+((tempMv[1]+8–(tempMv[1]> =0))>>4))

[0795] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0796] xColSb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0797] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0798] xColSb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+(tempMv[0]+8+(tempMV[0]> =0))>>4))

[0799] …

[0800] 8.5.5.4 Derivation of Temporal Merge Basic Motion Data Based on Sub-Blocks

[0801] …

[0802] The positions of the co-occurring blocks (xColCb, yColCb) within ColPic are derived as follows.

[0803] – The following applies:

[0804] yColCb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+8–(tempMv[1]> =0))>>4))

[0805] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0806] xColCb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0807] – Otherwise, if subpic_treated_as_pic_flag[SubPicIdx] equals 0, the following applies:

[0808] xColCb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0809] 5.3 Example #3: Example of rounding MV.

[0810] The syntax change is based on JVET-O2001-vE.

[0811] 8.5.5.3 Derivation of Sub-Block-Based Temporal Merge Candidates

[0812] …

[0813] The positions (xColSb, yColSb) of the colocation subblocks within –ColPic are derived as follows.

[0814] 1. The following applies:

[0815] -yColSb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+((tempMv[1]+8–(tempMv[1]> =0))>>4))

[0816] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0817] -xColSb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0818] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0819] -xColSb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+(tempMv[0]+8+(tempMV[0]> =0))>>4))

[0820] …

[0821] 8.5.5.4 Derivation of Temporal Merge Basic Motion Data Based on Sub-Blocks

[0822] …

[0823] The positions of the co-occurring blocks (xColCb, yColCb) within ColPic are derived as follows.

[0824] – The following applies:

[0825] -yColCb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+8–(tempMv[1]> =0))>>4))

[0826] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0827] -xColCb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0828] – Otherwise, if subpic_treated_as_pic_flag[SubPicIdx] equals 0, the following applies:

[0829] -xColCb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8+(tempMV[0]> =0))>>4))

[0830] 5.4 Example #4: A second example of MV rounding.

[0831] 8.5.5.3 Derivation of Sub-Block-Based Temporal Merge Candidates

[0832] The input to this process is:

[0833] – The brightness position (xCb, yCb) of the left-top sample of the current brightness codec block relative to the left-top brightness sample of the current image.

[0834] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0835] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0836] –Availability flag for adjacent codec units, availableFlagA1

[0837] – The reference index refIdxLXA1 of the adjacent encoding / decoding unit, where X is 0 or 1.

[0838] – The prediction list for adjacent codec units uses the flag predFlagLXA1, where X is 0 or 1.

[0839] – The motion vector mvLXA1 of adjacent codec units with 1 / 16 fractional sample accuracy, where X is 0 or 1.

[0840] The output of this process is:

[0841] –Availability flag availableFlagSbCol

[0842] – The number of luminance codec sub-blocks in the horizontal direction (numSbX) and the number of luminance codec sub-blocks in the vertical direction (numSbY).

[0843] –Refer to the indices refIdxL0SbCol and refIdxL1SbCol.

[0844] – The motion vectors mvL0SbCol[xSbIdx][ySbIdx] and mvL1SbCol[xSbIdx][ySbIdx] with a brightness accuracy of 1 / 16 fraction of a sample point, where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1,

[0845] – The prediction list uses the flags predFlagL0SbCol[xSbIdx][ySbIdx] and predFlagL1SbCol[xSbIdx][ySbIdx], where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1.

[0846] The availability flag availableFlagSbCol is derived as follows.

[0847] - AvailableFlagSbCol is set to 0 if one or more of the following conditions are true.

[0848] –slice_temporal_mvp_enabled_flag equals 0.

[0849] –sps_sbtmvp_enabled_flag equals 0.

[0850] –cbWidth is less than 8.

[0851] –cbHeight is less than 8.

[0852] – Otherwise, apply the following sequence of steps:

[0853] 1. The positions of the left top sample (xCtb, yCtb) and the right bottom center sample (xCtr, yCtr) of the luma codec block containing the current codec block are derived as follows:

[0854] -xCtb = (xCb >> CtuLog2Size) <CtuLog2Size (8-542)

[0855] -yCtb=(yCb>>CtuLog2Size)< <CtuLog2Size (8-543)

[0856] -xCtr=xCb+(cbWidth / 2) (8-544)

[0857] -yCtr=yCb+(cbHeight / 2) (8-545)

[0858] 2. The luminance position (xColCtrCb, yColCtrCb) is set to be equal to the left top luminance sample of the co-occurrence luminance block relative to the position given by (xCtr, yCtr) within the ColPic, which covers the left top luminance sample of the co-occurrence image specified by ColPic.

[0859] 3. Invoke the derivation procedure for the basic motion data of the temporal Merge based on sub-blocks as specified in Clause 8.5.5.4, with the positions (xCtb, yCtb), (xColCtrCb, yColCtrCb), availability flag availableFlagA1, prediction list usage flag predFlagLXA1, reference index refIdxLXA1, and motion vector mvLXA1, where X is 0 and 1, as inputs, and the motion vector ctrMvLX and the prediction list usage flag ctrPredFlagLX of the co-located block, where X is 0 and 1, and the temporal motion vector tempMv as outputs.

[0860] 4. The variable availableFlagSbCol is derived as follows:

[0861] – If both ctrPredFlagL0 and ctrPredFlagL1 are equal to 0, then availableFlagSbCol is set to 0.

[0862] Otherwise, availableFlagSbCol is set to 1.

[0863] When availableFlagSbCol equals 1, the following applies:

[0864] – The derivation of variables numSbX, numSbY, sbWidth, sbHeight, and refIdxLXSbCol is as follows:

[0865] -numSbX=cbWidth>>3 (8-546)

[0866] -numSbY=cbHeight>>3 (8-547)

[0867] -sbWidth=cbWidth / numSbX (8-548)

[0868] -sbHeight=cbHeight / numSbY (8-549)

[0869] -refIdxLXSbCol=0 (8-550)

[0870] –For xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1, the motion vector mvLXSbCol[xSbIdx][ySbIdx] and the prediction list using the flag predFlagLXSbCol[xSbIdx][ySbIdx] are derived as follows:

[0871] – Luminance position (xSb, ySb), specifies the left-top sample of the current codec sub-block relative to the left-top luminance sample of the current image, and is derived as follows:

[0872] -xSb=xCb+xSbIdx*sbWidth+sbWidth / 2 (8-551)

[0873] -ySb=yCb+ySbIdx*sbHeight+sbHeight / 2 (8-552)

[0874] The positions (xColSb, yColSb) of the colocation subblocks within –ColPic are derived as follows.

[0875] 1. The following applies:

[0876] -yColSb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+((tempMv[1]+8–(tempMv[1]> =0?1:0))>>4))

[0877] (8-553)

[0878] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0879] -xColSb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8–(tempMv[0]> =0?1:0))>>4))

[0880] (8-554)

[0881] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0882] -xColSb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+8–(tempMv[0]> =0?1:0))>>4))

[0883] (8-555)

[0884] The variable currCb specifies the luminance codec block that covers the current codec subblock within the current image.

[0885] – The variable colCb specifies the luminance codec block that covers the modified position within ColPic, given by ((xColSb>>3)<<3,(yColSb>>3)<<3).

[0886] – The luminance position (xColCb, yColCb) is set to be equal to the left top sample of the co-bit luminance codec block specified by colCb relative to the left top luminance sample of the co-bit image specified by ColPic.

[0887] – Call the derivation procedure for the co-position motion vector as specified in Clause 8.5.2.12, taking currCb, colCb, (xColCb, yColCb), refIdxL0 set to 0 and sbFlag set to 1 as inputs, and outputting the motion vectors mvL0SbCol[xSbIdx][ySbIdx] and availableFlagL0SbCol assigned to the sub-block.

[0888] – Call the derivation procedure for the co-position motion vector as specified in Clause 8.5.2.12, taking currCb, colCb, (xColCb, yColCb), refIdxL1 set to 0 and sbFlag set to 1 as inputs, and outputting the motion vectors mvL1SbCol[xSbIdx][ySbIdx] and availableFlagL1SbCol assigned to the sub-block.

[0889] –When both availableFlagL0SbCol and availableFlagL1SbCol are equal to 0, the following applies to X being 0 and 1:

[0890] -mvLXSbCol[xSbIdx][ySbIdx]=ctrMvLX (8-556)

[0891] -predFlagLXSbCol[xSbIdx][ySbIdx]=ctrPredFlagLX (8-557)

[0892] 8.5.5.4 Derivation of basic motion data in the temporal domain based on sub-blocks. The input to this process is:

[0893] – The position (xCtb, yCtb) of the left vertex sample of the luma codec block containing the current codec block.

[0894] – The position of the top left sample (xColCtrCb, yColCtrCb) of the co-bit luminance codec block covering the bottom right center sample.

[0895] –Availability flag for adjacent codec units, availableFlagA1

[0896] – The reference index of the adjacent encoding / decoding unit is refIdxLXA1.

[0897] – The prediction list for adjacent codec units uses the flag predFlagLXA1.

[0898] – Motion vector mvLXA1 of adjacent codec units with 1 / 16 fractional sample accuracy.

[0899] The output of this process is:

[0900] – Motion vectors ctrMvL0 and ctrMvL1

[0901] – The prediction list uses the flags ctrPredFlagL0 and ctrPredFlagL1.

[0902] –Time-domain motion vector tempMv.

[0903] The variable tempMv is set as follows:

[0904] -tempMv[0]=0 (8-558)

[0905] -tempMv[1]=0 (8-559)

[0906] The variable currPic specifies the current image.

[0907] When availableFlagA1 is true, the following applies:

[0908] – tempMv is set to equal mvL0A1 if all of the following conditions are true:

[0909] –predFlagL0A1 equals 1.

[0910] –DiffPicOrderCnt(ColPic, RefPicList[0][refIdxL0A1]) equals 0,

[0911] Otherwise, if all of the following conditions are true, then tempMv is set to equal mvL1A1:

[0912] –slice_type equals B,

[0913] –predFlagL1A1 equals 1,

[0914] –DiffPicOrderCnt(ColPic, RefPicList[1][refIdxL1A1]) equals 0. The positions of the co-occurring blocks (xColCb, yColCb) within ColPic are derived as follows.

[0915] – The following applies:

[0916] -yColCb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+8-(tempMv[1]> =0?1:0))>>4))

[0917] (8-560)

[0918] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0919] -xColCb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8–(tempMv[0]> =0?1:0))>>4))

[0920] (8-561)

[0921] – Otherwise, if subpic_treated_as_pic_flag[SubPicIdx] equals 0, then the following applies:

[0922] -xColCb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+8–(tempMv[0]> =0?1:0))>>4))

[0923] (8-562)

[0924] The array colPredMode is set to be equal to the prediction mode array CuPredMode[0] of the co-bit image specified by ColPic.

[0925] The motion vectors ctrMvL0 and ctrMvL1, as well as the prediction list using the flags ctrPredFlagL0 and ctrPredFlagL1, are derived as follows:

[0926] – If colPredMode[xColCb][yColCb] equals MODE_INTER, then the following applies:

[0927] – The variable currCb specifies the luminance codec block (xCtrCb, yCtrCb) that covers the current image.

[0928] The variable colCb specifies the luminance codec block that overwrites the modified position within ColPic given by ((xColCb>>3)<<3,(yColCb>>3)<<3).

[0929] – The luminance position (xColCb, yColCb) is set to be equal to the left top sample of the co-bit luminance codec block specified by colCb relative to the left top luminance sample of the co-bit image specified by ColPic.

[0930] – Call the derivation of the co-position motion vector specified in Clause 8.5.2.12, with currCb, colCb, (xColCb, yColCb), refIdxL0 set to 0 and sbFlag set to 1 as inputs, and assign the output to ctrMvL0 and ctrPredFlagL0.

[0931] – Call the derivation of the co-position motion vector specified in Clause 8.5.2.12, with currCb, colCb, (xColCb, yColCb), refIdxL1 set to 0 and sbFlag set to 1 as inputs, and assign the output to ctrMvL1 and ctrPredFlagL1.

[0932] – Otherwise, the following applies:

[0933] -ctrPredFlagL0=0 (8-563)

[0934] -ctrPredFlagL1 = 0 (8-564)

[0935] 5.5 Example #5: Third example of MV rounding.

[0936] 8.5.5.3 Derivation of Sub-Block-Based Temporal Merge Candidates

[0937] The input to this process is:

[0938] – The brightness position (xCb, yCb) of the left-top sample of the current brightness codec block relative to the left-top brightness sample of the current image.

[0939] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0940] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0941] –Availability flag for adjacent codec units, availableFlagA1

[0942] – The reference index refIdxLXA1 of the adjacent encoding / decoding unit, where X is 0 or 1.

[0943] – The prediction list for adjacent codec units uses the flag predFlagLXA1, where X is 0 or 1.

[0944] – The motion vector mvLXA1 of adjacent codec units with 1 / 16 fractional sample accuracy, where X is 0 or 1.

[0945] The output of this process is:

[0946] –Availability flag availableFlagSbCol

[0947] – The number of luminance codec sub-blocks in the horizontal direction (numSbX) and the number of luminance codec sub-blocks in the vertical direction (numSbY).

[0948] –Refer to the indices refIdxL0SbCol and refIdxL1SbCol.

[0949] – The motion vectors mvL0SbCol[xSbIdx][ySbIdx] and mvL1SbCol[xSbIdx][ySbIdx] with a brightness accuracy of 1 / 16 fraction of a sample point, where xSbIdx = 0..numSbX-1, ySbIdx = 0..numSbY-1,

[0950] – The prediction list uses the flags predFlagL0SbCol[xSbIdx][ySbIdx] and predFlagL1SbCol[xSbIdx][ySbIdx], where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1.

[0951] The availability flag availableFlagSbCol is derived as follows.

[0952] - AvailableFlagSbCol is set to 0 if one or more of the following conditions are true.

[0953] –slice_temporal_mvp_enabled_flag equals 0.

[0954] –sps_sbtmvp_enabled_flag equals 0.

[0955] –cbWidth is less than 8.

[0956] –cbHeight is less than 8.

[0957] – Otherwise, apply the following sequence of steps:

[0958] 1. The positions of the left top sample (xCtb, yCtb) and the right bottom center sample (xCtr, yCtr) of the luma codec block containing the current codec block are derived as follows:

[0959] -xCtb = (xCb >> CtuLog2Size) <CtuLog2Size (8-542)

[0960] -yCtb=(yCb>>CtuLog2Size)< <CtuLog2Size (8-543)

[0961] -xCtr=xCb+(cbWidth / 2) (8-544)

[0962] -yCtr=yCb+(cbHeight / 2) (8-545)

[0963] 2. The luminance position (xColCtrCb, yColCtrCb) is set to be equal to the left top luminance sample of the co-occurrence luminance block relative to the position given by (xCtr, yCtr) within the ColPic, which covers the left top luminance sample of the co-occurrence image specified by ColPic.

[0964] 3. Invoke the derivation procedure for the sub-block-based temporal Merge basic motion data as specified in Clause 8.5.5.4, with the positions (xCtb, yCtb), (xColCtrCb, yColCtrCb), availability flag availableFlagA1, prediction list usage flag predFlagLXA1, reference index refIdxLXA1, and motion vector mvLXA1, where X is 0 and 1, as inputs, and the motion vector ctrMvLX and the prediction list usage flag ctrPredFlagLX of the co-located block, where X is 0 and 1, and the temporal motion vector tempMv as outputs.

[0965] 4. The variable availableFlagSbCol is derived as follows:

[0966] – If both ctrPredFlagL0 and ctrPredFlagL1 are equal to 0, then availableFlagSbCol is set to 0.

[0967] Otherwise, availableFlagSbCol is set to 1.

[0968] When availableFlagSbCol equals 1, the following applies:

[0969] – The derivation of variables numSbX, numSbY, sbWidth, sbHeight, and refIdxLXSbCol is as follows:

[0970] -numSbX=cbWidth>>3 (8-546)

[0971] -numSbY=cbHeight>>3 (8-547)

[0972] -sbWidth=cbWidth / numSbX (8-548)

[0973] -sbHeight=cbHeight / numSbY (8-549)

[0974] -refIdxLXSbCol=0 (8-550)

[0975] – For xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1, the motion vector mvLXSbCol[xSbIdx][ySbIdx] and the prediction list using the flag predFlagLXSbCol[xSbIdx][ySbIdx] are derived as follows:

[0976] – Luminance position (xSb, ySb), specifies the left-top sample of the current codec sub-block relative to the left-top luminance sample of the current image, and is derived as follows:

[0977] -xSb=xCb+xSbIdx*sbWidth+sbWidth / 2 (8-551)

[0978] -ySb=yCb+ySbIdx*sbHeight+sbHeight / 2 (8-552)

[0979] The positions (xColSb, yColSb) of the colocation subblocks within –ColPic are derived as follows.

[0980] 1. The following applies:

[0981] -yColSb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+((tempMv[1]+(tempMv[1]> =0?7:8))>>4)) (8-553

[0982] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0983] -xColSb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4)) (8-554

[0984] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0985] -xColSb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4)) (8-555)

[0986] The variable currCb specifies the luminance codec block that covers the current codec subblock within the current image.

[0987] – The variable colCb specifies the luminance codec block that covers the modified position within ColPic, given by ((xColSb>>3)<<3,(yColSb>>3)<<3).

[0988] – The luminance position (xColCb, yColCb) is set to be equal to the left top sample of the co-bit luminance codec block specified by colCb relative to the left top luminance sample of the co-bit image specified by ColPic.

[0989] – Call the derivation procedure for the co-position motion vector as specified in Clause 8.5.2.12, taking currCb, colCb, (xColCb, yColCb), refIdxL0 set to 0 and sbFlag set to 1 as inputs, and outputting the motion vectors mvL0SbCol[xSbIdx][ySbIdx] and availableFlagL0SbCol assigned to the sub-block.

[0990] – Call the derivation procedure for the co-position motion vector as specified in Clause 8.5.2.12, taking currCb, colCb, (xColCb, yColCb), refIdxL1 set to 0 and sbFlag set to 1 as inputs, and outputting the motion vectors mvL1SbCol[xSbIdx][ySbIdx] and availableFlagL1SbCol assigned to the sub-block.

[0991] – When both availableFlagL0SbCol and availableFlagL1SbCol are equal to 0, the following applies to X being 0 and 1:

[0992] -mvLXSbCol[xSbIdx][ySbIdx]=ctrMvLX (8-556)

[0993] -predFlagLXSbCol[xSbIdx][ySbIdx]=ctrPredFlagLX (8-557)

[0994] 8.5.5.4 Derivation of basic motion data in the temporal domain based on sub-blocks. The input to this process is:

[0995] – The position (xCtb, yCtb) of the left vertex sample of the luma codec block containing the current codec block.

[0996] – The position of the top left sample (xColCtrCb, yColCtrCb) of the co-bit luminance codec block covering the bottom right center sample.

[0997] –Availability flag for adjacent codec units, availableFlagA1

[0998] – The reference index of the adjacent encoding / decoding unit is refIdxLXA1.

[0999] – The prediction list for adjacent codec units uses the flag predFlagLXA1.

[1000] – Motion vector mvLXA1 of adjacent codec units with 1 / 16 fractional sample accuracy.

[1001] The output of this process is:

[1002] – Motion vectors ctrMvL0 and ctrMvL1

[1003] – The prediction list uses the flags ctrPredFlagL0 and ctrPredFlagL1.

[1004] –Time-domain motion vector tempMv.

[1005] The variable tempMv is set as follows:

[1006] -tempMv[0]=0 (8-558)

[1007] -tempMv[1]=0 (8-559)

[1008] The variable currPic specifies the current image.

[1009] When availableFlagA1 is true, the following applies:

[1010] – tempMv is set to equal mvL0A1 if all of the following conditions are true:

[1011] –predFlagL0A1 equals 1.

[1012] –DiffPicOrderCnt(ColPic, RefPicList[0][refIdxL0A1]) equals 0,

[1013] Otherwise, if all of the following conditions are true, then tempMv is set to equal mvL1A1:

[1014] –slice_type equals B,

[1015] –predFlagL1A1 equals 1,

[1016] –DiffPicOrderCnt(ColPic, RefPicList[1][refIdxL1A1]) equals 0. The positions of the co-occurring blocks (xColCb, yColCb) within ColPic are derived as follows.

[1017] – The following applies:

[1018] -yColCb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+((tempMv[1]+(tempMv[1]> =0?7:8))>>4))

[1019] (8-560)

[1020] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[1021] -xColCb=Clip3(xCtb,Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4))

[1022] (8-561)

[1023] – Otherwise, if subpic_treated_as_pic_flag[SubPicIdx] equals 0, then the following applies:

[1024] -xColCb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+((tempMv[0]+(tempMv[0]> =0?7:8))>>4))

[1025] (8-562)

[1026] The array colPredMode is set to be equal to the prediction mode array CuPredMode[0] of the co-bit image specified by ColPic.

[1027] The motion vectors ctrMvL0 and ctrMvL1, as well as the prediction list using the flags ctrPredFlagL0 and ctrPredFlagL1, are derived as follows:

[1028] – If colPredMode[xColCb][yColCb] equals MODE_INTER, then the following applies:

[1029] – The variable currCb specifies the luminance codec block (xCtrCb, yCtrCb) that covers the current image.

[1030] The variable colCb specifies the luminance codec block that overwrites the modified position within ColPic given by ((xColCb>>3)<<3,(yColCb>>3)<<3).

[1031] – The luminance position (xColCb, yColCb) is set to be equal to the left top sample of the co-bit luminance codec block specified by colCb relative to the left top luminance sample of the co-bit image specified by ColPic.

[1032] – Call the derivation of the co-position motion vector specified in Clause 8.5.2.12, with currCb, colCb, (xColCb, yColCb), refIdxL0 set to 0 and sbFlag set to 1 as inputs, and assign the output to ctrMvL0 and ctrPredFlagL0.

[1033] – Call the derivation of the co-position motion vector specified in Clause 8.5.2.12, with currCb, colCb, (xColCb, yColCb), refIdxL1 set to 0 and sbFlag set to 1 as inputs, and assign the output to ctrMvL1 and ctrPredFlagL1.

[1034] – Otherwise, the following applies:

[1035] -ctrPredFlagL0=0 (8-563)

[1036] -ctrPredFlagL1 = 0 (8-564)

[1037] 8.5.6.3 Fractional Sample Interpolation Process

[1038] 8.5.6.3.1 Overall

[1039] The input to this process is:

[1040] – Luminance position (xSb, ySb), specifies the left top sample point of the current encoding / decoding sub-block relative to the left top luminance sample point of the current image.

[1041] – The variable sbWidth specifies the width of the current codec sub-block.

[1042] – The variable sbHeight specifies the height of the current encoding / decoding sub-block.

[1043] – Motion vector offset mvOffset

[1044] –Refine the motion vector refMvLX,

[1045] –Selected reference image sample array refPicLX,

[1046] – Half-sample interpolation filter index hpelIfIdx,

[1047] – Bidirectional optical flow flag bdofFlag

[1048] – The variable cIdx specifies the color component index of the current block.

[1049] The output of this process is:

[1050] – The array predSamplesLX represents the predicted sample values ​​(sbWidth+brdExtSize)x(sbHeight+brdExtSize).

[1051] The predicted block bound extension size brdExtSize is derived as follows:

[1052] -brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&

[1053] sps_affine_prof_enabled_flag))? 2:0(8-752)

[1054] The variable fRefWidth is set to be equal to the PicOutputWidthL of the reference image in the brightness sample.

[1055] The variable fRefHeight is set to be equal to the PicOutputHeightL of the reference image in the brightness sample.

[1056] The motion vector mvLX is set to (refMvLX-mvOffset).

[1057] – If cIdx equals 0, then the following applies:

[1058] – Scaling factors and their fixed-point representations are defined as

[1059] -hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL (8-753)

[1060] -vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[1061] – Let (xIntL, yIntL) be the luminance position given in whole sample units, and (xFracL, yFracL) be the offset given in 1 / 16 sample units. These variables are used only in this clause to specify the fractional sample positions within the reference sample array refPicLX.

[1062] – The left top coordinate of the boundary block filled with reference samples (xSbInt) L ySbInt L ) is set to (xSb+(mvLX[0]>>4), ySb+(mvLX[1]>>4)).

[1063] – For each luminance sample location (x) within the predicted luminance sample array predSamplesLX L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), corresponding to the predicted brightness sample value predSamplesLX[x L ][y L The derivation is as follows:

[1064] –make (refxSb) L ,refySb L ) and (refx L refy L ) represents the brightness position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in 1 / 16 sample units. The variable refxSb L refx L ,refySb L and refy L The derivation is as follows:

[1065] -refxSb L =((xSb<<4)+refMvLX[0])*hori_scale_fp

[1066] (8-755)

[1067] -refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6(8-756)

[1068] -refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp

[1069] (8-757)

[1070] -refyL=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*

[1071] ((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[1072] – Variable xInt L yInt L xFrac L and yFrac L The derivation is as follows:

[1073] -xInt L =refx L >>4 (8-759)

[1074] -yInt L =refy L >>4 (8-760)

[1075] -xFrac L =refx L &15 (8-761)

[1076] -yFrac L =refy L &15 (8-762)

[1077] – If all of the following conditions are met, then using6TapFlag is set to 1:

[1078] –cbWidth[0][xSb][ySb]<=4||cbHeight[0][xSb][ySb]<=4||

[1079] cbWidth[0][xSb][ySb]*cbHeight[0][xSb][ySb]<=64.

[1080] –PredFlagL0[xSb][ySb]==1&&PredFlagL1[xSb][ySb]==1.

[1081] – If bdofFlag equals true or (sps_affine_prof_enabled_flag equals true and inter_affine_flag[xSb][ySb] equals true), and one or more of the following conditions are true, then the luminance integer sample retrieval procedure as specified in Clause 8.5.6.3.3 is invoked to retrieve (xInt) the luminance integer sample. L +(xFrac L >>3)-1), yInt L +(yFrac L >>3)-1) and refPicLX are used as inputs to derive the predicted luminance sample values ​​predSamplesLX[x L ][y L ].

[1082] 1.x L It equals 0.

[1083] 2.x L It equals sbWidth+1.

[1084] 3.y L It equals 0.

[1085] 4.y L It equals sbHeight + 1.

[1086] – Otherwise, by invoking the luminance sample 8-tap interpolation filtering procedure specified in Clause 8.5.6.3.2, the results are obtained as follows: (xIntL-(brdExtSize>0?1:0), yIntL-(brdExtSize>0?1:0)), (xFracL, yFracL), (xSbIntL) L ySbInt L Using refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) and using 6TapFlag as input, the predicted luminance sample values ​​predSamplesLX[xL][yL] are derived.

[1087] Otherwise (cIdx is not equal to 0), the following applies:

[1088] – Let (xIntC, yIntC) be the chromaticity position given in whole sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify the overall fractional sample position within the reference sample array refPicLX.

[1089] – The left top coordinates (xSbIntC, ySbIntC) of the boundary block filled by the reference sample points are set to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[1090] – For each chromaticity sample position (xC = 0..sbWidth-1, yC = 0..sbHeight-1) within the predicted chromaticity sample array predSamplesLX, the corresponding predicted chromaticity sample value predSamplesLX[xC][yC] is derived as follows:

[1091] –make (refxSb) C ,refySb C ) and (refx C refy C ) represents the chromaticity position pointed to by the motion vector (mvLX[0], mvLX[1]) given in 1 / 32 sample units. The variable refxSb C ,refySb C refx C and refy C The derivation is as follows:

[1092] -refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp (8-763)

[1093] -refx C =((Sign(refxSb) C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5(8-764)

[1094] -refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp(8-765)

[1095] -refy C =((Sign(refySb)C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>>5 (8-766)

[1096] – Variable xInt C yInt C xFrac C and yFrac C The derivation is as follows:

[1097] -xInt C =refx C >>5 (8-767)

[1098] -yInt C =refy C >>5 (8-768)

[1099] -xFrac C =refy C &31 (8-769)

[1100] -yFrac C =refy C &31 (8-770)

[1101] – By invoking the procedure specified in Clause 8.5.6.3.4, the predicted sample values ​​predSamplesLX[xC][yC] are derived with (xIntC, yIntC), (xFracC, yFracC), (xSbIntC, ySbIntC), sbWidth, sbHeight, and refPicLX as input.

[1102] 8.5.6.3.2 Brightness Sample Interpolation and Filtering Process

[1103] The input to this process is:

[1104] – Brightness position in whole sample units (xInt) L yInt L ),

[1105] – Brightness position in fractional sample units (xFrac) L yFrac L ),

[1106] – Brightness position in whole sample units (xSbInt) L ySbInt L ), specifies the left top sample of the boundary block filled with the reference sample relative to the left top brightness sample of the reference image.

[1107] –Luminance reference sample array refPicLX L ,

[1108] – Half-sample interpolation filter index hpelIfIdx,

[1109] – The variable sbWidth specifies the width of the current child block.

[1110] – The variable sbHeight specifies the height of the current child block.

[1111] – Luminance position (xSb, ySb), specifies the left top sample of the current sub-block relative to the left top luminance sample of the current image.

[1112] – The flag using6TapFlag specifies whether to use 6-tap interpolation filtering.

[1113] The output of this process is the predicted luminance sample value, predSampleLX. L

[1114] The derivation of variables shift1, shift2, and shift3 is as follows:

[1115] – The variable shift1 is set to equal Min(4, BitDepth). Y -8), variable shift2 is set to 6, and variable shift3 is set to Max(2, 14-BitDepth). Y ).

[1116] – The variable picW is set to equal pic_width_in_luma_samples, and the variable picH is set to equal pic_height_in_luma_samples.

[1117] Luminance interpolation filter coefficient f L [p] For each 1 / 16 fractional sample location, p equals xFrac L or yFrac L The derivation is as follows:

[1118] – The luminance interpolation filter coefficient f is determined if at least one of the following conditions is met. L [p] is specified in Table 8-12.

[1119] –MotionModelIdc[xSb][ySb] is greater than 0, and both sbWidth and sbHeight are equal to 4.

[1120] –using6TapFlag equals 1.

[1121] Otherwise, the brightness interpolation filter coefficient f L [p]Specified in Table 8-11, depending on hpelIfIdx. Luminance position in integer sample units (xInt) i yInt i For i = 0..7, the derivation is as follows:

[1122] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[1123] -xInt i =Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xInt L +i-3)(8-771)

[1124] -yInt i =Clip3(SubPicTopBoundaryPos, SubPicBotBoundaryPos, yInt L +i-3) (8-772)

[1125] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[1126] -xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3): (8-773)

[1127] xInt L +i-3)

[1128] -yInt i =Clip3(0, picH-1, yInt) L +i-3) (8-774)

[1129] The brightness position in integer units for i = 0..7 is further modified as follows:

[1130] -xInt i =Clip3(xSbInt) L -3, xSbIntL +sbWidth+4,xInt i )

[1131] (8-775)

[1132] -yInt i =Clip3(ySbInt) L -3,ySbInt L +sbHeight+4,yInt i )

[1133] (8-776)

[1134] Predicted brightness sample value predSampleLX L The derivation is as follows:

[1135] –If xFrac L and yFrac L If both are equal to 0, then predSampleLX L The value is derived as follows:

[1136] -predSampleLX L =refPicLX L [xInt3][yInt3]< <shift3 (8-777)

[1137] Otherwise, if xFrac L Not equal to 0 and yFrac L If it equals 0, then predSampleLX L The value is derived as follows:

[1138]

[1139] Otherwise, if xFrac L Equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The value is derived as follows:

[1140]

[1141] Otherwise, if xFrac L Not equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The value is derived as follows:

[1142] The sample array temp[n], where n = 0..7, is derived as follows:

[1143]

[1144] –Predicted luminance sample value predSampleLX L The derivation is as follows:

[1145]

[1146] Table 8-11 – Brightness interpolation filter coefficients f for each 1 / 16 fractional sample location p L The specification of [p].

[1147]

[1148] Table 8-12 – Luminance interpolation filter coefficients f for each 1 / 16 fractional sample location p of the affine motion pattern L The specification of [p].

[1149]

[1150] Figure 29 This is a block diagram of a video processing device 2600. Device 2600 can be used to implement one or more of the methods described herein. Device 2600 can be implemented as a smartphone, tablet computer, computer, Internet of Things (IoT) receiver, etc. Device 2600 may include one or more processors 2602, one or more memories 2604, and video processing hardware 2606. The processors (multiple) 2602 can be configured to implement one or more methods described herein. The memories (multiple) 2604 can be used to store data and code to implement the methods and techniques described herein. The video processing hardware 2606 can be used to implement some of the techniques described herein in hardware circuitry.

[1151] Figure 30 This is a flowchart of an exemplary video processing method 3000. The method includes determining (3002) the maximum number of candidate MLs in the sub-block Merge candidate list or enabling the use of alternative temporal motion vector prediction (ATMVP) candidates, based on the state of the current video block, either using Temporal Motion Vector Prediction (TMVP) or Current Picture Reference Codec (CPR) of the current video block, where the state is one of enabled or disabled, and performing (3004) a conversion between the current video block and the bitstream representation of the current video block based on the result of the determination.

[1152] It should be understood that several technologies have been disclosed that will benefit video encoder and decoder embodiments integrated into video processing devices, such as smartphones, laptops, desktop computers, and similar devices, by allowing the use of the ATMVP codec tool in encoding or decoding video or images. Various embodiments and technologies may be used as described in the following terms-based description.

[1153] 1. A video processing method comprising: for a current video block, determining, based on the state of Temporal Motion Vector Prediction (TMVP) usage or the state of Current Picture Reference Codec (CPR) for the current video block, a maximum number of candidate MLs in a sub-block Merge candidate list or enabling the use of alternative Temporal Motion Vector Prediction (ATMVP) candidates, wherein the state is one of enabled or disabled; and based on the determined result, performing a conversion between the current video block and a bitstream representation of the current video block.

[1154] 2. The method described in Clause 1, wherein enabling the use of ATMVP candidates is also based on the current state of ATMVP usage for the video block.

[1155] 3. The method according to any one of Clauses 1-2, wherein determining ML comprises: determining that ATMVP and TMVP are disabled; and determining ML by excluding ATMVP candidates from the sub-block Merge candidate list.

[1156] 4. The method according to any one of clauses 1-2, wherein determining ML comprises:

[1157] It is determined that ATMVP is enabled for the current video block, TMVP is disabled for the current video block, and affine codec is disabled for the current video block; and

[1158] Based on the aforementioned conclusion, ML is set to zero.

[1159] 5. The method according to any one of clauses 1-2, wherein determining ML comprises:

[1160] It is determined that ATMVP is enabled for the current video block, TMVP is enabled for the current video block, and affine codec is disabled for the current video block; and

[1161] Based on the aforementioned conclusion, ML is set to 1.

[1162] 6. The method according to any one of Clauses 1-2, wherein determining ML is based on the state of ATMVP usage for the current block and whether the co-bit reference image of the current image of the current video block is the current image.

[1163] 7. The method according to any one of Clauses 1-2, wherein determining ML is based on the state of ATMVP and whether the reference image at reference index 0 in the list of reference images of the current image with the current video block is the current image.

[1164] 8. The method described in Clause 7, wherein the list of reference images is List 0 or List 1.

[1165] Additional descriptions of clauses 1-8 can be found in items 1-2 listed in section 4.

[1166] 9. A video processing method, comprising:

[1167] The conversion between the bitstream representing a sequence of video images and the pixel values ​​of the video images determines the relationship between Temporal Motion Vector Prediction (TMVP) codec tools and Alternative TMVP (ATMVP) codec tools, where the video images use block-based coding, wherein blocks of video images are organized into stripes or slices, and

[1168] Based on the established criteria, perform the conversion;

[1169] The relation specifies applicability, such as one or more of the following:

[1170] (a) If TMVP is not allowed at the strip, slice, or picture level, then ATMVP is disabled at the strip, slice, or picture level.

[1171] (b) If the TMVP is notified at the strip, slice, or picture level signaling, then the ATMVP is notified at the strip, slice, or picture level signaling after the TMVP.

[1172] (c) Specify the relationship only at the sequence level, and not at the strip, slice, or picture level.

[1173] 10. The method according to Clause 9, wherein the relationship is signaled in a single bit of the bitstream representation.

[1174] Additional descriptions of clauses 9-10 can be found in items 3-4 listed in section 4.

[1175] 11. A video encoding / decoding method, comprising:

[1176] The amount of masking used to determine the location of the coordinates of the current block or sub-block of the current video region for the conversion between video regions and bitstream representations; and

[1177] By applying a mask based on this quantity, the conversion between video regions and their corresponding bitstream representations is performed.

[1178] 12. The method according to Clause 11, wherein the amount of the mask is signaled in the bitstream representation.

[1179] 13. The method according to clauses 11-12, wherein when the indication in the signaling notification parameter set of the bitstream representation indicates that motion compression is disabled, it is set to no mask.

[1180] 14. The method according to any one of clauses 11-13, wherein the mask is applied by bitwise AND operation of an integer MASK with a value equal to ~(2M-1), where M is an integer and "~" denotes a bitwise compensation operator.

[1181] 15. The method according to any one of clauses 11-14, wherein the applied mask comprises a mask of a 2K x 2K block of video regions that result in the application of the same motion information, wherein K and M are different integers.

[1182] 16. The method according to any one of clauses 11-15, wherein M = 3 or M = 4.

[1183] 17. The method according to any one of clauses 11-16, wherein M is signaled in the bitstream representation.

[1184] Additional descriptions of clauses 11-17 can be found in items 3-4 listed in Chapter 4.

[1185] 18. A method for video processing, comprising:

[1186] During the conversion between the current video region and the bitstream representation, the valid corresponding region of the alternative temporal motion vector prediction (ATMVP) for the current video region is determined, and

[1187] The conversion is performed based on the effective corresponding region.

[1188] The determination is based on one or more of the following:

[1189] (a) Adaptive determination based on the characteristics of the current video region;

[1190] (b) A basic region having a size of M x N pixels, where M and N are integers;

[1191] (c) Determine the motion vector used to locate the valid corresponding area, wherein the determination of the motion vector is based on rules; or

[1192] (d) Determine the default motion vector for the ATMVP codec.

[1193] 19. The method according to Clause 18, wherein the current video region is a valid corresponding region is a function of: (a) the width or height of the current video region, or (b) a motion vector compression tool applied to the current video region.

[1194] 20. The method according to any one of Clauses 18-19, wherein the current video region corresponds to a basic region of size M x N smaller than the codec tree unit (CTU) region.

[1195] 21. The method according to Clause 20, wherein the current video block is within a basic region, and wherein the valid corresponding region is a co-located basic region.

[1196] 22. The method according to Clause 20, wherein the current video block is within the basic region, and wherein the effective corresponding region does not overlap at least partially with the co-located basic region.

[1197] 23. The method described in Clause 18, wherein the valid corresponding region is in an image different from the current image containing the current video region.

[1198] 24. The method according to Clause 23, wherein the current video region is the current video block, and the temporal motion vector (TMV) corresponding to the current video block is (1) set to a default value, or (2) set to a motion vector based on a historical motion vector predictor table.

[1199] 25. The method according to Clause 23, wherein determining the default motion vector of the ATMVP codec comprises (1) setting the default motion vector to the value (0,0), or (2) deriving the default motion vector from a history-based motion vector predictor table, or (3) deriving the default motion vector from adjacent blocks.

[1200] Additional descriptions of clauses 18-25 can be found in items 7-10 listed in Chapter 4.

[1201] 26. A video processing method, comprising:

[1202] Use rules to generate a candidate list or motion vectors for video regions; and

[1203] The candidate list performs conversions between video regions and their bitstream representations, where the rules include one or more of the following:

[1204] (a) Rule 1: During the transformation, candidates corresponding to the Alternate Time-Domain Motion Vector Prediction (ATMVP) mode are always considered;

[1205] (b) Rule 2: Use the enabled state of the affine mode of the video region when determining the availability of zero motion affine merge candidates in the sub-block merge candidate list of the sub-blocks of the video region.

[1206] (c) Rule 3: Use non-affine fill candidates for the sub-block merge candidate list of sub-blocks in the video region; or

[1207] (d) Rule 4: Derive the motion vector of the video region from the motion vector of the block covering the corresponding position according to the alternative temporal motion vector predictor method.

[1208] 27. The method described in Clause 26, wherein the video region is the current video block, and Rule 1 further specifies that the ATMVP default motion vector is determined by identifying the block at the corresponding position of the corresponding position covering the center point of the current video block.

[1209] 28. The method according to Clause 26, wherein Rule 2 further specifies that zero-motion affine merge candidates should not be added to the sub-block merge candidate list when the affine mode enabled state indicates that the affine mode is off for the current video block.

[1210] Additional descriptions of clauses 26-28 can be found in items 11-12 listed in Chapter 4.

[1211] 29. The method according to any one of clauses 1-28, wherein the conversion comprises generating a bitstream from the current video block or the current video region.

[1212] 30. The method according to any one of clauses 1-28, wherein the conversion comprises generating pixel values ​​of the current video block or the current video region from the bitstream representation.

[1213] 31. The method of any one of Clauses 1-28 further comprises: determining that a temporal block in a co-occurring image associated with the current video block is encoded in CPR mode, and wherein a default motion candidate is used instead of an ATMVP candidate based on the determination that the temporal block is encoded in CPR mode.

[1214] 32. The method according to Clause 31, wherein the default motion candidate indicates the center position of the current video block.

[1215] 33. The method according to Clause 31, wherein the default motion candidate is a (0,0) motion vector and the reference image index is equal to zero for both reference image lists.

[1216] 34. The method of any one of Clauses 1-28 further comprises: determining the position of the current video block, wherein the default motion information used for the ATMVP candidate is based on the determination of the position of the current video block.

[1217] 35. The method described in accordance with Clause 34, wherein the position is based on the position of a sub-block of the current video block.

[1218] 36. The method of any one of Clauses 1-28, wherein the use of an ATMVP candidate is based on a flag indicating the use of an ATMVP candidate provided by a strip, sheet, or picture header.

[1219] 37. The method according to Clause 36, wherein the current picture of the current video block is not an intra-frame random access point (IRAP) picture, and wherein the current picture is not inserted into RefPicList0 with a reference index equal to zero.

[1220] 38. The method of any one of Clauses 1-28, wherein ATMVP or TMVP is enabled or disabled based on a flag that infers a strip or slice to be false, based on a current picture containing the current video block and a list of reference pictures X, and the current picture is a reference picture with an index set to M.

[1221] 39. The method of any one of Clauses 1-28, further comprising: determining a reference image with an index set to M in a reference image list X having the current video block as the current image, and wherein ATMVP is enabled based on determining that the reference image with an index set to M in a reference image list X having the current video block is the current image.

[1222] 40. The method described in Clause 39, wherein sub-block motion information is defined as pointing to the current image.

[1223] 41. The method according to Clause 39 further includes: determining that the sub-block motion information is from a temporal block, wherein the temporal block is encoded and decoded using a reference image pointing to the current image of the temporal block.

[1224] 42. The method described in Clause 41, wherein the sub-block motion information is not scaled.

[1225] 43. The method of any one of clauses 1-28, wherein the conversion includes aligning the sub-block Merge index for encoding and decoding.

[1226] 44. The method according to Clause 43, wherein the first number of binary numbers is context-coded and the second number of binary numbers is bypass-coded and is not context-coded.

[1227] 45. The method described in Clause 43, wherein the binary number is context-encoded.

[1228] 46. ​​A video processing method, comprising: for a current video block, determining motion vectors for locating corresponding blocks in different images during an Alternate Temporal Motion Vector Prediction (ATMVP) process, for a conversion between the current video block and its codec representation based on a right-shift rounding process, and performing the conversion between the current video block and its codec representation based on the determined result.

[1229] 47. The method according to Clause 46, wherein the right-shift rounding procedure is the same as the right-shift rounding procedure used for motion vector scaling during the transformation.

[1230] 48. The method described in Clause 46, wherein the right-shifting rounding process results in an integer value by rounding towards zero.

[1231] 49. The method of any one of Clauses 46-48, wherein the motion vectors used in the ATMVP process are used to derive default motion information.

[1232] The following terms provide some exemplary solutions to the technologies described in the items in the preceding chapters (e.g., items 24-29).

[1233] 50. A video processing method, comprising: converting a video using sub-pictures between a video picture and a codec representation of a video segment; determining that rules relating to constraints of the sub-blocks are satisfied by the conversion; and performing the conversion according to the rules of the constraints.

[1234] 51. The method described in Clause 50, wherein the constraint rule specifies that the width of the sub-image S ending at column (j-1) is set to be equal to j minus the leftmost column of the sub-image S, since positions (i, j) and (i, j-1) belong to different sub-images within the video image.

[1235] 52. The method as described in any one of Clauses 50-51, wherein the constraint rule specifies that the height of the sub-image S ending at (NumSubPicGridRows-1) rows is set to (NumSubPicGridRows–1) minus the top row of the sub-image S plus one.

[1236] 53. The method as described in any one of clauses 50-52, wherein the constraint rule specifies that the width of the sub-picture S ending at column (NumSubPicGridColumns-1) is set to (NumSubPicGridColumns–1) minus the leftmost column of sub-picture S plus 1.

[1237] 54. The method of any one of clauses 50-53, wherein the constraint rule specifies that the size grid of the sub-picture in the video picture is an integer multiple of the codec tree unit size used during the conversion.

[1238] 55. The method as described in any one of clauses 50-54, wherein the constrained rules specify that all sub-pictures in the video picture do not overlap and together cover the entire video picture.

[1239] The following terms provide some exemplary solutions to the technologies described in the items in the previous chapters (e.g., items 30-34).

[1240] 56. A video processing method, comprising: a conversion between video units of a video and a codec representation of a video; determining whether to use a reference picture resampling (RPR) mode during the conversion; and performing the conversion based on the determination.

[1241] 57. The method described in Clause 56, wherein a flag in the codec representation indicates the use of an RPR mode.

[1242] 58. The method of any one of Clauses 56-57, wherein the RPR mode is indicated at the sequence level or the video parameter set level.

[1243] 59. The method of any one of clauses 56-59, wherein the RPR mode includes: during the conversion, using interpolation filtering to derive a predicted block of the current block of the video unit based on whether the resolution of the reference image is different from that of the current image, or whether the width and / or height of the reference image is greater than that of the current image, during the motion compensation process.

[1244] 60. The method according to any one of clauses 1 to 59, wherein the conversion comprises generating a codec representation from the pixel values ​​of the current video block.

[1245] 61. The method of any one of Clauses 1 to 59, wherein the conversion comprises pixel values ​​from the current video block represented by the codec.

[1246] 62. A video encoder device including a processor configured to implement the method according to any one of claims 1 to 61.

[1247] 63. A video decoder device including a processor configured to implement the method according to any one of clauses 1 to 61.

[1248] 64. A computer-readable medium having code stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of clauses 1 to 61.

[1249] Figure 32This is a block diagram illustrating an exemplary video processing system 3200, in which various techniques described herein can be implemented. Various implementations may include some or all of the components of system 3200. System 3200 may include input 3202 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or may be received in a compressed or encoded format. Input 3202 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Networking (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[1250] System 3200 may include codec component 3204, which can implement the various codec or encoding methods described in this document. Codec component 3204 can reduce the average bit rate of video from input 3202 to the output of codec component 3204 to produce a codec representation of the video. Codec techniques are therefore sometimes referred to as video compression or transcoding techniques. The output of codec component 3204 can be stored or transmitted via a connected communication, as represented by component 3206. The stored or communicated bitstream (or codec) representation of the video received at input 3202 can be used by component 3208 to generate pixel values ​​or displayable video that are sent to display interface 3210. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are called “codec” operations or tools, it should be understood that codec tools or operations are used by the encoder, and the corresponding decoding tools or operations that reverse the results of codec are performed by the decoder.

[1251] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Resolution Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described in this document can be implemented in various electronic devices, such as smartphones, laptops, or other devices capable of digital data processing and / or video display.

[1252] Some embodiments of the disclosed technology are discussed in a terms-based format.

[1253] Some exemplary embodiments of the technology described in item 22 of section 4 include:

[1254] A1. A method for visual media processing (e.g., Figure 33The method 3300 shown includes: for the current video block, determining (3302) a motion vector used in a sub-block-based motion vector prediction (sbTMVP) process to locate a corresponding block in a co-bit image used for the conversion between the current video block and the bitstream representation of the current video block, wherein the motion vector used in the sbTMVP process is calculated according to a scaling operation; and performing (3304) a conversion between the current video block and the bitstream representation of the visual media data based on the use of the motion vector.

[1255] A2. The method according to Clause A1, wherein the scaling operation includes at least one of the following: a right shift operation, a left shift operation, and / or a rounding operation applied to the motion vector used in the sbTMVP process.

[1256] A3. The method according to clause A2, wherein the right-shifting rounding operation causes the motion vector to be rounded to an integer value by rounding towards zero.

[1257] A4. The method described under Clause A1 further includes:

[1258] Once it is determined that the dimension of the co-located image or the dimension of the window associated with the co-located image is different from the dimension of the current image associated with the current block or the dimension of the window associated with the current image, a scaling operation is applied to the motion vector used in the sbTMVP process.

[1259] A5. The method according to clause A4, wherein the result of the scaling operation on the motion vector used in the sbTMVP process is expressed as:

[1260] MVx' = MVx * W1 / W2 and MVy' = MVy * H1 / H2, where MVx and MVy are the motion vectors of the block corresponding to the current video block, and MVx' and MVy' are the scaling motion vectors of the current video block. The dimensions of the co-positioned image or the dimensions of the window associated with the co-positioned image are referred to as W1 and H1, and the dimensions of the current image or the dimensions of the window associated with the current image are referred to as W2 and H2, respectively.

[1261] Some exemplary embodiments of the technology described in the project of Chapter 4 include:

[1262] B1. A method for visual media processing (e.g., Figure 34 The method shown in 3400 includes:

[1263] For the current video block, determine (3402) the motion vector used in the sub-block-based temporal motion vector prediction (sbTMVP) process to locate the corresponding block in the co-bit image used for the conversion between the current video block and the bitstream representation of the current video block, wherein the motion vector used in the sbTMVP process is calculated with respect to the center point of the current video block;

[1264] The center point of the current video block is modified by applying one or more operations; and

[1265] Based on the center point modified by applying the one or more of the operations, (3408) a conversion is performed between the current video block and the bitstream representation of the visual media data.

[1266] B2. The method according to clause B1, wherein one or more operations include a scaling operation and / or an operation corresponding to adding an offset to the center point.

[1267] B3. The method described under Clause B1 further includes:

[1268] Once it is determined that the dimension of the co-positioned image or the dimension of the window associated with the co-positioned image is different from the dimension of the current image associated with the current block or the dimension of the window associated with the current image, the center point of the current video block is modified.

[1269] B4. As described in Clause B3, the additional modification to the center point of the current video block is expressed as follows:

[1270] x0' = (x0-X2)*W1 / W2+X1 and y0' = (y0-Y2)*H1 / H2+Y1,

[1271] The center point of the current video block is (x0, y0), and the coordinates of the center point of the current video block after the additional modification are (x0', y0'). The dimension of the co-position image or the dimension of the window associated with the co-position image is designated as W1 and H1, respectively. The dimension of the current image or the dimension of the window associated with the current image is designated as W2 and H2, respectively. The top left position of the window associated with the current image is designated as (X2, Y2), and the top left position of the window associated with the co-position image is designated as (X1, Y1).

[1272] B5. As described in Clause B3, the additional modification to the center point of the current video block is expressed as follows:

[1273] x0' = (x0 - X2) * W1 / W2 and y0' = (y0 - Y2) * H1 / H2,

[1274] The center point of the current video block is (x0, y0), and the coordinates of the center point of the current video block after the additional modification are (x0', y0'). The dimension of the co-position image or the dimension of the window associated with the co-position image is designated as W1 and H1, the top left position of the window associated with the current image is designated as (X2, Y2), and the dimension of the current image or the dimension of the window associated with the current image is designated as W2 and H2, respectively.

[1275] Some exemplary embodiments 24 of the technology described in the project of Chapter 4 include:

[1276] C1. A method for visual media processing (e.g., Figure 35 The method shown in 3500 includes:

[1277] For the current video block, determine (3502) the motion vectors used in the sub-block-based temporal motion vector prediction (sbTMVP) process to locate the corresponding block in the co-bit image used for the conversion between the current video block and the bitstream representation of the current video block, wherein the motion vectors used in the sbTMVP process are calculated with respect to points in the corresponding block in the co-bit image;

[1278] Modify the points in the corresponding blocks of the co-located image (3504) by applying one or more operations; and

[1279] Based on the points in the corresponding blocks of the co-bit image modified by applying the one or more of the operations, (3506) a conversion is performed between the current video block and the bitstream representation of the visual media data.

[1280] C2. The method according to Clause C1, wherein one or more operations include a scaling operation and / or an operation corresponding to adding an offset to the point in the corresponding block of the co-positioned image.

[1281] C3. The method described under Clause C1 further includes:

[1282] Once it is determined that the dimension of the co-positioned image or the dimension of the window associated with the co-positioned image is different from the dimension of the current image associated with the current video block or the dimension of the window associated with the current image, additional modifications are made to the points in the corresponding block of the co-positioned image.

[1283] C4. The method described in clause C3, wherein the additional modification of the point in the corresponding block of the co-location image is expressed as:

[1284] x' = (x-X2)*W1 / W2+X1 and y' = (y-Y2)*H1 / H2+Y1,

[1285] The point in the corresponding block of the co-located image is (x, y), and the coordinates of the point in the corresponding block of the co-located image after the additional modification are (x', y'). The width and height of the window associated with the current image are designated as W2 and H2, respectively, the width and height of the window associated with the co-located image are designated as W1 and H1, respectively, the top left position of the window associated with the current image is designated as (X2, Y2), and the top left position of the window associated with the co-located image is designated as (X1, Y1).

[1286] C5. The method described in clause C3, wherein the additional modification of the point in the corresponding block of the co-location image is expressed as:

[1287] x' = (x-X2)*W1 / W2 and y' = (y-Y2)*H1 / H2, wherein the point in the corresponding block of the co-located image is (x, y), and the coordinates of the point in the corresponding block of the co-located image after the additional modification are (x', y'), wherein the width and height of the window associated with the current image are respectively designated as W2 and H2, the width and height of the window associated with the co-located image are respectively designated as W1 and H1, and the top left position of the window associated with the current image is designated as (X2, Y2).

[1288] Some exemplary embodiments of the technology described in the project of Chapter 4 include:

[1289] D1. A method for visual media processing (e.g., Figure 36 The method 3600 shown includes: converting a video picture contained in visual media data to a bitstream representation of the visual media data using subpictures; determining (3602) that a rule associated with one or more subpictures is satisfied by the conversion; and performing (3604) the conversion according to the constrained rule, wherein the rule specifies that the size of the subpicture in the video picture is an integer multiple of the codec tree unit size associated with the video picture.

[1290] D2. The method described in Clause D1, wherein the rule specifies that the bitstream representation includes a variable corresponding to the width of the sub-image.

[1291] D3. The method described in Clause D1, wherein the rule specifies that the bitstream representation contains a variable corresponding to the height of the sub-image.

[1292] D4. The method according to any one or more of clauses D2-D3, wherein the variables corresponding to the height of the sub-image and / or the variables corresponding to the width of the sub-image are expressed in units of luminance codec tree unit size.

[1293] D5. The method according to any one or more of clauses D2 and D4, wherein the variable corresponding to the width of the sub-image is equal to the difference between the width of the sub-image in units of the luminance-encoded tree block size and 1.

[1294] D6. The method according to any one or more of clauses D3 and D4, wherein the variable corresponding to the height of the sub-image is equal to the difference between the height of the sub-image in units of the luminance codec tree cell size and 1.

[1295] D7. The method described in Clause D4, wherein the size of the luma codec tree unit is defined as the width or height of the luma codec tree block array.

[1296] D8. The method described in Clause D4, wherein the size of the luma codec tree unit is defined as the variable CtbSizeY, expressed as:

[1297] CtbSizeY = (1 << (sps_log2_ctu_size_minus5 + 5)), where sps_log2_ctu_size_minus5 refers to the size of the syntax element.

[1298] D9. The method according to clause D8, wherein the variable corresponding to the width of the sub-image is calculated as the width of the sub-image multiplied by the width of CtbSizeY.

[1299] D10. The method according to clause D8, wherein the variable corresponding to the height of the sub-image is calculated as the width of the sub-image multiplied by the height of CtbSizeY.

[1300] D11. The method described in Clause D1, wherein the value of the flag specified by the rule indicates the use of the sub-image during the conversion.

[1301] D12. The method described in Clause D11, wherein the flag is a Boolean value.

[1302] D13. The method according to clause D12, wherein when the flag is 1, it indicates that a sub-image is used, and when the flag is 0, it indicates that a sub-image is not used.

[1303] Some exemplary embodiments of the technology described in the project of Chapter 4 include:

[1304] E1. A method for video processing (e.g., Figure 37 The method 3700 shown includes: for a conversion between a video picture contained in visual media data and a bitstream representation of the visual media data using subpictures, determining (3702) that a rule relating to one or more subpictures is satisfied by the conversion; and performing (3704) the conversion according to the constrained rule, wherein the rule specifies that all subpictures in the video picture are non-overlapping and that all subpictures in the video picture together cover the entire video picture.

[1305] Some exemplary embodiments 30 of the technology described in the project of Chapter 4 include:

[1306] F1. A method for visual media processing (e.g., Figure 38 The method 3800 shown includes: converting between video units of visual media and a bitstream representation of the visual media data; (3802) determining whether a reference picture resampling (RPR) technique is used during the conversion; and (3804) performing the conversion based on the determination, wherein a flag corresponding to the determination is included in the bitstream representation at the sequence parameter set level.

[1307] F2. The method according to Clause F1, wherein the flag indicates the use of the RPR technology, and wherein the flag takes a Boolean value.

[1308] F3. The method according to clause F2, wherein the flag equal to 1 indicates that the spatial resolution of the codec image with respect to the sequence parameter set is inferred to be changeable, and the flag equal to 0 indicates that the spatial resolution of the codec image with respect to the sequence parameter set is inferred to be unchanged.

[1309] F4. The method according to clause F5, wherein the flag equal to 1 indicates that the image spatial resolution is changeable within the codec layer video sequence referencing the sequence parameter set, and the flag equal to 0 indicates that the image spatial resolution is not changeable within the codec layer video sequence referencing the sequence parameter set.

[1310] F5. The method according to any one or more of clauses F1-F4, wherein whether window information of the picture associated with the video unit is included in the bitstream representation is independent of whether the RPR technology is used.

[1311] F6. The method according to any one or more of clauses F1-F4, wherein the height and / or width of the video unit indicated in the picture parameter set (PPS) is inferred to be the same as the height and / or width of the video unit in another picture parameter set (PPS) associated with the same sequence parameter set.

[1312] F7. The method according to any one or more of clauses F1-F6, wherein the bitstream representation is configured to include window information of the picture associated with the video unit, regardless of the value of the flag.

[1313] F8. The method according to any one or more of clauses F1-F6, wherein the bitstream represents window information configured to contain a picture associated with the video unit, and wherein the flag indicates whether the window information is used during the conversion.

[1314] F9. The method according to any one or more of clauses F1-F4, wherein the height and / or width of the video unit indicated in the picture parameter set (PPS) is inferred to be the same as the height and / or width of the video unit in another picture parameter set (PPS) associated with the same sequence parameter set.

[1315] F10. The methods described under any one or more of clauses F-F4 also include:

[1316] Once it is determined that the RPR technology is not used during the conversion, the height and / or width of the video unit indicated in the Picture Parameters Set (PPS) are inferred to be the default values ​​for the height and / or width of the video unit.

[1317] F11. The method according to clause F10, wherein the default values ​​for the height and / or width of the video unit are inferred to be the maximum height and / or maximum width of the video unit indicated in the sequence parameter set (SPS) associated with the video unit.

[1318] Some exemplary embodiments of the technology described in the project of Chapter 4 include:

[1319] G1. A method for visual media processing (e.g., Figure 39 The method 3900 shown includes: selecting (3902) an interpolation filter during a motion compensation process to derive a predicted block of the current block of visual media data based on satisfying conditions, wherein the conditions are at least in part based on determining that the resolution of a reference image is different from the resolution of the current image and / or the dimension of a window associated with the reference image is different from the dimension of a window associated with the current image; and performing (3904) a conversion between the current block of visual media data and a bitstream representation of the current block.

[1320] G2. The method described under Clause G1 further includes:

[1321] Once it is determined that the resolution of the reference image is different from the resolution of the current image or the dimension of the window associated with the reference image is different from the dimension of the window associated with the current image, an interpolation filter different from the one used when the reference image has the same resolution as the current image is used.

[1322] G3. The method according to clause G1, wherein the condition is based on determining W1>a*W2 and / or H1>b*H2, where (W1, H1) represents the width and height of the reference image or the window associated with the reference image, and (W2, H2) represents the width and height of the current image or the window associated with the reference image, and a and b are scaling factors.

[1323] G4. The method described in Clause G3, where a = b = 1.5.

[1324] G5. The method described in accordance with clause G1, wherein the condition is further based on the height and / or width of the current block.

[1325] G6. The method described in accordance with clause G5, wherein the height and / or width of the current block individually implement one or more threshold conditions.

[1326] G7. The method described in accordance with clause G5, wherein the mathematical combination of the height and / or width of the current block achieves one or more threshold conditions.

[1327] G8. The method according to clause G6, wherein fulfilling the one or more threshold conditions includes the height and / or width of the current block exceeding the one or more threshold conditions.

[1328] G9. The method according to clause G6, wherein fulfilling the one or more threshold conditions includes the height and / or width of the current block missing the one or more threshold conditions.

[1329] G10. The method according to clause G7, wherein fulfilling the one or more threshold conditions comprises a mathematical combination of the height and / or width of the current block exceeding the one or more threshold conditions.

[1330] G11. The method according to clause G7, wherein fulfilling the one or more threshold conditions comprises a mathematical combination of the height and / or width of the current block that misses the one or more threshold conditions.

[1331] G12. The method described in any one or more of clauses G1-G11, wherein the selected interpolation filter is exclusively applied to the chroma color components of the current video block.

[1332] G13. The method described in any one or more of clauses G1-G11, wherein the selected interpolation filter is exclusively applied to the luminance and color components of the current video block.

[1333] G14. The method described in any one or more of clauses G1-G11, wherein the selected interpolation filter is exclusively applied in the horizontal filtering direction.

[1334] G15. The method described in any one or more of clauses G1-G11, wherein the selected interpolation filter is applied exclusively in the vertical filtering direction.

[1335] G16. The method according to any one or more of clauses G1-G11, wherein a single-tap filter is selected as the interpolation filter such that the result of applying the interpolation filter is equivalent to replicating integer samples.

[1336] G17. The method described in any one or more of clauses G1-G11, wherein a 4-tap interpolation filter or a 6-tap interpolation filter is selected when the condition is met.

[1337] G18. The method according to any one or more of clauses G1-G11, wherein a bilinear filter is selected as the interpolation filter when the condition is met.

[1338] Some exemplary embodiments 32 of the technology described in the project of Chapter 4 include:

[1339] H1. A method for visual media processing (e.g., Figure 40 The method 4000 shown includes: a conversion between a current block of visual media data and a bitstream representation of the visual media data; determining (4002) that the current block is a combined inter-frame intra-prediction (CIIP) block, wherein an intra-prediction block of the CIIP block is generated using the size of a transform unit (TU), wherein, in the combined inter-frame intra-prediction (CIIP) block, the final prediction of the current block is based on a weighted sum of the inter-frame prediction and the intra-frame prediction of the current block; and performing (4004) the conversion based on the determination.

[1340] H2. The method described under clause H1 further includes:

[1341] Once it is determined that the dimension of the encoding / decoding unit associated with the current block exceeds the maximum size of the transform unit, the encoding / decoding unit is divided into multiple transform units; and

[1342] For each of the plurality of transform units, an intra-frame prediction block and an inter-frame prediction block are generated.

[1343] H3. The method described under clause H2 further includes:

[1344] Once it is determined that the maximum size of the transform unit is less than a threshold and the dimension of the codec unit associated with the current block exceeds the maximum size of the transform unit, the codec unit is recursively divided into multiple transform units; and

[1345] For each of the plurality of transform units, an intra-frame prediction block and an inter-frame prediction block are generated.

[1346] H4. The method according to clause H3, wherein the first intra-prediction block is a reconstructed sample based on the second intra-prediction block, wherein the first intra-prediction block and the second intra-prediction block are generated based on the partitioning of the current block.

[1347] I1. The method according to any one or more of clauses A1 to H4, wherein the conversion comprises generating the bitstream representation from the current video block.

[1348] I2. The method according to any one or more of clauses A1 to H4, wherein the transformation comprises generating samples of the current video block from the bitstream representation.

[1349] 13. A video encoder device including a processor, the processor being configured to implement the method described in any one or more of clauses A1 to H4.

[1350] 14. A video decoder device including a processor configured to implement the method described in any one or more of clauses A1 to H4.

[1351] 15. A computer-readable medium having code stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one or more of clauses A1 to H4.

[1352] I6. A computer-readable storage medium storing a bitstream representation generated by the method described in any one or more of clauses A1 to H4.

[1353] Figure 41 This is a block diagram illustrating an exemplary video encoding / decoding system 100 that can utilize the techniques of this disclosure.

[1354] like Figure 41 As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, which can be referred to as a video encoding device. The destination device 120 can decode the encoded video data generated by the source device 110, which can be referred to as a video decoding device.

[1355] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[1356] Video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. Video data may include one or more pictures. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may contain a sequence of bits forming a codec representation of the video. The bitstream may contain codec pictures and associated data. A codec picture is a codec representation of a picture. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. Encoded video data may be transmitted directly to destination device 120 via network 130a through I / O interface 116. Encoded video data may also be stored on storage medium / server 130b for access by destination device 120.

[1357] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[1358] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may decode and display video data to the user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120, which is configured to connect to an external display device.

[1359] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Codec (HEVC) standard, the Multi-Functional Video Codec (VVC) standard, and other current and / or other standards.

[1360] Figure 42 This is a block diagram illustrating an example of a video encoder 200, which can be... Figure 41 The video encoder 114 in the system 100 shown in the figure.

[1361] The video encoder 200 can be configured to perform any or all of the techniques disclosed herein. Figure 42 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some embodiments, the processor can be configured to perform any or all of the techniques described in this disclosure.

[1362] The video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra-frame prediction unit 206), a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[1363] In other examples, the video encoder 200 may contain more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in IBC mode, where at least one reference picture is the picture in which the current video block is located.

[1364] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, can be highly integrated, but... Figure 5 The examples are presented separately for illustrative purposes.

[1365] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[1366] The mode selection unit 203 can select one of the encoding / decoding modes (intra-frame or inter-frame) based, for example, on the error result, and provide the resulting intra-frame or inter-frame encoded / decoded block to the residual generation unit 207 to generate residual block data, which is then provided to the reconstruction unit 212 to reconstruct the encoded block for use as a reference picture. In some examples, the mode selection unit 203 can select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on inter-frame prediction signaling notification and intra-frame prediction signaling notification. The mode selection unit 203 can also select the resolution of the block's motion vector (e.g., sub-pixel or integer pixel precision) in the case of inter-frame prediction.

[1367] To perform inter-frame prediction for the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on motion information and decoded samples from images from buffer 213 other than the image associated with the current video block.

[1368] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-band, a P-band, or a B-band.

[1369] In some embodiments, the motion estimation unit 204 can perform unidirectional prediction on the current video block, and can search for a reference video block for the current video block in the reference images of list 0 or list 1. The motion estimation unit 204 can then generate a reference index indicating the reference image in list 0 or list 1 containing the reference video block and a motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. The motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[1370] In other examples, motion estimation unit 204 can perform bidirectional prediction of the current video block. Motion estimation unit 204 can search for a reference video block for the current video block in the reference images in list 0, and can also search for another reference video block for the current video block in the reference images in list 1. Motion estimation unit 204 can then generate reference indices indicating the reference images in lists 0 and 1 containing reference video blocks, and motion vectors indicating the spatial displacement between the reference video blocks and the current video block. Motion estimation unit 204 can output the reference index and motion vector of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information of the current video block.

[1371] In some embodiments, the motion estimation unit 204 can output motion information of the complete group during the decoding process of the decoder.

[1372] In some embodiments, the motion estimation unit 204 may not output the complete set of motion information for the current video block. Instead, the motion estimation unit 204 may signal the motion information of the current video block to another video block. For example, the motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of adjacent video blocks.

[1373] In one example, the motion estimation unit 204 may indicate a value in the syntax structure associated with the current video block that indicates to the video decoder 300 that the current video block has the same motion information as another video block.

[1374] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) within the syntactic structure associated with the current video block. The motion vector difference indicates the difference between the motion vectors of the current video block and the motion vectors of the indicated video block. Video decoder 300 can use the motion vectors of the indicated video block and the motion vector difference to determine the motion vectors of the current video block.

[1375] As discussed above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling notification techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling Notification.

[1376] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[1377] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) multiple predicted video blocks from the current video block. The residual data for the current video block can contain residual video blocks, which correspond to different sample components of the samples in the current video block.

[1378] In other examples, residual data for the current video block may not exist (e.g., in skip mode), and the residual generation unit 207 may not perform subtraction operations.

[1379] The transform processing unit 208 can generate one or more transform coefficient video blocks of the current video block by applying one or more transforms to the residual video block associated with the current video block.

[1380] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[1381] Inverse quantization unit 210 and inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video blocks respectively to reconstruct the residual video blocks from the transform coefficient video blocks. Reconstruction unit 212 can add the reconstructed residual video blocks to the corresponding samples from one or more predicted video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current block and store it in buffer 213.

[1382] After the video block is reconstructed by the reconstruction unit 212, loop filtering can be performed to reduce video block artifacts in the video block.

[1383] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy encoded data and output a bit stream containing the entropy encoded data.

[1384] Figure 43 This is a block diagram illustrating an example of a video decoder 300, which can be... Figure 41 The video decoder 114 in the system 100 shown.

[1385] The video decoder 300 can be configured to perform any or all of the techniques disclosed herein. Figure 43 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some embodiments, the processor can be configured to perform any or all of the techniques described in this disclosure.

[1386] exist Figure 43 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306, as well as a buffer 307. In some embodiments, the video decoder 300 can perform decoding through which, generally, the reverse is achieved with respect to the video encoder 200 (e.g., Figure 42 The encoding described is valid.

[1387] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may contain entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy-coded video data, and the motion compensation unit 302 can determine motion information, including motion vectors, motion vector precision, reference image list index, and other motion information, from the entropy-coded video data. The motion compensation unit 302 can determine such information, for example, by performing AMVP and Merge modes.

[1388] The motion compensation unit 302 can generate motion compensation blocks, possibly based on interpolation filtering. The identifier of the interpolation filter to be used with sub-pixel precision can be included in the syntax element.

[1389] The motion compensation unit 302 can use interpolation filtering as the video encoder 200 does during video block encoding to calculate the interpolated values ​​of sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filtering used by the video encoder 200 based on the received syntax information and use the interpolation filtering to generate predictive blocks.

[1390] The motion compensation unit 302 may use some syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how to divide the pictures of the encoded video sequence into segments, a mode indicating how to encode each segment, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence.

[1391] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 303 inverse-quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[1392] The reconstruction unit 306 can sum the residual block with the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to form a decoded block. If desired, deblocking filtering can also be applied to filter the decoded block to remove blocky artifacts. The decoded video block is then stored in buffer 307, which provides a reference block for subsequent motion compensation.

[1393] In this document, the term "video processing" or "visual media processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from the pixel representation of a video to the corresponding bitstream representation, and vice versa. As defined in the syntax, the bitstream representation of the current video block can, for example, correspond to bits that are co-occurring within the bitstream or distributed at different positions in the bitstream. For example, a macroblock can be encoded based on the error residual values ​​of the transformation and encoding / decoding, and also using bits in the header and other fields in the bitstream. Furthermore, as described in the solutions above, during the conversion, the decoder can resolve the bitstream based on determination, knowing that some fields may or may not be present. Similarly, the encoder can determine whether certain syntax fields will be included or not included, and generate the codec representation accordingly by including or excluding syntax fields from the codec representation.

[1394] The disclosed and other solutions, examples, embodiments, modules, and functional operations described herein can be implemented as digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or combinations thereof. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material composition that enables machine-readable propagated signals, or a combination thereof. The term "data processing apparatus" encompasses all devices, apparatuses, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. The propagated signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[1395] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form, including standalone programs or modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on a single computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[1396] The processes and logic described in this document can be executed by one or more programmable processors, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic can also be executed by dedicated logic circuitry, and the devices can be implemented as dedicated logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[1397] For example, processors suitable for executing computer programs include general-purpose microprocessors and special-purpose microprocessors, as well as any one or more processors in any kind of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, receiving data from or transferring data to one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data. However, a computer does not necessarily need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[1398] Although this patent document includes numerous details, these details should not be construed as limiting any subject matter or the scope of the claims, but rather as descriptions of features specific to particular embodiments of a particular technology. Certain features described in this patent document in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Moreover, although features may be described above as functioning in certain combinations and even initially claimed in this way, in some cases, one or more features from the claimed combination may be removed from the combination, and the claimed combination may be used for sub-combinations or variations thereof.

[1399] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or in a sequential order, or to perform all shown operations to achieve the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[1400] Only some implementation methods and examples are described, and other implementation methods, enhancements and variations may be made based on the content described and shown in this patent document.

Claims

1. A method for visual media processing, comprising: A first transformation between a first current block of visual media data and the bitstream of the visual media data, determining that the first current block is a combined inter-frame intra-frame prediction (CIIP) block, wherein an intra-frame prediction block of the CIIP block is generated using the size of a transform unit (TU), wherein, in the combined inter-frame intra-frame prediction (CIIP) block, the final prediction of the first current block is based on a weighted sum of the inter-frame prediction and the intra-frame prediction of the first current block; and The first conversion is performed based on the determination. Specifically, the intra-prediction signaling notification used in the combined inter-frame intra-prediction process is performed at the TU level rather than the CU level. The method further includes: For the second current block of the visual media data, determine the motion vector used in the sub-block-based temporal motion vector prediction (sbTMVP) process to locate the corresponding block in the co-bit image used for the second transformation between the second current block and the bitstream of the second current block, wherein the motion vector used in the sbTMVP process is calculated according to a scaling operation; Once it is determined that the dimension of the co-located image or the dimension of the window associated with the co-located image is different from the dimension of the current image associated with the second current block or the dimension of the window associated with the current image, a scaling operation is applied to the motion vector used in the sbTMVP process; and The second conversion between the second current block and the bitstream of the visual media data is performed based on the motion vector.

2. The method according to claim 1, further comprising: Once it is determined that the dimension of the coding unit associated with the first current block exceeds the maximum size of the transform unit, the coding unit is divided into multiple transform units; as well as For each of the plurality of transform units, an intra-frame prediction block and an inter-frame prediction block are generated.

3. The method according to claim 2, further comprising: Once it is determined that the maximum size of the transform unit is less than a threshold and the dimension of the coding unit associated with the first current block exceeds the maximum size of the transform unit, the coding unit is recursively divided into multiple transform units; and For each of the plurality of transform units, an intra-frame prediction block and an inter-frame prediction block are generated.

4. The method according to claim 3, wherein, The first intra-prediction block is a reconstructed sample based on the second intra-prediction block, wherein the first intra-prediction block and the second intra-prediction block are generated based on the division of the first current block.

5. The method according to any one of claims 1 to 4, wherein the first conversion comprises generating the bit stream from the first current block.

6. The method according to any one of claims 1 to 4, wherein the first conversion comprises generating samples of the first current block from the bitstream.

7. A video encoder device including a processor, the processor being configured to implement the method according to any one of claims 1 to 6.

8. A video decoder device including a processor, the processor being configured to implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having code stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 6.

10. A method for storing a video bitstream, comprising: For a first current block of visual media data, the first current block is determined to be a combined inter-frame intra-frame prediction (CIIP) block, wherein an intra-frame prediction block of the CIIP block is generated using the size of a transform unit (TU), wherein, in the combined inter-frame intra-frame prediction (CIIP) block, the final prediction of the first current block is based on a weighted sum of the inter-frame prediction and the intra-frame prediction of the first current block; and Based on the determination, the bit stream is generated; The bit stream is stored in a non-transitory computer-readable recording medium. Specifically, the intra-prediction signaling notification used in the combined inter-frame intra-prediction process is performed at the TU level rather than the CU level. The method further includes: For the second current block of the visual media data, determine the motion vector used in the sub-block-based temporal motion vector prediction (sbTMVP) process to locate the corresponding block in the co-bit image used for the second transformation between the second current block and the bitstream of the second current block, wherein the motion vector used in the sbTMVP process is calculated according to a scaling operation; Once it is determined that the dimension of the co-located image or the dimension of the window associated with the co-located image is different from the dimension of the current image associated with the second current block or the dimension of the window associated with the current image, a scaling operation is applied to the motion vector used in the sbTMVP process; and The second conversion between the second current block and the bitstream of the visual media data is performed based on the motion vector.

Images