Efficient methods, apparatus, and storage media for deriving affine Merge motion vectors

Abstract

A video processing method for efficient affine Merge motion vector derivation is disclosed. In one aspect, a video processing method is provided, comprising: dividing a current video block into sub-blocks; deriving motion vectors for each sub-block, wherein the motion vector of each sub-block is associated with the position of the sub-block according to a positional rule; and using the motion vectors of the sub-blocks to process the bitstream representation of the current video block.

Description

[0001] Cross-reference to related applications

[0002] This application is a divisional application of Chinese Patent Application No. 20190586584.8, filed on July 1, 2019, entitled "Efficient Method, Apparatus, and Storage Medium for Affine Merge Motion Vector Derivation". Pursuant to applicable patent law and / or the Paris Convention, this application promptly claims priority and benefit to International Patent Application No. PCT / CN2018 / 093943, filed on July 1, 2018, and International Patent Application No. PCT / CN2018 / 095568, filed on July 13, 2018. The entire disclosure of Chinese Application No. 20190586584.8, International Patent Application No. PCT / CN2018 / 093943, and International Patent Application No. PCT / CN2018 / 095568 is incorporated herein by reference as a part of this application disclosure. Technical Field

[0003] This patent document relates to video encoding and decoding technologies, devices, and systems. Background Technology

[0004] Despite advancements in video compression, digital video still accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This document discloses techniques that can be used in video encoding and decoding implementations to improve the performance of sub-block-based encoding, particularly when using affine motion coding modes.

[0006] In one example aspect, a video processing method is provided, including dividing a current block into sub-blocks; deriving motion vectors for each sub-block, wherein the motion vector of each sub-block is associated with the position of the sub-block according to a positional rule; and using the motion vectors of the sub-blocks to process the bitstream representation of the current block.

[0007] In another aspect, a video processing method is provided, comprising: for a conversion between a current block and a bitstream representation of the current block using an affine mode, deriving a motion vector at a control point of the current block based on a position rule; and using the motion vector to perform the conversion between the current block and the bitstream representation, wherein the position rule specifies that non-adjacent neighboring blocks are excluded from the derivation.

[0008] In another aspect, a video processing method is provided, comprising: for a conversion between a current block and a bitstream representation of the current block, determining an affine Merge candidate list for the conversion by including Merge candidates from one or more neighboring blocks that satisfy a validity criterion based on the positions of the one or more neighboring blocks; and performing the conversion between the current block and the bitstream representation using motion vectors.

[0009] In yet another example, a video encoder device that implements the video encoding method described herein is disclosed.

[0010] In another representative aspect, the various techniques described herein can be implemented as a computer program product stored on a non-transitory computer-readable medium. This computer program product includes program code for implementing the methods described herein.

[0011] In another representative aspect, video decoder devices can implement the methods described herein.

[0012] Details of one or more implementations are set forth in the accompanying appendices, drawings, and the following description. Other features will be apparent from the specification, drawings, and claims. Attached Figure Description

[0013] Figure 1 An example of prediction based on sub-blocks is shown.

[0014] Figure 2 An example of a simplified affine motion model is shown.

[0015] Figure 3 An example of the affine motion vector field (MVF) for each sub-block is shown.

[0016] Figure 4 An example of motion vector prediction (MVP) in the AF_INTER mode is shown.

[0017] Figure 5A and Figure 5B Examples of candidate encoding modes for AF_MERGE are described.

[0018] Figure 6 An example process for motion prediction using the Advanced Time Motion Vector Predictor (ATMVP) for a coding unit (CU) is shown.

[0019] Figure 7 An example of a CU with four sub-blocks (AD) and their neighboring blocks (ad) is shown.

[0020] Figure 8 An example of optical flow trajectories in video encoding is shown.

[0021] Figure 9A and 9B An example of a bidirectional optical flow (BIO) coding technique without block expansion is shown. Figure 9A An example of an access location outside the block is shown, and Figure 9B An example of padding to avoid additional memory access and computation is shown.

[0022] Figure 10 An example of bilateral matching is shown.

[0023] Figure 11 An example of template matching is shown.

[0024] Figure 12 An example of unidirectional motion estimation (ME) in frame rate upconversion (FRUC) is shown.

[0025] Figure 13 An example implementation of interleaved prediction is shown.

[0026] Figure 14 Examples are shown of different positions of the MV for different sub-blocks, where the stars represent different positions.

[0027] Figure 15 An example is shown for deriving the neighboring blocks of v0x and v0y.

[0028] Figure 16A and Figure 16B Examples are shown of MVs derived from the left adjacent block encoded in affine mode (a) or from the top adjacent block encoded in affine mode.

[0029] Figure 17 An example is shown of neighboring blocks and the current block belonging to different coding tree unit (CTU) rows, where affine Merge candidates from such neighboring blocks are considered invalid.

[0030] Figure 18 An example of interleaving prediction based on two partitioning modes according to the disclosed technology is shown.

[0031] Figure 19A An example partitioning pattern for dividing a block into 4×4 sub-blocks according to the disclosed technique is shown.

[0032] Figure 19B An example partitioning pattern for dividing a block into 8×8 sub-blocks according to the disclosed technique is shown.

[0033] Figure 19C An example partitioning pattern for dividing a block into 4×8 sub-blocks according to the disclosed technique is shown.

[0034] Figure 19D An example partitioning pattern for dividing a block into 8×4 sub-blocks according to the disclosed technique is shown.

[0035] Figure 19E An example partitioning pattern for dividing a block into non-uniform sub-blocks according to the disclosed technique is shown.

[0036] Figure 19F Another example partitioning pattern for dividing a block into non-uniform sub-blocks according to the disclosed technique is shown.

[0037] Figure 19G This illustrates yet another example partitioning pattern for dividing a block into non-uniform sub-blocks according to the disclosed technique.

[0038] Figure 20 This is a block diagram of an example hardware platform used to implement the visual media decoding or visual media encoding techniques described in this document.

[0039] Figure 21 This is a flowchart of an example method for video processing.

[0040] Figure 22 This is a flowchart of another example method used for video processing.

[0041] Figure 23 This is a flowchart of another example method used for video processing. Detailed Implementation

[0042] Section headings are used in this document to improve readability, but are not intended to limit the techniques and embodiments described in the sections to those sections only.

[0043] To improve video compression rates, researchers are constantly searching for new technologies for encoding video.

[0044] 1. Introduction

[0045] This patent document relates to video / image coding technology. Specifically, it concerns sub-block-based prediction in video / image coding. It can be applied to existing video coding standards (such as HEVC) or to standards that will eventually be finalized (Multi-Functional Video Coding). It can also be applied to future video / image coding standards or video / image codecs.

[0046] Brief discussion

[0047] Sub-block-based prediction was first introduced into video coding standards by HEVC Annex I (3D-HEVC). Using sub-block-based prediction, blocks such as coding units (CUs) or prediction units (PUs) are divided into several non-overlapping sub-blocks. Different motion information, such as reference indices or motion vectors (MVs), can be assigned to different sub-blocks, and motion compensation (MC) is performed separately for each sub-block. Figure 1 The concept of sub-block-based prediction is illustrated.

[0048] To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into a reference software called the Joint Exploration Model (JEM).

[0049] In JEM, sub-block-based prediction is used for several coding tools, such as affine prediction, optional temporal motion vector prediction (ATMVP), space-time motion vector prediction (STMVP), bidirectional optical flow (BIO), and frame rate upconversion (FRUC).

[0050] 2.1. Affine Prediction

[0051] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, in the real world, many types of motion exist, such as zooming in / out, rotation, perspective motion, and other irregular motions. In JEM, a simplified affine transformation motion compensation prediction is applied. For example... Figure 2 As shown, the affine motion field of the block is described by two control point motion vectors.

[0052] The motion vector field (MVF) of the block is described by the following equation:

[0053]

[0054] Where (v 0x ,v 0y (v) is the motion vector of the left top corner control point. 1x ,v 1y ) is the motion vector of the right top corner control point.

[0055] To further simplify motion compensation prediction, a sub-block-based affine transformation prediction is applied. The sub-block size M×N is derived from equation (2), where MvPre is the fractional precision of the motion vector (1 / 16 in JEM), (v 2x ,v 2y ) is the motion vector of the lower left control point, calculated according to equation (1).

[0056]

[0057] After deriving from equation (2), M and N should be adjusted downwards if necessary so that they are the divisors of w and h, respectively.

[0058] like Figure 3 As shown, in order to derive the motion vector of each M×N sub-block, the motion vector of the center sample point of each sub-block is calculated according to equation (1) and rounded to 1 / 16 fractional precision. Then, a motion-compensated interpolation filter is applied to generate a prediction for each sub-block using the derived motion vector.

[0059] After MCP, the high-precision motion vector of each sub-block is rounded and saved with the same precision as the normal motion vector.

[0060] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. AF_INTER mode can be applied to CUs with both width and height greater than 8. Signaling in the bitstream informs the CU-level affine flag to indicate whether AF_INTER mode is used. In this mode, neighboring blocks are used to construct motion vector pairs {(v0,v1)|v0={v...}. A ,v B ,v C},v1={v D ,v E The candidate list. For example... Figure 4 As shown, v0 is selected from the motion vectors of blocks A, B, or C. Motion vectors from neighboring blocks are scaled according to a reference list and the relationship between the POC of the neighboring blocks' references, the POC of the current CU's references, and the POC of the current CU. The method for selecting v1 from neighboring blocks D and E is similar. If the number of candidates in the candidate list is less than two, the list is populated by motion vector pairs constructed by repeating each AMVP candidate. When the candidate list is greater than two, candidates are first classified based on the consistency of neighboring motion vectors (the similarity between the two motion vectors in the candidate pair), and only the top two candidates are retained. An RD cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) for the current CU. Furthermore, the signaling notification in the bitstream indicates the index of the CPMVP position in the candidate list. After determining the CPMVP of the current affine CU, affine motion estimation is applied, and the control point motion vector (CPMV) is found. The difference between the CPMV and the CPMVP is then signaled in the bitstream.

[0061] When CU is applied in AF_MERGE mode, it obtains the first block encoded using affine mode from the reconstructed blocks of valid neighbors. For example... Figure 5A As shown, the selection order for candidate blocks is from left, top, upper right, lower left to upper left. Figure 5BAs shown, if the neighboring lower-left block A is encoded in affine mode, the motion vectors v2, v3, and v4 of the CU containing block A are derived for its top-left, top-right, and bottom-left corners. Then, the motion vector v0 of the current CU's top-left corner is calculated based on v2, v3, and v4. Next, the motion vector v1 of the current CU's top-right corner is calculated.

[0062] After deriving the CPMVv0 and v1 of the current CU, the MVF of the current CU is generated according to the simplified affine motion model equation (1). In order to identify whether the current CU is encoded using the AF_MERGE mode, an affine flag is signaled in the bitstream when at least one neighboring block is encoded in affine mode.

[0063] 2.2.ATMVP

[0064] In the Optional Temporal Motion Vector Prediction (ATMVP) method, the Temporal Motion Vector Prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. For example... Figure 6 As shown, the sub-CU is a square N×N block (N is set to 4 by default).

[0065] ATMVP predicts motion vectors for sub-CUs within a CU in two steps. The first step is to identify corresponding blocks in a reference image using so-called temporal vectors. This reference image is called the motion source image. The second step is to divide the current CU into sub-CUs and obtain the motion vectors from the blocks corresponding to each sub-CU, along with the reference index for each sub-CU, such as... Figure 6 As shown.

[0066] In the first step, reference images and corresponding blocks are determined using motion information from the spatial neighboring blocks of the current CU. To avoid repeated scanning of neighboring blocks, the first merge candidate in the current CU's merge candidate list is used. The first available motion vector and its associated reference index are set to the index of the time vector and the motion source image. In this way, in ATMVP, corresponding blocks can be identified more accurately than in TMVP, where the corresponding block (sometimes called a collocated block) is always located in the lower right or center position relative to the current CU.

[0067] In the second step, the corresponding block of the sub-CU is identified by adding a time vector to the coordinates of the current CU, using the time vector in the motion source image. For each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample) is used to derive the sub-CU's motion information. After identifying the motion information of the corresponding N×N block, it is converted into the reference index and motion vector of the current sub-CU in the same way as the TMVP of HEVC, where motion scaling and other processes are also applied. For example, the decoder checks whether the low-latency condition is met (i.e., the POC of all reference images of the current image is less than the POC of the current image) and may use the motion vector MV. x (The motion vector corresponding to the reference image list X) is used to predict the motion vector MV of each sub-CU. y (where X equals 0 or 1 and Y equals 1-X).

[0068] 3.STMVP

[0069] In this method, the motion vector of the sub-CU is recursively derived according to the raster scan sequence. Figure 7 This concept is illustrated. Let's consider an 8×8 CU containing four 4×4 sub-CUs A, B, C, and D. The neighboring 4×4 blocks in the current frame are labeled a, b, c, and d.

[0070] Motion derivation for sub-CU A begins by identifying its two spatial neighbors. The first neighbor is the N×N block (block c) above sub-CU A. If block c is unavailable or intra-coded, the other N×N blocks above sub-CU A are checked (starting from block c, from left to right). The second neighbor is the block (block b) to the left of sub-CU A. If block b is unavailable or intra-coded, the other blocks to the left of sub-CU A are checked (starting from block b, from top to bottom). Motion information obtained from neighboring blocks in each list is scaled to the first reference frame for the given list. Next, the Temporal Motion Vector Predictor (TMVP) for sub-block A is derived by following the same procedure as the TMVP derivation specified in HEVC. Motion information for the co-occurring block at position D is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The averaged motion vector is designated as the motion vector for the current sub-CU.

[0071] 4. BIO

[0072] Bidirectional optical flow (BIO) is a per-sample motion refinement performed on top of block-by-block motion compensation used for bidirectional prediction. Sample-level motion refinement does not use signaling notification.

[0073] Let I (k)The brightness value of reference k (k=0,1) after block motion compensation, and I (k) The horizontal and vertical components of the gradient. Assuming optical flow is effective, the motion vector field (v) x ,v y The following formula is given:

[0074]

[0075] Combining this optical flow equation with the Hermitian interpolation of each sample's motion trajectory yields a unique third-order polynomial, which ultimately matches the function value I. (k) and its derivative Both. The value of the third-order polynomial at t=0 is the BIO prediction:

[0076]

[0077] Here, τ0 and τ1 represent the distances to the reference frame, such as... Figure 8 As shown. Distances τ0 and τ1 are calculated based on the POC of Ref0 and Ref1: τ0 = POC(current) - POC(Ref0), τ1 = POC(Ref1) - POC(current). If both predictions originate from the same time direction (both from the past or both from the future), then the sign is different (i.e., τ0·τ1 < 0). In this case, BIO is applied only when the predictions do not originate from the same time (i.e., τ0 ≠ τ1), both reference regions have non-zero motion (MVx0, MVy0, MVx1, MVy1 ≠ 0), and the block motion vector is proportional to the time distance (MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0078] By minimizing points A and B ( Figure 9A and 9B The difference Δ between the values ​​in the motion trajectory and the intersection of the reference frame plane is used to determine the motion vector field (v). x ,v y The model uses only the first linear term of the local Taylor expansion of Δ:

[0079]

[0080] All values ​​in the above equations depend on the sample location (i′, j′), which has been omitted from the notation so far. Assuming the motion is consistent in the local surrounding region, we minimize Δ within a square window Ω of (2M+1)×(2M+1) centered at the current prediction point, where M equals 2:

[0081]

[0082] For this optimization problem, JEM uses a simplification method, first minimizing in the vertical direction and then minimizing in the horizontal direction. This results in:

[0083]

[0084]

[0085] in,

[0086]

[0087] To avoid division by zero or very small values, regularization parameters r and m are introduced in equations (7) and (8).

[0088] r = 500·4 d-8 (10)

[0089] m = 700·4 d-8 (11)

[0090] Here, d is the bit depth of the video sample.

[0091] To ensure that BIO memory access remains the same as regular bidirectional predictive motion compensation, all predicted and gradient values ​​I are computed only for the position within the current block. (k) , In equation (9), a (2M+1)×(2M+1) square window Ω centered on the current prediction point on the boundary of the prediction block needs to access locations outside the block (e.g., Figure 9A (As shown). In JEM, the I outside the block... (k) , The value is set to be equal to the nearest available value within the block. For example, this can be implemented as padding, such as... Figure 9B As shown.

[0092] Using BIO, the motion field can be refined for each sample. To reduce computational complexity, a block-based BIO design is used in JEM. Motion refinement is calculated based on 4×4 blocks. In block-based BIO, the s in equation (9) of all samples in the 4×4 block are aggregated. n The value of s, then s n The aggregated values ​​are used to derive the BIO motion vector offset for a 4×4 block. More specifically, the following formula is used for block-based BIO derivation:

[0093]

[0094] Where b k Let represent the sample set belonging to the k-th 4×4 block of the prediction block. Then, in equations (7) and (8), s... nReplace with ((s) n,bk )>>4), to derive the associated motion vector offset.

[0095] In some cases, the MV regiment of BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the size of the MV regiment is truncated by a threshold thBIO. The threshold is determined based on whether all reference images of the current image come from the same direction. If all reference images of the current image come from the same direction, the threshold value is set to 12×2. 14 -d Otherwise, set it to 12×2 13-d .

[0096] The gradient of the block inversion (BIO) is simultaneously computed using motion-compensated interpolation with operations consistent with the HEVC motion compensation process (2D separable FIR). The input to this 2D separable FIR is a reference frame sample with the same motion compensation process and fractional positions (fracX, fracY) as the fractional part of the block motion vector. The gradient at the horizontal level is calculated... In this case, the vertical interpolation signal is first used using BIOfilterS corresponding to the fractional position fracY with a descaling offset of d-8, and then a gradient filter BIOfilterG is applied in the horizontal direction, corresponding to the fractional position fracX with a descaling offset of 18-d. In the vertical gradient... In this case, a gradient filter is first applied vertically using BIOfilterG, corresponding to the fractional position fracY with a descaling offset of d-8. Then, signal displacement is performed horizontally using BIOfilterS, corresponding to the fractional position fracX with a descaling offset of 18-d. The interpolation filter BIOfilterG for gradient calculation and the interpolation filter BIOfilterS for signal displacement are relatively short (6 taps) to maintain reasonable complexity. Table 1 shows the filters used for gradient calculation at different fractional positions of the block motion vector in BIO. Table 2 shows the interpolation filters used for predictive signal generation in BIO.

[0097] Table 1 shows the filters used for gradient calculation in BIO.

[0098]

[0099]

[0100] Table 2 shows the interpolation filters used in BIO for predictive signal generation.

[0101] Fractional pixel position Interpolation filters for predicted signals (BIOfilters) 0 {0,0,64,0,0,0} 1 / 16 {1,-3,64,4,-2,0} 1 / 8 {1,-6,62,9,-3,1} 3 / 16 {2,-8,60,14,-5,1} 1 / 4 {2,-9,57,19,-7,2} 5 / 16 {3,-10,53,24,-8,2} 3 / 8 {3,-11,50,29,-9,2} 7 / 16 {3,-11,44,35,-10,3} 1 / 2 {3,-10,35,44,-11,3}

[0102] In JEM, BIO is applied to all bidirectional prediction blocks when two predictions come from different reference images. BIO is disabled when LIC is enabled for the CU.

[0103] In JEM, OBMC is applied to blocks after the normal MC process. To reduce computational complexity, BIO is not applied during OBMC. This means that BIO is only applied to the block's MC process when its own MV is used, and not during OBMC when the MV of a neighboring block is used.

[0104] 2.5.FRUC

[0105] When the Merge flag of the CU is true, the FRUC flag is signaled to that CU. When the FRUC flag is false, the Merge index is signaled, and the regular Merge mode is used. When the FRUC flag is true, the additional FRUC mode flag is signaled to indicate which method (bilateral matching or template matching) will be used to derive the block's motion information.

[0106] On the encoder side, the decision on whether to use the FRUC Merge mode for the CU is based on the RD cost selection, as done for normal Merge candidates. In other words, the two matching modes of the CU (bilateral matching and template matching) are validated using RD cost selection. The matching mode that results in the minimum cost is further compared with other CU modes. If the FRUC matching mode is the most efficient, the FRUC flag is set to true for the CU, and the relevant matching mode is used.

[0107] The motion derivation process in the FRUC Merge mode involves two steps. First, a CU-level motion search is performed, followed by sub-CU-level motion refinement. At the CU level, initial motion vectors are derived for the entire CU based on bilateral matching or template matching. First, a candidate MV list is generated, and the candidate resulting in the minimum matching cost is selected as the starting point for further CU-level refinement. Then, a local search based on bilateral matching or template matching is performed around the starting point, and the MV resulting in the minimum matching cost is taken as the MV for the entire CU. Subsequently, the motion information is further refined at the sub-CU level, with the derived CU motion vectors serving as the starting point.

[0108] For example, the following derivation process is performed for the derivation of motion information for W×HCU. In the first stage, the MV of the entire W×HCU is derived. In the second stage, the CU is further divided into M×M sub-CUs. The value of M is calculated as in (13), where D is a predefined partitioning depth, which is set to 3 by default in JEM.

[0109] Then export the MV for each sub-CU.

[0110]

[0111] like Figure 10 As shown, bilateral matching is used to derive the motion information of the current CU by finding the closest match between two blocks along the current CU's motion trajectory in two different reference images. Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 pointing to the two reference blocks should be proportional to the temporal distances between the current image and the two reference images—i.e., TD0 and TD1. As a special case, when the current image is temporally between the two reference images and the temporal distances from the current image to the two reference images are the same, bilateral matching becomes a mirror-based bidirectional MV.

[0112] like Figure 11 As shown, template matching is used to derive the motion information of the current CU by finding the closest match between a template in the current image (the top neighbor block and / or left neighbor block of the current CU) and a block in the reference image (having the same size as the template). In addition to the FRUC Merge mode described above, template matching also applies to the AMVP mode. In JEM, as in HEVC, AMVP has two candidates. Using the template matching method, new candidates are derived. If the newly derived candidate by template matching is different from the first existing AMVP candidate, it is inserted at the beginning of the AMVP candidate list, and then the list size is set to 2 (meaning the second existing AMVP candidate is removed). When applied to the AMVP mode, only the CU-level search is applied.

[0113] CU-level MV candidate set

[0114] The candidate set of MVs at the CU level consists of the following:

[0115] (i) If the current CU is in AMVP mode, then it is an original AMVP candidate.

[0116] (ii) All Merge candidates

[0117] (iii) Interpolating several MVs in the MV domain (described later),

[0118] (iv) Motion vectors of the top and left adjacent sides.

[0119] When using bilateral matching, each valid MV of the Merge candidate is used as input to generate MV pairs under the assumption of bilateral matching. For example, a valid MV of the Merge candidate is (MVa, refa) in reference list A. Then, a reference image refb for its paired bilateral MV is found in another reference list B, such that refa and refb are on different sides of the current image in time. If such a refb is not available in reference list B, then refb is determined to be a different reference from refa, and the temporal distance of refb to the current image is the minimum in list B. After determining refb, MVb is derived by scaling MVa based on the temporal distance between the current image and refa and refb.

[0120] Four MVs from the interpolated MV domain are also added to the CU-level candidate list. More specifically, the interpolated MVs at the current CU positions (0,0), (W / 2,0), (0,H / 2), and (W / 2,H / 2) are added.

[0121] When FRUC is applied to the AMVP pattern, the original AMVP candidates are also added to the CU-level MV candidate set.

[0122] At the CU level, up to 15 MVs for AMVP CU and up to 13 MVs for Merge CU are added to the candidate list.

[0123] Sub-CU level MV candidate set

[0124] The candidate set of MVs at the sub-CU level consists of the following:

[0125] (i) Search for the determined MV at the CU level.

[0126] (ii) The adjacent MVs of the top, left, top left, and top right,

[0127] (iii) A scaled version of the juxtaposed MV from the reference image.

[0128] (iv) A maximum of 4 ATMVP candidates

[0129] (v) A maximum of 4 STMVP candidates.

[0130] The scaled MV from the reference image is derived as follows. All reference images in both lists are iterated over. The MV at the side-by-side position of a sub-CU in the reference image is scaled to the reference of the starting CU-level MV.

[0131] The ATMVP and STMVP candidates are limited to the top four.

[0132] At the sub-CU level, up to 17 MVs are added to the candidate list.

[0133] Generation of interpolated MV domain

[0134] Before encoding the frames, an interpolated motion domain is generated for the entire image based on a one-sided ME. The motion domain can then be used later as a CU-level or sub-CU-level MV candidate.

[0135] First, the motion domain of each reference image in both reference lists is traversed at a 4×4 block level. For each 4×4 block, if the motion associated with the block passes through the 4×4 blocks in the current image (e.g., ...), ... Figure 12 (As shown) and the block has not yet been assigned any interpolated motion, the motion of the reference block is scaled to the current image according to the time distances TD0 and TD1 (in the same way that the MV is scaled in TMVP in HEVC), and the scaled motion is assigned to the block in the current frame. If an unscaled MV is assigned to a 4×4 block, the motion of the block is marked as unavailable in the interpolated motion domain.

[0136] Interpolation and matching costs

[0137] When the motion vector points to the fractional sample location, motion-compensated interpolation is required. To reduce complexity, both bilateral matching and template matching use bilinear interpolation instead of the conventional 8-tap HEVC interpolation.

[0138] The calculation of matching cost differs slightly at different steps. When selecting candidates from the candidate set at the CU level, the matching cost is the sum of absolute differences (SAD) between bilateral matches or template matches. After determining the starting MV, the matching cost C for bilateral matches in the sub-CU level search is calculated as follows:

[0139]

[0140] Where w is a weighting factor, and is empirically set to 4, MV and MV s These indicate the current MV and the starting MV, respectively. SAD is still used as the matching cost for template matching in sub-CU level searches.

[0141] In FRUC mode, the motion signature (MV) is derived using only luma samples. The derived motion is used for luma and chroma prediction in inter-frame MC. After determining the MV, the final MC is performed using an 8-tap interpolation filter for luma and a 4-tap interpolation filter for chroma.

[0142] MV refinement

[0143] MV refinement is a pattern-based MV search based on bilateral matching cost or template matching cost. JEM supports two search modes: unrestricted center-biased diamond search (UCBDS) and adaptive cross search, respectively, for MV refinement at the CU level and sub-CU level. For MV refinement at both the CU and sub-CU levels, the MV is directly searched with 1 / 4 luminance sample MV precision, followed by refinement with 1 / 8 luminance sample MV precision. The search range for MV refinement at both the CU and sub-CU steps is set to equal 8 luminance samples.

[0144] Selection of prediction direction in template matching FRUC Merge pattern

[0145] In the bilateral matching merge mode, bidirectional prediction is always applied because the motion information of the CU is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference images. The template matching merge mode does not have this limitation. In the template matching merge mode, the encoder can choose between unidirectional prediction from list 0, unidirectional prediction from list 1, or bidirectional prediction for the CU. The choice is based on the template matching cost, as follows:

[0146] If costBi <= factor * min(cost0, cost1)

[0147] Use two-way forecasting;

[0148] Otherwise, if cost0 <= cost1

[0149] Use one-way prediction from list 0;

[0150] otherwise,

[0151] Use one-way prediction from List 1;

[0152] Where cost0 is the SAD matching of template 0, cost1 is the SAD matching of template 1, and costBi is the SAD matching of template 1 for bidirectional prediction. The value of factor is 1.25, which means that the selection process is biased towards bidirectional prediction.

[0153] Inter-frame prediction direction selection is only applied to the CU-level template matching process.

[0154] Interleaved Prediction Example

[0155] For interleaved prediction, a block is divided into sub-blocks using more than one partitioning pattern. A partitioning pattern is defined as the way a block is divided into sub-blocks, including the size and position of the sub-blocks. For each partitioning pattern, the motion information of each sub-block can be derived from the partitioning pattern to generate the corresponding prediction block. Therefore, even for a single prediction direction, multiple prediction blocks can be generated using multiple partitioning patterns. Optionally, only one partitioning pattern can be applied for each prediction direction.

[0156] Suppose there are X partitioning patterns, and using these X partitioning patterns, X predicted blocks, denoted as P0, P1, ..., P, are generated for the current block through sub-block-based prediction. X-1 The final prediction for the current block can be generated using the following formula, denoted as P.

[0157]

[0158] Where (x, y) are the coordinates of the pixels in the block, w i (x,y) is P i The weighted values. Without loss of generalization, assume... Where N is a non-negative value. Figure 13 An example of interleaved prediction for two partitioning patterns is shown.

[0159] 3. Example problems solved by the described embodiments

[0160] like Figure 5A and Figure 5B The affine Merge MV derivation process shown has two potential drawbacks.

[0161] First, the coordinates of the left vertex of the CU and the size of the CU must be stored by each 4×4 block belonging to that CU. This information does not need to be stored in HEVC.

[0162] Secondly, the decoder must access the MV of the 4×4 block that is not adjacent to the current CU. In HEVC, the decoder only needs to access the MV of the 4×4 block that is adjacent to the current CU.

[0163] 4. Example Implementation

[0164] We propose several methods to further improve sub-block-based prediction, including interleaved prediction and affine MergeMV derivation.

[0165] The techniques and embodiments listed below should be considered as examples used to explain general concepts. Furthermore, these techniques can be combined to operate together during the video encoding process or the corresponding decoding process. Note that in this document, the term "encoding" includes "encoding conversion," where an uncompressed source video is encoded into another encoding format.

[0166] MV derivation of sub-blocks

[0167] 1. In one embodiment, the MV of the sub-block is derived with respect to the center of the sub-block.

[0168] a. Optional, derive the MV of the sub-block for any position within the sub-block, which may not be at the center of the sub-block.

[0169] b. Optionally, the location where the MV is derived can be different for each sub-block. (This location depends on each sub-block.)

[0170] c. It can be deduced that the position of MV can depend on the position of its sub-blocks. Figure 14 An example is shown.

[0171] d. Let MxN represent the size of the sub-block, where the center position can be defined as ((M>>1)+a)x((N>>1)+b), where a and b can be 0 or -1.

[0172] Figure 14 Examples of different positions for deriving the MV for different sub-blocks are shown. Stars represent positions. It can be seen that the MV derivation can use a variety of different positions.

[0173] Derivation of the efficient affine Merge MV

[0174] 2. In one embodiment, during the affine Merge MV derivation, the MVs at control points (such as mv0 at the left vertex and mv1 at the right vertex) are derived using only information from adjacent neighboring blocks. In one example, using the affine Merge pattern, deriving the MV of the current CU does not require the coordinates of the left vertex, the size of neighboring CUs, or the MVs of 4×4 blocks that are not adjacent to the current CU.

[0175] a. In one embodiment, affine parameters (such as the four-parameter affine modes a, b, c and d in equation (1)) are stored in each block encoded using affine modes (including affine inter-frame mode and affine Merge mode).

[0176] i. If a block is encoded using the affine Merge pattern, it inherits four parameters from neighboring blocks that are encoded using the affine pattern.

[0177] ii. In one example, the four arguments of list 0 and list 1 are different.

[0178] iii. In one example, parameters for two lists of reference images can be stored. Optionally, even for bidirectional prediction, only one set of affine parameters can be stored. Optionally, for multiple hypotheses, two sets of affine parameters can be stored, with each set corresponding to a list of reference images for bidirectional prediction.

[0179] b. In one embodiment, only a portion of the set of affine parameters (e.g., two parameters (a and b) in the four-parameter affine mode of equation (1)) is stored in each block encoded using affine modes (including affine inter-frame mode and affine Merge mode). If a block is encoded using affine Merge mode, it inherits a portion of the stored parameters from neighboring blocks encoded using affine modes.

[0180] i. In one example, different reference images or a different list of reference images can store all the relevant partial affine parameters.

[0181] ii. The two arguments for list 0 and list 1 are different.

[0182] c. In one embodiment, v in equation (1) 0x and v 0y (Also denoted as c and d) are derived from the block adjacent to the top left corner of the current block. In the following example, it is assumed that the current block is merged into a neighboring block G encoded using an affine pattern.

[0183] i. In one example, such as Figure 15 The three neighboring blocks R, S, and T shown are used to derive (v0x, v0y). The MVs in the three blocks are labeled as MV(R), MV(S), and MV(T).

[0184] (a) In one example, if X is inter-frame coded, then (v 0x ,v 0y Set it to be equal to MV(X) (X can be R, S or T).

[0185] (b) In one example, if R, S, and T are inter-frame coded, then (v 0x ,v 0y ) is set to be equal to the average of MV(R), MV(S), and MV(T).

[0186] (c) In one example, if X and Y are inter-frame coded, then (v 0x ,v 0y ) is set to be equal to the average of MV(X) and MV(Y) (X and Y can be R, S or T).

[0187] (d) In one example, (v) 0x ,v 0y) is set to be equal to MV(X) and MV(X) should refer to the same reference as block G.

[0188] ii. In one example, (v0x, v0y) is derived from the MV of the temporally neighboring block.

[0189] iii. In one example, (v0x, v0y) is scaled to the reference of block G.

[0190] d. In one embodiment, the MV of a block encoded using the affine Merge pattern is derived from the S left adjacent blocks encoded using the affine pattern (S=2 for a four-parameter affine pattern and S=3 for a six-parameter affine pattern). Figure 16A An example is shown. L0 and L1 are two left adjacent blocks encoded using affine patterns. Δ is the distance between the two left adjacent blocks. The motion vectors of the two blocks are (mvL0) and (mvL1) respectively. x ,mvL0 y ) and (mvL1 x mvL1 y (mv0) x mv0 y ) is the MV at the left top control point of the current block (also called (v) in equation (1) 0x ,v 0y The y-distance between one of the two blocks (e.g., L0) and the top left control point is denoted as Φ. Note that the distance can be measured from the top, middle, or bottom of the block. Figure 16A In the middle, this distance is measured from the bottom.

[0191] i. In one example, a and b in equation (1) can be derived as

[0192] a=(mvL1 y -mvL0 y ) / Δ,b=-(mvL1 x -mvL0 x ) / Δ.

[0193] ii. Δ can be a fixed number.

[0194] (a) Δ can be 2 N The numbers can take the form of 1, 4, 8, 16, etc. In this case, the division operation used to calculate a and b above can be implemented as a shift operation.

[0195] iii. Δ can be a number that depends on the height of the block.

[0196] iv. Δ can be derived to satisfy the following maximum length: all left adjacent blocks between L0 and L1 (inclusive) are encoded using affine patterns and share the same reference picture.

[0197] v.(mv0 x mv0 y This can be deduced as

[0198] mv0 x =mvL0 x +bΦ,mv0 y =mvL0 y -aΦ.

[0199] vi. If Φ is the y-distance between L1 and the left top control point, then (mv0) x mv0 y This can be deduced as

[0200] mv0 x =mvL1 x +bΦ,mv0 y =mvL1 y -aΦ.

[0201] e. In one embodiment, the MV of a block encoded using the affine Merge pattern is derived from the top neighboring blocks encoded using the affine pattern (S=2 for four-parameter affine pattern, S=3 for six-parameter affine pattern). Figure 16B An example is shown. T0 and T1 are two top-neighbor blocks encoded using an affine pattern. Δ is the distance between the two top-neighbor blocks. The motion vectors of the two blocks are (mvT0) and (mvT1) respectively. x ,mvT0 y ) and (mvT1 x mvT1 y (mv0) x mv0 y ) is the MV at the left top control point of the current block (also called (v) in equation (1) 0x ,v 0y The x-distance between one of the two blocks (e.g., T0) and the top left control point is denoted as Φ. Note that the distance can be measured from the left, middle, or right of the block. Figure 16B In the middle, this distance is measured from the right.

[0202] i. In one example, a and b in equation (1) can be derived as

[0203] a=(mvT1 x -mvT0 x ) / Δ,b=(mvT1 y -mvT0 y ) / Δ.

[0204] ii. Δ can be a fixed number.

[0205] (a) Δ can be 2N in the form of, such as 1, 4, 8, 16, etc. In this case, the division operation for calculating the above a and b can be implemented as a shift operation.

[0206] iii. Δ can be a number depending on the height of the block.

[0207] iv. Δ can be derived as the maximum length that satisfies the following: all top adjacent blocks between T0 and T1 (including T0 and T1) are encoded using the affine mode and share the same reference picture

[0208] v. (mv0 x , mv0 y ) can be derived as

[0209] mv0 x = mvT0 x - aΦ, mv0 y = mvT0 y - bΦ.

[0210] vi. If Φ is the x distance between T1 and the left top control point, then (mv0 x , mv0 y ) can be derived as

[0211] mv0 x = mvT1 x - aΦ, mv0 y = mvT1 y - bΦ.

[0212] Figure 16A and Figure 16B show examples of deriving the MV of the affine Merge mode from the left adjacent block ( Figure 16A ) or from the top adjacent block ( Figure 16B ) encoded using the affine mode.

[0213] f. The MV of a block encoded using the affine Merge mode can be derived from non - adjacent blocks encoded using the affine mode.

[0214] g. Which adjacent blocks are used to derive the MV of a block encoded using the affine Merge mode can depend on the block shape.

[0215] i. For a block with dimensions MxN and M > N, the MV of the block encoded using the affine Merge mode is derived from the top adjacent block encoded using the affine mode.

[0216] ii. For a block with dimensions MxN and M < N, the MV of the block encoded using the affine Merge mode is derived from the left adjacent block encoded using the affine mode.

[0217] iii. For a block of size MxN where M=N, the MV of the block encoded using the affine Merge pattern is derived from the block adjacent to the left top corner of the current block.

[0218] 3. In one embodiment, whether an affine merge candidate from a neighboring block is a valid affine merge candidate depends on the location of the neighboring block.

[0219] a. In one example, if a neighboring block belongs to a different CTU than the current Coding Tree Unit (CTU) (e.g., the largest CU (LCU)), then an affine Merge candidate from the neighboring block is considered invalid (and is not added to the Merge candidate list).

[0220] b. Optionally, if a neighboring block belongs to a different CTU line than the current CTU line, then an affine merge candidate from the neighboring block is considered invalid (and not added to the merge candidate list), such as... Figure 17 As shown.

[0221] c. Optionally, if a neighboring block belongs to a different stripe, an affine Merge candidate from the neighboring block is considered invalid (and is not added to the Merge candidate list).

[0222] d. Optionally, if a neighboring block belongs to a different tile, an affine Merge candidate from the neighboring block is considered invalid (and is not added to the Merge candidate list).

[0223] Figure 17 An example is shown where the neighboring block belongs to a different CTU row than the current block. In this example, if the neighboring block belongs to a different CTU row than the current CTU row, the affine merge candidate from the neighboring block is considered invalid (and is not added to the merge candidate list).

[0224] Examples of interleaved predictions

[0225] Figure 18 An example of interleaved prediction with two partitioning modes based on the disclosed technique is shown. The current block 1300 can be partitioned into multiple modes. For example, as... Figure 18 As shown, the current block is divided into mode 0 (1301) and mode 1 (1302). Two prediction blocks P0 (1303) and P1 (1304) are generated. By calculating the weighted sum of P0 (1303) and P1 (1304), the final prediction block P (1305) of the current block 1300 can be generated.

[0226] More generally, given X partition patterns, the X predicted blocks of the current block (denoted as P0, P1, ..., P2) X-1The prediction for the current block can be generated from sub-block-based predictions using X partitioning patterns. The final prediction (denoted as P) for the current block can be generated as follows:

[0227]

[0228] Here, (x, y) are the coordinates of the pixels in the block, and w i (x,y) is P i The weighting coefficients. By way of example rather than constraint, the weights can be expressed as:

[0229]

[0230] N is a non-negative value. Alternatively, the bit shift operation in equation (16) can also be expressed as:

[0231]

[0232] The sum of the weights is a power of 2. By performing a shift operation instead of floating-point division, the weighted sum P can be calculated more efficiently.

[0233] The partitioning pattern can have different sub-block shapes, sizes, or positions. In some embodiments, the partitioning pattern can include irregular sub-block sizes. Figures 19A-19G Examples of several partitioning patterns for 16×16 blocks are shown. Figure 19A In this approach, the block is divided into 4×4 sub-blocks according to the disclosed technique. This pattern is also used in JEM. Figure 19B An example of a partitioning pattern for dividing a block into 8×8 sub-blocks according to the disclosed technique is shown. Figure 19C An example of a partitioning pattern for dividing a block into 8×4 sub-blocks according to the disclosed technique is shown. Figure 19D An example of a partitioning pattern for dividing a block into 4×8 sub-blocks according to the disclosed technique is shown. Figure 19E In this method, a portion of a block is divided into 4×4 sub-blocks according to the disclosed technique. Pixels on the block boundaries are divided into smaller sub-blocks, such as 2×4, 4×2, or 2×2. Some sub-blocks can be merged to form larger sub-blocks. Figure 19F Examples of adjacent sub-blocks (such as 4×4 and 2×4 sub-blocks) are shown, which, when merged, form larger sub-blocks of size 6×4, 4×6, or 6×6. Figure 19G In this process, a portion of a block is divided into 8×8 sub-blocks. Pixels at the block boundaries are divided into smaller sub-blocks such as 8×4, 4×8, or 4×4.

[0234] In sub-block-based prediction, the shape and size of a sub-block can be determined based on the shape and / or size and / or information of the coded block. For example, in some embodiments, when the current block size is M×N, the sub-block size is 4×N (or 8×N, etc.), meaning the sub-block has the same height as the current block. In some embodiments, when the current block size is M×N, the sub-block size is M×4 (or M×8, etc.), meaning the sub-block has the same width as the current block. In some embodiments, when the current block size is M×N (where M>N), the sub-block size is A×B, where A>B (e.g., 8×4). Alternatively, the sub-block size is B×A (e.g., 4×8).

[0235] In some embodiments, the size of the current block is M×N. When M×N<=T (or min(M,N)<=T, or max(M,N)<=T, etc.), the size of the sub-block is A×B; when M×N>T (or min(M,N)>T, or max(M,N)>T, etc.), the size of the sub-block is C×D, where A<=C, B<=D. For example, if M×N<=256, the size of the sub-block can be 4×4. In some implementations, the size of the sub-block is 8×8.

[0236] In some embodiments, the application of interleaving prediction can be determined based on the direction of inter-frame prediction. For example, in some embodiments, interleaving prediction may be applicable to bidirectional prediction but not to unidirectional prediction. As another example, when multiple hypotheses are applied, interleaving prediction may be applied to a prediction direction when there is more than one reference block.

[0237] In some embodiments, the application of interleaving prediction can also be determined based on the inter-frame prediction direction. In some embodiments, for two different reference lists, a bidirectional prediction block based on sub-block prediction is divided into sub-blocks with two different partitioning modes. For example, when predicting from reference list 0 (L0), the bidirectional prediction block is divided into 4×8 sub-blocks, such as... Figure 19D As shown. When predicting from reference list 1 (L1), the same block is divided into 8×4 sub-blocks, as follows. Figure 19C As shown. The final prediction P is calculated as follows:

[0238]

[0239] Here, P 0 and P 1 These are the predicted values ​​from L0 and L1, respectively. 0 and w 1 These are the weighted values ​​from L0 and L1, respectively. As shown in equation (16), the weighted values ​​can be determined as: w 0 (x,y)+w 1(x, y) = 1 << N (where N is a non - negative integer value). Since fewer sub - blocks are used for prediction in each direction (e.g., 4×8 sub - blocks instead of 8×8 sub - blocks), the calculation requires less bandwidth compared to existing block - based methods. By using larger sub - blocks, the prediction results are also less susceptible to the influence of noise.

[0240] In some embodiments, for the same reference list, a unidirectional prediction block with block - based prediction is divided into sub - blocks with two or more different partitioning patterns. For example, for the prediction of list L (L = 0 or 1), P L is calculated as follows:

[0241]

[0242] Here, XL is the number of partitioning patterns of list L. is the prediction generated using the i - th partitioning pattern, and is the weighting value of. For example, when XL is 2, two partitioning patterns are applied to list L. In the first partitioning pattern, the block is divided into 4×8 sub - blocks as shown in Figure 19D and in the second partitioning pattern, the block is divided into 8×4 sub - blocks as shown in Figure 19C shown.

[0243] In some embodiments, a bidirectional prediction block based on block - based prediction is regarded as a combination of two unidirectional prediction blocks respectively from L0 and L1. The predictions from each list can be derived as described in the example above. The final prediction P can be calculated as:

[0244]

[0245] Here, the parameters a and b are two additional weights applied to the two internal prediction blocks. In this particular example, both a and b can be set to 1. Similar to the example above, since fewer sub - blocks are used for prediction in each direction (e.g., 4×8 sub - blocks instead of 8×8 sub - blocks), the bandwidth usage is better than or the same as existing block - based methods. At the same time, the prediction results can be improved by using larger sub - blocks.

[0246] In some embodiments, a separate non - uniform pattern can be used in each unidirectional prediction block. For example, for each list L (e.g., L0 or L1), the block is divided into different patterns (e.g., as shown in Figure 19E or Figure 19F shown). Using a smaller number of sub - blocks reduces the demand for bandwidth. The non - uniformity of the sub - blocks also increases the robustness of the prediction results.

[0247] In some embodiments, for a multi-hypothesis coding block, there can be multiple prediction blocks generated by different partitioning patterns for each prediction direction (or reference image list). Multiple prediction blocks can be used and additional weights applied to generate the final prediction. For example, the additional weights can be set to 1 / M, where M is the total number of prediction blocks generated.

[0248] In some embodiments, the encoder can determine whether and how to apply interleaving prediction. The encoder can then send information corresponding to this determination to the decoder at the sequence level, picture level, view level, stripe level, coding tree unit (CTU) (also known as maximum coding unit (LCU)) level, CU level, PU level, tree unit (TU) level, or region level (which may include multiple CUs / PUs / TUs / LCUs). This information can be signaled in the first block of the sequence parameter set (SPS), view parameter set (VPS), picture parameter set (PPS), stripe header (SH), CTU / LCU, CU, PU, ​​TU, or region.

[0249] In some implementations, interleaving prediction is applicable to existing sub-block methods, such as affine prediction, ATMVP, STMVP, FRUC, or BIO. In this case, no additional signaling cost is required. In some implementations, new sub-block merge candidates generated by interleaving prediction can be inserted into the merge list, such as interleaving prediction + ATMVP, interleaving prediction + STMVP, interleaving prediction + FRUC, etc.

[0250] In some embodiments, the partitioning pattern to be used for the current block can be derived based on information from spatially and / or temporally neighboring blocks. For example, both the encoder and decoder can employ a predetermined set of rules to obtain a partitioning pattern based on temporal adjacency (e.g., the partitioning pattern previously used in the same block) or spatial adjacency (e.g., the partitioning pattern used by neighboring blocks), rather than relying on the encoder to send relevant information.

[0251] In some embodiments, the weighting value w can be fixed. For example, all partitioning patterns can be weighted equally: w i (x,y) = 1. In some embodiments, the weighting value can be determined based on the block location and the partitioning pattern used. For example, for different (x,y) values, w i (x,y) may differ. In some embodiments, the weighting values ​​may further depend on the sub-block prediction-based coding technique (e.g., affine or ATMVP) and / or other coding information (e.g., skip or non-skip modes and / or MV information).

[0252] In some embodiments, the encoder may determine weighting values ​​and send these values ​​to the decoder at the sequence level, picture level, stripe level, CTU / LCU level, CU level, PU level, or region level (which may include multiple CUs / PUs / TUs / LCUs). The weighting values ​​may be signaled in the first block of the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Stripe Header (SH), CTU / LCU, CU, PU, ​​or region. In some embodiments, the weighting values ​​may be derived from the weighting values ​​of spatially and / or temporally neighboring blocks.

[0253] It should be noted that the interleaving prediction technique disclosed herein can be applied to one, some, or all of the sub-block prediction-based coding techniques. For example, the interleaving prediction technique can be applied to affine prediction, while other sub-block prediction-based coding techniques (e.g., ATMVP, STMVP, FRUC, or BIO) do not use interleaving prediction. As another example, all affine, ATMVP, and STMVP use the interleaving prediction technique disclosed herein.

[0254] Figure 20 This is a block diagram of an example video bitstream processing apparatus 2000. Apparatus 2000 can be used to implement one or more methods described herein. Apparatus 2000 can be implemented in smartphones, tablets, computers, Internet of Things (IoT) receivers, etc. Apparatus 2000 may include one or more processors 2002, one or more memories 2004, and video processing hardware 2006. Processor 2002 can be configured to implement one or more methods described herein. One or more memories 2004 can be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 2006 can be used to implement some of the techniques described herein in hardware circuitry. Note that some or all of the memory 2004 and circuitry 2006 external to the processor 2002 electronics is optional and is an implementation choice.

[0255] Figure 21 A flowchart of an example method 2100 for video processing is shown. Method 2100 includes dividing the current block (2102) into sub-blocks. Method 2100 further includes deriving (2104) motion vectors for each sub-block, wherein the motion vector of each sub-block is associated with the position of the sub-block according to a positional rule. Method 2100 further includes using the motion vectors of the sub-blocks to process the bitstream representation of the current block.

[0256] Figure 22This is a flowchart of an example method 2200 for video processing. Method 2200 includes deriving (2202) a motion vector at a control point of the current block based on a positional rule for a conversion between the current block and its bitstream representation using an affine pattern. Method 2200 further includes using the motion vector to perform (2204) a conversion between the current block and the bitstream representation. In some implementations, the positional rule specifies the exclusion of non-adjacent neighboring blocks for the derivation. In some implementations, the motion vector can be derived without using information about neighboring coding units, including at least one non-adjacent 4×4 block of the current block. In some implementations, the method further includes storing and reusing at least some affine parameters of the neighboring blocks from the previous conversion. In some implementations, the storage and reuse of at least some affine parameters can be performed in two separate steps.

[0257] Figure 23 This is a flowchart of an example method 2300 for video processing. Method 2300 includes determining an affine Merge candidate list for the conversion between the current block and its bitstream representation by including Merge candidates from one or more neighboring blocks that satisfy a validity criterion based on the positions of the one or more neighboring blocks. Method 2300 further includes performing (2304) the conversion between the current block and the bitstream representation using motion vectors.

[0258] The following uses a clause-based description format to describe additional features and embodiments of the above methods / techniques.

[0259] 1. A video processing method (e.g., Figure 21 The method shown in 2000 includes: dividing the current block into sub-blocks; deriving a motion vector for each sub-block, wherein the motion vector of each sub-block is associated with the position of the sub-block according to a position rule; and using the motion vector of the sub-blocks to process the bitstream representation of the current block.

[0260] 2. According to the method of Clause 1, wherein the position rule specifies that the position is the center of the corresponding sub-block.

[0261] 3. The method according to Clause 2, wherein the corresponding sub-block size is M x N and the center is defined as ((M>>1)+a)x((N>>1)+b), where M and N are natural numbers and a and b are 0 or -1.

[0262] 4. The method described in Clause 1, wherein the position rule specifies that the position is a non-center position of the corresponding sub-block.

[0263] 5. The method according to Clause 1, wherein the position specified by the position rule results in the derivation of motion vectors at different positions in different sub-blocks.

[0264] 6. A video processing method (e.g., Figure 22 The method 2200 shown includes: for a conversion between a current block using an affine mode and a bitstream representation of the current block, deriving a motion vector at a control point of the current block based on a position rule; and performing the conversion between the current block and the bitstream representation using the motion vector, wherein the position rule specifies that non-adjacent neighboring blocks are excluded from the derivation.

[0265] 7. The method according to Clause 6, wherein the motion vector is derived without using information from neighboring coding units of at least one non-adjacent 4×4 block including the current block.

[0266] 8. The method described in Clause 7 further includes:

[0267] Store and reuse at least some of the affine parameters of the neighboring blocks of the previous transformation.

[0268] 9. The method according to Clause 8, wherein the current block inherits at least some affine parameters from neighboring blocks encoded in affine mode.

[0269] 10. The method according to Clause 8, wherein for reference frames in List 0 and List 1, at least some of the affine parameters are different.

[0270] 11. The method according to Clause 8, wherein the at least some affine parameters comprise two sets, each set for one of a plurality of hypothetical reference images.

[0271] 12. The method according to any one of clauses 8 to 11, wherein the at least some affine parameters include two of four affine parameters.

[0272] 13. The method described in Clause 6, wherein the motion vector (v) of the left top corner of the current block 0x ,v 0y The current block is derived from the block adjacent to the left top corner of the current block, and the current block is merged into a neighboring block encoded using the affine pattern.

[0273] 14. The method described in Clause 13 further includes:

[0274] The motion vector (v) is derived using three neighboring blocks R, S, and T, each with a corresponding motion vector MV(R), MV(S), and MV(T). 0x ,v 0y ),and

[0275] If X is inter-frame coded, then the motion vector (v) 0x ,v 0y Set ) to be equal to MV(X), and X is R, S or T.

[0276] 15. The method described in Clause 13 further includes:

[0277] The motion vector (v) is derived using three neighboring blocks R, S, and T, each with a corresponding motion vector MV(R), MV(S), and MV(T). 0x ,v 0y ),and

[0278] If R, S, and T are inter-frame coded, then the motion vector (v) 0x ,v 0y ) is set to be equal to the average of MV(R), MV(S), and MV(T).

[0279] 16. The method described in Clause 13 further includes:

[0280] The motion vector (v) is derived using three neighboring blocks R, S, and T, each with a corresponding motion vector MV(R), MV(S), and MV(T). 0x ,v 0y ),and

[0281] If X and Y are inter-frame coded, then the motion vector (v) 0x ,v 0y ) is set to be equal to the average of MV(X) and MV(Y), and X and Y are R, S or T.

[0282] 17. The method according to Clause 13, wherein the motion vector (v) is derived from the motion vector of the temporally neighboring block. 0x ,v 0y ).

[0283] 18. The method according to Clause 13, wherein the motion vector (v) 0x ,v 0y The reference is scaled to the neighboring block.

[0284] 19. The method according to Clause 6, wherein the motion vector is derived from the left adjacent block encoded using the affine pattern.

[0285] 20. The method according to Clause 6, wherein the motion vector of the block encoded using the affine pattern is derived from the S top neighboring blocks encoded using the affine pattern, wherein S equals 2 for a four-parameter affine pattern.

[0286] 21. The method according to clause 20, wherein the distance between two top adjacent blocks is 2 N a fixed number of the form, and N is an integer.

[0287] 22. The method according to clause 20, wherein the distance between the top adjacent blocks depends on the height of the block encoded using the affine mode.

[0288] 23. The method according to clause 20, wherein the distance between the top adjacent blocks is derived as the maximum length that satisfies all top adjacent blocks being encoded using the affine mode and sharing the same reference picture.

[0289] 24. The method according to clause 20, wherein the motion vector (mv0 x , mv0 y ) at the left top control point of the current block is derived as i) mv0 x = mvT0 x - aΦ and mv0 y = mvT0 y - bΦ or ii) mv0 x = mvT1 x , mv0 y = mvT1 y - bΦ, where Φ is the distance between the left top control point and one of the two top adjacent blocks T0 and T1 encoded using the affine mode.

[0290] 25. The method according to clause 6, wherein the current block size is MxN pixels, where M and N are integers, and when M < N, the motion vector is derived from the left adjacent block.

[0291] 26. The method according to clause 6, wherein the current block size is MxN pixels, where M and N are integers, and when M > N, the motion vector is derived from the top adjacent block.

[0292] 27. The method according to clause 6, wherein the current block size is MxN pixels, where M and N are integers, and when M = N, the motion vector is derived from the block adjacent to the left top corner.

[0293] 28. A video processing method (e.g., Figure 23 the method 2300 shown in), comprising: for the conversion between the current block and the bitstream representation of the current block, determining an affine Merge candidate list for the conversion by including Merge candidates from one or more neighboring blocks that satisfy a validity criterion based on the positions of the one or more neighboring blocks; and performing the conversion between the current block and the bitstream representation using the motion vector.

[0294] 29. The method according to Clause 28, wherein the neighboring block comes from a CTU different from the current CTU (Coding Tree Unit), and wherein the affine Merge pattern candidate from the neighboring block is invalid.

[0295] 30. The method according to Clause 28, wherein the current CTU belongs to the current CTU row, wherein the neighboring block belongs to a different CTU row from the current CTU row, and wherein the affine Merge pattern candidate from the neighboring block is invalid.

[0296] 31. The method according to Clause 28, wherein the current block belongs to the current strip, wherein the neighboring block belongs to a strip different from the current strip, and wherein the affine Merge pattern candidate from the neighboring block is invalid.

[0297] 32. The method according to Clause 28, wherein the current block belongs to the current slice, wherein the neighboring block belongs to a slice different from the current slice, and wherein the affine Merge pattern candidate from the neighboring block is invalid.

[0298] 33. A video processing method, comprising:

[0299] For the conversion between the current block using affine mode and the bitstream representation of the current block, a motion vector at the control point of the current block is derived based on one or more adjacent blocks; and

[0300] The motion vector is used to perform the conversion between the current block and the bitstream representation.

[0301] 34. The method according to Clause 33, wherein the motion vector at the control point of the current block using the affine mode is inherited from one or more adjacent blocks above.

[0302] 35. The method according to Clause 33, wherein the adjacent neighbor blocks are also encoded by affine pattern.

[0303] 36. The method according to Clause 35, wherein the adjacent neighbor blocks are encoded using a 4-parameter affine pattern.

[0304] 37. The method according to Clause 36, wherein the motion vector at the control point of the current block using the affine mode is inherited from the bottom-left and bottom-right control point motion vectors of the one or more adjacent blocks.

[0305] 38. The method according to any one of the preceding clauses, wherein the transformation includes generating the bitstream representation from the current block.

[0306] 39. The method according to any one of the preceding clauses, wherein the transformation includes generating the current block from the bitstream representation.

[0307] 40. A video decoding apparatus comprising a processor configured to implement one or more of the methods described in clauses 1 to 39.

[0308] 41. A video encoding apparatus comprising a processor configured to implement one or more of the methods described in clauses 1 to 39.

[0309] 42. A computer-readable program medium having code stored thereon, the code including instructions that, when executed by a processor, cause the processor to perform one or more of the methods described in clauses 1 to 39.

[0310] As can be understood from the foregoing, specific embodiments of the technology disclosed herein have been described for illustrative purposes, but various modifications can be made without departing from the scope of the invention. Therefore, the technology disclosed herein is not limited except for the appended claims.

[0311] The disclosures and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in a combination of one or more of them. The disclosures and other embodiments can be implemented as one or more computer program products, i.e., one or more computer program instruction modules encoded on a computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances influencing machine-readable propagation signals, or a combination of one or more of them. The term "data processing apparatus" encompasses all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. Propagation signals are artificially generated signals, such as machine-generated electrical signals, optical signals, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[0312] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or code portions). Computer programs can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communication network.

[0313] The processes and logic flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by dedicated logic circuits, and the devices can be implemented as dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0314] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from, transfer data to, or both receive and transfer data from such mass storage devices. However, a computer does not need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0315] While this patent document contains numerous details, these details should not be construed as limiting any invention or the scope of the claims, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of separate embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. Furthermore, although the features described above may be described as functioning in certain combinations and even initially claimed in this way, in some cases, one or more features from a claimed combination may be removed from that combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0316] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or sequentially, or to perform all the shown operations to achieve the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0317] Only a few implementations and examples are described, and other implementations, enhancements and variations can be made based on what is described and shown in this patent document.

Claims

1. A video data processing method, comprising: For the conversion between the current video block and the bitstream of the video, a motion vector CPMV at the control point of the current video block is determined based on rules, wherein the rules specify the exclusion of non-adjacent neighboring blocks from one or more neighboring blocks of the current video block; The conversion between the current video block and the bitstream is performed based on the motion vector CPMV at the control point. The current video block contains multiple sub-blocks, and performing the conversion includes: Based on the CPMV and the specified position of the corresponding sub-block, a motion vector is determined for each of the plurality of sub-blocks; Round the motion vector of each sub-block to 1 / 16 fractional precision; The rounded motion vector of each sub-block is filtered using a filter to generate a prediction for each sub-block, wherein the maximum precision of the filter is 1 / 16 fraction of a pixel. The rule specifies that, for affine Merge mode, a CPMV candidate for the current video block is derived from the motion vector of the top-neighbor block encoded using the affine mode, disregarding the left-neighbor block encoded using the affine mode, and the CPMV candidate is used to derive the motion vector CPMV of the current video block. Where the current video block belongs to the current slice, and the spatial neighboring block in one or more neighboring blocks of the current video block belongs to a different slice from the current slice, the spatial neighboring block is invalid for determining the CPMV of the current video block.

2. The method of claim 1, wherein the rule further specifies excluding invalid neighbor blocks from the one or more neighbor blocks based on the location of the one or more neighbor blocks.

3. The method of claim 2, wherein if a neighboring block in one or more neighboring blocks originates from a CTU different from the current coding tree unit (CTU), the neighboring block is invalid.

4. The method of claim 2, wherein the current video block belongs to the current slice, and the neighboring block in the one or more neighboring blocks belongs to a slice different from the current slice, the neighboring block is invalid.

5. The method of claim 1, wherein the current CTU belongs to the current CTU row, and the neighboring block in the one or more neighboring blocks belongs to a CTU row different from the current CTU row, the neighboring block is invalid.

6. The method of claim 1, wherein the designated position is the center of the corresponding sub-block.

7. The method of claim 6, wherein the size of the corresponding sub-block is M x N and the center is defined as ((M>>1) + a) x ((N>>1) + b), where M and N are natural numbers, and a and b are 0 or -1.

8. The method of claim 6, wherein the designated position is the non-center position of the corresponding sub-block.

9. The method of claim 6, wherein the specified position results in the derivation of motion vectors at different positions in different sub-blocks.

10. The method according to claim 1, further comprising: Based on the position of the current video block, determine the CPMV candidate of the current video block derived from the motion vector of the top neighbor block encoded and decoded using the affine mode.

11. The method of claim 1, wherein the motion vector is derived without using information from neighboring coding units of at least one non-adjacent 4×4 block comprising the current video block.

12. The method of claim 11, further comprising: Store and reuse at least some of the affine parameters of the neighboring blocks of the previous transformation.

13. The method of claim 12, wherein the current video block inherits at least some affine parameters from neighboring blocks encoded and decoded in the affine mode.

14. The method of claim 12, wherein the at least some affine parameters are different for reference frames in list 0 and list 1.

15. The method of claim 12, wherein the at least some affine parameters comprise two sets, each set for one of a plurality of hypothetical reference images.

16. The method of claim 12, wherein the at least some affine parameters include two of four affine parameters.

17. The method of claim 11, wherein the motion vector (v0x, v0y) of the top left corner of the current video block is derived from the block adjacent to the top left corner of the current video block, and the current video block is merged into a neighboring block encoded and decoded using the affine mode.

18. The method of claim 17, further comprising: The motion vector (v0x, v0y) is derived using three neighboring blocks R, S, and T, each with a corresponding motion vector MV(R), MV(S), and MV(T). If X is inter-frame coded, then the motion vector (v0x, v0y) is set to be equal to MV(X), and X is R, S, or T.

19. The method of claim 17, further comprising: The motion vector (v0x, v0y) is derived using three neighboring blocks R, S, and T, each with a corresponding motion vector MV(R), MV(S), and MV(T). If R, S, and T are inter-frame coded, then the motion vector (v0x, v0y) is set to be equal to the average of MV(R), MV(S), and MV(T).

20. The method of claim 17, further comprising: The motion vector (v0x, v0y) is derived using three neighboring blocks R, S, and T, each with a corresponding motion vector MV(R), MV(S), and MV(T). If X and Y are inter-frame coded, then the motion vector (v0x, v0y) is set to be equal to the average of MV(X) and MV(Y), and X and Y are R, S, or T.

21. The method of claim 17, wherein the motion vector (v0x, v0y) is derived from the motion vector of the temporally neighboring block.

22. The method of claim 17, wherein the motion vector (v0x, v0y) is scaled to a reference of the neighboring block.

23. The method of claim 11, wherein the motion vector of the block encoded using the affine mode is derived from the S top neighboring blocks encoded using the affine mode, wherein S equals 2 for a four-parameter affine mode.

24. The method of claim 23, wherein the distance between two top adjacent blocks is a fixed number of the form 2N, where N is an integer.

25. The method of claim 23, wherein the distance between the top adjacent blocks depends on the height of the blocks encoded and decoded using the affine pattern.

26. The method of claim 23, wherein the distance between the top adjacent blocks is derived as the maximum length that satisfies all top adjacent blocks using the affine pattern for encoding and decoding and sharing the same reference image.

27. The method of claim 23, wherein the motion vector (mv0x, mv0y) at the top left control point of the current video block is derived as i) mv0x = mvT0x - aΦ and mv0y = mvT0y - bΦ or ii) mv0x = mvT1x - aΦ, mv0y = mvT1y - bΦ, Φ is the distance between the top left control point and one of the two top adjacent blocks T0 and T1 encoded and decoded using the affine mode, (mvT0x, mvT0y) represents the motion vector of T0, (mvT1x, mvT1y) represents the motion vector of T1, and a and b are affine parameters.

28. The method of claim 11, wherein the current video block size is M x N pixels, where M and N are integers, and the motion vector is derived from the top-side adjacent block when M > N.

29. The method of claim 11, wherein the current video block size is M x N pixels, where M and N are integers, and when M = N, the motion vector is derived from the block adjacent to the top left corner.

30. The method of claim 1, wherein the CPMV of the current video block using the affine mode is derived from one or more adjacent blocks above.

31. The method of claim 1, wherein the adjacent neighboring blocks are also encoded and decoded using the affine pattern.

32. The method of claim 31, wherein the adjacent neighbor blocks are encoded and decoded using a 4-parameter affine mode.

33. The method of claim 32, wherein the motion vector at the control point of the current video block using the affine mode is derived from the left bottom motion vector and right bottom motion vector of the one or more adjacent neighboring blocks.

34. The method according to any one of claims 1-33, wherein the conversion comprises encoding the current video block into the bitstream.

35. The method according to any one of claims 1-33, wherein the conversion comprises decoding the current video block from the bitstream.

36. A video data processing apparatus, comprising: A determination module is configured to determine, based on rules, a motion vector CPMV at a control point of the current video block for a conversion between the current video block and the bitstream of the video, wherein the rules specify the exclusion of non-adjacent neighboring blocks from one or more neighboring blocks of the current video block. The conversion module is used to perform conversion between the current video block and the bitstream based on the motion vector CPMV at the control point. The current video block contains multiple sub-blocks, and performing the conversion includes: Based on the CPMV and the specified position of the corresponding sub-block, a motion vector is determined for each of the plurality of sub-blocks; Round the motion vector of each sub-block to 1 / 16 fractional precision; The rounded motion vector of each sub-block is filtered using a filter to generate a prediction for each sub-block, wherein the maximum precision of the filter is 1 / 16 fraction of a pixel. The rule specifies that, for affine Merge mode, a CPMV candidate for the current video block is derived from the motion vector of the top-neighbor block encoded using the affine mode, disregarding the left-neighbor block encoded using the affine mode, and the CPMV candidate is used to derive the motion vector CPMV of the current video block. Where the current video block belongs to the current slice, and the spatial neighboring block in one or more neighboring blocks of the current video block belongs to a different slice from the current slice, the spatial neighboring block is invalid for determining the CPMV of the current video block.

37. The apparatus of claim 36, wherein the rule further specifies the exclusion of invalid neighboring blocks from the one or more neighboring blocks based on the location of the one or more neighboring blocks.

38. The apparatus of claim 37, wherein the current video block belongs to the current slice, and the neighboring block in the one or more neighboring blocks belongs to a slice different from the current slice, the neighboring block is invalid.

39. The apparatus of claim 36, wherein the designated position is the center of the corresponding sub-block.

40. The apparatus of claim 39, wherein the size of the corresponding sub-block is M x N and the center is defined as ((M>>1) + a) x ((N>>1) + b), where M and N are natural numbers and a and b are 0 or -1.

41. The apparatus of claim 36, further comprising: Based on the position of the current video block, determine CPMV candidates for the current video block that are to be derived from the motion vectors of the top neighboring block using the affine mode encoding and decoding.

42. A non-transitory computer-readable storage medium for storing instructions, said instructions causing a processor to: For the conversion between the current video block and the bitstream of the video, a motion vector CPMV at the control point of the current video block is determined based on rules, wherein the rules specify the exclusion of non-adjacent neighboring blocks from one or more neighboring blocks of the current video block; and The conversion between the current video block and the bitstream is performed based on the motion vector CPMV at the control point. The current video block contains multiple sub-blocks, and performing the conversion includes: Based on the CPMV and the specified position of the corresponding sub-block, a motion vector is determined for each of the plurality of sub-blocks; Round the motion vector of each sub-block to 1 / 16 fractional precision; The rounded motion vector of each sub-block is filtered using a filter to generate a prediction for each sub-block, wherein the maximum precision of the filter is 1 / 16 fraction of a pixel. The rule specifies that, for affine Merge mode, a CPMV candidate for the current video block is derived from the motion vector of the top-neighbor block encoded using the affine mode, disregarding the left-neighbor block encoded using the affine mode, and the CPMV candidate is used to derive the motion vector CPMV of the current video block. Where the current video block belongs to the current slice, and the spatial neighboring block in one or more neighboring blocks of the current video block belongs to a different slice from the current slice, the spatial neighboring block is invalid for determining the CPMV of the current video block.

43. A method for storing a video bitstream, the method comprising: The motion vector CPMV at the control point of the current video block is determined based on rules, wherein the rules specify the exclusion of non-adjacent neighboring blocks from one or more neighboring blocks of the current video block; as well as Based on determining that the bitstream is generated from the current video block; The bitstream is stored in a non-transitory computer-readable recording medium; The current video block contains multiple sub-blocks, and generating the bitstream includes: Based on the CPMV and the specified position of the corresponding sub-block, a motion vector is determined for each of the plurality of sub-blocks; Round the motion vector of each sub-block to 1 / 16 fractional precision; The rounded motion vector of each sub-block is filtered using a filter to generate a prediction for each sub-block, wherein the maximum precision of the filter is 1 / 16 fraction of a pixel. The rule specifies that, for affine Merge mode, a CPMV candidate for the current video block is derived from the motion vector of the top-neighbor block encoded using the affine mode, disregarding the left-neighbor block encoded using the affine mode, and the CPMV candidate is used to derive the motion vector CPMV of the current video block. Where the current video block belongs to the current slice, and the spatial neighboring block in one or more neighboring blocks of the current video block belongs to a different slice from the current slice, the spatial neighboring block is invalid for determining the CPMV of the current video block.

Images