History-based affine parameter inheritance
By introducing history-based motion vector prediction and affine model in video encoding and decoding technology, the problem of inefficiency in the existing technology when dealing with complex motion scenarios is solved, and more efficient video encoding and decoding and bandwidth usage efficiency is achieved.
Patent Information
- Application Number
- CN201980075765.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-02-22
- Filing Date
- 2019-11-18
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2039-11-18
AI Technical Summary
Existing video encoding and decoding technologies are inefficient in handling non-translational motions and cannot effectively utilize more codec modes to reduce bit rate, especially when dealing with complex motion scenarios, resulting in low video quality and bandwidth usage efficiency.
Using history-based motion vector prediction (HMVP) technology, affine model and control point motion vectors are used to improve the encoding and decoding of video blocks through affine prediction and advanced motion vector prediction (AMVP), including the construction of affine Merge candidate list and the derivation of motion vectors to improve the efficiency of video encoding and decoding.
It improves the efficiency and quality of video encoding and decoding, reduces the bit rate, and is suitable for HEVC and future video encoding and decoding standards, especially when dealing with complex motion scenarios, which significantly improves bandwidth usage efficiency.
Smart Images

Figure CN113170105B_ABST
Abstract
Description
[0001] This application claims priority and the benefits of International Patent Application No. PCT / CN2018 / 115935 filed on November 16, 2018, No. PCT / CN2019 / 070003 filed on January 1, 2019, No. PCT / CN2019 / 071221 filed on January 10, 2019, No. PCT / CN2019 / 074600 filed on February 2, 2019, and No. PCT / CN2019 / 075867 filed on February 22, 2019, in a timely manner, according to applicable patent law and / or pursuant to the provisions of the Paris Convention. The entire disclosures of International Patent Applications No. PCT / CN2018 / 115935, PCT / CN2019 / 070003, PCT / CN2019 / 071221, No. PCT / CN2019 / 074600 filed on February 2, 2019, and PCT / CN2019 / 075867 are incorporated by reference as part of the disclosure of this application. Technical Field
[0002] This document deals with video and image encoding and decoding technologies. Background Art
[0003] Digital video accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention
[0004] The disclosed techniques may be used by video or image decoder or encoder embodiments where geometric partitioning using history based Motion Vector Prediction (HMVP) is used.
[0005] In one example aspect, a method of video processing is disclosed. The method includes: storing, by a processor, parameters associated with a motion vector of a first video block in a memory, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; and performing further processing of the first video block using the parameters.
[0006] In another example aspect, a method of video processing includes: storing, by a processor, a control point motion vector (CPMV) in a memory representing coordinates of a representative point relative to a top left sample of a first video block; and performing further processing of the first video block using the CPMV.
[0007] In another example aspect, a method of video processing includes: performing, by a processor, determining parameters associated with a motion vector of a first video block, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; determining that a buffer for storing the parameters is full; modifying the buffer so that the buffer is not full; storing the parameters in the not-full buffer; and performing further processing of the first video block using the parameters stored in the buffer.
[0008] In another example aspect, a method of video processing includes: refreshing, by a processor, a buffer for storing parameters associated with a motion vector of a first video block, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; and performing further processing of the first video block using the parameters stored in the buffer.
[0009] In another example aspect, a method of video processing includes: identifying, by a processor, parameters associated with a motion vector of a first video block in a memory, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; and processing a second video block using the parameters associated with the first video block.
[0010] In another example aspect, a method of video processing includes: identifying, by a processor, parameters associated with a motion vector of a first video block in a memory, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; determining, by the processor based on the parameters, that a second video block uses one of the 4-parameter affine model or the 6-parameter affine model; generating an affine merge candidate list for the second video block based on the 4-parameter affine model or the 6-parameter affine model used by the second video block; and performing further processing of the second video block using the affine merge candidate list.
[0011] In another example aspect, a method of video processing includes: identifying, by a processor, parameters associated with a motion vector (MV) of a first video block; deriving, by the processor, an affine model based on the parameters of the first video block, the affine model being one or both of a 4-parameter affine model or a 6-parameter affine model; and performing further processing of the first video block using the affine model.
[0012] In another example aspect, a method of video processing includes: deriving affine Merge candidates from an affine history-based motion vector prediction (HMVP) buffer; and modifying a list, which is an affine Merge list or a sub-block Merge list, by inserting the affine Merge candidates into the list and by removing inherited affine Merge candidates from the list.
[0013] In another example aspect, a method of video processing includes: deriving an affine Merge Advanced Motion Vector Prediction (AMVP) candidate from an affine history-based motion vector prediction (HMVP) buffer; and modifying an affine AMVP candidate list by inserting the affine AMVP candidate into the list and by removing inherited affine AMVP candidates from the list.
[0014] In one example aspect, a method of video processing includes: deriving affine-related information associated with an affine model to be used by a current block for a conversion between a current block of a video and a bitstream representation of the current block based on affine-related information associated with an affine model used by a first block stored in a buffer, wherein the first block is affine-encoded prior to the current block; and performing the conversion by using the derived affine-related information.
[0015] In another example aspect, a method of video processing includes calculating a first set of affine parameters associated with an affine model of a first block of affine encoding and decoding based on a motion vector (MV) associated with the affine model of the first block, a width of the first block, and a height of the first block, and cropping the calculated first set of affine parameters.
[0016] In another example aspect, a method of video processing includes: for conversion between a current block of a video and a bitstream representation of the current block, deriving a virtual affine model of the current block from a plurality of affine models stored in a buffer for storing affine models; and performing the conversion by using the virtual affine model.
[0017] In another example aspect, a method of video processing includes: deriving a first set of affine parameters associated with an affine model of a current block for a conversion between a current block of video and a bitstream representation of the current block; comparing the first set of affine parameters with one or more of a set of affine parameters stored in a buffer for storing affine parameters to determine whether the first set of affine parameters is the same as or similar to at least one set of affine parameters; and deciding whether or how to update the buffer for storing affine parameters based on the determination.
[0018] In another example aspect, a method of video processing includes: deriving affine related information associated with an affine model of a current block for conversion between a current block and a bitstream representation of the current block based on affine parameters stored in a buffer for storing affine related information of a block encoded and decoded before a current block of a video; and performing the conversion by using the derived affine related information.
[0019] In another example aspect, a method of video processing includes: deriving an affine model of a current block for conversion between a current block and a bitstream representation of the current block based on a set of affine parameters stored in a buffer for affine-related information of a block encoded and decoded before a current block of a video; generating an affine Merge candidate list, wherein the affine Merge candidate list includes affine Merge candidates of the current block based on the derived affine model of the current block; and performing the conversion based on the affine Merge candidate list.
[0020] In another example aspect, a method of video processing includes: generating an affine advanced motion vector prediction (AMVP) candidate list for a current block of a video as an affine AMVP codec block for conversion between a current block of the video and a bitstream representation of the current block, wherein the candidate list includes at least one AMVP candidate based on affine parameters stored in a buffer for storing affine-related information of a block encoded and decoded before the current block; and performing the conversion based on the affine AMVP candidate list.
[0021] In another example aspect, a method of video processing includes: deriving an affine candidate for a current block for conversion between a current block and a bitstream representation of the current block based on affine parameters stored in a buffer for affine-related information of a block encoded and decoded before a current block of a video and one or more unit blocks; inserting the derived affine candidate into an affine candidate list; and performing the conversion by using the affine candidate list.
[0022] In another example aspect, a method of video processing includes: deriving an affine candidate for a current block of a video for a conversion between a current block and a bitstream representation of the current block based on affine parameters stored in an affine history-based motion vector prediction (HMVP) buffer; inserting the derived affine candidate into an affine candidate list; and performing the conversion by using the affine candidate list.
[0023] In another example aspect, the above method may be implemented by a video encoder apparatus including a processor.
[0024] In yet another example aspect, the methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.
[0025] These and other aspects are described further in this document. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 An example of sub-block based prediction is shown.
[0027] Figure 2 Examples of a 4-parameter affine model and a 6-parameter affine model are shown.
[0028] Figure 3 An example of an affine motion vector field (MVF) for each sub-block is shown.
[0029] Figure 4 An example of a candidate for AF_MERGE is shown.
[0030] Figure 5 An example of candidate positions for the affine merge mode is shown.
[0031] Figure 6 Examples of 135 degree division and 45 degree division are shown.
[0032] Figure 7 The positions of neighboring blocks are shown.
[0033] Figure 8 An example of a codec unit (CU) to which the first weighting factor group is applied is shown.
[0034] Fig. 9 An example of motion vector storage is shown.
[0035] Fig.10 An example of a decoding flow chart with the proposed History-Based Motion Vector Prediction (HMVP) method is shown.
[0036] Fig.11 An example of updating a table in the HMVP method is shown.
[0037] Fig.12 An example of a final motion vector expression (UMVE) search process is shown.
[0038] Fig.13 An example of a UMVE search point is shown.
[0039] Fig.14 An example of a distance index and distance offset mapping is shown.
[0040] Fig.15 An example of deriving the CPMV from the motion vectors (MV) of neighboring blocks and a set of parameters stored in a buffer is shown.
[0041] Fig.16 is a block diagram of an example of a video processing device.
[0042] Fig.17 A block diagram of an example implementation of a video encoder is shown.
[0043] Fig.18 is a flow chart of an example of a video processing method.
[0044] Fig.19is a flow chart of an example of a video processing method.
[0045] Fig. 20 is a flow chart of an example of a video processing method.
[0046] Fig.21 is a flow chart of an example of a video processing method.
[0047] Fig. 22 is a flow chart of an example of a video processing method.
[0048] Fig.23 is a flow chart of an example of a video processing method.
[0049] Fig.24 is a flow chart of an example of a video processing method.
[0050] Fig.25 Examples of possible locations of collocated cell blocks are shown.
[0051] Fig.26 is a flow chart of an example of a video processing method.
[0052] Fig. 27 Example locations in a 4x4 basic block are shown.
[0053] Fig.28 is a flow chart of an example of a video processing method.
[0054] Fig.29 is a flow chart of an example of a video processing method.
[0055] Fig.30 is a flow chart of an example of a video processing method.
[0056] Fig.31 is a flow chart of an example of a video processing method.
[0057] Fig.32 is a flow chart of an example of a video processing method.
[0058] Fig.33 is a flow chart of an example of a video processing method.
[0059] Fig.34 is a flow chart of an example of a video processing method.
[0060] Fig.35 is a flow chart of an example of a video processing method.
[0061] Fig.36 is a flow chart of an example of a video processing method. DETAILED DESCRIPTION
[0062] This document provides various techniques that can be used by a decoder of an image or video bitstream to improve the quality of a decompressed or decoded digital video or image. For brevity, the term "video" is used herein to include both sequences of pictures (traditionally referred to as video) and individual images. In addition, a video encoder may also implement these techniques during the encoding process in order to reconstruct decoded frames for further encoding.
[0063] The section titles used in this document are for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. Thus, the embodiments of one section can be combined with the embodiments of other sections.
[0064] 1. Overview
[0065] This patent document relates to video / image coding technology. Specifically, it relates to affine prediction in video / image coding. It can be applied to existing video coding standards (such as HEVC) or standards to be finalized (multi-function video codec). It can also be applied to future video / image coding standards or video / image codecs.
[0066] 2. Background
[0067] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that utilizes temporal prediction plus transform codecs. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and put them into reference software named the Joint Exploration Model (JEM). In April 2018, the Joint Video Expert Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard with the goal of a 50% bitrate reduction compared to HEVC.
[0068] Sub-block based prediction was first introduced into the video codec standard by HEVC Annex I (3D-HEVC). With sub-block based prediction, a block such as a coding unit (CU) or a prediction unit (PU) is divided into several non-overlapping sub-blocks. Different sub-blocks can be assigned different motion information, such as reference index or motion vector (MV), and motion compensation (MC) is performed separately for each sub-block. Figure 1 The concept of sub-block based prediction is demonstrated.
[0069] To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and put them into reference software named Joint Exploration Model (JEM).
[0070] In JEM, sub-block-based prediction is used for a variety of codec tools, such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), bidirectional optical flow (BIO) and frame rate up conversion (FRUC). Affine prediction is also adopted in VVC.
[0071] Fig.17 is a block diagram of an example implementation of a video encoder. Fig.17 An encoder implementation is shown with a built-in feedback path, where the video encoder also performs a video decoding function (reconstructing a compressed representation of the video data for use in the encoding of the next video data).
[0072] 2.1 Affine Prediction
[0073] In HEVC, only the translational motion model is used for motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, a simplified affine transformation motion compensation prediction is applied. Figure 2 As shown, the affine motion field of the block consists of two (in Figure 2 ) or three (in the 4-parameter affine model on the left side of Figure 2The 6-parameter affine model on the right side of ) is described by the control point motion vector.
[0074] Figure 3 An example of MVs in sub-blocks of an affine coded block is shown.
[0075] The motion vector field (MVF) of the block is described by the following equations using a 4-parameter affine model in equation (1) (where the 4 parameters are defined as variables a, b, e, and f) and a 6-parameter affine model in equation (2) (where the 4 parameters are defined as variables a, b, c, d, e, and f):
[0076]
[0077]
[0078] Among them, (mv h 0,mv h 0) is the motion vector of the upper left control point (Control Point, CP), (mv h 1,mv h 1) is the motion vector of the upper right control point, and (mv h 2,mv h 2) is the motion vector of the lower left control point. All three motion vectors are called control point motion vectors (CPMV). (x, y) represents the coordinates of the representative point relative to the upper left sample point in the current block. The CP motion vector can be signaled (like in affine AMVP mode) or derived on the fly (like in affine Merge mode). w and h are the width and height of the current block. In practice, the division is implemented by right shift and rounding operations. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample point in the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2).
[0079] In the divide-free design, (1) and (2) are implemented as:
[0080]
[0081] For the 4-parameter affine model shown in (1):
[0082]
[0083] For the 6-parameter affine model shown in (2):
[0084]
[0085] final,
[0086]
[0087]
[0088] Where S represents the calculation precision. For example, in VVC, S=7. In VVC, the MV for the subblock with the upper left sample at (xs, ys) in MC is calculated by (6), where x=xs+2 and y=ys+2.
[0089] To derive the motion vector for each 4×4 sub-block, Figure 3 As shown, the motion vector of the center sample of each sub-block is calculated according to equation (1) or (2) and rounded to fractional precision of 1 / 16. Then, a motion compensated interpolation filter is applied to generate a prediction of each sub-block using the derived motion vector.
[0090] The affine model can be inherited from the spatially adjacent affine codec blocks (such as the left, top, top right, bottom left, and top left neighboring blocks), as Figure 4 For example, if you are encoding and decoding in affine mode Figure 4 The lower left block A in the neighborhood is Figure 4 If A0 is represented by A, the control point (CP) motion vector mv0 of the upper left corner, upper right corner and lower left corner of the adjacent CU / PU containing block A is extracted N 、mv1 N and mv2 N . And based on mv0 N 、mv1 N and mv2 N Calculate the motion vector mv0 of the upper left corner / upper right side / lower left side of the current CU / PU C 、mv1 C and mv2 C (Only for 6-parameter affine model). It should be noted that in VTM-2.0, if the current block is affine coded, the sub-block (e.g., 4×4 block in VTM) LT stores mv0 and RT stores mv1. If the current block is coded with 6-parameter affine model, LB stores mv2; otherwise (with 4-parameter affine model), LB stores mv2'. Other sub-blocks store MV for MC.
[0091] It should be noted that when the CU is encoded and decoded in affine Merge mode (i.e., in AF_MERGE mode), it obtains the first block encoded and decoded in affine mode from the valid neighborhood reconstructed block. And the selection order of candidate blocks is from left, top, top right, bottom left to top left, such as Figure 4 as shown in .
[0092] In affine Merge mode, the derived CP MV mv0 of the current block can be C 、mv1 C and mv2 C Used as CP MV. Or they can be used as MVP of affine inter mode in VVC. It should be noted that for Merge mode, if the current block is encoded and decoded in affine mode, after deriving the CP MV of the current block, the current block can be further divided into multiple sub-blocks, and each block will derive its motion information based on the derived CP MV of the current block.
[0093] 2.2 Affine Merge Candidate List
[0094] Different from VTM where only one affine spatial neighbor can be used to derive the affine motion of a block, it is proposed to build a separate affine candidate list for AF_MERGE mode.
[0095] Insert the inherited affine candidate into the candidate list
[0096] Inherited affine candidates are candidates derived from valid neighborhood reconstructed blocks encoded and decoded using the affine mode.
[0097] like Figure 5 As shown, the scanning order of candidate blocks is A1, B1, B0, A0 and B2. When a block (for example A1) is selected, a two-step process is applied:
[0098] First, two / three control points of the current block are derived using the three angular motion vectors of the CU covering the block.
[0099] The sub-block motion of each sub-block within the current block is derived based on the control points of the current block.
[0100] Insert the constructed affine candidate
[0101] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand, the constructed affine candidate is inserted into the candidate list.
[0102] Constructed affine candidates means that the candidates are constructed by combining the neighborhood motion information of each control point.
[0103] The motion information of the control point is first obtained from Figure 5 CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial locations used to predict CPk (k = 1, 2, 3); T is the temporal location used to predict CP4.
[0104] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.
[0105] The motion information for each control point is obtained according to the following priority order:
[0106] For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is available, use B3. If neither B2 nor B3 is available, use A2. If all three candidates are unavailable, motion information for CP1 cannot be obtained.
[0107] For CP2, the check priority is B1->B0;
[0108] For CP3, the check priority is A1->A0;
[0109] For CP4, use T.
[0110] Second, a combination of control points is used to construct a motion model.
[0111] The motion vectors of three control points are required to calculate the transformation parameters in the 6-parameter affine model. The three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). For example, the 6-parameter affine motion model is constructed using the CP1, CP2, and CP3 control points, denoted as Affine(CP1, CP2, CP3).
[0112] The motion vectors of two control points are required to calculate the transformation parameters in the 4-parameter affine model. The two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). For example, the 4-parameter affine motion model is constructed using the CP1 and CP2 control points, denoted as Affine(CP1, CP2).
[0113] The combinations of constructed affine candidates are inserted into the candidate list in the following order:
[0114] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}
[0115] Insert zero motion vector
[0116] If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand, zero motion vectors are inserted into the candidate list until the list is full.
[0117] 2.3 Affine Merge Candidate List
[0118] 2.5.1 Affine Merge Mode
[0119] In the affine merge mode, only the first available affine neighborhood can be used to derive motion information for the affine merge mode. The candidate list for the affine merge mode is constructed by searching for valid affine neighborhoods and combining the neighborhood motion information for each control point.
[0120] The affine merge candidate list is constructed as follows:
[0121] Insert inheritance affine candidates
[0122] The inherited affine candidate means that the candidate is derived from the affine motion model of its valid neighboring affine codec blocks. In a common basis, such as Figure 5 As shown, the scanning order of candidate positions is: A1, B1, B0, A0 and B2.
[0123] After deriving a candidate, a complete pruning process is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate is discarded.
[0124] Insert the constructed affine candidate
[0125] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (set to 5 in this contribution), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate means that the candidate is constructed by combining the neighborhood motion information of each control point.
[0126] The motion information of the control point is first obtained from Figure 5 The specified spatial and temporal neighborhoods are derived as shown. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial locations used to predict CPk (k = 1, 2, 3); T is the temporal location used to predict CP4.
[0127] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.
[0128] The motion information for each control point is obtained according to the following priority order:
[0129] For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is available, use B3. If neither B2 nor B3 is available, use A2. If all three candidates are unavailable, motion information for CP1 cannot be obtained.
[0130] For CP2, the check priority is B1->B0;
[0131] For CP3, the check priority is A1->A0;
[0132] For CP4, use T.
[0133] Second, combinations of control points are used to construct affine merge candidates.
[0134] The motion information of three control points is required to construct a 6-parameter affine candidate. The three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the top left, top right and bottom left control points.
[0135] The motion information of two control points is required to construct a 4-parameter affine candidate. The two control points can be selected from one of the following six combinations ({CP1,CP4}, {CP2,CP3}, {CP1,CP2}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4}). The combination {CP1,CP4}, {CP2,CP3}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4} will be converted into a 4-parameter motion model represented by the top left and top right control points.
[0136] The combinations of constructed affine candidates are inserted into the candidate list in the following order:
[0137] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}
[0138] For a combined reference list X (X is 0 or 1), the reference index with the highest usage in the control point is selected as the reference index of list X, and the motion vectors pointing to different reference pictures will be scaled.
[0139] After deriving a candidate, a complete pruning process is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate is discarded.
[0140] Fill with zero motion vectors
[0141] If the number of candidates in the affine merge candidate list is less than 5, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.
[0142] 2.5.2 Affine Merge Mode
[0143] The following simplification of the affine Merge mode is proposed.
[0144] The pruning of inherited affine candidates is simplified by comparing codec units covering adjacent positions instead of comparing derived affine candidates. Up to 2 inherited affine candidates are inserted into the affine merge list. The pruning of constructed affine candidates is completely removed.
[0145] Remove the MV scaling operation in the constructed affine candidates. If the reference indices of the control points are different, the constructed motion model is discarded.
[0146] The number of constructed affine candidates is reduced from 10 to 6.
[0147] It is also proposed that other Merge candidates using sub-block prediction such as ATMVP are also put into the affine Merge candidate list. In this case, the affine Merge candidate list can be renamed with some other names such as sub-block Merge candidate list.
[0148] 2.4 Control point MV offset in affine merge mode
[0149] Generate new affine Merge candidates based on the CPMV offset of the first affine Merge candidate. If the first affine Merge candidate enables the 4-parameter affine model, 2 CPMVs are derived for each new affine Merge candidate by offsetting the 2 CPMVs of the first affine Merge candidate; otherwise (the 6-parameter affine model is enabled), 3 CPMVs are derived for each new affine Merge candidate by offsetting the 3 CPMVs of the first affine Merge candidate. In unidirectional prediction, the CPMV offset is applied to the CPMV of the first candidate. In bidirectional prediction with list 0 and list 1 in the same direction, the CPMV offset is applied to the first candidate as follows:
[0150] MVnew(L0),i=MVold(L0)+MVoffset(i)
[0151] MVnew(L1),i=MVold(L1)+MVoffset(i)
[0152] In bidirectional prediction with list 0 and list 1 in opposite directions, the CPMV offset is applied to the first candidate as follows:
[0153] MVnew(L0),i=MVold(L0)+MVoffset(i)
[0154] MVnew(L1),i=MVold(L1)-MVoffset(i)
[0155] In this paper, various offset directions with various offset magnitudes can be used to generate new affine merge candidates. Two implementations are tested:
[0156] Generate 16 new affine merge candidates with 8 different offset directions and 2 different offset magnitudes, as shown in the following offset set:
[0157] Offset set = {(4,0),(0,4),(-4,0),(0,-4),(-4,-4),(4,-4),(4,4),(-4,4),(8,0),(0,8),(-8,0),(0,-8),(-8,-8),(8,-8),(8,8),(-8,8)}.
[0158] For this design, the affine merge list increases to 20. The number of potential affine merge candidates is 31 in total.
[0159] Generate 4 new affine merge candidates with 4 different offset directions and 1 offset magnitude, as shown in the following offset set:
[0160] Offset set = {(4,0),(0,4),(-4,0),(0,-4)}.
[0161] The affine merge list is kept at 5. Remove the affine merge candidates constructed in the four time domains to keep the number of potential affine merge candidates unchanged, that is, 15 in total. Assume that the coordinates of CPMV1, CPMV2, CPMV3, and CPMV4 are (0, 0), (W, 0), (H, 0), and (W, H). Note that CPMV4 is derived from Figure 6 The removed candidates are the following four time-domain related constructed affine merge candidates: {CP2, CP3, CP4}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}.
[0162] 2.5 Improvement of generalized bidirectional prediction
[0163] The proposed generalized bidirectional prediction improvement (GBi) is adopted.
[0164] In the example, GBi is proposed. The gain-complexity trade-off of GBi is improved. GBi applies unequal weights to the prediction values from L0 and L1 in bidirectional prediction mode. In inter-prediction mode, multiple weight pairs including equal weight pairs (1 / 2, 1 / 2) are evaluated based on rate-distortion optimization (RDO), and the GBi index of the selected weight pair is signaled to the decoder. In Merge mode, the GBi index is inherited from the adjacent CU. In BMS2.1 GBi, the prediction value generation in bidirectional prediction mode is as shown in equation (1).
[0165] PGBi=(w0*PL0+w1*PL1+RoundingOffsetGBi)>>shiftNumGBi,
[0166] Where PGBi is the final prediction value of GBi. w0 and w1 are the selected GBi weight pairs and are applied to the prediction values of list 0 (L0) and list 1 (L1), respectively. RoundingOffsetGBi and shiftNumGBi are used to normalize the final prediction values in GBi. The supported w1 weight sets are {-1 / 4, 3 / 8, 1 / 2, 5 / 8, 5 / 4}, where five weights correspond to one equal weight pair and four unequal weight pairs. The mixing gain, which is the sum of w1 and w0, is fixed to 1.0. Therefore, the corresponding w0 weight set is {5 / 4, 5 / 8, 1 / 2, 3 / 8, -1 / 4}. The weight pairs are selected at the CU level.
[0167] For non-low-latency images, the weight set size is reduced from five to three, where the w1 weight set is {3 / 8, 1 / 2, 5 / 8} and the w0 weight set is {5 / 8, 1 / 2, 3 / 8}. The weight set size reduction for non-low-latency images applies to the BMS2.1 GBi and all GBi tests in this contribution.
[0168] In the example, a solution is proposed to further improve GBi performance. Specifically, the following modifications are applied on the basis of the existing GBi design.
[0169] 2.5.1GBi encoder bug fix
[0170] In order to reduce GBi encoding time, in the current encoder design, the encoder will store the unidirectional prediction motion vectors estimated from GBi weight equal to 4 / 8 and reuse them for unidirectional prediction search for other GBi weights. This fast encoding method is applied to both translational motion models and affine motion models. A 6-parameter affine model and a 4-parameter affine model are adopted. When the GBi weight is equal to 4 / 8, the BMS2.1 encoder does not distinguish between the 4-parameter affine model and the 6-parameter affine model when storing the unidirectional prediction affine MV. Therefore, after encoding with GBi weight 4 / 8, the 4-parameter affine MV may be overwritten by the 6-parameter affine MV. For other GBi weights, the stored 6-parameter affine MV can be used for the 4-parameter affine ME, or the stored 4-parameter affine MV can be used for the 6-parameter affine ME. The proposed GBi encoder error repair is to separate the 4-parameter and 6-parameter affine MV storage. When the GBi weight is equal to 4 / 8, the encoder stores those affine MVs based on the affine model type, and for other GBi weights, the corresponding affine MV is reused based on the affine model type.
[0171] CU size constraints of 2.5.2GBi
[0172] In this method, GBi is disabled for small CUs. In inter prediction mode, if bidirectional prediction is used and the CU area is smaller than 128 luma samples, GBi is disabled without any signaling.
[0173] 2.5.3 Using GBi's Merge Mode
[0174] In Merge mode, the GBi index is not signaled. Instead, it is inherited from the neighboring block it is merged into. When a TMVP candidate is selected, GBi is turned off in that block.
[0175] 2.5.4 Affine prediction using GBi
[0176] GBi can be used when the current block is coded with affine prediction. For affine inter mode, the GBi index is signaled. For affine merge mode, the GBi index is inherited from the neighboring block it is merged into. If the constructed affine model is selected, GBi is turned off in this block.
[0177] 2.6 Triangle Prediction Mode
[0178] The concept of Triangular Prediction Mode (TPM) is to introduce a new triangle partitioning for motion compensation prediction. Figure 6 As shown, it divides the CU into two triangular prediction units in the diagonal direction or the anti-diagonal direction. Each triangular prediction unit in the CU is inter-predicted using its own unidirectional prediction motion vector and reference frame index derived from the unidirectional prediction candidate list. After predicting the triangular prediction unit, adaptive weighting is performed on the diagonal edges. Then, the transform and quantization process is applied to the entire CU. Note that this mode is only applied to Skip and Merge modes.
[0179] 2.6.1 TPM’s One-Way Prediction Candidate List
[0180] The unidirectional prediction candidate list consists of five unidirectional prediction motion vector candidates. Figure 7 As shown, it is derived from seven adjacent blocks including five spatial adjacent blocks (1 to 5) and two temporal collocated blocks (6 to 7). The motion vectors of the seven adjacent blocks are collected and put into the unidirectional prediction candidate list in the order of unidirectional prediction motion vector, L0 motion vector of bidirectional prediction motion vector, L1 motion vector of bidirectional prediction motion vector and average motion vector of L0 and L1 motion vector of bidirectional prediction motion vector. If the number of candidates is less than five, zero motion vector is added to the list. The motion candidate added to the list is called TPM motion candidate.
[0181] More specifically, the following steps are involved:
[0182] From A1, B1, B0, A0, B2, Col and Col2 (corresponding to Figure 8 Blocks 1-7 in (a) obtain motion candidates, Without any repair Cutting operation .
[0183] Set the variable numCurrMergeCand = 0
[0184] For each motion candidate derived from A1, B1, B0, A0, B2, Col and Col2, if the motion candidate is unidirectionally predicted (from list 0 or list 1), it is added to the Merge list, where numCurrMergeCand is increased by 1. This added motion candidate is named "original unidirectionally predicted candidate".
[0185] application Fully trimmed .
[0186] For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is bidirectionally predicted, the motion information from list 0 is added to the Merge list (i.e., modified to be unidirectionally predicted from list 0), and numCurrMergeCand is increased by 1. This added motion candidate is named "truncated list 0 predicted candidate".
[0187] application Fully trimmed .
[0188] For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is bidirectionally predicted, the motion information from list 1 is added to the Merge list (i.e., modified to be unidirectionally predicted from list 1), and numCurrMergeCand is increased by 1. This added motion candidate is named "truncated list 1 predicted candidate".
[0189] application Fully trimmed .
[0190] For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand is less than 5, if the motion candidate is bi-predicted,
[0191] If the slice QP of the list 0 reference picture is less than the slice QP of the list 1 reference picture, the motion information of list 1 is first scaled to the list 0 reference picture, and the average of the two MVs (one from the original list 0 and the other is the scaled MV from list 1) is added to the Merge list, that is, the average unidirectional prediction from the list 0 motion candidate, and numCurrMergeCand is increased by 1.
[0192] Otherwise, the motion information of list 0 is first scaled to the list 1 reference picture, and the average of the two MVs (one from the original list 1 and the other is the scaled maximum value from list 0) is added to the Merge list, that is, the average unidirectional prediction from the list 1 motion candidate, and numCurrMergeCand is increased by 1.
[0193] application Fully trimmed .
[0194] If numCurrMergeCand is less than 5, a zero motion vector candidate is added.
[0195] 2.6.1.1 Adaptive Weighting Process
[0196] After predicting each triangle prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangle prediction units to derive the final prediction for the entire CU. Two groups of weighting factors are defined as follows:
[0197] The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luminance and chrominance samples respectively;
[0198] The second weighting factor group: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luminance and chrominance samples respectively.
[0199] The weighting factor set is selected based on the comparison of the motion vectors of the two triangle prediction units. When the reference pictures of the two triangle prediction units are different from each other or the difference of their motion vectors is greater than 16 pixels, the second weighting factor set is used. Otherwise, the first weighting factor set is used. Example Figure 8 shown.
[0200] 2.6.1.2 Motion Vector Storage
[0201] Motion vector of triangle prediction unit ( Fig. 9 Mv1 and Mv2 in CU are stored in 4×4 grids. For each 4×4 grid, a unidirectional prediction motion vector or a bidirectional prediction motion vector is stored, depending on the position of the 4×4 grid in the CU. Fig. 9 As shown, a unidirectional prediction motion vector Mv1 or Mv2 is stored for a 4×4 grid located in a non-weighted area (i.e., not located at a diagonal edge). On the other hand, a bidirectional prediction motion vector is stored for a 4×4 grid located in a weighted area. The bidirectional prediction motion vector is derived from Mv1 and Mv2 according to the following rule:
[0202] In the case where Mv1 and Mv2 have motion vectors from different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector.
[0203] In the case where both Mv1 and Mv2 come from the same L0 (or L1) direction,
[0204] If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, Mv2 is scaled to that picture. Mv1 and the scaled Mv2 are combined to form a bidirectional prediction motion vector.
[0205] If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bidirectional prediction motion vector.
[0206] Otherwise, only Mv1 is stored for the weighted region.
[0207] 2.7 History-based Motion Vector Prediction
[0208] A history-based MVP (HMVP) method is proposed, where the HMVP candidate is defined as the motion information of the previously coded block. During the encoding / decoding process, a table with multiple HMVP candidates is maintained. When a new slice is encountered, the table is cleared. Whenever there is a non-affine block for inter-frame coding, the associated motion information is added as the last entry of the table as a new HMVP candidate. The overall coding and decoding process is as follows Fig.10 and Fig.11 Depicted.
[0209] In this document, the table size S is set to 6, which indicates that at most 6 HMVP candidates can be added to the table. When a new motion candidate is inserted into the table, the constrained FIFO rule is used, where a redundancy check is first applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table, and then all the following HMVP candidates are moved forward, that is, the index is reduced by 1.
[0210] HMVP candidates can be used in the Merge candidate list construction process. Check the last few HMVP candidates in the table in order and insert them into the candidate list after the TMVP candidate. Apply pruning to the HMVP candidates to merge candidates in the spatial or temporal domain, excluding sub-block motion candidates (i.e. ATMVP).
[0211] To reduce the number of pruning operations, three simplifications are introduced:
[0212] The number of HMPV candidates to be checked, denoted by L, is set as follows:
[0213] L=(N<=4)?M:(8-N) (1)
[0214] Wherein, N indicates the number of available non-sub-block Merge candidates in the table, and M indicates the number of available HMVP candidates in the table.
[0215] In addition, once the total number of available Merge candidates reaches the signaled maximum allowed Merge candidate minus 1, the Merge candidate list construction process based on the HMVP list is terminated.
[0216] Furthermore, the number of pairs used for combined bi-predictive Merge candidate derivation is reduced from 12 to 6.
[0217] Similarly, HMVP candidates can also be used in the AMVP candidate list construction process. The motion vectors of the last K HMVP candidates in the table are inserted after the TMVP candidates. Only HMVP candidates with the same reference picture as the AMVP target reference picture are used to construct the AMVP candidate list. Pruning is applied to HMVP candidates. In this document, K is set to 4, while the AMVP list size remains unchanged, that is, equal to 2.
[0218] 2.8 Final Motion Vector Expression (UMVE)
[0219] In this document, the Ultimate Motion Vector Expression (UMVE) is proposed. UMVE is also called Merge with MVD (MMVD) in VVC. UMVE is used in Skip or Merge mode using the proposed motion vector expression method.
[0220] UMVE reuses the same Merge candidates as in VVC. Among the Merge candidates, candidates can be selected and further extended by the proposed motion vector representation method.
[0221] UMVE provides a new motion vector representation with simplified signaling. The expression method includes the starting point, motion magnitude and motion direction.
[0222] Fig.12 An example of a UMVE search process is shown. Fig.13 An example of a UMVE search point is shown.
[0223] The proposed technique uses the Merge candidate list as is. However, for the extension of UMVE, only candidates of the default Merge type (MRG_TYPE_DEFAULT_N) are considered.
[0224] The base candidate index defines the starting point. The base candidate index indicates the best candidate among the candidates in the list, as shown below.
[0225] Table 1: Basic candidate IDX
[0226] Basic Candidate IDX 0 1 2 3 The Nth MVP 1st MVP 2nd MVP No.3MVP 4th MVP
[0227] If the number of basic candidates is equal to 1, the basic candidate IDX is not signaled.
[0228] The distance index is the motion amplitude information. The distance index indicates the predefined distance from the start point information. The predefined distance is as follows:
[0229] Table 2: Distance IDX
[0230]
[0231] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions as shown below.
[0232] Table 3: Direction IDX
[0233] Directions to IDX 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –
[0234] The UMVE flag is signaled immediately after the Skip flag and the Merge flag are sent. If the Skip and Merge flags are true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. However, if it is not 1, the AFFINE flag is parsed. If the AFFINE flag is equal to 1, this is AFFINE mode, but if it is not 1, the Skip / Merge index is parsed for the Skip / Merge mode of the VTM.
[0235] No additional line buffering is needed for UMVE candidates, because software's Skip / Merge candidates are used directly as base candidates. The complement of the MV is decided before motion compensation, using the input UMVE index. No need to keep a long line buffer for this.
[0236] 2.9 Inter-Intra Mode
[0237] In inter-intra mode, multi-hypothesis prediction combines an intra prediction and a Merge index prediction. In MergeCU, a flag is signaled for Merge mode to select the intra mode from the intra candidate list when the flag is true. For the luma component, the intra candidate list is derived from 4 intra prediction modes including DC mode, planar mode, horizontal mode and vertical mode, and the size of the intra candidate list can be 3 or 4, depending on the block shape. When the CU width is greater than twice the CU height, the horizontal mode does not include the intra mode list, and when the CU height is greater than twice the CU width, the vertical mode is removed from the intra mode list. Weighted averaging is used to combine one intra prediction mode selected by the intra mode index and one Merge index prediction selected by the Merge index. For chroma components, DM is always applied without additional signaling. The weights used for combined prediction are described as follows. When DC mode or planar mode is selected, or the CB width or height is less than 4, equal weights are applied. For those CBs whose CB width and height are greater than or equal to 4, when horizontal / vertical mode is selected, a CB is first divided vertically / horizontally into four equal-area regions. Each weight set, denoted as (w_intra i ,w_inter i ), where i is 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5) and (w_intra4, w_inter4) = (2, 6) will be applied to the corresponding areas. (w_intra1, w_inter1) is used for the area closest to the reference sample, and (w_intra4, w_inter4) is used for the area farthest from the reference sample. Then, the combined prediction can be calculated by adding the two weighted predictions and shifting right by 3 bits. In addition, the intra prediction mode of the intra-frame hypothesis of the prediction value can be saved for reference by subsequent adjacent CUs.
[0238] 2.10 Affine Merge Mode with Prediction Offset
[0239] The proposed method selects the first available affine Merge candidate as the base predictor. Then, it applies motion vector offset to the motion vector value of each control point from the base predictor. If no affine Merge candidate is available, the proposed method is not used.
[0240] The inter prediction direction of the selected basic prediction value and the reference index for each direction are used without change.
[0241] In the current implementation, the affine model of the current block is assumed to be a 4-parameter model, and only 2 control points need to be derived. Therefore, only the first 2 control points of the basic prediction value will be used as the control point prediction value.
[0242] For each control point, the zero_MVD flag is used to indicate whether the control point of the current block has the same MV value as the corresponding control point prediction value. If the zero_MVD flag is true, no other signaling is required for the control point. Otherwise, the distance index and offset direction index are signaled for the control point.
[0243] A distance offset table of size 5 is used, as shown in the following table. The distance index is signaled to indicate which distance offset to use. The mapping between distance index and distance offset value is as follows: Fig.14 shown.
[0244] Table 1 - Distance offset table
[0245] Distance DX 0 1 2 3 4 Distance Offset 1 / 2 pixel 1 pixel 2 pixels 4 pixels 8 pixels
[0246] The direction index can represent four directions as shown below, where only the x or y direction may have MV differences, but not both.
[0247] Offset direction IDX 00 01 10 11 x-dir-factor +1 –1 0 0 y-dir-factor 0 0 +1 –1
[0248] If the inter prediction is unidirectional, the signaled distance offset is applied to the offset direction of the prediction value of each control point. The result will be the MV value of each control point.
[0249] For example, when the basic prediction value is a unidirectional prediction, and the motion vector value of the control point is MVP (v px ,v py ). When the distance offset and direction index are signaled, the motion vector of the corresponding control point of the current block will be calculated as follows.
[0250] MV(vx,vy)=MVP(vpx,vpy)+MV(x-dir-factor*distance-offset,y-dir-factor*distance-offset);
[0251] If the inter prediction is bi-directional, the signaled distance offset is applied in the signaled offset direction of the L0 motion vector of the control point prediction value; and the same distance offset with the opposite direction is applied to the L1 motion vector of the control point prediction value. The result will be the MV value of each control point in each inter prediction direction.
[0252] For example, when the basic prediction value is bidirectional prediction, and the motion vector value of the control point on L0 is MVPL0 (v0px, v0py), and the motion vector of the control point on L1 is MVPL1 (v1px, v1py). When the distance offset and direction index are signaled, the motion vector of the corresponding control point of the current block will be calculated as follows.
[0253] MVL0(v0x,v0y)=MVPL0(v0px,v0py)+MV(x-dir-factor*distance-offset,y-dir-factor*distance-offset);
[0254] MVL1(v0x,v0y)=MVPL1(v0px,v0py)+MV(-x-dir-factor*distance-offset,-y-dir-factor*distance-offset);
[0255] A simplified method is proposed to reduce the signaling overhead by signaling the distance offset index and offset direction index for each block. The same offset will be applied to all available control points in the same way. In this method, the number of control points is determined by the affine type of the basic prediction value, 3 control points of the 6-parameter type, and 2 control points of the 4-parameter type. The distance offset table and the offset direction table are the same as in 2.1.
[0256] Since all control points of a block are signaled at once, the zero_MVD flag is not used in this method.
[0257] 2.11 Representation of affine motion data in P1809115501
[0258] In P1809115501, it is proposed to store affine parameters instead of CPMV to predict the affine model of the subsequent codec block.
[0259] 2.12Merge list design
[0260] There are three different Merge list construction processes supported in VVC:
[0261] Sub-block Merge candidate list: It includes ATMVP and affine Merge candidates. Affine mode and ATMVP mode share a Merge list construction process. Here, ATMVP and affine Merge candidates can be added in sequence. The sub-block Merge list size is signaled in the slice header and the maximum value is 5.
[0262] Unidirectional prediction TPM Merge list: For triangular prediction mode, even though the two partitions can choose their own merge candidate indexes, a merge list construction process is shared for both partitions. When building the merge list, the spatial neighbors and two temporal blocks of the block are checked. The motion information derived from the spatial neighbors and temporal blocks is called regular motion candidates in our IDF. These regular motion candidates are further used to derive multiple TPM candidates. Note that the transformation is performed at the whole block level, even though the two partitions may use different motion vectors to generate their own prediction blocks.
[0263] The one-way predictive TPM Merge list size is fixed to 5.
[0264] Regular Merge list: For the remaining codec blocks, a Merge list construction process is shared. Here, spatial / temporal / HMVP, paired combined bidirectional prediction Merge candidates and zero motion candidates can be inserted in sequence. The regular Merge list size is signaled in the slice header and the maximum value is 6.
[0265] 2.12.1 Sub-block Merge Candidate List
[0266] It is recommended to put all sub-block related motion candidates in a separate Merge list in addition to the regular Merge list of non-sub-block Merge candidates.
[0267] Sub-block related motion candidates are placed in a separate Merge list, which is called "sub-block Merge candidate list".
[0268] In one example, the sub-block Merge candidate list includes affine Merge candidates and ATMVP candidates, and / or sub-block-based STMVP candidates.
[0269] 2.12.2 Affine Merge Candidate List
[0270] In this document, the ATMVP Merge candidate in the normal Merge list is moved to the first position of the affine Merge list. In this way, all the Merge candidates in the new list (ie, the sub-block based Merge candidate list) are based on the sub-block codec tool.
[0271] The affine merge candidate list is constructed by the following steps:
[0272] Insert inheritance affine candidates
[0273] An inherited affine candidate means that the candidate is derived from the affine motion model of its valid neighboring affine codec block. Up to two inherited affine candidates are derived from the affine motion model of the neighboring block and inserted into the candidate list. For the left prediction value, the scanning order is {A0, A1}; for the top prediction value, the scanning order is {B0, B1, B2}.
[0274] Insert the constructed affine candidate
[0275] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (set to 5), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate means that the candidate is constructed by combining the neighboring motion information of each control point.
[0276] The motion information of the control point is first obtained from Figure 5 The specified spatial neighborhood and temporal neighborhood derivation are shown. CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial locations used to predict CPk (k = 1, 2, 3); T is the temporal location used to predict CP4.
[0277] The coordinates of CP1, CP2, CP3 and CP4 are (0, 0), (W, 0), (H, 0) and (W, H), respectively, where W and H are the width and height of the current block.
[0278] The motion information for each control point is obtained according to the following priority order:
[0279] For CP1, the priority of checking is B2->B3->A2. If B2 is available, use B2. Otherwise, if B2 is available, use B3. If both B2 and B3 are unavailable, use A2. If all three candidates are unavailable, the motion information of CP1 cannot be obtained.
[0280] For CP2, the check priority is B1->B0.
[0281] For CP3, the check priority is A1->A0.
[0282] For CP4, use T.
[0283] Second, combinations of control points are used to construct affine merge candidates.
[0284] The motion information of three control points is required to construct a 6-parameter affine candidate. The three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the top left, top right and bottom left control points.
[0285] The motion information of two control points is required to construct a 4-parameter affine candidate. The two control points can be selected from one of two combinations ({CP1, CP2}, {CP1, CP3}). These two combinations will be converted into a 4-parameter motion model represented by the top left and top right control points.
[0286] The combinations of constructed affine candidates are inserted into the candidate list in the following order:
[0287] {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}
[0288] Available combinations of motion information of CPs are added to the affine Merge list only when the CPs have the same reference index.
[0289] Fill with zero motion vectors
[0290] If the number of candidates in the affine merge candidate list is less than 5, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.
[0291] Shared Merge List
[0292] It is proposed that all leaf CUs of an ancestor node in the CU partition tree share the same Merge candidate list to enable parallel processing of small skip / merge codec CUs. The ancestor node is called a Merge shared node. In the case where the Merge shared node pretends to be a leaf CU, a shared Merge candidate list is generated at the Merge shared node.
[0293] 3. Examples of Problems Solved by Embodiments
[0294] It remains unclear how to juxtapose affine prediction with other new codecs such as HMVP.
[0295] 4. Examples of Embodiments
[0296] In this document, we propose methods to control the bandwidth required for affine prediction in a more flexible way. We also propose to juxtapose affine prediction with other codec tools.
[0297] The following detailed techniques should be considered as examples to explain the general concept. These techniques should not be interpreted in a narrow sense. In addition, these techniques can be combined in any way. Combinations between these techniques and other techniques are also applicable.
[0298] In the following discussion, it is assumed that the coordinates of the upper left corner / upper right corner / lower left corner / lower right corner of the neighboring block of the current block (for example, the upper or left neighboring CU) are (LTNx, LTNy) / (RTNx, RTNy) / (LBNx, LBNy) / (RBNx, RBNy) respectively; the coordinates of the upper left corner / upper right corner / lower left corner / lower right corner of the current CU are (LTCx, LTCy) / (RTCx, RTCy) / (LBCx, LBCy) / (RBCx, RBCy) respectively; the width and height of the upper or left neighboring CU of affine coding are w' and h' respectively; the width and height of the current CU of affine coding are w and h respectively.
[0299] The CPMVs of the upper left corner, upper right corner, and lower left corner are represented as MV0=(MV0x, MV0y), MV1=(MV1x, MV1y), and MV2=(MV2x, MV2y), respectively.
[0300] In the following discussion, SignShift(x, n) is defined as
[0301]
[0302] In one example, offset0 and offset1 are set to (1<<(n-1)). In another example, they are set to 0.
[0303] Shift is defined as
[0304] Shift(x,n)=(x+offsset)>>n,
[0305] In one example, offset is set to (1<<(n-1)). In another example, it is set to 0.
[0306] Clip3(min, max, x) is defined as
[0307]
[0308] It should be noted that when other types of sub-block Merge candidates such as ATMVP candidates are also put into the list or other types of Merge lists that may include at least one affine Merge candidate, the term "affine Merge candidate list" may be renamed (e.g., "sub-block Merge candidate list").
[0309] The proposed technique may also be applicable to other kinds of motion candidate lists, such as affine AMVP candidate lists.
[0310] History-based affine parameter inheritance
[0311] 1. The parameters a, b, c, d, e and f of the affine codec block defined in equation (2) can be stored in a buffer (the buffer can be a table, or a lookup table, or a First-In-First-Out (FIFO) table, or a stack, or a queue, or a list, or a link, or an array, or any other storage with any data structure) or a constrained FIFO table, where each affine model is unique. In the following discussion, an entry in the buffer is denoted as H[i], where i is the index of the reference entry.
[0312] a. Alternatively, a, b, c, and d defined in equation (2) may be stored in a buffer; in this case, e and f are no longer stored.
[0313] b. Alternatively, if encoding and decoding is performed in 4-parameter affine mode, a and b defined in equation (1) can be stored in a buffer;
[0314] c. Alternatively, if encoding and decoding is performed in 4-parameter affine mode, a, b, e and f defined in equation (1) can be stored in a buffer;
[0315] d. If encoding and decoding is performed in 4-parameter affine mode, the parameters a, b, c, d, e and f defined in equation (2) are always stored in the buffer, but are restricted to c = -b, d = a.
[0316] e. If encoding and decoding is performed in 4-parameter affine mode, the parameters a, b, c and d defined in equation (2) are always stored in the buffer, but are restricted to c = -b, d = a.
[0317] f. The same number of parameters can be stored for 4-parameter and 6-parameter affine models, for example, a, b, c, d, e, and f are stored. In another example, a, b, c, and d are stored.
[0318] g. Alternatively, different numbers of parameters may be stored for 4-parameter and 6-parameter affine models, and the affine model type (ie, 4-parameter or 6-parameter) may also be stored.
[0319] h. Which parameters are to be stored in the buffer may depend on Affine mode, Inter or Merge mode, block size, picture type, etc.
[0320] i. Side information associated with the affine parameters may also be stored in the buffer together with the affine parameters, such as the inter prediction direction (list 0 or list 1, or bidirectional), and the reference index of list 0 and / or list 1. In the present disclosure, when discussing a set of affine parameters stored in a buffer, the associated side information may also be included.
[0321] i. If the affine codec block is bi-directionally predicted, the set of affine parameters to be stored includes the parameters for list 0 and the parameters for list 1.
[0322] (a) Parameters of two reference lists (List 0 and List 1) are stored.
[0323] (b) In one example, the parameters of the two reference lists are stored independently (in two different buffers);
[0324] (c) Alternatively, the parameters of both reference lists may be stored together with the prediction from one to the other.
[0325] j. As an alternative storage method, the CPMV {MV0, MV1} or {MV0, MV1, MV2} of the affine encoded block is stored in the buffer instead of the parameters. The parameters for encoding and decoding a new block can be calculated from {MV0, MV1} or {MV0, MV1, MV2} when needed.
[0326] i. The width of the affine codec block can be stored in the buffer together with the CPMV.
[0327] ii. The height of the affine codec block can be stored in a buffer together with the CPMV.
[0328] iii. The upper left coordinate of the affine codec block can be stored in the buffer together with the CPMV.
[0329] k. In one example, the basic MV in equation (1) Stored together with parameters a and b.
[0330] i. In one example, the coordinates of the location where the base MV is located are also stored along with the parameters a and b.
[0331] l. In one example, the MV in the basic equation (2) Stored together with parameters a, b, c and d.
[0332] i. In one example, the coordinates of the location where the base MV is located are also stored along with the parameters a, b, c, and d.
[0333] m. In one example, if the stored parameter sets and their base MVs refer to the same reference picture list, they should refer to the same reference pictures.
[0334] n. The buffer used to store affine related information for encoding / decoding (such as CPMV, affine parameters, basic point position coordinates, block width and height) is also referred to as "affine HMVP buffer" in this document.
[0335] 2. In one example, the parameters to be stored in the buffer can be calculated as follows
[0336] a.
[0337] b.
[0338] c.
[0339] d.
[0340] e. For 4-parameter affine prediction c = -b.
[0341] f. For 4-parameter affine prediction d=a.
[0342] g.
[0343] h.
[0344] i. (e, f) = (mvx, mvy), where (mvx, mvy) can be any MV used to encode and decode a block.
[0345] 3. Propose to calculate the affine model parameters without division operation. Assume that the width and height of the current block, marked as w and h, are equal to 2 WB and 2 HB . P is an integer that defines the precision of the calculation, for example, P is set to 7.
[0346] a.
[0347] b.
[0348] c.
[0349] d.
[0350] 4. The affine model parameters can be further clipped before being stored in the buffer.
[0351] a. In one example, assume that the parameter x (e.g., x=a or b or c or d) is
[0352] K bits are used for storage, then x = Clip3 (-2 K-1 , 2 K-1 -1, x).
[0353] b. For example, a=Clip(-128, 127, a), then a is stored as an 8-bit signed integer.
[0354] 5. Affine model parameters may be pruned before being used to encode / decode an affine codec block (such as to derive the MV of a sub-block).
[0355] a. In one example, a=Clip3(Min_a, Max_a, a), b=Clip3(Min_b, Max_b, b), c=Clip3(Min_c, Max_c, c), d=Clip3(Min_d, Max_d, d) where Min_a / b / c / d and Max_a / b / c / d are called clipping boundaries.
[0356] b. In one example, the cropping boundary may depend on the precision (eg, bit depth) of the affine parameters.
[0357] c. In one example, the cropping boundary may depend on the width and height of the block.
[0358] d. In one example, the cropping boundary may be signaled such as in the VPS / SPS / PPS / picture header / slice header / slice group header.
[0359] e. In one example, the clipping boundaries may depend on the profile and / or level of the standard.
[0360] 6. The affine model parameters of each affine codec block may be stored in a buffer after decoding or encoding the block.
[0361] a. Whether to store the affine model parameters of the affine codec block may depend on the codec affine mode (eg, affine AMVP or affine Merge), the number of affine codec blocks, the location of the affine codec blocks, the block size, etc.
[0362] b. In one example, the affine model parameters of every K-th affine codec block are stored in a buffer after decoding or encoding every K-th affine codec block. That is, the affine model parameters of every first, second, ... (K-1)-th affine codec block are not stored in the buffer.
[0363] iK is a number such as 2 or 4.
[0364] ii. K can be signaled from the encoder to the decoder in VPS / SPS / PPS / slice header / slice group header / slice.
[0365] 7. The buffer used to store affine parameters can have a maximum capacity.
[0366] a. The buffer can store at most M sets of affine parameters, i.e., for H[i], i>=0 and i <M。
[0367] iM is an integer such as 8 or 16.
[0368] ii.M can be signaled from the encoder to the decoder in VPS / SPS / PPS / slice header / slice group header / slice / CTU line / CTU.
[0369] iii.M may be different for different standard profiles / levels / tiers.
[0370] 8. When the buffer for affine parameter storage is not full, (i.e., the number S of sets of stored affine parameters is less than the maximum capacity M), and a new set of affine parameters needs to be stored in the buffer, H[S-1] is used to store the new parameters and then S=S+1.
[0371] 9. When the buffer is full (i.e., the number S of sets of stored affine parameters is equal to the maximum capacity M), and a new set of affine parameters needs to be stored in the buffer, one or some of the following strategies may be performed:
[0372] a. The new set of affine parameters cannot be stored in the buffer;
[0373] b. An entry already in the buffer is removed and a new set of affine parameters is stored in the buffer.
[0374] i. In one example, the oldest entry stored in the buffer, such as H[0], is removed from the buffer.
[0375] ii. In one example, the last entry stored in the buffer, eg, H[M-1], is removed from the buffer.
[0376] iii. In one example, any entry stored in the buffer, such as H[T], is removed from the buffer for which T>=0 and T <M。
[0377] iv. If H[T] is removed, the new set of affine parameters is stored as H[T].
[0378] v. If H[T] is removed, all entries after H[T] are moved forward. For example, for X in ascending order from T to M-1, H[X] = H[X+1]. Then the new set of affine parameters is placed at the last entry in the buffer, for example H[M-1];
[0379] vi. If H[T] is removed, all entries before H[T] are moved backwards. For example, for X in descending order from T to 1, H[X] = H[X-1]. Then the new set of affine parameters is placed in the first entry in the buffer, such as H[0];
[0380] 10. When a new set of affine parameters needs to be stored in the buffer, it can be compared with all or some of the sets of affine parameters already in the buffer. If it is determined to be the same or similar to at least one set of affine parameters already in the buffer, it should not be stored in the buffer. This process is known as "pruning".
[0381] a. For one reference picture list (one prediction direction), the affine parameters {a, b, c, d} or {a, b, c, d, e, f} and the affine parameters {a', b', c', d'} or {a', b', c', d', e', f'} are considered to be the same or similar if:
[0382] i. In one example, a == a'.
[0383] ii. In one example, b == b'.
[0384] iii. In one example, c == c'.
[0385] iv. In one example, d == d'.
[0386] v. In one example, a == a' and b == b'.
[0387] vi. In one example, c == c' and d == d'.
[0388] vii. In one example, a == a' and b == b' and c == c'.
[0389] viii. In one example, a == a' and b == b' and c == c' and d = d'.
[0390] ix. In one example, |a - a'| < delta0.
[0391] x. In one example, |b - b'| < delta0.
[0392] xi. In one example, |c - c'| < delta0.
[0393] xii. In one example, |d - d'| < delta0.
[0394] xiii. In one example, |a - a'| < delta0 and |b - b'| < delta1.
[0395] xiv. In one example, |c - c'| < delta0 and |d - d'| < delta1.
[0396] xv. In one example, |a - a'| < delta0 and |b - b'| < delta1 and |c - c'| < delta2.
[0397] xvi. In one example, |a - a'| < delta0 and |b - b'| < delta1 and |c - c'| < delta2 and |d - d'| < delta3.
[0398] xvii. Variables (e.g., delta0, delta1, delta2, delta3) can be predefined numbers, or they can depend on codec information such as block width / height. They can be different for different standard profiles / levels / layers. They can be signaled from the encoder to the decoder in the VPS / SPS / PPS / slice header / picture group header / picture / CTU row / CTU.
[0399] b. In the following cases, two sets of affine parameters are considered not to be the same or similar:
[0400] i. They are associated with different inter - prediction directions (list 0 or list 1, or bi - directional),
[0401] ii. When list 0 is one of the prediction directions in use, they are associated with different reference indices of list 0.
[0402] iii. When list 1 is one of the prediction directions in use, they are associated with different reference indices of list 1.
[0403] iv. They have different numbers of affine parameters or they use different affine models.
[0404] c. If both sets of affine parameters are related from bi-directional prediction, then they are judged to be the same (or similar) if the parameters of list 0 are judged to be the same (or similar) and the parameters of list 1 are also judged to be the same (or similar).
[0405] d. The new set of affine parameters can be compared with every set of affine parameters already in the buffer.
[0406] i. Alternatively, the new set of affine parameters is compared only with some of the sets of affine parameters already in the buffer. For example, it is compared with the first W entries, such as H[0]...H[W-1]. In another example, it is compared with the last W entries, such as H[MW], H[M-1]. In another example, it is compared with one entry in every W entries, such as H[0], H[W], H[2*W]...
[0407] e. If an entry in the buffer (denoted as H[T]) is found to be the same or similar to the new set of affine parameters that need to be stored in the buffer, then
[0408] iH[T] is removed and the new set of affine parameters is stored as H[T].
[0409] ii. H[T] is removed, and then all entries after H[T] are moved forward. For example, for X in ascending order from T to M-1, H[X] = H[X+1]. Then the new set of affine parameters is placed at the last entry in the buffer, for example H[M-1].
[0410] iii. H[T] is removed, and then all entries before H[T] are moved backwards. For example, for X in descending order from T to 1, H[X] = H[X-1]. Then the new set of affine parameters is placed in the first entry in the buffer, such as H[0].
[0411] 11. The buffer storing the affine parameters can be flushed.
[0412] a. The buffer is cleared when it is flushed.
[0413] b. The buffer is cleared on refresh, and then one or more default affine parameters are placed in the buffer on refresh.
[0414] i. The default affine parameters can be different for different sequences;
[0415] ii. The default affine parameters can be different for different images;
[0416] iii. The default affine parameters can be different for different strips;
[0417] iv. The default affine parameters can be different for different slices;
[0418] v. The default affine parameters can be different for different CTU (i.e. LCU) lines;
[0419] vi. The default affine parameters can be different for different CTUs;
[0420] vii. Default affine parameters may be signaled from the encoder to the decoder in VPS / SPS / PPS / slice header / slice group header / slice / CTU line / CTU.
[0421] c. The buffer is flushed when the following is done
[0422] i. Start encoding / decoding the first block of the picture;
[0423] ii. Start encoding / decoding the first block of the strip;
[0424] iii. Start encoding / decoding the first block of the slice;
[0425] iv. Start encoding / decoding the first block of the CTU (ie LCU) line;
[0426] v. Start encoding / decoding the first block of CTU;
[0427] 12. The affine model parameters stored in the buffer can be used to derive the affine prediction of the current block.
[0428] a. In one example, the parameters stored in the buffer may be used for motion vector prediction or motion vector encoding and decoding of the current block.
[0429] b. In one example, the parameters stored in the buffer can be used to derive the control point MV (CPMV) of the current affine codec block.
[0430] c. In one example, the parameters stored in the buffer can be used to derive the MV used in motion compensation of the sub-blocks of the current affine coded block.
[0431] d. In one example, the parameters stored in the buffer can be used to derive a prediction of the CPMV of the current affine-encoded block. When the CPMV needs to be encoded, this prediction of the CPMV can be used to predict the CPMV of the current block.
[0432] i. In one example, if the current block is encoded with a 4-parameter affine model, a higher priority is assigned to the 4-parameter affine model, and a lower priority is assigned to the 6-parameter affine model.
[0433] ii. In one example, if the current block is encoded and decoded using a 6-parameter affine model, a higher priority is assigned to the 6-parameter affine model, and a lower priority is assigned to the 4-parameter affine model.
[0434] 13. The motion information of the adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer can be used together to derive the affine model of the current block. For example, they can be used to derive the CPMV or MV of the sub-block used in motion compensation, such as Fig.15 shown.
[0435] a. Assume that the MV stored in the unit block is (mv h 0,mv v 0), and derive MV (mv h (x,y),mv v The coordinates of the position of (x, y)) are expressed as (x, y). Assuming that the coordinates of the upper left corner of the current block are (x0', y0'), and the width and height of the current block are w and h, then
[0436] i. To derive CPMV, (x, y) can be (x0', y0'), or (x0'+w, y0'), or (x0', y0'+h), or (x0'+w, y0'+h).
[0437] ii. To derive the MV of the sub-block of the current block, (x, y) can be the center of the sub-block. Assuming (x00, y00) is the upper left position of the sub-block and the sub-block size is M×N, then
[0438] (a)xm=x00+M / 2, ym=y00+N / 2;
[0439] (b)xm=x00+M / 2-1, ym=y00+N / 2-1;
[0440] (c)xm=x00+M / 2-1, ym=y00+N / 2;
[0441] (d)xm=x00+M / 2, ym=y00+N / 2-1;
[0442] iii. In one example,
[0443]
[0444] If the parameters in the buffer come from a block encoded with 4-parameter affine mode.
[0445] iv. In one example,
[0446]
[0447] If the parameters in the buffer come from a block encoded with 6-parameter affine mode.
[0448] v. In one example,
[0449]
[0450] Regardless of whether the parameters in the buffer come from a block encoded or decoded using the 4-parameter affine mode or the 6-parameter affine mode.
[0451] b. In one example, the CPMVs of the current block are derived from the motion vectors and the parameters stored in the buffer, and these CPMVs serve as the MVPs of the signaled CPMVs of the current block.
[0452] c. In one example, the CPMV of the current block is derived from the motion vector and the parameters stored in the buffer, and these CPMVs are used to derive the MV of each sub-block for motion compensation.
[0453] d. In one example, if the current block is affine-merge coded, the MV of each sub-block used for motion compensation is derived from the motion vector and parameters stored in neighboring blocks.
[0454] e. In one example, the set of motion vectors and parameters of neighboring unit blocks used to derive the CPMV or MV of a sub-block used in motion compensation of the current block should comply with some or all of the following constraints:
[0455] i. they are associated with the same inter prediction direction (list 0 or list 1, or bidirectional),
[0456] ii. When list 0 is the one prediction direction in use, they are associated with the same reference index of list 0.
[0457] iii. When List 1 is the one prediction direction in use, they are associated with the same reference index of List 1.
[0458] Fig.15 is an example of deriving the CPMV from the MVs of neighboring blocks and a set of parameters stored in a buffer.
[0459] 14. The affine model of the current block derived from the set of affine parameters stored in the buffer can be used to generate an affine Merge candidate.
[0460] a. In one example, side information such as inter prediction direction and reference index of list0 / list1 associated with stored parameters is inherited by the generated affine merge candidate.
[0461] b. The affine Merge candidate derived from the set of affine parameters stored in the buffer can be inserted into the affine Merge candidate list after the affine Merge candidate inherited from the adjacent block and before the constructed affine Merge candidate;
[0462] c. Affine Merge candidates derived from the set of affine parameters stored in the buffer can be inserted into the affine Merge candidate list after the constructed affine Merge candidates and before the filled candidates.
[0463] d. Affine Merge candidates derived from the set of affine parameters stored in the buffer can be used without using temporal motion prediction ( Figure 5 After constructing the affine Merge candidate of the block T in Figure 5 The constructed affine merge candidates of the block T in are inserted into the affine merge list before;
[0464] e. Affine Merge candidates derived from the set of affine parameters stored in the buffer may be inserted into the affine Merge candidate list, and they may be interleaved with constructed affine Merge candidates and / or fill candidates.
[0465] 15. The affine parameters stored in the buffer can be used to generate affine AMVP candidates.
[0466] a. In one example, the stored parameters used to generate the affine AMVP candidate should reference the same reference picture as the target reference picture of the affine AMVP codec block.
[0467] i. In one example, the reference picture list associated with the stored parameters should be the same as the target reference picture list.
[0468] ii. In one example, the reference index associated with the stored parameters should be the same as the target reference index.
[0469] b. The affine AMVP candidate derived from the set of affine parameters stored in the buffer can be inserted into the affine AMVP candidate list after the affine AMVP candidate inherited from the neighboring block and before the constructed affine AMVP candidate;
[0470] c. The affine AMVP candidate derived from the set of affine parameters stored in the buffer may be inserted into the affine AMVP candidate list after the constructed affine AMVP candidate and before the HEVC-based affine AMVP candidate.
[0471] d. The affine AMVP candidate derived from the set of affine parameters stored in the buffer may be inserted into the affine AMVP candidate list after the HEVC-based affine AMVP candidate and before the padding affine AMVP candidate.
[0472] e. Affine AMVP candidates derived from a set of affine parameters stored in a buffer can be generated without using temporal motion prediction ( Figure 5 After constructing the affine AMVP candidate of the block T in Figure 5 The constructed affine AMVP candidates of block T in are previously inserted into the affine AMVP list;
[0473] f. In one example, if the current block is encoded with a 4-parameter affine model, a higher priority is assigned to the 4-parameter affine model, and a lower priority is assigned to the 6-parameter affine model.
[0474] g. In one example, if the current block is encoded with a 6-parameter affine model, a higher priority is assigned to the 6-parameter affine model, and a lower priority is assigned to the 4-parameter affine model.
[0475] 16. How many sets of affine model parameters in the buffer are added to the candidate list (denoted by N) can be predefined.
[0476] aN may be signaled from the encoder to the decoder in VPS / SPS / PPS / slice header / slice group header / slice.
[0477] bN may depend on block size, coding mode information (eg, AMVP / Merge), etc.
[0478] cN may depend on the standard profile / level / tier.
[0479] dN may depend on the available candidates in the list.
[0480] iN may depend on the specific type of available candidates (e.g., inherited affine motion candidates)
[0481] 17. How to select the portion of the total set of affine model parameters in the buffer to be inserted into the candidate list (eg, N as in bullet 15) may be predefined.
[0482] a. In one example, the latest several sets in the buffer (eg, the last N entries).
[0483] b. It may depend on the index of the set of affine model parameters in the buffer.
[0484] 18. When multiple sets of affine model parameters need to be inserted into the candidate list, they can be added in ascending order of index.
[0485] a. Alternatively, they can be added in descending order of index.
[0486] b. Alternatively, the rule that determines the insertion order depends on the number of available candidates in the candidate list before adding those parameters from the buffer.
[0487] 19. The set of affine parameters stored in the buffer and their associated basic MVs and the locations of the basic MVs can be used together to derive the affine model of the current block. For example, they can be used to derive the CPMV or MV of the sub-block used in motion compensation.
[0488] a. Assume that the associated basic MV is (mv h 0,mv v 0), and derive MV (mv h (x,y),mv v The coordinates of the position of (x, y)) are expressed as (x, y). Assuming that the coordinates of the upper left corner of the current block are (x0', y0'), and the width and height of the current block are w and h, then
[0489] i. To derive CPMV, (x, y) can be (x0', y0'), or (x0'+w, y0'), or (x0', y0'+h), or (x0'+w, y0'+h).
[0490] ii. To derive the MV of a sub-block of the current block, (x, y) may be the center of the sub-block.
[0491] iii. Assume that (xm, ym) is the stored coordinates of the location (basic location) where the basic MV is located.
[0492] iv. In one example,
[0493]
[0494] If the parameters in the buffer come from a block encoded with 4-parameter affine mode.
[0495] v. In one example,
[0496]
[0497] If the parameters in the buffer come from a block encoded with 6-parameter affine mode.
[0498] vi. In one example,
[0499]
[0500] Regardless of whether the parameters in the buffer come from a block encoded or decoded using the 4-parameter affine mode or the 6-parameter affine mode.
[0501] b. In one example, the CPMVs of the current block are derived from the motion vectors and the parameters stored in the buffer, and these CPMVs serve as the MVPs of the signaled CPMVs of the current block.
[0502] c. In one example, the CPMV of the current block is derived from the associated base MV and parameters stored in a buffer, and these CPMVs are used to derive the MV of each sub-block for motion compensation.
[0503] d. In one example, if the current block is affine-merge coded, the MV of each sub-block used for motion compensation is derived from the associated basic MV and parameters stored in neighboring blocks.
[0504] 20. The motion information of spatially adjacent / non-adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer can be used together to derive the affine model of the current block. For example, they can be used to derive the CPMV or MV of the sub-block used in motion compensation.
[0505] a. Assume that the MV stored in the unit block is (mv h 0,mv v 0), and derive MV (mv h (x,y),mv v The coordinates of the position of (x, y)) are expressed as (x, y). Assuming that the coordinates of the upper left corner of the current block are (x0', y0'), and the width and height of the current block are w and h, then
[0506] i. To derive CPMV, (x, y) can be (x0', y0'), or (x0'+w, y0'), or (x0', y0'+h), or (x0'+w, y0'+h).
[0507] ii. To derive the MV of a sub-block of the current block, (x, y) may be the center of the sub-block.
[0508] iii. Assuming (x00, y00) is the upper left position of the spatial adjacent M×N unit block, the basic position (xm, ym) can be derived as:
[0509] (a)xm=x00+M / 2, ym=y00+N / 2;
[0510] (b)xm=x00+M / 2-1, ym=y00+N / 2-1;
[0511] (c)xm=x00+M / 2-1, ym=y00+N / 2;
[0512] (d)xm=x00+M / 2, ym=y00+N / 2-1;
[0513] iv. In one example,
[0514]
[0515] If the parameters in the buffer come from a block encoded with 4-parameter affine mode.
[0516] v. In one example,
[0517]
[0518] If the parameters in the buffer come from a block encoded with 6-parameter affine mode.
[0519] vi. In one example,
[0520]
[0521] Regardless of whether the parameters in the buffer come from a block encoded or decoded using the 4-parameter affine mode or the 6-parameter affine mode.
[0522] b. In one example, the CPMV of the current block is derived from the motion vectors of the spatial neighboring unit blocks and the parameters stored in the buffer, and these CPMVs serve as the MVP of the signaled CPMV of the current block.
[0523] c. In one example, the CPMV of the current block is derived from the motion vectors of spatially neighboring unit blocks and the parameters stored in a buffer, and these CPMVs are used to derive the MV of each sub-block for motion compensation.
[0524] d. In one example, if the current block is affine merge coded, the MV of each sub-block used for motion compensation is derived from the motion vector of the spatial neighboring unit block and the parameters stored in the neighboring block.
[0525] e. In one example, the set of motion vectors and parameters of the spatial neighboring unit blocks used to derive the CPMV or the MV of the sub-block used in motion compensation of the current block should comply with some or all of the following constraints:
[0526] i. they are associated with the same inter prediction direction (list 0 or list 1, or bidirectional),
[0527] ii. When list 0 is the one prediction direction in use, they are associated with the same reference index of list 0.
[0528] iii. When List 1 is the one prediction direction in use, they are associated with the same reference index of List 1.
[0529] f. In one example, if the MV of the spatially adjacent M×N unit block and the stored affine parameters refer to different reference pictures, the MV of the spatially adjacent M×N unit block is scaled to refer to the same reference picture as the stored affine parameters to derive the affine model of the current block.
[0530] 21. It is proposed that temporal motion vector prediction (TMVP) can be used together with the affine parameters stored in the buffer. For example, they can be used to derive the CPMV or MV of a sub-block used in motion compensation.
[0531] a. The motion information of the collocated M×N unit blocks (e.g., 4×4 blocks in VTM) in the collocated pictures and the set of affine parameters stored in the buffer can be used together to derive the affine model of the current block. For example, they can be used to derive the CPMV or MV of the sub-block used in motion compensation.
[0532] i. Fig.25 Examples of possible locations of juxtaposed unit blocks are shown. (A1-A4, B1-B4, ... F1-F4, J1-J4, K1-K4, and L1-L4.
[0533] b. Assume that the MV stored in the collocated unit block is (mv h 0,mv v 0), and derive MV (mv h (x,y),mv v The coordinates of the position of (x, y)) are expressed as (x, y). Assuming that the coordinates of the upper left corner of the current block are (x0', y0'), and the width and height of the current block are w and h, then
[0534] i. To derive CPMV, (x, y) can be (x0', y0'), or (x0'+w, y0'), or (x0', y0'+h), or (x0'+w, y0'+h).
[0535] ii. To derive the MV of a sub-block of the current block, (x, y) may be the center of the sub-block.
[0536] iii. Assuming (x00, y00) is the upper left position of the concatenated M×N unit block, the basic position (xm, ym) can be derived as:
[0537] (a)xm=x00+M / 2, ym=y00+N / 2;
[0538] (b)xm=x00+M / 2-1, ym=y00+N / 2-1;
[0539] (c)xm=x00+M / 2-1, ym=y00+N / 2;
[0540] (d)xm=x00+M / 2, ym=y00+N / 2-1;
[0541] iv. In one example,
[0542]
[0543] If the parameters in the buffer come from a block encoded with 4-parameter affine mode.
[0544] v. In one example,
[0545]
[0546] If the parameters in the buffer come from a block encoded with 6-parameter affine mode.
[0547] vi. In one example,
[0548]
[0549] Regardless of whether the parameters in the buffer come from a block encoded or decoded using the 4-parameter affine mode or the 6-parameter affine mode.
[0550] c. In one example, the CPMV of the current block is derived from the motion vectors of temporally neighboring blocks and the parameters stored in the buffer, and these CPMVs serve as the MVP of the signaled CPMV of the current block.
[0551] d. In one example, the CPMV of the current block is derived from the motion vectors of temporally neighboring blocks and parameters stored in a buffer, and these CPMVs are used to derive the MV of each sub-block for motion compensation.
[0552] e. In one example, if the current block is affine Merge coded, the MV of each sub-block used for motion compensation is derived from the motion vector of the temporal neighboring block and the parameters stored in the neighboring block.
[0553] f. In one example, the set of motion vectors and parameters of temporal neighboring unit blocks used to derive the CPMV or MV of a sub-block used in motion compensation of the current block should comply with some or all of the following constraints:
[0554] i. they are associated with the same inter prediction direction (list 0 or list 1, or bidirectional),
[0555] ii. When list 0 is the one prediction direction in use, they are associated with the same reference index of list 0.
[0556] iii. When List 1 is the one prediction direction in use, they are associated with the same reference index of List 1.
[0557] g. In one example, if the MV of the temporally adjacent M×N unit block and the stored affine parameters refer to different reference pictures, the MV of the spatial-temporal M×N unit block is scaled to refer to the same reference picture as the stored affine parameters to derive the affine model of the current block.
[0558] i. For example, if the POC of the collocated picture is POCx; the POC of the reference picture referenced by the MV of the temporally adjacent M×N unit block is POCy; the POC of the current picture is POCz; the POC of the reference picture referenced by the stored affine parameters is POCw, then (mv h 0,mv v 0) Scale to
[0559] mv h 0 = mv h 0×(POCw-POCz) / (POCy-POCx) and
[0560] mv v 0 = mv v 0×(POCw-POCz) / (POCy-POCx).
[0561] 22. Affine Merge candidates derived from the parameters stored in the buffer and one or more spatially adjacent / non-adjacent unit blocks can be placed in an affine Merge candidate list.
[0562] a. In one example, these candidates are placed just after the inherited affine Merge candidates.
[0563] b. In one example, these candidates are placed just after the first constructed Affine Merge candidates.
[0564] c. In one example, these candidates are placed just after the first affine Merge candidate constructed from spatially neighboring blocks.
[0565] d. In one example, these candidates are placed just after all constructed affine merge candidates.
[0566] e. In one example, these candidates are placed just before the all-zero affine Merge candidates.
[0567] f. In one example, if another affine Merge candidate is inherited from a spatial neighboring unit block, the spatial neighboring unit block is not used to derive the affine Merge candidate together with the parameters stored in the buffer.
[0568] g. In one example, a spatially adjacent unit block can be used to derive an affine Merge candidate together with only one set of parameters stored in a buffer. In other words, if a spatially adjacent unit block and a set of parameters stored in a buffer have already derived an affine Merge candidate, it cannot be used to derive another affine Merge candidate together with another set of parameters stored in a buffer.
[0569] h. In one example, at most N affine merge candidates derived from the parameters stored in the buffer and the spatial neighboring unit blocks can be put into the affine merge candidate list. N is an integer such as 3.
[0570] i. In one example, if an affine Merge candidate derived from the parameters stored in the buffer and the spatial neighboring unit blocks is selected, the GBI index of the current block is inherited from the GBI index of the spatial neighboring blocks.
[0571] j. In one example, the affine merge candidates derived from the parameters stored in the buffer and the spatial neighboring blocks are sequentially placed into the affine merge candidate list.
[0572] i. For example, a two-level nested loop method is used to search for available affine merge candidates derived from the parameters stored in the buffer and the spatial neighboring blocks, and put them into the affine merge candidate list.
[0573] (a) In the first level loop, each set of parameters stored in the buffer is accessed sequentially. They can be accessed from the beginning to the end of the list, or from the end to the beginning of the list, or in any other predefined or adaptive order.
[0574] a. In the example, some sets of parameters stored in the buffer are skipped in the first cycle. For example, the first N or last N sets in the table are skipped. Alternatively, if k%S==0, then H[k] is skipped. Alternatively, if k%S!=0, then H[k] is skipped.
[0575] (b) For each set of parameters stored in the buffer, a second level loop is applied. In the second level loop, each spatially adjacent block is accessed sequentially. For example, Figure 5 Blocks A1, B1, B0, A0 and B2 are shown. In a pseudo-code implementation, the nested loops can be described as:
[0576]
[0577] a. In one example, only one spatial neighboring block may be included in the second loop. For example, only A1 is included.
[0578] b. Using the set of parameters given in the first-level loop and the spatial neighboring blocks given in the second-level loop, if all or some of the following conditions are met, an affine merge candidate is generated and placed in the affine merge candidate list.
[0579] i. Spatial adjacent blocks are available;
[0580] ii. Spatial adjacent blocks are inter-frame coded;
[0581] iii. The spatial neighboring blocks are not outside the current CTU row.
[0582] iv. The inter prediction of the parameter set (list 0, list 1, or bidirectional) and the inter prediction of the spatial neighboring block (list 0, list 1, or bidirectional) are the same;
[0583] v. The reference index of list 0 of the set of parameters and the reference index of list 0 of the spatial neighbor block are the same;
[0584] vi. The reference index of list 1 of the set of parameters and the reference index of list 1 of the spatial neighboring block are the same;
[0585] vii. The POC of the reference picture of list 0 of the set of parameters is the same as the POC of one of the reference pictures of the spatial neighboring block.
[0586] viii. The POC of the reference picture of list 1 of the set of parameters is the same as the POC of one of the reference pictures of the spatial neighboring block.
[0587] c. In one example, if a neighboring block has been used to derive an inherited affine Merge candidate, it is skipped in the second cycle and is not used to derive an affine Merge candidate together with the stored affine parameters.
[0588] d. In one example, if a neighboring block has been used to derive an affine Merge candidate together with a set of stored affine parameters, it is skipped in the second loop and is not used to derive an affine Merge candidate together with another set of stored affine parameters.
[0589] e. In one example, if a neighboring block is used to derive an affine Merge candidate, all other neighboring blocks after the neighboring block are skipped, and the second loop is jumped out and returned to the first loop. The next set of parameters is accessed in the first loop.
[0590] 23. Affine Merge candidates derived from the parameters stored in the buffer and one or more time-domain unit blocks can be placed in an affine Merge candidate list.
[0591] a. In one example, these candidates are placed just after the inherited affine Merge candidates.
[0592] b. In one example, these candidates are placed just after the first constructed Affine Merge candidates.
[0593] c. In one example, these candidates are placed just after the first affine Merge candidate constructed from spatially neighboring blocks.
[0594] d. In one example, these candidates are placed just after all constructed affine merge candidates.
[0595] e. In one example, these candidates are placed just after all affine Merge candidates derived from the parameters stored in the buffer and the spatial neighboring cells.
[0596] f. In one example, these candidates are placed just before the all-zero affine Merge candidates.
[0597] g. In one example, at most N affine merge candidates derived from the parameters stored in the buffer and the temporal neighboring unit blocks can be put into the affine merge candidate list. N is an integer such as 3.
[0598] h. In one example, if an affine Merge candidate derived from the parameters stored in the buffer and the temporal neighboring unit block is selected, the GBI index of the current block is inherited from the GBI index of the temporal neighboring block.
[0599] i. In one example, the affine merge candidates derived from the parameters stored in the buffer and the temporal neighboring blocks are sequentially placed into the affine merge candidate list.
[0600] i. For example, a two-level nested loop method is used to search for available affine merge candidates derived from parameters stored in a buffer and temporal neighboring blocks, and put them into an affine merge candidate list.
[0601] (a) In the first level loop, each set of parameters stored in the buffer is accessed sequentially. They can be accessed from the beginning to the end of the list, or from the end to the beginning of the list, or in any other predefined or adaptive order.
[0602] a. In the example, some sets of parameters stored in the buffer are skipped in the first cycle. For example, the first N or last N sets in the table are skipped. Alternatively, if k%S==0, then H[k] is skipped. Alternatively, if k%S!=0, then H[k] is skipped.
[0603] (b) For each set of parameters stored in the buffer, a second level loop is applied. In the second level loop, each temporal neighboring block is accessed sequentially. For example, Fig.25 Blocks L4 and E4 are shown. In a pseudo-code implementation, the nested loops can be described as:
[0604]
[0605] a. In one example, only one temporal neighboring block may be included in the second loop. For example, only L4 is included.
[0606] b. Using the set of parameters given in the first-level loop and the neighboring blocks given in the second-level loop, if all or some of the following conditions are met, an affine merge candidate is generated and put into the affine merge candidate list.
[0607] i. Adjacent blocks are available;
[0608] ii. The adjacent blocks are inter-frame coded;
[0609] iii. The neighboring block is not outside the current CTU row.
[0610] iv. The inter prediction of the set of parameters (list 0, list 1, or bidirectional) is the same as the inter prediction of the neighboring block;
[0611] v. The reference index of list 0 of the set of parameters and the reference index of list 0 of the neighboring block are the same;
[0612] vi. The reference index of list 1 of the set of parameters and the reference index of list 1 of the adjacent block are the same;
[0613] vii. The POC of the reference picture of list 0 of the set of parameters is the same as the POC of one of the reference pictures of the neighboring block.
[0614] viii. The POC of the reference picture of list 1 of the set of parameters is the same as the POC of one of the reference pictures of the neighboring block.
[0615] c. In one example, if a neighboring block has been used to derive an inherited affine Merge candidate, it is skipped in the second cycle and is not used to derive an affine Merge candidate together with the stored affine parameters.
[0616] d. In one example, if a neighboring block has been used to derive an affine Merge candidate together with a set of stored affine parameters, it is skipped in the second loop and is not used to derive an affine Merge candidate together with another set of stored affine parameters.
[0617] e. In one example, if a neighboring block is used to derive an affine Merge candidate, all other neighboring blocks after the neighboring block are skipped, and the second loop is jumped out and returned to the first loop. The next set of parameters is accessed in the first loop.
[0618] 24. Affine AMVP candidates derived from the parameters stored in the buffer and one or more spatially adjacent / non-adjacent unit blocks may be placed into an affine AMVP candidate list.
[0619] a. In one example, these candidates are placed just after the inherited affine AMVP candidates.
[0620] b. In one example, these candidates are placed just after the first built AMVP Merge candidates.
[0621] c. In one example, these candidates are placed just after the first affine AMVP candidate constructed from spatial neighbors.
[0622] d. In one example, these candidates are placed just after all constructed affine AMVP candidates.
[0623] e. In one example, these candidates are placed just after the first translational affine AMVP candidate.
[0624] f. In one example, these candidates are placed just after all translational affine AMVP candidates.
[0625] g. In one example, these candidates are placed just before the all zero-affine AMVP candidates.
[0626] h. In one example, if another affine AMVP candidate is inherited from a spatial neighboring unit block, the spatial neighboring unit block is not used to derive the affine AMVP candidate together with the parameters stored in the buffer.
[0627] i. In one example, a spatial neighboring unit block can be used to derive an affine AMVP candidate together with only one set of parameters stored in a buffer. In other words, if a spatial neighboring unit block and a set of parameters stored in a buffer have already derived an affine AMVP candidate, it cannot be used to derive another affine AMVP candidate together with another set of parameters stored in a buffer.
[0628] j. In one example, at most N affine AMVP candidates derived from the parameters stored in the buffer and the spatial neighboring unit blocks can be put into the affine AMVP candidate list. N is an integer such as 1.
[0629] k. In one example, the affine AMVP candidates derived from the parameters stored in the buffer and the spatial neighboring blocks are sequentially placed into the affine AMVP candidate list.
[0630] i. For example, a two-level nested loop method is used to search for available affine AMVP candidates derived from the parameters stored in the buffer and the spatial neighboring blocks, and put them into the affine AMVP candidate list.
[0631] (a) In the first level loop, each set of parameters stored in the buffer is accessed sequentially. They can be accessed from the beginning to the end of the list, or from the end to the beginning of the list, or in any other predefined or adaptive order.
[0632] a. In the example, some sets of parameters stored in the buffer are skipped in the first cycle. For example, the first N or last N sets in the table are skipped. Alternatively, if k%S==0, then H[k] is skipped. Alternatively, if k%S!=0, then H[k] is skipped.
[0633] (b) For each set of parameters stored in the buffer, a second level loop is applied. In the second level loop, each spatially adjacent block is accessed sequentially. For example, Figure 5 Blocks A1, B1, B0, A0 and B2 are shown. In a pseudo-code implementation, the nested loops can be described as:
[0634]
[0635] a. In one example, only one spatial neighboring block may be included in the second loop. For example, only A1 is included.
[0636] b. Using the set of parameters given in the first-level loop and the spatial neighboring blocks given in the second-level loop, if some or all of the following conditions are met, an affine AMVP candidate is generated and placed in the affine AMVP candidate list.
[0637] i. Spatial adjacent blocks are available;
[0638] ii. Spatial adjacent blocks are inter-frame coded;
[0639] iii. The spatial neighboring blocks are not outside the current CTU row.
[0640] iv. The reference index of list 0 of the set of parameters and the reference index of list 0 of the spatial neighboring block are the same;
[0641] v. The reference index of list 1 of the set of parameters and the reference index of list 1 of the spatial neighboring block are the same;
[0642] vi. The reference index of list 0 of the set of parameters is equal to the reference index of the AMVP signaling notification of list 0.
[0643] vii. The reference index of List 1 of the set of parameters is equal to the reference index of the AMVP signaling notification of List 1.
[0644] viii. The reference index of List 0 of the spatial neighboring block is equal to the reference index of List 0 notified by AMVP signaling.
[0645] ix. The reference index of List 1 of the spatial neighboring blocks is equal to the reference index of List 1 notified by AMVP signaling.
[0646] x. The POC of the reference picture of list 0 of the set of parameters is the same as the POC of one of the reference pictures of the spatial neighboring block.
[0647] xi. The POC of the reference picture of list 1 of the set of parameters is the same as the POC of one of the reference pictures of the spatial neighboring block.
[0648] xii. The POC of the reference picture signaled by AMVP of List 0 is the same as the POC of one of the reference pictures of the spatial neighboring block.
[0649] xiii. The POC of the reference picture signaled by AMVP of List 0 is the same as the POC of one of the reference pictures of the parameter set.
[0650] c. In one example, if a neighboring block has been used to derive an inherited affine AMVP candidate, it is skipped in the second cycle and is not used to derive an affine AMVP candidate together with the stored affine parameters.
[0651] d. In one example, if a neighboring block has been used to derive an affine AMVP candidate with a set of stored affine parameters, it is skipped in the second loop and is not used to derive an affine AMVP candidate with another set of stored affine parameters.
[0652] e. In one example, if a neighboring block is used to derive an affine AMVP candidate, all other neighboring blocks after the neighboring block are skipped, and the second loop is jumped out and returned to the first loop. The next set of parameters is accessed in the first loop.
[0653] 25. Affine AMVP candidates derived from the parameters stored in the buffer and one or more time-domain unit blocks may be placed into an affine AMVP candidate list.
[0654] a. In one example, these candidates are placed just after the inherited affine AMVP candidates.
[0655] b. In one example, these candidates are placed just after the first built AMVP Merge candidates.
[0656] c. In one example, these candidates are placed just after the first affine AMVP candidate constructed from spatial neighbors.
[0657] d. In one example, these candidates are placed just after all constructed affine AMVP candidates.
[0658] e. In one example, these candidates are placed just after the first translational affine AMVP candidate.
[0659] f. In one example, these candidates are placed just after all translational affine AMVP candidates.
[0660] g. In one example, these candidates are placed just before the all zero-affine AMVP candidates.
[0661] h. In one example, these candidates are placed just after all affine AMVP candidates derived from the parameters stored in the buffer and the spatial neighbors.
[0662] i. In one example, at most N affine AMVP candidates derived from the parameters stored in the buffer and the temporal neighboring unit blocks can be put into the affine Merge candidate list. N is an integer such as 1.
[0663] j. In one example, the affine AMVP candidates derived from the parameters stored in the buffer and the temporal neighboring blocks are sequentially placed into the affine AMVP candidate list.
[0664] i. For example, a two-level nested loop method is used to search for available affine AMVP candidates derived from parameters stored in a buffer and temporal neighboring blocks, and put them into an affine AMVP candidate list.
[0665] (a) In the first level loop, each set of parameters stored in the buffer is accessed sequentially. They can be accessed from the beginning to the end of the list, or from the end to the beginning of the list, or in any other predefined or adaptive order.
[0666] a. In the example, some sets of parameters stored in the buffer are skipped in the first cycle. For example, the first N or last N sets in the table are skipped. Alternatively, if k%S==0, then H[k] is skipped. Alternatively, if k%S!=0, then H[k] is skipped.
[0667] (b) For each set of parameters stored in the buffer, a second level loop is applied. In the second level loop, each temporal neighboring block is accessed sequentially. For example, Figure 5 Blocks A1, B1, B0, A0 and B2 are shown. In a pseudo-code implementation, the nested loops can be described as:
[0668]
[0669] a. In one example, only one temporal neighboring block may be included in the second loop. For example, only A1 is included.
[0670] b. Using the set of parameters given in the first-level loop and the temporal neighboring blocks given in the second-level loop, if some or all of the following conditions are met, an affine AMVP candidate is generated and put into the affine AMVP candidate list.
[0671] i. Time domain adjacent blocks are available;
[0672] ii. The temporal adjacent blocks are inter-frame coded;
[0673] iii. The temporal neighboring blocks are not outside the current CTU row.
[0674] iv. The reference index of list 0 of the set of parameters and the reference index of list 0 of the temporal neighboring block are the same;
[0675] v. The reference index of list 1 of the set of parameters and the reference index of list 1 of the temporal neighboring block are the same;
[0676] vi. The reference index of list 0 of the set of parameters is equal to the reference index of the AMVP signaling notification of list 0.
[0677] vii. The reference index of List 1 of the set of parameters is equal to the reference index of the AMVP signaling notification of List 1.
[0678] viii. The reference index of list 0 of the time-domain neighboring block is equal to the reference index of list 0 notified by AMVP signaling.
[0679] ix. The reference index of List 1 of the time domain neighboring blocks is equal to the reference index of List 1 notified by AMVP signaling.
[0680] x. The POC of the reference picture of list 0 of the set of parameters is the same as the POC of one of the reference pictures of the temporal neighboring block.
[0681] xi. The POC of the reference picture of list 1 of the set of parameters is the same as the POC of one of the reference pictures of the temporal neighboring block.
[0682] xii. The POC of the reference picture signaled by AMVP of List 0 is the same as the POC of one of the reference pictures of the temporal neighboring block.
[0683] xiii. The POC of the reference picture signaled by AMVP of List 0 is the same as the POC of one of the reference pictures of the parameter set.
[0684] c. In one example, if a neighboring block has been used to derive an inherited affine AMVP candidate, it is skipped in the second cycle and is not used to derive an affine AMVP candidate together with the stored affine parameters.
[0685] d. In one example, if a neighboring block has been used to derive an affine AMVP candidate with a set of stored affine parameters, it is skipped in the second loop and is not used to derive an affine AMVP candidate with another set of stored affine parameters.
[0686] e. In one example, if a neighboring block is used to derive an affine AMVP candidate, all other neighboring blocks after the neighboring block are skipped, and the second loop is jumped out and returned to the first loop. The next set of parameters is accessed in the first loop.
[0687] 26. It is proposed that the affine merge candidates derived from the affine HMVP buffer are placed in the affine merge list / sub-block merge list, and the inherited affine merge candidates can be removed from the list.
[0688] a. In one example, affine Merge candidates derived from the affine HMVP buffer are put into the affine Merge list / sub-block Merge list, and inherited affine Merge candidates are excluded from the list.
[0689] b. In an alternative example, the affine Merge candidates derived from the affine HMVP buffer are put into the affine Merge list / sub-block Merge list, and the affine Merge candidates inherited from the blocks in the current CTU row are removed from the list.
[0690] i. For example, affine Merge candidates derived from the affine HMVP buffer are put into the affine Merge list / sub-block Merge list after affine Merge candidates inherited from blocks in a CTU row different from the current CTU row.
[0691] c. Alternatively, whether to add an inherited affine merge candidate may depend on the affine HMVP buffer.
[0692] i. In one example, an affine Merge candidate derived from an affine HMVP buffer may be inserted into the candidate list before an inherited affine Merge candidate.
[0693] ii. In one example, when the affine HMVP buffer is empty, the inherited affine Merge candidate may be added; otherwise (if the affine HMVP buffer is not empty), the inherited affine Merge candidate may be excluded.
[0694] d. Alternatively, whether to apply the proposed method may depend on the block size.
[0695] 27. It is proposed that affine AMVP candidates derived from the affine HMVP buffer be placed into the affine AMVP list, and inherited affine AMVP candidates can be removed from the list.
[0696] a. In one example, affine AMVP candidates derived from an affine HMVP buffer are put into an affine AMVP list, and inherited affine AMVP candidates are excluded from the list.
[0697] b. In an alternative example, the derived affine AMVP candidates stored in the affine HMVP buffer are put into the affine AMVP list, and the affine AMVP candidates inherited from the blocks in the current CTU row are removed from the list.
[0698] i. For example, affine AMVP candidates derived from the affine HMVP buffer are placed in the affine AMVP list after affine AMVP candidates inherited from blocks in a CTU row different from the current CTU row.
[0699] c. Alternatively, whether to add an inherited affine AMVP candidate may depend on the affine HMVP buffer.
[0700] d. Alternatively, whether to apply the proposed method may depend on the block size.
[0701] 28. In one example, if the affine merge candidates derived from the parameters stored in the buffer can be put into the list, the size of the affine merge candidate list is increased by N (eg, N=1).
[0702] 29. In one example, if the affine AMVP candidates derived from the parameters stored in the buffer can be put into the list, the size of the affine AMVP candidate list is increased by N (eg, N=1).
[0703] 30. The virtual affine model can be derived from multiple existing affine models stored in the buffer. Assume that the buffer already includes several affine models, and the i-th candidate is represented by Candi, with parameters (ai, bi, ci, di, ei, fi).
[0704] a. In one example, the parameters of Candi and Candj can be combined by taking some parameters from Candi and the remaining parameters from Candj to form a virtual affine model. An example of a virtual affine model is (ai, bi, cj, dj, ei, fi).
[0705] b. In one example, the parameters of Candi and Candj can be used jointly to generate a virtual affine model using a function (such as averaging). An example of a virtual affine model is ((ai+aj) / 2, (bi+bj) / 2, (ci+cj) / 2, (di+dj) / 2, (ei+ej) / 2, (fi+fj) / 2).
[0706] c. Virtual affine models can be used in a similar manner as stored affine models, such as in the bullet points above.
[0707] 31. It is proposed that the affine Merge candidates inherited from spatially adjacent blocks are not placed in the sub-block based Merge candidate list, and the disclosed history-based affine Merge candidates are placed in the sub-block based Merge candidate list.
[0708] a. In one example, the disclosed history-based affine Merge candidate is placed in the sub-block based Merge candidate list right after the ATMVP candidate.
[0709] b. In one example, the disclosed history-based affine merge candidates are put into the sub-block-based merge candidate list before the constructed affine merge candidates.
[0710] c. It is proposed that whether the affine Merge candidate inherited from the spatial neighboring block is placed in the sub-block based Merge candidate list may depend on the position of the spatial neighboring block.
[0711] i. In one example, if the spatial neighboring block is in the same CTU or CTU row as the current block, the affine Merge candidate inherited from the spatial neighboring block is placed in the sub-block based Merge candidate list; otherwise, it is not placed in the list.
[0712] ii. Alternatively, if the spatial neighboring block is not in the same CTU or CTU row as the current block, the affine Merge candidate inherited from the spatial neighboring block is placed in the sub-block based Merge candidate list; otherwise, it is not placed in the list.
[0713] 32. It is proposed that the affine AMVP candidates inherited from spatially adjacent blocks are not placed in the affine MVP candidate list, and the disclosed history-based affine MVP candidates are placed in the affine MVP candidate list.
[0714] a. In one example, the disclosed history-based affine MVP candidates are first put into the affine MVP candidate list.
[0715] b. It is proposed that whether the affine AMVP candidate inherited from the spatial neighboring block is placed in the affine MVP candidate list may depend on the position of the spatial neighboring block.
[0716] i. In one example, if the spatial neighboring block is in the same CTU or CTU row as the current block, the affine AMVP candidate inherited from the spatial neighboring block is put into the affine MVP candidate list; otherwise, it is not put into the list.
[0717] ii. Alternatively, if the spatial neighboring block is not in the same CTU or CTU row as the current block, the affine AMVP candidate inherited from the spatial neighboring block is placed in the affine MVP candidate list; otherwise, it is not placed in the list.
[0718] 33. More than one affine HMVP buffer is used to store affine parameters or CPMVs in different categories.
[0719] a. For example, two buffers are used to store the affine parameters in reference list 0 and reference list 1 respectively.
[0720] i. In one example, after decoding an affine-coded CU, the CPMV or parameters of reference list 0 are used to update the HMVP buffer of reference list 0.
[0721] ii. In one example, after decoding the affine-coded CU, the CPMV or parameters of reference list 1 are used to update the HMVP buffer of reference list 1.
[0722] iii. In one example, if the motion information of the spatially adjacent / non-adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer are used together to derive the affine model of the current block, the MV of the spatially adjacent / non-adjacent unit blocks of the reference reference list X is combined with the affine parameters stored in the buffer of the reference reference list X. X=0 or 1.
[0723] iv. In one example, if the motion information of the temporally adjacent M×N unit block (e.g., a 4×4 block in a VTM) and a set of affine parameters stored in a buffer are used together to derive an affine model of the current block, the MV of the temporally adjacent unit block of the reference list X is combined with the affine parameters stored in the buffer of the reference list X. X=0 or 1.
[0724] b. For example, N (eg, N=6) buffers are used to store affine parameters referring to different reference indices in different reference lists. In the following discussion, "reference K" means that the reference index of the reference picture is K.
[0725] i. In one example, after decoding an affine-coded CU, the CPMV or parameters of reference K in reference list X are used to update the HMVP buffer of reference K in list X. X=0 or 1. K can be 0, 1, 2, etc.
[0726] ii. In one example, after decoding an affine-coded CU, the CPMV or parameters of reference K in reference list X are used to update the HMVP buffer of reference L in list X, where K>=L. X=0 or 1. M can be 1, 2, 3, etc.
[0727] iii. In one example, if the motion information of the spatially adjacent / non-adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer are used together to derive the affine model of the current block, the MV of the spatially adjacent / non-adjacent unit blocks of the reference K in the reference list X is combined with the affine parameters of the reference K in the reference list X stored in the buffer. X=0 or 1. K can be 0, 1, 2, etc.
[0728] iv. In one example, if the motion information of the temporally adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer are used together to derive the affine model of the current block, the MV of the temporally adjacent unit blocks of reference K in reference list X is combined with the affine parameters stored in the buffer of reference K in reference list X. X=0 or 1. K can be 0, 1, 2, etc.
[0729] v. In one example, if the motion information of the spatially adjacent / non-adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer are used together to derive the affine model of the current block, the MV of the spatially adjacent / non-adjacent unit blocks of reference K in reference list X is combined with the affine parameters of reference L in reference list X stored in the buffer, where K>=L. X=0 or 1. L can be 1, 2, 3, etc.
[0730] vi. In one example, if the motion information of the temporally adjacent M×N unit blocks (e.g., 4×4 blocks in VTM) and the set of affine parameters stored in the buffer are used together to derive the affine model of the current block, the MV of the temporally adjacent unit blocks of reference K in reference list X is combined with the affine parameters of reference L in reference list X stored in the buffer, where K>=L. X=0 or 1. L can be 1, 2, 3, etc.
[0731] c. The size of each affine HMVP buffer of a class can be different.
[0732] i. In one example, the size may depend on the reference picture index.
[0733] ii. For example, the size of the affine HMVP buffer of reference 0 is 3, the size of the affine HMVP buffer of reference 1 is 2, and the size of the affine HMVP buffer of reference 2 is 1.
[0734] 34. Whether and / or how to update the affine HMVP buffer may depend on the codec mode and / or other codec information of the current CU.
[0735] a. For example, if a CU is encoded and decoded in affine Merge mode and the Merge candidate is derived from the affine HMVP buffer, the HMVP buffer is not updated after decoding the CU.
[0736] i. Alternatively, the affine HMVP buffer is updated by removing the affine parameters associated with the last entry of the affine HMVP buffer.
[0737] b. In one example, the affine HMVP buffer may be updated each time a block is encoded or decoded in affine mode.
[0738] c. In one example, when a block is encoded and decoded in affine merge mode and the block uses a shared merge list, the update of the affine HMVP buffer is skipped.
[0739] 35. In one example, the affine HMVP buffer can be divided into M (M>1) sub-buffers: HB 0 HB 1 ,…HB M-1 .
[0740] a. Alternatively, multiple affine HMVP buffers (ie, multiple affine HMVP tables) may be allocated, each of which may correspond to one of the above-mentioned sub-buffers HB i .
[0741] b. In one example, an operation on one sub-buffer (eg, a sub-buffer update process, a use of a sub-buffer) may not affect other sub-buffers.
[0742] c. In one example, M is predefined, such as 10.
[0743] d. In one example, the first M0 buffers are associated with storage of affine parameters of reference picture list X, and the remaining (M-M0) buffers are associated with storage of affine parameters of reference picture list Y, where Y=1-X and X is 0 or 1.
[0744] i. Alternatively, the affine parameters of reference picture list X may be stored in an interleaved manner with those of reference picture list Y.
[0745] ii. In one example, the affine parameters of the reference picture list X may be stored in HB i In the example, i is an odd value, and the affine parameters of the reference picture list X can be stored in HB j In this case, j is an even value.
[0746] e. In one example, M can be signaled from the encoder to the decoder, such as at the video level (e.g., VPS), sequence level (e.g., SPS), picture level (e.g., PPS or picture header), slice level (e.g., slice header), slice group level (e.g., slice group header).
[0747] f. In one example, M may depend on the number of reference pictures.
[0748] i. For example, M may depend on the number of reference pictures in reference list 0;
[0749] ii. For example, M may depend on the number of reference pictures in reference list 1;
[0750] g. In one example, each sub-buffer may have the same maximum allowed number of entries, denoted as N. For example, N=1 or N=2;
[0751] h. In one example, each sub-buffer may have a different maximum allowed number of entries. For example, sub-buffer HB K Can have at most N K The allowed number of entries. For different K, N K Can be different.
[0752] i. When a set of affine parameters is used to update an HMVP buffer, a sub-buffer with a sub-buffer index SI may be selected, and then the set of affine parameters may be used to update the corresponding sub-buffer HB SI .
[0753] i. In one example, the selection of the sub-buffer may be based on the codec information of the block on which the set of affine parameters is applied.
[0754] (a) In one example, the coding information may include a reference list index (or prediction direction) and / or a reference index associated with a set of affine parameters.
[0755] (b) For example, assuming that the reference list index and reference index of the set of affine parameters are represented as X (e.g., X is 0 or 1) and RIDX, the selected sub-buffer index SI can be calculated as SI = f(X, RIDX), where f is a function.
[0756] a. In one example, SI=X*MaxR0+min(RIDX, MaxRX-1), where MaxR0 and MaxR1 are integers, for example MaxR0=MaxR1=5.
[0757] b. Alternatively, SI=2*min(RIDX,MaxRX-1)+X.
[0758] c. In one example, X can only be 0 or 1, and RIDX must be greater than or equal to 0.
[0759] d. In one example, MaxR0 and MaxR1 may be different.
[0760] e. In one example, MaxR0 / MaxR1 may depend on the temporal layer index, slice / slice group / picture type, low delay check flag, etc.
[0761] f. In one example, MaxR0 may depend on the total number of reference pictures in reference list 0.
[0762] g. In one example, MaxR1 may depend on the total number of reference pictures in reference list 1.
[0763] h. In one example, MaxR0 and / or MaxR1 can be signaled from the encoder to the decoder, such as at the video level (e.g., VPS), sequence level (e.g., SPS), picture level (e.g., PPS or picture header), slice level (e.g., slice header), slice group level (e.g., slice group header).
[0764] j. When the set of affine parameters is used to update the sub-buffer HB SI When , it can be regarded as updating a regular affine HMVP buffer, and the method of updating an affine HMVP buffer disclosed in this document can be applied to update a sub-buffer.
[0765] k. Spatially or temporally adjacent or non-adjacent neighboring blocks (which may also be simply referred to as "neighboring blocks") may be used in combination with one or more sets of affine parameters stored in one or more HMVP affine sub-buffers.
[0766] 36. In one example, the maximum allowed size of an affine HMVP buffer and / or an affine HMVP sub-buffer may be equal to 1.
[0767] a. In one example, no counter is needed to record the number of sets of affine parameters stored in the affine HMVP buffer or the affine HMVP sub-buffer.
[0768] 37. Whether and / or how to operate on an affine HMVP buffer or an affine HMVP sub-buffer may depend on whether all affine parameters of the set are zero.
[0769] a. In one example, when an affine HMVP buffer or an affine HMVP sub-buffer is flushed, all affine parameters stored in the buffer or sub-buffer are set to zero.
[0770] i. The affine HMVP buffer or affine HMVP sub-buffer may be flushed before encoding / decoding each picture and / or slice and / or slice group and / or CTU row and / or CTU and / or CU.
[0771] b. In one example, when a set of affine parameters is used to update an affine HMVP buffer or an affine HMVP sub-buffer, if all affine parameters in the set are equal to zero, the buffer or sub-buffer is not updated.
[0772] c. In one example, when parameters of a set of affine parameters stored in an affine HMVP buffer or an affine HMVP sub-buffer are all zero, the set of affine parameters cannot be used to generate an affine Merge candidate or an affine AMVP candidate.
[0773] i. For example, in combination with neighboring blocks, a set of affine parameters cannot be used to generate an affine Merge candidate or an affine AMVP candidate.
[0774] ii. For example, when the parameters of a set of affine parameters stored in an entry of an affine HMVP buffer or an affine HMVP sub-buffer are all zero, the entry is marked as "invalid" or "unavailable".
[0775] iii. For example, when the parameters of a set of affine parameters stored in all entries of an affine HMVP buffer or an affine HMVP sub-buffer are all zero, the affine HMVP buffer or affine HMVP sub-buffer is marked as "invalid" or "unavailable", and / or the counter of the buffer or sub-buffer is set to zero.
[0776] 38. When spatially or temporally adjacent or non-adjacent neighboring blocks (which may also be referred to as "neighboring blocks") are used to generate affine Merge candidates by combining affine parameters stored in an affine HMVP buffer, only the affine parameters stored in one or several related sub-buffers may be accessed.
[0777] a. For example, the relevant sub-buffer may be determined by the coding information of the adjacent block. For example, the coding information may include a reference list and / or a reference index of the adjacent block.
[0778] b. For example, one or more sets of affine parameters stored in the relevant sub-buffer can be used to generate affine merge candidates in combination with neighboring blocks.
[0779] i. For example, the set of affine parameters stored as the first entry in the relevant sub-buffer may be used.
[0780] ii. For example, the set of affine parameters stored as the last entry in the relevant sub-buffer may be used.
[0781] c. For example, determine a related sub-buffer HB for the MV of the adjacent block of reference list 0 S0 .
[0782] d. For example, determine a related sub-buffer HB for the MV of the adjacent block of reference list 1 S1 .
[0783] i.HB S0 and HB S1 Can be different.
[0784] e. For an MV of a neighboring block that refers to a reference picture with a reference index RIDX in the reference list LX, the relevant sub-buffer index SI is calculated as SI = g(LX, RIDX), where g is a function.
[0785] i. For example, function g is the same as function f in bullet 35.d.
[0786] ii. In one example, SI=LX*MaxR0+min(RIDX, MaxRX-1), where MaxR0 and MaxR1 are integers, for example MaxR0=MaxR1=5.
[0787] (a) In one example, LX can only be 0 or 1, and RIDX must be greater than or equal to 0.
[0788] (b) MaxR0 and MaxR1 can be different.
[0789] (c) MaxR0 may depend on the total number of reference pictures in reference list 0.
[0790] (d) MaxR1 may depend on the total number of reference pictures in reference list 1.
[0791] (e) MaxR0 and / or MaxR1 can be signaled from the encoder to the decoder, such as at the video level (e.g., VPS), sequence level (e.g., SPS), picture level (e.g., PPS or picture header), slice level (e.g., slice header), slice group level (e.g., slice group header).
[0792] f. In one example, when a neighboring block is inter-coded using unidirectional prediction with reference to a reference picture having a reference index RIDX in a reference list LX, if there is at least one entry available in the sub-buffer, and / or the counter of the sub-buffer is not equal to 0, an affine Merge candidate can be generated from the neighboring block in combination with a set of affine parameters stored in the associated affine HMVP sub-buffer.
[0793] i. Referring to the reference picture with reference index RIDX in reference list LX, the generated affine Merge candidate should also be unidirectionally predicted.
[0794] g. In one example, when a neighboring block is inter-coded using bidirectional prediction with reference to a reference picture having a reference index RIDX0 in reference list 0 and a reference index RIDX1 in reference list 1, an affine Merge candidate can be generated from the neighboring block in combination with one or more sets of affine parameters stored in one or more related affine HMVP sub-buffers.
[0795] i. In one example, referring to the reference picture with reference index RID0 in reference list 0 and reference index RID1 in reference list 1, the generated affine Merge candidate should also be bi-directionally predicted.
[0796] (a) A bidirectionally predicted affine merge candidate can be generated only when there is at least one entry available in the sub-buffer associated with the reference index RID0 in the reference list 0 (and / or the counter of the sub-buffer is not equal to 0), and there is at least one entry available in the sub-buffer associated with the reference index RID1 in the reference list 1 (and / or the counter of the sub-buffer is not equal to 0).
[0797] (b) In one example, if the following conditions are not met, then no affine Merge candidate may be generated from the neighboring blocks in combination with the affine parameters stored in the affine HMVP buffer and / or sub-buffer.
[0798] a. There is at least one entry available in the sub-buffer associated with reference index RID0 in reference list 0 (and / or the counter of the sub-buffer is not equal to 0), and there is at least one entry available in the sub-buffer associated with reference index RID1 in reference list 1 (and / or the counter of the sub-buffer is not equal to 0).
[0799] ii. In an alternative example, with reference to a reference picture having a reference index RID0 in reference list 0 or a reference index RID1 in reference list 1, the generated affine Merge candidate may also be uni-directionally predicted.
[0800] (a) If there is at least one entry available in the sub-buffer associated with the reference index RID0 in the reference list 0 (and / or the counter of the sub-buffer is not equal to 0), and there is no entry available in the sub-buffer associated with the reference index RID1 in the reference list 1 (and / or the counter of the sub-buffer is equal to 0), then the generated affine merge candidate is unidirectionally predicted with reference to the reference picture with the reference index RID0 in the reference list 0.
[0801] (b) If there is at least one entry available in the sub-buffer associated with the reference index RID1 in the reference list 1 (and / or the counter of the sub-buffer is not equal to 0), and there is no entry available in the sub-buffer associated with the reference index RID0 in the reference list 0 (and / or the counter of the sub-buffer is equal to 0), then the generated affine Merge candidate is unidirectionally predicted with reference to the reference picture having the reference index RID1 in the reference list 1.
[0802] h. In one example, all methods disclosed in this document can be used to generate affine merge candidates by combining affine parameters stored in one or several related sub-buffers.
[0803] 39. When spatially or temporally adjacent or non-adjacent neighboring blocks (which may also be referred to as "neighboring blocks") are used to generate affine AMVP candidates by combining affine parameters stored in an affine HMVP buffer, only the affine parameters stored in one or several relevant sub-buffers may be accessed.
[0804] a. For example, the relevant sub-buffer may be determined by the coding information of the adjacent block. For example, the coding information may include a reference list and / or a reference index of the adjacent block.
[0805] b. For example, one or more sets of affine parameters stored in the relevant sub-buffer may be used to generate affine AMVP candidates in combination with neighboring blocks.
[0806] i. For example, the set of affine parameters stored as the first entry in the relevant sub-buffer may be used.
[0807] ii. For example, the set of affine parameters stored as the last entry in the relevant sub-buffer may be used.
[0808] c. For a target reference picture with a target reference index RIDX in the target reference list LX, the relevant sub-buffer index SI is calculated as SI = h(LX, RIDX), where g is a function.
[0809] i. For example, function g is the same as function f in bullet 35.d.
[0810] ii. For example, function g is the same as function g in bullet point 38.
[0811] iii. In one example, SI=LX*MaxR0+min(RIDX, MaxRX-1), where MaxR0 and MaxR1 are integers, for example MaxR0=MaxR1=5.
[0812] (a) In one example, LX can only be 0 or 1, and RIDX must be greater than or equal to 0.
[0813] (b) MaxR0 and MaxR1 can be different.
[0814] (c) MaxR0 may depend on the total number of reference pictures in reference list 0.
[0815] (d) MaxR1 may depend on the total number of reference pictures in reference list 1.
[0816] (e) MaxR0 and / or MaxR1 can be signaled from the encoder to the decoder, such as at the video level (e.g., VPS), sequence level (e.g., SPS), picture level (e.g., PPS or picture header), slice level (e.g., slice header), slice group level (e.g., slice group header).
[0817] d. In one example, if no entry is available in the sub-buffer associated with the target reference index RIDX of the target reference list LX (and / or the counter of the sub-buffer is equal to 0), no affine AMVP candidate may be generated from the affine parameters stored in the affine HMVP buffer / sub-buffer.
[0818] e. In one example, when the neighboring block is inter-coded and has an MV that refers to a target reference index RIDX in a target reference list LX, then the MV is used to generate an affine AMVP candidate in combination with affine parameters stored in the relevant sub-buffer.
[0819] f. In one example, when the neighboring block is inter-coded and does not have an MV that refers to the target reference index RIDX in the target reference list LX, then no affine AMVP candidate may be generated from the neighboring block.
[0820] i. Alternatively, when the neighboring block is inter-coded and does not have an MV that refers to a target reference index RIDX in the target reference list LX, the neighboring block is checked to determine whether it has a second MV that refers to a second reference picture in the reference list 1-LX, and the second reference has the same POC as the target reference picture.
[0821] (a) If it has a second MV that refers to a second reference picture in reference list 1-LX, and the second reference has the same POC as the target reference picture, then the second MV is used to generate an affine AMVP candidate in combination with the affine parameters stored in the relevant sub-buffer. Otherwise, no affine AMVP candidate may be generated from a neighboring block.
[0822] g. In one example, all methods disclosed in this document can be applied to generate affine Merge / AMVP candidates by combining affine parameters stored in one or several related sub-buffers.
[0823] 40. If encoding and decoding is performed in intra block copy (IBC) mode, neighboring blocks cannot be combined with affine parameters stored in the affine HMVP buffer or affine HMVP sub-buffer to generate affine Merge / AMVP candidates.
[0824] 41. If the spatial neighboring blocks are used to generate inherited Merge / AMVP candidates, the spatial neighboring blocks cannot be used in combination with the affine parameters stored in the affine HMVP buffer / sub-buffer to generate the affine Merge / AMVP candidates.
[0825] 42. Spatially and / or temporally adjacent / non-adjacent blocks may be divided into K groups (e.g., K=2), and how the parameters in the affine HMVP buffer / sub-buffer are combined with the motion information of the spatially and / or temporally adjacent / non-adjacent blocks used to encode and decode the current block may be based on the group.
[0826] a. Affine Merge candidates generated from affine parameters stored in the affine HMVP buffer / sub-buffer in combination with spatially adjacent blocks in different groups can be placed in different positions in the affine Merge candidate list;
[0827] b. Affine AMVP candidates generated from affine parameters stored in the affine HMVP buffer / sub-buffer in combination with spatially adjacent blocks in different groups may be placed in different positions in the affine AMVP candidate list;
[0828] c. In one example, spatially adjacent blocks can be grouped based on their codec information.
[0829] i. For example, neighboring blocks can be put into a group based on whether they are affine coded or not.
[0830] ii. For example, neighboring blocks can be put into a group based on whether they are affine coded in AMVP mode.
[0831] iii. For example, neighboring blocks can be put into a group based on whether they are affine-encoded in Merge mode.
[0832] d. In one example, spatially neighboring blocks can be divided into groups based on their locations.
[0833] e. In one example, not all neighboring blocks are put into K groups.
[0834] f. In one example, spatially adjacent blocks are divided into two groups as follows:
[0835] i. The first encountered left neighboring block of the affine codec can be put into group X.
[0836] (a) The left neighboring blocks are checked sequentially, e.g. Figure 5 Block A0 and block A1 are shown.
[0837] (b) In one example, if the first encountered affine coded left neighbor block is used to generate the inherited Merge / AMVP candidate, it is not put into group X.
[0838] ii. The first encountered upper neighboring block of the affine codec is put into group X.
[0839] (a) The upper neighboring blocks are checked sequentially. Figure 5 Shown are block B0, block B1 and block B2.
[0840] (b) In one example, if the first encountered inter-coded and affine-coded upper neighboring block is used to generate the inherited Merge / AMVP candidate, it is not put into group X.
[0841] iii. Adjacent blocks of other inter-frame codecs can be put into group Y, where Y is not equal to X.
[0842] g. In one example, an affine merge candidate generated from affine parameters stored in the affine HMVP buffer / sub-buffer in combination with spatially adjacent blocks in group X may be placed in the affine merge candidate list before the Kth constructed affine merge candidate. For example, K may be 1 or 2.
[0843] h. In one example, an affine merge candidate generated from affine parameters stored in an affine HMVP buffer / sub-buffer in combination with spatially adjacent blocks in group Y can be placed in the affine merge candidate list after the Kth constructed affine merge candidate. For example, K can be 1 or 2.
[0844] i. In one example, the affine AMVP candidate generated from the affine parameters stored in the affine HMVP buffer / sub-buffer in combination with the spatial neighboring blocks in group X can be placed in the affine AMVP candidate list before the K-th constructed affine Merge candidate. For example, K can be 1 or 2.
[0845] j. In one example, the affine AMVP candidate generated from the affine parameters stored in the affine HMVP buffer / sub-buffer in combination with the spatial neighboring blocks in group Y can be placed in the affine AMVP candidate list after the Kth constructed affine Merge candidate. For example, K can be 1 or 2.
[0846] k. In one example, affine AMVP candidates generated from affine parameters stored in the affine HMVP buffer / sub-buffer in combination with spatially neighboring blocks in group X may be put into the affine AMVP candidate list before zero candidates.
[0847] 43. The basic position (xm, ym) in bullet point 20 can be any position inside a basic neighboring block (e.g., a 4×4 basic block), such as Fig. 27 as shown in .
[0848] a. For example, (xm, ym) can be Fig. 27 P22 in.
[0849] b. Assume that the coordinates of the upper left sample point of the current block are (xPos00, yPos00), the coordinates of the upper right sample point of the current block are (xPos10, yPos00), and the coordinates of the upper right sample point of the current block are (xPos00, yPos01). Figure 5 middle:
[0850] i. The (xm, ym) of the adjacent basic block A1 is (xPos00-2, yPos01-1);
[0851] ii. The (xm, ym) of the adjacent basic block A0 is (xPos00-2, yPos01+3);
[0852] iii. The (xm, ym) of the adjacent basic block B1 is (xPos10-1, yPos00-2);
[0853] iv. The (xm, ym) of the adjacent basic block B0 is (xPos10+3, yPos00-2);
[0854] v. The (xm, ym) of the adjacent basic block B2 is (xPos00-2, yPos00-2);
[0855] Fig.16 1 is a block diagram of a video processing device 1700. Device 1700 may be used to implement one or more methods described herein. Device 1700 may be embodied in a smart phone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. Device 1700 may include one or more processors 1702, one or more memories 1704, and video processing hardware 1706. Processor(s) 1702 may be configured to implement one or more methods described in this document. Memory(s) 1704 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1706 may be used to implement some of the techniques described in this document in hardware circuits.
[0856] Fig.18 1900 is a flow chart of a method 1900 for processing a video. The method 1900 includes storing (1905) parameters associated with a motion vector of a first video block in a memory, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; and performing (1910) further processing of the first video block using the parameters.
[0857] Fig.19 is a flow chart of a method 2000 for processing a video. The method 2000 comprises: storing (2005) a control point motion vector (CPMV) representing the coordinates of a representative point relative to an upper left sample point of a first video block; and performing (2010) further processing of the first video block using the CPMV.
[0858] Fig. 20 21 is a flow chart of a method 2100 for processing a video. The method 2100 includes: performing a determination (2105) of parameters associated with a motion vector of a first video block, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; determining (2110) that a buffer for storing the parameters is full; modifying (2115) the buffer so that the buffer is not full; storing (2120) the parameters in the not full buffer; and performing (2125) further processing of the first video block using the parameters stored in the buffer.
[0859] Fig.212200. The method 2200 includes: performing a determination 2205 of a first parameter associated with a motion vector of a first video block, the first parameter being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; determining 2210 that a buffer includes a second parameter similar to the first parameter; and performing 2215 further processing of the first video block using the second parameter instead of the first parameter.
[0860] Fig. 22 23 is a flow chart of a method 2300 for processing a video. The method 2300 includes: refreshing (2305) a buffer for storing parameters associated with a motion vector of a first video block, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; and performing (2310) further processing of the first video block using the parameters stored in the buffer.
[0861] Fig.23 24 is a flow chart of a method 2400 for processing a video. The method 2400 includes: identifying (2405) parameters in a memory associated with a motion vector of a first video block, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; and performing (2410) further processing of a second video block using the parameters associated with the first video block.
[0862] Fig.24 25 is a flowchart of a method 2500 for processing a video. The method 2500 includes: identifying (2505) parameters associated with a motion vector of a first video block in a memory, the parameters being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; determining (2510) that a second video block uses one of the 4-parameter affine model or the 6-parameter affine model based on the parameters; generating (2515) an affine merge candidate list for the second video block based on the 4-parameter affine model or the 6-parameter affine model used by the second video block; and performing (2520) further processing of the second video block using the affine merge candidate list.
[0863] Fig.26 26 is a flow chart of a method 2600 for processing a video. The method 2600 includes: identifying (2605) parameters associated with a motion vector (MV) of a first video block; deriving (2610) an affine model based on the parameters of the first video block, the affine model being one or both of a 4-parameter affine model or a 6-parameter affine model; and performing (2615) further processing of the first video block using the affine model.
[0864] Some examples of determining candidates for encoding and their use are described in Section 4 of this document with reference to methods 1900, 2000, 2100, 2200, 2300, 2400, 2500, and 2600. For example, as described in Section 4, a video block may be processed using parameters associated with a 4-parameter affine model or a 6-parameter affine model.
[0865] Referring to methods 1900 , 2000 , 2100 , 2200 , 2300 , 2400 , 2500 , and 2600 , a video block may be encoded in a video bitstream in which bit efficiency may be achieved by using a bitstream generation rule associated with motion information prediction.
[0866] The method may include where four parameters associated with the 6-parameter affine model are stored in memory and two other parameters associated with the 6-parameter affine model are not stored in memory.
[0867] The method may include where two parameters associated with the 4-parameter affine model are stored in memory and two other parameters associated with the 4-parameter affine model are not stored in memory.
[0868] The method may include determining that the first video block is encoded using a 4-parameter affine model, wherein storing the parameters includes storing four parameters associated with the 4-parameter affine model.
[0869] The method may include wherein a first number of parameters associated with a 4-parameter affine model are stored and a second number of parameters associated with a 6-parameter affine model are stored, the first number and the second number being the same.
[0870] The method may include wherein a first number of parameters associated with a 4-parameter affine model are stored and a second number of parameters associated with a 6-parameter affine model are stored, the first number and the second number being different.
[0871] The method may include determining characteristics of encoding and decoding the first video block, the characteristics including one or more of the following: affine mode, use of inter-frame or merge mode, size of the first video block, picture type, wherein the stored parameters are based on the characteristics of the encoding and decoding of the first video block.
[0872] The method may comprise storing one or more of: an inter prediction direction, or a reference index.
[0873] The method may include determining that the first video block is bi-directionally predicted; and based on determining that the first video block is bi-directionally predicted, storing parameters for list 0 and list 1.
[0874] The method may include wherein parameters for list 0 are stored in a first buffer and parameters for list 1 are stored in a second buffer, the first buffer and the second buffer being different.
[0875] The method may include where parameters for List 0 and List 1 are stored together with a prediction from one of List 0 or List 1 to the other.
[0876] The method may include storing a width of the first video block in a memory, wherein performing further processing of the first video block is based on the width.
[0877] The method may include storing a height of the first video block in a memory, wherein performing further processing of the first video block is based on the height.
[0878] The method may include storing in a memory information indicating an upper left coordinate of the first video block, wherein performing further processing of the first video block is based on the upper left coordinate.
[0879] The method may include determining the parameter without using a division operation.
[0880] The method may include clipping the parameters before storing the parameters in the memory.
[0881] The method may include where the parameters are stored after the first video block is decoded or encoded.
[0882] The method may include determining characteristics of encoding and decoding the first video block, the characteristics including one or more of: a codec affine mode, a number of affine coded video blocks, a position of the first video block, or a size of the first video block, wherein the stored parameters are based on the characteristics of the codec of the first video block.
[0883] The method may include: determining that the second video block is affine coded; and determining that the first video block is a Kth affine coded block, wherein based on the first video block being the Kth affine coded block, parameters associated with the first video block are stored in a memory, and the second video block is not stored in the memory.
[0884] The method may include: wherein K is 2 or 4.
[0885] The method may include: wherein K is signaled from the encoder to the decoder in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a slice group header, or a slice.
[0886] The method may include where the memory stores a maximum number M of sets of parameters.
[0887] The method may include: wherein M is 8 or 16.
[0888] The method may include where M is signaled from the encoder to the decoder in a VPS, SPS, picture parameter set PPS, slice header, slice group header, slice, codec tree unit (CTU) line, or CTU.
[0889] The method may include where modifying the buffer so that the buffer is not full comprises removing the oldest stored parameter from the buffer.
[0890] The method may include where modifying the buffer so that the buffer is not full comprises removing a most recently stored parameter from the buffer.
[0891] The method may include performing, by a processor, determining a third parameter associated with a motion vector of a first video block, the third parameter being related to one or both of a 4-parameter affine model or a 6-parameter affine model of the motion vector of the first video block; determining that a second parameter stored in a buffer is different from the third parameter; based on determining that the second parameter is different from the third parameter, storing the third parameter in a buffer; and performing further processing of the first video block using the third parameter.
[0892] The method may include where determining that the second parameter stored in the buffer is different from the third parameter comprises determining that the second parameter and the third parameter are associated with different inter-prediction directions.
[0893] The method may include where determining that the second parameter stored in the buffer is different from the third parameter comprises determining that the second parameter and the third parameter are associated with different reference indices of list 0 when list 0 is used as the prediction direction.
[0894] The method may include where determining that the second parameter stored in the buffer is different from the third parameter comprises determining that the second parameter and the third parameter are associated with different reference indices of list 1 when list 1 is used as a prediction direction.
[0895] The method may include where determining that the second parameter stored in the buffer is different from the third parameter includes determining that the second parameter includes a first number of parameters and the third parameter includes a second number of parameters, the first number and the second number being different.
[0896] The method may include where determining that the second parameter stored in the buffer is different from the third parameter includes determining that the second parameter includes a first number of parameters and the third parameter includes a second number of parameters, the first number and the second number being different.
[0897] The method may include where determining that the second parameter stored in the buffer is different from the third parameter includes determining that one of the second parameter or the third parameter is associated with a 4-parameter affine model and the other is associated with a 6-parameter affine model.
[0898] The method may include where flushing the buffer comprises not storing the parameter in the buffer.
[0899] The method may include where refreshing the buffer includes configuring the buffer to be in an empty state associated with not storing parameters, and storing default parameters in the buffer based on the buffer being in the empty state.
[0900] The method may include where the default parameter is based on the sequence.
[0901] The method may include where the default parameters are based on the picture.
[0902] The method may include where the default parameters are based on the stripe.
[0903] The method may include where the default parameters are based on the slice.
[0904] The method may include: wherein the default parameters are based on the CTU line.
[0905] The method may include where the default parameters are based on the CTU.
[0906] The method may include where the default parameters are signaled from the encoder to the decoder in a VPS, SPS, picture parameter set PPS, slice header, slice group header, slice, CTU line, or CTU.
[0907] The method may include determining that the first video block is a first block of a picture, wherein flushing the buffer is based on determining that the first video block is the first block of the picture.
[0908] The method may include determining that the first video block is a first block of the slice, wherein flushing the buffer is based on determining that the first video block is the first block of the slice.
[0909] The method may include determining that the first video block is a first block of the slice, wherein flushing the buffer is based on determining that the first video block is the first block of the slice.
[0910] The method may include determining that the first video block is a first block of a CTU line, wherein flushing the buffer is based on determining that the first video block is the first block of the CTU line.
[0911] The method may include determining that the first video block is a first block of a CTU, wherein flushing the buffer is based on determining that the first video block is the first block of the CTU.
[0912] The method may include wherein the parameter is used for motion vector prediction or motion vector encoding and decoding of the second video block.
[0913] The method may include where the parameters are used to derive a control point motion vector (CPMV) for the second video block.
[0914] The method may include determining that the second video block is encoded using a 4-parameter affine model, and performing further processing of the second video block to prioritize parameters associated with the 4-parameter affine model over parameters associated with the 6-parameter affine model.
[0915] The method may include determining that the second video block is encoded using a 6-parameter affine model, and performing further processing of the second video block to prioritize parameters associated with the 6-parameter affine model over parameters associated with the 4-parameter affine model.
[0916] The method may include identifying motion information of a first video block, the first video block being adjacent to a second video block, and wherein performing further processing of the second video block includes using the motion information of the first video block.
[0917] The method may include wherein motion information is also stored in the buffer, and the motion information and the parameters are used to derive the CPMV of the second video block.
[0918] The method may include wherein the motion vector is also stored in the buffer, and the motion vector and the parameter are used to derive the CPMV of the second video block.
[0919] The method may include where the affine Merge candidate list inherits the inter prediction direction and reference index associated with the parameters.
[0920] The method may include where the parameters are used to generate an Advanced Motion Vector Prediction (AMVP) candidate.
[0921] The method may include: wherein the parameters refer to the same reference picture.
[0922] The method may include where a reference picture list associated with the parameter is the same as a target reference picture list.
[0923] The method may include wherein a reference index associated with the parameter is the same as the target reference index.
[0924] The method may include: wherein the number of sets of parameters stored in the memory used to generate the affine Merge candidate list is N.
[0925] The method may include: wherein N is signaled from the encoder to the decoder in a VPS, an SPS, a picture parameter set PPS, a slice header, a slice group header, or a slice.
[0926] The method may include: wherein N is based on a size of the second video block or codec mode information of the second video block.
[0927] The method may include: wherein N is based on a standard profile, level or hierarchy.
[0928] The method may include: wherein N is predefined.
[0929] The method may include: wherein the affine merge candidate list is in ascending order of index.
[0930] The method may include: wherein the affine merge candidate list is in descending order of index.
[0931] The method may include: wherein the affine merge candidate list is in descending or ascending order of index based on the number of available candidates.
[0932] The method may include where the 4-parameter affine model is stored with the same number of parameters as the 6-parameter affine model, and some of the stored parameters of the 6-parameter affine model are derived from parameters of the 4-parameter affine model.
[0933] The method may include converting a basic motion vector (MV) Stored in memory, and where further processing is also based on the basic MV.
[0934] The method may comprise storing coordinates of the position of a base motion vector (MV) in a memory, and wherein the further processing is also based on the base MV, and wherein the further processing is also based on the base MV.
[0935] The method may include wherein parameters indicative of a and b are also stored in the memory.
[0936] The method may include converting a basic motion vector (MV) Stored in a memory, and parameters indicating a, b, c and d are stored in a memory, and wherein further processing is also based on the basic MV.
[0937] The method may comprise storing the coordinates of the position of a base motion vector (MV) and the parameters a, b, c and d in a memory, and wherein the further processing is also based on the base MV.
[0938] The method may include identifying characteristics of the parameter, wherein the affine model is derived based on the characteristics, the characteristics including one or more of a basis motion vector (MV) of the parameter or a position of the MV.
[0939] The method may include where deriving the affine model includes deriving one or both of a control point motion vector (CPMV) or a MV.
[0940] The method may include where deriving the affine model includes deriving the CPMV, and the CPMV of the first video block is derived from the MV and the parameters.
[0941] The method may include where deriving the affine model includes using motion information of neighboring video blocks of the first video block.
[0942] The method may include where a CPMV associated with a first video block is derived from MVs of neighboring video blocks of the first video block.
[0943] The method may include where the CPMV is used to derive the MV of each sub-block of the first video block.
[0944] The method may include where a temporal motion vector prediction (TMVP) is also used along with the parameters to derive the affine model.
[0945] The method may include where the TMVP is used to derive the CPMV or the MV of a sub-block of the first video block.
[0946] The method may include where deriving the affine model is further based on motion information of a collocated video block in a collocated picture related to the first video block.
[0947] The method may include deriving an affine Merge candidate from the parameters; and adding a spatially adjacent video block that is not adjacent to the first video block to an affine Merge candidate list including the affine Merge candidate.
[0948] The method may include: wherein the affine merge candidate and the spatially neighboring video block are included in the affine merge candidate list after the inherited affine merge candidate.
[0949] The method may include: wherein the affine merge candidate and the spatially neighboring video block are included in the affine merge candidate list after the first constructed affine merge candidate.
[0950] The method may include: wherein the affine merge candidate and the spatially neighboring video block are included in the affine merge candidate list after the first merge candidate constructed from the spatially neighboring blocks.
[0951] The method may include: wherein the affine merge candidate and the spatially neighboring video block are included in an affine merge candidate list after the constructed affine merge candidate.
[0952] The method may include: wherein the affine merge candidate and the spatially neighboring video block are included in the affine merge candidate list after the zero affine merge candidate.
[0953] The method may include: wherein the affine merge candidate and the spatially neighboring video blocks are included in the affine merge candidate list in sequence.
[0954] The method may include adding the time-domain unit block to an affine merge candidate list.
[0955] The method may include: wherein the time domain unit block is added in the Merge candidate list after the inherited affine Merge candidate.
[0956] The method may include: wherein the time domain unit block is added to the Merge candidate list after the first constructed affine Merge candidate.
[0957] The method may include: wherein the temporal unit block is added in the Merge candidate list after a first affine Merge candidate constructed from spatial neighboring blocks of the first video block.
[0958] The method may include: spatially adjacent or non-adjacent video blocks of the first video block are added to an affine merge candidate list.
[0959] The method may include where spatially adjacent or non-neighboring video blocks are added to the affine Merge candidate list after the inherited affine AMVP candidate.
[0960] The method may include: wherein a size of the affine merge candidate list is increased based on determining that an affine merge candidate derived from the parameter can be placed in the affine merge candidate list.
[0961] The method may include: wherein a size of the affine merge candidate list is increased based on a determination that an affine AMVP candidate derived from the parameters can be placed in the affine merge candidate list.
[0962] The method may include deriving a virtual affine model based on a plurality of affine models stored in a memory, wherein performing further processing of the first video block includes using the virtual affine model.
[0963] The method may include: wherein generating the affine Merge candidate list includes excluding affine Merge candidates inherited from spatially neighboring video blocks from being inserted into the sub-block based Merge candidate list, and the history based Merge candidates are inserted into the sub-block Merge candidate list.
[0964] The method may include: wherein a history-based Merge candidate is inserted after the ATMVP candidate.
[0965] The method may include: wherein the history-based Merge candidate is inserted before the constructed affine Merge candidate.
[0966] The method may include: wherein generating the affine Merge candidate list includes inserting the affine Merge candidates inherited from the spatially adjacent video blocks into the sub-block based Merge candidates based on the positions of the spatially adjacent video blocks.
[0967] The method may include where the location is within the same codec tree unit (CTU) or CTU row as the current video block.
[0968] The method may include where the location is outside of the same codec tree unit (CTU) or CTU row as the current video block.
[0969] The method may include: wherein generating the affine Merge candidate list includes excluding affine AMVP candidates inherited from spatially neighboring video blocks from being inserted into an affine motion vector prediction (MVP) candidate list, and history-based Merge candidates are inserted into the affine MVP candidate list.
[0970] The method may include: wherein history-based Merge candidates are first inserted into an affine MVP candidate list.
[0971] The method may include: wherein generating the affine Merge candidate list includes inserting the affine AMVP candidate inherited from the spatially adjacent video block into the affine MVP candidate list based on the position of the spatially adjacent video block.
[0972] The method may include where the location is within the same codec tree unit (CTU) or CTU row as the current video block.
[0973] The method may include where the location is not within the same CTU or CTU row as the current video block.
[0974] The method may include: wherein the affine model parameters are further pruned before being used for encoding or decoding the affine codec block.
[0975] The method may include where the clipping boundary is based on a precision of the affine model parameters.
[0976] The method may include where the cropping boundary is based on a width and a height of the first video block.
[0977] The method may include where the cropping boundary is signaled in a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, or a slice group header.
[0978] The method may include where the clipping boundaries are based on a profile or level of criteria.
[0979] The method may include where the inherited affine merge candidate is from a video block in a current codec tree unit (CTU).
[0980] The method may include: wherein the affine merge candidate is inserted before the inherited affine merge candidate.
[0981] The method may include determining that the HMVP buffer is empty; and based on determining that the HMVP buffer is empty, adding an inherited affine merge candidate to a list.
[0982] The method may include where the inherited affine AMVP candidate is from a video block in a current codec tree unit (CTU) row.
[0983] The method may include where the affine HMVP buffer includes a plurality of buffers, each of the plurality of buffers corresponding to storage of affine parameters or control point motion vectors (CPMVs) in a different category.
[0984] The method may include: wherein the plurality of buffers include a first buffer for storing affine parameters in reference list 0 and a second buffer for storing affine parameters in reference list 1.
[0985] The method may include where a plurality of buffers are used to store affine parameters that reference different indices in different reference lists.
[0986] The method may include: wherein the CPMV or affine parameters of reference K in reference list X are used to update the HMVP buffer of reference K in list X, where X is 0 or 1, and K is 0, 1, or 2.
[0987] The method may include where the CPMV or affine parameters of reference K are used to update the HMVP buffer of reference L in list X, where K>=L, X is 0 or 1, and K is an integer.
[0988] The method may include where each of the plurality of buffers is of a different size.
[0989] The method may include where the size is based on a reference picture index.
[0990] The method may include: wherein the affine HMVP buffer is updated based on a codec mode or codec information of a current codec unit (CU).
[0991] The method may include: wherein the current CU is encoded and decoded in the affine Merge mode, and the affine HMVP buffer is not updated based on that the current CU is encoded and decoded in the affine Merge mode.
[0992] The method may include where the affine HMVP buffer is updated based on the current CU being encoded in affine mode.
[0993] The method may include where updating the HMPV buffer is skipped based on a current CU encoded and decoded in an affine Merge mode, and a shared Merge list is used.
[0994] Fig.282800 is a flow chart of a method 2800 for processing a video. The method 2800 includes: deriving (2802) affine-related information associated with an affine model to be used by a current block for a conversion between a current block of a video and a bitstream representation of the current block based on affine-related information associated with an affine model used by a first block stored in a buffer, wherein the first block is affine-encoded before the current block; and performing (2804) the conversion by using the derived affine-related information.
[0995] In some examples, the affine model includes at least one of a 4-parameter affine model and a 6-parameter affine model.
[0996] In some examples, the affine-related information stored in the buffer includes a first set of affine parameters associated with the affine model of the first block.
[0997] In some examples, when a 6-parameter affine model is applied to the transformation of the first block, six parameters associated with the 6-parameter affine model are stored in the buffer, or when a 4-parameter affine model is applied to the transformation of the first block, four parameters associated with the 4-parameter affine model are stored in the buffer.
[0998] In some examples, when the 6-parameter affine model is applied to the transformation of the first block, four parameters associated with the 6-parameter affine model are stored in the buffer, and two other parameters associated with the 6-parameter affine model are not stored in the buffer.
[0999] In some examples, when the 4-parameter affine model is applied to the transformation of the first block, two parameters related to the 4-parameter affine model are stored in the buffer, and two other parameters related to the 4-parameter affine model are not stored in the buffer.
[1000] In some examples, six parameters associated with the 6-parameter affine model are always stored in a buffer, and when the 4-parameter affine model is applied to the transformation of the first block, a first parameter of the six parameters is set to the same as a second parameter of the six parameters, and a third parameter of the six parameters is set to the negative value of a fourth parameter of the six parameters.
[1001] In some examples, four parameters associated with the 6-parameter affine model are always stored in a buffer, and when the 4-parameter affine model is applied to the transformation of the first block, a first parameter of the four parameters is set to the same as a second parameter of the four parameters, and a third parameter of the four parameters is set to the negative value of a fourth parameter of the four parameters.
[1002] In some examples, a first number of parameters associated with a 4-parameter affine model are stored and a second number of parameters associated with a 6-parameter affine model are stored, the first number and the second number being the same.
[1003] In some examples, the first number and the second number are 4 or 6.
[1004] In some examples, a first number of parameters associated with a 4-parameter affine model is stored and a second number of parameters associated with a 6-parameter affine model is stored, the first number and the second number being different.
[1005] In some examples, the first number is four and the second number is six.
[1006] In some examples, the method also includes: determining characteristics of a first block of the video, wherein the characteristics include one or more of: an affine mode, use of an inter-frame or merge mode, a size of the first block, a picture type, wherein the affine parameters to be stored in the buffer are determined based on the characteristics of the first block of the video.
[1007] In some examples, side information and / or an affine model type associated with affine parameters of an affine model is stored in a buffer, wherein the affine model type includes a 4-parameter affine model or a 6-parameter affine model, and the side information includes at least one of: an inter-frame prediction direction including list 0 or list 1 or bidirectional, and a list 0 and / or list 1 reference index.
[1008] In some examples, when the first block is bi-predicted, the first set of affine parameters also includes affine parameters for list 0 and affine parameters for list 1.
[1009] In some examples, the affine parameters for list 0 are stored in a first buffer and the affine parameters for list 1 are stored in a second buffer different from the first buffer.
[1010] In some examples, one of the parameters for list 0 and the parameters for list 1 are stored, and the stored affine parameters are used to predict the other parameter.
[1011] In some examples, the affine-related information includes a first set of control point motion vectors (CPMVs) associated with the affine model of the first block.
[1012] In some examples, the method also includes: storing the width of the first block in the buffer together with the first set of CPMVs; and / or storing the height of the first block in the buffer together with the first set of CPMVs; and / or storing information indicating the upper left coordinate of the first block in the buffer together with the first set of CPMVs.
[1013] In some examples, the affine-related information includes a base motion vector (MV) associated with the affine model of the first block.
[1014] In some examples, the method further includes: when the 4-parameter affine model is applied to the transformation of the first block, storing the base MV together with two parameters related to the 4-parameter affine model in a buffer.
[1015] In some examples, the method further includes storing the coordinates of the base MV together with two parameters associated with the 4-parameter affine model in a buffer.
[1016] In some examples, the method further includes: when the 6-parameter affine model is applied to the transformation of the first block, storing the base MV together with four parameters associated with the 6-parameter affine model in a buffer.
[1017] In some examples, the coordinates of the base MV are stored in a buffer along with four parameters associated with a 6-parameter affine model.
[1018] In some examples, if a stored set of affine parameters and its associated base MV reference the same reference picture list, then the set and the base MV reference the same reference picture.
[1019] In some examples, the buffer is an affine history-based motion vector prediction (HMVP) buffer.
[1020] In some examples, the method also includes: determining whether to store affine-related information associated with the affine model of the first block based on characteristics of the first block, the characteristics including one or more of the following: a codec affine mode including affine advanced motion vector prediction (AMVP) and / or affine Merge, the number of affine codec video blocks, the position of the first block, or the size of the first block.
[1021] In some examples, the method further includes: for every K-th block of the video, storing affine-related information associated with the affine model of the block after decoding or encoding the block, K being an integer.
[1022] In some examples, K is 2 or 4.
[1023] In some examples, K is signaled in at least one of: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group header, a slice.
[1024] In some examples, the number of sets of affine-related information stored in the buffer is less than or equal to a preset maximum value M, where M is an integer.
[1025] In some examples, M is 8 or 16.
[1026] In some examples, M is signaled in at least one of: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group header, a slice, a codec tree unit (CTU), a CTU line.
[1027] In some examples, M is different for different standard profiles / levels / tiers.
[1028] In some examples, the method also includes: when the number of sets of affine-related information stored in the buffer is less than a preset maximum value M, storing a new set of affine-related information to be stored after the last set of affine-related information stored in the buffer.
[1029] In some examples, the method further includes: when the number of sets of affine-related information stored in the buffer is equal to a preset maximum value M, discarding a new set of affine-related information to be stored.
[1030] In some examples, the method also includes: when the number of sets of affine-related information stored in the buffer is equal to a preset maximum value M, removing the entry used to store the set of affine-related information stored in the buffer; and storing a new set of affine-related information to be stored in the buffer.
[1031] In some examples, the oldest entry stored in the buffer is removed.
[1032] In some examples, the last entry stored in the buffer is removed.
[1033] In some examples, any entries stored in the buffer are removed.
[1034] In some examples, in response to an entry being removed, a new set of affine-related information is stored in the buffer at a location corresponding to the removed entry.
[1035] In some examples, in response to an entry being removed, all entries following the removed entry are moved forward in the buffer in ascending order, and a new set of affine-related information is stored in a last entry of the buffer.
[1036] In some examples, in response to an entry being removed, all entries preceding the removed entry are moved back in the buffer in descending order, and a new set of affine-related information is stored in a first entry of the buffer.
[1037] In some examples, the method also includes flushing the buffer.
[1038] In some examples, the buffer is cleared on flush.
[1039] In some examples, the buffer is cleared on a refresh and then one or more default affine related information is placed in the buffer on a refresh.
[1040] In some examples, the default affine related information is different for at least one of: different sequences, different pictures, different slices, different patches, different CTU lines, different CTUs.
[1041] In some examples, default affine related information is signaled in at least one of: sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), slice header, slice group header, slice, CTU, CTU line.
[1042] In some examples, the buffer is flushed when encoding or decoding the first block of a picture starts, or when encoding or decoding the first block of a slice starts, or when encoding or decoding the first block of a slice starts, or when encoding or decoding the first block of a CTU line starts, or when encoding or decoding the first block of a CTU starts.
[1043] In some examples, the affine-related information includes at least one of an affine parameter, a CPMV, a width, a height, or a top-left position of the first block.
[1044] In some examples, the method also includes: making a decision about a codec mode and / or other codec information of a codec unit (CU) of the first block; and determining whether to update the buffer based on the decision.
[1045] In some examples, the method further includes: in response to a decision indicating that the CU is encoded and decoded in affine Merge mode and the Merge candidate is derived from the buffer, the buffer is not updated after decoding the CU.
[1046] In some examples, the method further includes, in response to a decision indicating that the CU is encoded in affine Merge mode and the Merge candidate is derived from the buffer, updating the buffer by moving the associated affine parameters to a last entry of the buffer.
[1047] In some examples, the method further includes updating the buffer each time a block is encoded in affine mode.
[1048] In some examples, the method further includes: when a block is encoded and decoded in Affine Merge mode and the block uses a shared Merge list, updating of the Affine HMVP buffer is skipped.
[1049] In some examples, the maximum allowed size of the buffer is equal to 1.
[1050] In some examples, no counter is used to record the number of sets of affine parameters stored in the buffer.
[1051] In some examples, the buffer is at least one of: a table, or a lookup table, or a first-in-first-out (FIFO) table, or a stack, or a queue, or a list, or a linked list, or an array, or any other storage having any data structure.
[1052] In some examples, for a 4-parameter affine model, the first set of affine parameters includes at least one variable of a, b, c, d, Where (mv h 0,mv v 0) is the motion vector of the upper left control point, (mv h 1, mv v 1) is the motion vector of the upper right control point, and (mv h 2.mv v 2) is the motion vector of the lower left control point, w and h are the width and height of the first block.
[1053] In some examples, for a 6-parameter affine model, the first set of affine parameters includes at least one variable of a, b, c, d, e, f, or (e, f) = (mv x ,mv y ), where (mv x ,mv y ) is any MV used to encode and decode the first block, where (mv h 0,mv v 0) is the motion vector of the upper left control point, (mv h 1, mv v 1) is the motion vector of the upper right control point, and (mv h 2.mv v 2) is the motion vector of the lower left control point, w and h are the width and height of the first block.
[1054] Fig.29 2900 is a flow chart of a method for processing a video. The method 2900 includes: calculating (2902) a first set of affine parameters associated with the affine model of a first block of affine coding based on a motion vector (MV) associated with the affine model of the first block, a width of the first block, and a height of the first block; and cropping (2904) the calculated first set of affine parameters.
[1055] In some examples, calculating a first set of affine parameters includes calculating a first set of affine parameters associated with the affine model of the first block based on a motion vector (MV) associated with the affine model of the first block, a width of the first block, and a height of the first block, and an integer defining a calculation precision without using a division operation.
[1056] In some examples, the method further includes: clipping the first set of affine parameters before storing the affine parameters in the buffer.
[1057] In some examples, the first set of affine parameters is clipped based on a bit depth of the affine parameters.
[1058] In some examples, the affine parameters in the first set of affine parameters are clipped to [-2 K-1 , 2 K-1 -1], where K is an integer.
[1059] In some examples, K is equal to 7.
[1060] In some examples, the method further includes: clipping the first set of affine parameters prior to deriving a motion vector (MV) for a sub-block of the first block.
[1061] In some examples, the first set of affine parameters is cropped based on a set of cropping boundaries associated with the affine parameters.
[1062] In some examples, the cropping boundary depends on at least one of: a precision of the affine parameters determined by a bit depth of the affine parameters, a width and height of the first block, a profile of the standard, or a level of the standard.
[1063] In some examples, the cropping boundary is signaled in at least one of: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a slice group header.
[1064] Fig.30 3000 is a flow chart of a method 3000 for processing a video. The method 3000 includes: for a conversion between a current block of a video and a bitstream representation of the current block, deriving (3002) a virtual affine model of the current block from a plurality of affine models stored in a buffer for storing affine models; and performing (3004) the conversion by using the virtual affine model.
[1065] In some examples, by obtaining one or more parameters from a first affine model among multiple affine models and obtaining remaining parameters from a second affine model among the multiple affine models, the parameters of the first affine model and the second affine model are combined to form a virtual affine model.
[1066] In some examples, parameters of a first affine model and a second affine model among a plurality of affine models are jointly used to generate a virtual affine model by using a function, where the function is an average.
[1067] In some examples, the virtual affine model of the current block includes at least one of a 4-parameter affine model and a 6-parameter affine model.
[1068] In some examples, the transform generates the current block of video from the bitstream representation.
[1069] In some examples, the transform generates a bitstream representation from the current block of video.
[1070] Fig.31 3100 is a flow chart of a method 3100 for processing a video. The method 3100 includes: deriving (3102) a first set of affine parameters associated with an affine model of a current block of video for a conversion between a current block and a bitstream representation of the current block; comparing (3104) the first set of affine parameters with one or more of a set of affine parameters stored in a buffer for storing affine parameters to determine whether the first set of affine parameters is the same as or similar to at least one set of affine parameters; and deciding (3106) based on the determination whether or how to use the first set of affine parameters to update the buffer for storing affine parameters.
[1071] In some examples, the affine model of the current block includes at least one of a 4-parameter affine model and a 6-parameter affine model.
[1072] In some examples, for a prediction direction, in response to one or more parameters in the first set of affine parameters being the same as corresponding one or more parameters in at least one set of affine parameters stored in a buffer, it is determined that the first set of affine parameters is the same or similar to at least one set of affine parameters.
[1073] In some examples, in response to an absolute difference between each of at least one parameter in the first set of affine parameters and a corresponding parameter in at least one set of affine parameters stored in a buffer being less than a threshold corresponding to each parameter, it is determined that the first set of affine parameters is the same or similar to at least one set of affine parameters for a prediction direction.
[1074] In some examples, the first set of affine parameters includes variables a, b, c, or d, or variables a, b, c, d, e, f, Or (e, f) = (mv x , mv y ), where (mv x , mv y) is any MV used to encode and decode the first block, where (mv h 0,mv v 0) is the motion vector of the upper left control point, (mv h 1, mv v 1) is the motion vector of the upper right control point, and (mv h 2.mv v 2) is the motion vector of the lower left control point, w and h are the width and height of the current block.
[1075] In some examples, the number of at least one parameter is one, and the at least one parameter includes a variable a, b, c, or d.
[1076] In some examples, the number of at least one parameter is two, and the at least one parameter includes variables a and b or includes variables c and d.
[1077] In some examples, the number of the at least one parameter is three, and the at least one parameter includes variables a, b, and c.
[1078] In some examples, the number of at least one parameter is four, and the at least one parameter includes variables a, b, c, and d.
[1079] In some examples, the threshold is a predefined number.
[1080] In some examples, the threshold value depends on codec information of the current block, where the codec information includes a width and / or a height of the current block.
[1081] In some examples, thresholds corresponding to different parameters are different for different standard profiles or levels or tiers.
[1082] In some examples, the threshold is signaled in at least one of: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group header, a slice, a codec tree unit (CTU), a CTU line.
[1083] In some examples, in response to a first set of affine parameters to be compared and another set of affine parameters being associated with different inter-frame prediction directions, it is determined that the two sets of affine parameters are not the same or similar, wherein the inter-frame prediction direction includes list 0, list 1, or bidirectional.
[1084] In some examples, when inter-prediction direction list 0 is one of the prediction directions in use, in response to a first set of affine parameters to be compared and another set of affine parameters being associated with different reference indices of list 0, it is determined that the two sets of affine parameters are not identical or similar.
[1085] In some examples, when inter-prediction direction list 1 is one of the prediction directions in use, in response to a first set of affine parameters to be compared and another set of affine parameters being associated with different reference indices of list 1, it is determined that the two sets of affine parameters are not the same or similar.
[1086] In some examples, it is determined that the two sets of affine parameters are not the same in response to the first set of affine parameters and the other set of affine parameters to be compared having different numbers of affine parameters or using different affine models.
[1087] In some examples, when a first set of affine parameters to be compared and another set of affine parameters are both associated with bidirectional prediction in an inter-prediction direction, in response to parameters in list 0 and parameters in list 1 of the two sets of affine parameters being the same or similar, it is determined that the two sets of affine parameters are the same or similar.
[1088] In some examples, the first set of affine parameters is compared to each set of affine parameters stored in a buffer for storing affine parameters.
[1089] In some examples, the first set of affine parameters is compared to a subset of the set of affine parameters stored in a buffer for storing affine parameters.
[1090] In some examples, the first set of affine parameters is compared to a previous W or a next W sets of affine parameters stored in a buffer for storing affine parameters, W being an integer.
[1091] In some examples, the first set of affine parameters is compared to one of every W sets of affine parameters stored in a buffer for storing affine parameters, W being an integer.
[1092] In some examples, the method further includes: in response to a determination indicating that the first set of affine parameters is the same as or similar to at least one set of affine parameters stored in the buffer, the first set of affine parameters is not stored in the buffer.
[1093] In some examples, the method also includes: in response to a determination indicating that the first set of affine parameters is the same as or similar to a set of affine parameters stored in the buffer, removing a set of affine parameters from the buffer; and storing the first set of affine parameters in the buffer at a position corresponding to the removed set of affine parameters.
[1094] In some examples, the method also includes: in response to a determination indicating that the first set of affine parameters is the same as or similar to a set of affine parameters stored in a buffer, removing the set of affine parameters from the buffer, moving all sets of affine parameters following the removed set of affine parameters forward in the buffer in ascending order, and storing the first set of affine parameters in the buffer at a position corresponding to the last set of affine parameters.
[1095] In some examples, the method also includes: in response to a determination indicating that the first set of affine parameters is the same as or similar to a set of affine parameters stored in a buffer, removing the set of affine parameters from the buffer, moving all sets of affine parameters preceding the removed set of affine parameters backward in the buffer in descending order, and storing the first set of affine parameters in the buffer at a position corresponding to the first set of affine parameters.
[1096] In some examples, the method further includes storing the first set of affine parameters in a buffer in response to a determination indicating that the first set of affine parameters is not the same as or similar to at least one set of affine parameters stored in the buffer.
[1097] In some examples, the method further includes: performing the transforming using a first set of affine parameters.
[1098] In some examples, the transform generates the current block of video from the bitstream representation.
[1099] In some examples, the transform generates a bitstream representation from the current block of video.
[1100] Fig.32 3200 is a flow chart of a method for processing a video. The method 3200 includes: deriving (3202) affine related information associated with an affine model of a current block for conversion between a current block and a bitstream representation of the current block based on affine parameters stored in a buffer for affine related information of a block encoded and decoded before a current block of the video; and performing (3204) the conversion by using the affine related information.
[1101] In some examples, the affine model of the current block includes at least one of a 4-parameter affine model and a 6-parameter affine model.
[1102] In some examples, the affine parameters stored in the buffer are used for motion vector prediction or motion vector encoding and decoding of the current block.
[1103] In some examples, the affine related information includes a control point motion vector (CPMV) for the current block.
[1104] In some examples, the affine-related information includes a motion vector (MV) used in motion compensation of a sub-block of the current block.
[1105] In some examples, the affine-related information includes a prediction of a control point motion vector (CPMV) of the current block, wherein the prediction of the CPMV is used to predict the CPMV of the current block when encoding and decoding the CPMV is required.
[1106] In some examples, when the current block is encoded and decoded with a 4-parameter affine model, a higher priority is assigned to the 4-parameter affine model stored in the buffer, and a lower priority is assigned to the 6-parameter affine model stored in the buffer.
[1107] In some examples, when the current block is encoded and decoded using a 6-parameter affine model, a higher priority is assigned to the 6-parameter affine model stored in the buffer, and a lower priority is assigned to the 4-parameter affine model stored in the buffer.
[1108] In some examples, deriving affine-related information associated with the affine model of the current block includes: deriving affine-related information associated with the affine model of the current block based on a set of affine parameters stored in a buffer and additional motion information.
[1109] In some examples, the additional motion information includes at least one of the following: motion information of adjacent blocks, or motion information associated with a set of affine parameters, or motion information of spatially adjacent or non-adjacent unit blocks, or motion information of temporally adjacent unit blocks / collocated unit blocks in collocated pictures, the motion information including at least a motion vector, a reference list indication, and a reference picture index.
[1110] In some examples, deriving the CPMV of the current block is based on a set of affine parameters, the coordinates of the position for which the CPMV is to be derived, the coordinates of the base position, and one of the following: the base MV (mv) of a neighboring block in the spatial or temporal domain h 0,mv v 0), or the basic MV associated with a set of affine parameters (mv h 0,mv v 0), or the basic MV of adjacent or non-adjacent unit blocks in the spatial domain (mv h 0,mv v 0), or the basic MV of the adjacent unit block in the time domain (mv h 0,mv v 0), where the base position is associated with the base MV.
[1111] In some examples, deriving the CPMV for the current block is based on the difference between the position for which the CPMV is to be derived and the base position.
[1112] In some examples, the position is determined based on the coordinates of the upper left corner of the current block, the width and height of the current block, and the basic position is determined based on the coordinates of the upper left corner of the current block, the size of the neighboring block or the neighboring unit block.
[1113] In some examples, deriving an MV used in motion compensation for a sub-block of a current block is based on a set of affine parameters, the MV to be derived for which (mv h (x,y),mv v (x, y)), the coordinates of the location of the base location, and one of the following: the base MV (mv) of a neighboring block in the spatial or temporal domain h 0,mv v 0), or the basic MV associated with a set of affine parameters (mv h 0,mv v 0), or the basic MV of adjacent or non-adjacent unit blocks in the spatial domain (mv h 0,mv v 0), or the basic MV of the adjacent unit block in the time domain (mv h 0,mv v 0).
[1114] In some examples, deriving an MV used in motion compensation for a sub-block of the current block is based on the MV to be derived for the sub-block (mv h (x,y),mv v The difference between the position (x, y)) and the base position.
[1115] In some examples, the position is determined based on the coordinates of the upper left corner of the sub-block, the center of the sub-block, and the block size of the sub-block.
[1116] In some examples, when the set of affine parameters stored in the buffer is from a block encoded and decoded with a 4-parameter affine model, deriving the CPMV or MV (mv h (x,y),mv v (x, y)) based on a 4-parameter affine model.
[1117] In some examples, when the set of affine parameters stored in the buffer is from a block encoded and decoded with a 6-parameter affine model, deriving the CPMV or MV (mv h (x,y),mv v (x, y)) based on a 6-parameter affine model.
[1118] In some examples, deriving CPMV or MV (mv h (x,y),mv v (x, y)) is based on a 6-parameter affine model, regardless of whether the set of affine parameters stored in the buffer comes from a block encoded or decoded with a 4-parameter affine model or a 6-parameter affine model.
[1119] In some examples, the CPMV of the current block is derived based on a set of affine parameters stored in a buffer and a motion vector, and the CPMV of the current block serves as a motion vector predictor (MVP) of the signaled CPMV of the current block.
[1120] In some examples, deriving the CPMV of the current block is based on a set of affine parameters stored in a buffer and a motion vector, and the CPMV of the current block is used to derive the MV of each sub-block of the current block for motion compensation.
[1121] In some examples, when the current block is affine-merge encoded, deriving the MV of each sub-block of the current block for motion compensation is based on a set of affine parameters stored in a buffer and a motion vector.
[1122] In some examples, the sets of motion vectors and affine parameters of the neighboring unit blocks or neighboring blocks are associated with the same inter prediction direction, where the inter prediction direction includes list 0, list 1, or bidirectional.
[1123] In some examples, when inter prediction direction list 0 is the one prediction direction in use, the set of motion vectors and affine parameters are associated with the same reference index of list 0.
[1124] In some examples, when inter prediction direction list 1 is the one prediction direction in use, the set of motion vectors and affine parameters are associated with the same reference index of list 1.
[1125] In some examples, when the MV of a spatial neighboring unit block or a neighboring block and a set of affine parameters stored in a buffer refer to different reference pictures, the MV of the spatial neighboring unit block or the neighboring block is scaled to refer to the same reference picture as the set of affine parameters stored in the buffer to derive the affine model of the current block.
[1126] In some examples, when the MV of a time-domain neighboring unit block or a neighboring block and a set of affine parameters stored in a buffer reference different reference pictures, the MV of the time-domain unit block or the neighboring block is scaled to reference the same reference picture as the set of affine parameters stored in the buffer to derive an affine model of the current block.
[1127] Fig.333300 is a flowchart of a method for processing a video. The method 3300 includes: deriving (3302) an affine model of the current block for conversion between the current block and a bitstream representation of the current block based on a set of affine parameters stored in a buffer for affine related information of a block encoded and decoded before the current block of the video; generating (3304) an affine Merge candidate list, wherein the affine Merge candidate list includes affine Merge candidates of the current block based on the derived affine model of the current block; and performing (3306) the conversion based on the affine Merge candidate list.
[1128] In some examples, side information associated with a set of stored affine parameters is inherited by an affine merge candidate based on a derived affine model, wherein the side information includes an inter prediction direction and a reference index of list 0 / list 1, wherein the inter prediction direction includes list 0, list 1, and bidirectional.
[1129] In some examples, affine merge candidates based on the derived affine model are inserted into the affine merge candidate list before the constructed affine merge candidates in the list and after the affine merge candidates inherited from the neighboring blocks in the list.
[1130] In some examples, an affine merge candidate based on the derived affine model is inserted into the affine merge candidate list before all other affine merge candidates.
[1131] In some examples, affine merge candidates based on the derived affine model are inserted into the affine merge candidate list before the fill candidates in the list and after the constructed affine merge candidates in the list.
[1132] In some examples, an affine Merge candidate based on a derived affine model is inserted into the affine Merge candidate list before an affine Merge candidate constructed using temporal motion prediction in the list and after an affine Merge candidate constructed without using temporal motion prediction in the list.
[1133] In some examples, affine merge candidates based on the derived affine model are inserted into the affine merge candidate list and interleaved with the constructed affine merge candidates in the list and the fill candidates in the list.
[1134] In some examples, the method further includes: adding N candidates derived based on the affine parameters stored in the buffer to an affine Merge candidate list, where N is a predefined integer.
[1135] Fig.343400 is a flowchart of a method 3400 for processing a video. The method 3400 includes: generating (3402) an affine advanced motion vector prediction (AMVP) candidate list for a current block of a video as an affine AMVP codec block for conversion between the current block and a bitstream representation of the current block, wherein the candidate list includes at least one AMVP candidate based on an affine parameter stored in a buffer for storing affine related information of a block coded before the current block; and performing (3404) the conversion based on the affine AMVP candidate list.
[1136] In some examples, the stored affine parameters used to generate the affine AMVP candidate reference the same reference picture as the target reference picture of the current block.
[1137] In some examples, the reference picture list associated with the stored affine parameters is the same as the target reference picture list for the current block.
[1138] In some examples, the reference index associated with the stored affine parameters is the same as the target reference index for the current block.
[1139] In some examples, the affine AMVP candidates based on affine parameters are inserted into the affine AMVP candidate list before the constructed affine AMVP candidates in the list and after the affine AMVP candidates inherited from neighboring blocks in the list.
[1140] In some examples, an affine AMVP candidate based on affine parameters is inserted into the affine AMVP candidate list before all other affine AMVP candidates.
[1141] In some examples, the affine parameter-based affine AMVP candidates are inserted into the affine AMVP candidate list before the High Efficiency Video Codec (HEVC)-based AMVP candidates in the list and after the constructed affine AMVP candidates in the list.
[1142] In some examples, the affine parameter-based affine AMVP candidates are inserted into the affine AMVP candidate list after the High Efficiency Video Codec (HEVC) based AMVP candidates in the list, before the populated affine AMVP candidates in the list.
[1143] In some examples, the affine AMVP candidates based on affine parameters are inserted into the affine AMVP candidate list before the constructed affine AMVP candidates in the list that use temporal motion prediction and after the constructed affine AMVP candidates in the list that do not use temporal motion prediction.
[1144] In some examples, when the current block is encoded and decoded with a 4-parameter affine model, a higher priority is assigned to the 4-parameter affine model stored in the buffer, and a lower priority is assigned to the 6-parameter affine model stored in the buffer.
[1145] In some examples, when the current block is encoded and decoded using a 6-parameter affine model, a higher priority is assigned to the 6-parameter affine model stored in the buffer, and a lower priority is assigned to the 4-parameter affine model stored in the buffer.
[1146] In some examples, the method further includes: adding N candidates derived based on the affine parameters stored in the buffer to an affine AMVP candidate list, N being a predefined integer.
[1147] In some examples, N is signaled in at least one of: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group header, a slice.
[1148] In some examples, N depends on at least one of: a block size of the current block, coding mode information including AMVP mode and / or Merge mode.
[1149] In some examples, N depends on the available candidates in the candidate list.
[1150] In some examples, N depends on the particular type of available candidates in the candidate list.
[1151] In some examples, when N is greater than 1, N candidates derived based on affine parameters stored in the buffer are added to the candidate list in ascending order of index, or in descending order of index, or in an order determined based on the number of available candidates in the candidate list.
[1152] In some examples, the method further includes: selecting N sets of affine parameters from all sets of affine parameters stored in the buffer to derive N candidates, N being an integer.
[1153] In some examples, the selected N sets of affine parameters are the latest N sets of affine parameters in the buffer.
[1154] In some examples, the N sets of affine parameters selected depend on the index of the set of affine model parameters in the buffer.
[1155] Fig.353500 is a flowchart of a method for processing a video. The method 3500 includes: deriving (3502) an affine candidate for a current block for a conversion between a current block and a bitstream representation of the current block based on affine parameters stored in a buffer for affine related information of a block encoded and decoded before a current block of the video and one or more unit blocks; inserting (3504) the derived affine candidate into an affine candidate list; and performing (3506) the conversion by using the affine candidate list.
[1156] In some examples, the unit block includes at least one of the group consisting of spatially adjacent / non-adjacent unit blocks and temporally adjacent / non-adjacent unit blocks.
[1157] In some examples, the derived affine candidate is placed before all other affine candidates, or just after the inherited affine candidate in the list, or just after the first constructed affine candidate in the list, or just after the first affine candidate constructed from spatially neighboring blocks / non-neighboring blocks in the list, or just after all affine candidates derived from spatially neighboring blocks / non-neighboring blocks in the list, or just after all constructed affine candidates in the list, or just before all zero affine candidates.
[1158] In some examples, when another affine candidate is inherited from a spatially or temporally adjacent / non-adjacent unit block, the spatially or temporally adjacent / non-adjacent unit block is not used to derive the affine candidate using the affine parameters stored in the buffer.
[1159] In some examples, spatially or temporally adjacent / non-adjacent blocks of cells are used to derive affine candidates using only one set of affine parameters stored in a buffer.
[1160] In some examples, at most N affine candidates derived from the affine parameters stored in the buffer and the unit block can be put into the affine candidate list, where N is an integer.
[1161] In some examples, if an affine candidate derived from affine parameters stored in a buffer and a unit block is selected, a generalized bi-prediction improvement (GBI) index of the current block is inherited from a GBI index of a spatially or temporally adjacent / non-neighboring unit block.
[1162] In some examples, affine candidates derived from affine parameters stored in a buffer and unit blocks are sequentially placed into an affine candidate list.
[1163] In some examples, the method further includes: searching for available affine candidates derived from the affine parameters stored in the buffer and the unit block by using a two-level nested loop method; and inserting the available affine candidates into the affine candidate list.
[1164] In some examples, in the first level loop, each set of affine parameters stored in the buffer is accessed in order from the beginning to the end of the table, or from the end to the beginning of the table, or any other predefined or adaptive order.
[1165] In some examples, some sets of affine parameters stored in the buffer are skipped in the first loop.
[1166] In some examples, for each set of affine parameters stored in the buffer, a second level loop is applied where each cell block is visited sequentially.
[1167] In some examples, only one spatially adjacent / non-adjacent unit block is included in the second level loop, or only one temporally adjacent / non-adjacent unit block is included in the second level loop.
[1168] In some examples, affine candidates are generated and placed in an affine candidate list using a set of affine parameters given in a first-level loop and a spatial or temporal neighboring unit block given in a second-level loop if all or part of the following conditions are met:
[1169] i. Spatial or temporal adjacent / non-adjacent unit blocks are available;
[1170] ii. Spatial or temporal adjacent / non-adjacent unit blocks are inter-frame coded;
[1171] iii. The spatial or temporal adjacent / non-adjacent unit blocks are not outside the current CTU row;
[1172] iv. the set of affine parameters including list 0, list 1 or bidirectional inter-frame prediction and spatial or temporal adjacent / non-adjacent unit block inter-frame prediction is the same;
[1173] v. The reference index of list 0 of the set of affine parameters is the same as the reference index of the adjacent / non-adjacent unit block in the spatial domain or temporal domain;
[1174] vi. The reference index of the list 1 of the set of affine parameters is the same as the reference index of the adjacent / non-adjacent unit block in the spatial domain or the temporal domain;
[1175] vii. The POC of the reference picture of list 0 of the set of affine parameters is the same as the POC of one of the reference pictures of the adjacent / non-adjacent unit block in the spatial domain or the temporal domain.
[1176] viii. The POC of the reference picture of list 1 of the set of affine parameters is the same as the POC of one of the reference pictures of the adjacent / non-adjacent unit block in the spatial domain or the temporal domain.
[1177] In some examples, when a spatial or temporal neighboring unit block has been used to derive inherited affine candidates, the spatial or temporal neighboring unit block is skipped in the second loop and is not used to derive affine candidates with stored affine parameters.
[1178] In some examples, when a spatial or temporal neighboring block has been used to derive an affine candidate using a set of affine parameters stored in a buffer, the spatial or temporal neighboring block is skipped in a second loop and is not used to derive an affine candidate using another set of affine parameters stored in the buffer.
[1179] In some examples, when a spatial or temporal neighboring block is used to derive an affine candidate, all other neighboring blocks after the neighboring block are skipped and the second loop is jumped out to return to the first loop and the next set of affine parameters is accessed in the first loop.
[1180] In some examples, the size of the affine candidate list is increased by N, where N is an integer, when an affine candidate derived from the affine parameters stored in the buffer can be placed into the list.
[1181] In some examples, the affine candidate is an affine merge candidate, and the affine candidate list is an affine merge candidate list.
[1182] In some examples, the affine candidate is an affine AMVP candidate and the affine candidate list is an affine AMVP candidate list.
[1183] In some examples, the derived affine candidate is placed just after the first translation-affine AMVP candidate in the list or after all translation-affine AMVP candidates in the list.
[1184] Fig.36 is a flow chart of a method 3600 for processing a video. The method 3600 includes: deriving (3602) an affine candidate for a current block of a video for a conversion between a current block and a bitstream representation of the current block based on affine parameters stored in an affine history-based motion vector prediction (HMVP) buffer; inserting (3604) the derived affine candidate into an affine candidate list; and performing (3606) the conversion by using the affine candidate list.
[1185] In some examples, affine candidates inherited from spatially neighboring blocks of the current block are not put into the list.
[1186] In some examples, affine candidates inherited from spatially neighboring blocks of the current block are excluded from the list.
[1187] In some examples, derived affine candidates are inserted into the list after affine candidates inherited from blocks in a different CTU row than the current CTU row.
[1188] In some examples, affine candidates inherited from blocks in the current CTU row are removed from the list.
[1189] In some examples, whether an affine candidate inherited from a spatial neighboring block of the current block is inserted into the list depends on the position of the spatial neighboring block.
[1190] In some examples, when the spatial neighboring block is in the same CTU or CTU row as the current block, the affine candidates inherited from the spatial neighboring block are inserted into the list.
[1191] In some examples, when the spatial neighboring block is not in the same CTU or CTU row as the current block, the affine candidates inherited from the spatial neighboring block are put into the list.
[1192] In some examples, whether to insert an affine Merge candidate inherited from a spatial neighboring block of the current block depends on the affine HMVP buffer.
[1193] In some examples, when the affine HMVP buffer is empty, the inherited affine Merge candidate is added to the list; and when the affine HMVP buffer is not empty, the inherited affine Merge candidate is excluded from the list.
[1194] In some examples, whether to insert affine merge candidates inherited from spatially neighboring blocks of the current block depends on the block size.
[1195] In some examples, the affine candidate is an affine Merge candidate, and the affine candidate list is an affine Merge candidate list / sub-block based Merge candidate list.
[1196] In some examples, the affine candidate is an affine AMVP candidate and the affine candidate list is an affine AMVP candidate list.
[1197] In some examples, the derived affine Merge candidate is inserted into the list just after the optional temporal motion vector prediction (ATMVP) candidate.
[1198] In some examples, the derived affine merge candidates are inserted into the list before the constructed affine merge candidates.
[1199] In some examples, the derived affine AMVP candidates are inserted into the list first.
[1200] In some examples, the base location is any location within the vicinity of a cell block.
[1201] In some examples, the base position is determined based on the coordinates of the upper left corner of the current block and the size of the neighboring unit blocks.
[1202] It should be understood that when the shape of the compressed codec unit is significantly different from the traditional square block or half-square rectangular block, the disclosed technology can be embodied in a video encoder or decoder to improve compression efficiency. For example, new codec tools that use long or tall codec units such as units with a size of 4×32 or 32×4 can benefit from the disclosed technology.
[1203] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this document and their structural equivalents), or in a combination of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, which are used to be executed by a data processing device or control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect machine-readable propagation signals, or a combination of one or more of them. The term "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal generated to encode information for transmission to a suitable receiver device, such as a machine-generated electrical signal, an optical signal, or an electromagnetic signal.
[1204] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages), and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers located on one site or distributed across multiple sites and interconnected by a communication network.
[1205] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).
[1206] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operably coupled to receive data from or transfer data to or from the one or more mass storage devices. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.
[1207] Although this patent document contains many details, these details should not be interpreted as limitations on any subject matter or the scope of possible protection, but as descriptions of features of specific embodiments of specific technologies. Certain features described in the context of separate embodiments in this patent document may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments or in any suitable sub-combination, respectively. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.
[1208] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
[1209] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and shown in this patent document.
Claims
1. A method for processing a video, comprising: deriving affine-related information associated with an affine model to be used by a current block for conversion between a current block of video and a bitstream of the current block based on affine-related information associated with an affine model used by a first block stored in a buffer, wherein the first block is affine-encoded before the current block; and performing the transformation by using the derived affine related information; The buffer is an affine history-based motion vector prediction HMVP buffer, The affine related information associated with the affine model used by the first block includes: a first set of control point motion vectors CPMV associated with the affine model of the first block, or The basic motion vectors associated with the affine model of the first block The method further comprises: When a 4-parameter affine model is applied to the transformation of the first block, the basic MV is stored in the buffer together with two parameters related to the 4-parameter affine model.
2. The method according to claim 1, wherein: The affine model includes at least one of a 4-parameter affine model and a 6-parameter affine model.
3. The method according to claim 1, wherein: The affine-related information stored in the buffer includes a first set of affine parameters associated with an affine model of the first block.
4. The method according to claim 3, wherein: When a 6-parameter affine model is applied to the transformation of the first block, six parameters associated with the 6-parameter affine model are stored in the buffer, or when a 4-parameter affine model is applied to the transformation of the first block, four parameters associated with the 4-parameter affine model are stored in the buffer.
5. The method according to claim 3, wherein: When a 6-parameter affine model is applied to the transformation of the first block, four parameters related to the 6-parameter affine model are stored in the buffer, and two other parameters related to the 6-parameter affine model are not stored in the buffer.
6. The method according to claim 3, wherein: When a 4-parameter affine model is applied to the transformation of the first block, two parameters related to the 4-parameter affine model are stored in the buffer, and two other parameters related to the 4-parameter affine model are not stored in the buffer.
7. The method according to claim 3, wherein: Six parameters associated with the 6-parameter affine model are always stored in the buffer, and when the 4-parameter affine model is applied to the transformation of the first block, a first parameter of the six parameters is set to be the same as a second parameter of the six parameters, and a third parameter of the six parameters is set to a negative value of a fourth parameter of the six parameters.
8. The method according to claim 3, wherein: Four parameters associated with the 6-parameter affine model are always stored in the buffer, and when the 4-parameter affine model is applied to the transformation of the first block, a first parameter of the four parameters is set to be the same as a second parameter of the four parameters, and a third parameter of the four parameters is set to a negative value of a fourth parameter of the four parameters.
9. The method according to claim 3, wherein: A first number of parameters associated with a 4-parameter affine model is stored, and a second number of parameters associated with a 6-parameter affine model is stored, the first number and the second number being the same.
10. The method according to claim 9, wherein: The first number and the second number are 4 or 6.
11. The method according to claim 3, wherein: A first number of parameters associated with a 4-parameter affine model is stored and a second number of parameters associated with a 6-parameter affine model is stored, the first number and the second number being different.
12. The method according to claim 11, wherein: The first number is four and the second number is six.
13. The method according to claim 3, further comprising: Determine characteristics of the first block of the video, wherein the characteristics include one or more of the following: an affine mode, use of inter-frame or merge mode, a size of the first block, a picture type, wherein the first set of affine parameters to be stored in the buffer is determined based on the characteristics of the first block of the video.
14. The method according to claim 3, wherein: Side information and / or an affine model type associated with the first set of affine parameters of the affine model is stored in the buffer, wherein the affine model type comprises a 4-parameter affine model or a 6-parameter affine model, and the side information comprises at least one of: an inter-frame prediction direction comprising list 0 or list 1 or bidirectional, and a list 0 and / or list 1 reference index.
15. The method according to claim 14, wherein: When the first block is bi-predicted, the first set of affine parameters further includes affine parameters for list 0 and affine parameters for list 1.
16. The method according to claim 15, wherein: The parameters for list 0 and the parameters for list 1 are both stored in the buffer.
17. The method according to claim 15, wherein: The affine parameters for list 0 are stored in a first buffer, and the affine parameters for list 1 are stored in a second buffer different from the first buffer.
18. The method according to claim 15, wherein: One of the parameters for list 0 and the parameters for list 1 is stored, and the stored affine parameters are used to predict the other parameter.
19. The method of claim 1, further comprising: storing the width of the first block in the buffer along with the first set of CPMVs; and / or storing the height of the first block together with the first set of CPMVs in the buffer; and / or Information indicating an upper left coordinate of the first block is stored in the buffer along with the first set of CPMVs.
20. The method of claim 1, further comprising: The coordinates of the base MV are stored in the buffer together with two parameters associated with the 4-parameter affine model.
21. The method according to claim 1, further comprising: When a 6-parameter affine model is applied to the transformation of the first block, the basic MV is stored in the buffer together with four parameters related to the 6-parameter affine model.
22. The method according to claim 21, wherein: The coordinates of the base MV are stored in the buffer together with four parameters associated with the 6-parameter affine model.
23. The method according to any one of claims 1 to 22, wherein: If a set of stored affine parameters and its associated base MV refer to the same reference picture list, then the set and the base MV refer to the same reference picture.
24. The method according to any one of claims 1 to 3, further comprising: Determine whether to store affine-related information associated with the affine model of the first block based on characteristics of the first block, the characteristics including one or more of the following: a codec affine mode including affine advanced motion vector prediction (AMVP) and / or affine Merge, the number of affine codec video blocks, the position of the first block, or the size of the first block.
25. The method according to any one of claims 1 to 3, further comprising: For every K-th block of the video, affine-related information associated with an affine model of the K-th block is stored after decoding or encoding the K-th block, where K is an integer.
26. The method according to claim 25, wherein: K is 2 or 4.
27. The method according to claim 25, wherein: K is signaled in at least one of the following: sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), slice header, slice group header, slice.
28. The method according to any one of claims 1 to 22, wherein: The number of sets of affine related information stored in the buffer is less than or equal to a preset maximum value M, where M is an integer.
29. The method according to claim 28, wherein: M is 8 or 16.
30. The method of claim 28, wherein: M is signaled in at least one of the following: sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), slice header, slice group header, slice, codec tree unit (CTU), CTU line.
31. The method of claim 28, wherein: M is different for different standard profiles / levels / tiers.
32. The method of claim 28, further comprising: When the number of sets of affine related information stored in the buffer is less than a preset maximum value M, a new set of affine related information to be stored is stored after the last set of affine related information stored in the buffer.
33. The method of claim 28, further comprising: When the number of sets of affine-related information stored in the buffer is equal to a preset maximum value M, a new set of affine-related information to be stored is discarded.
34. The method of claim 28, further comprising: When the number of sets of affine-related information stored in the buffer is equal to a preset maximum value M, removing an entry for storing the set of affine-related information stored in the buffer; and A new set of affine related information to be stored is stored in the buffer.
35. The method of claim 34, wherein: Removes the oldest entry stored in the buffer.
36. The method of claim 34, wherein: Removes the last entry stored in the buffer.
37. The method of claim 34, wherein: Removes any entry stored in the buffer.
38. The method of claim 37, wherein: In response to the entry being removed, the new set of affine related information is stored in the buffer at a position corresponding to the removed entry.
39. The method of claim 37, wherein: In response to the entry being removed, all entries following the removed entry are moved forward in the buffer in ascending order, and the new set of affine related information is stored in the last entry of the buffer.
40. The method of claim 37, wherein: In response to the entry being removed, all entries preceding the removed entry are moved backward in the buffer in descending order, and the new set of affine-related information is stored in a first entry of the buffer.
41. The method of any one of claims 1 to 3, further comprising flushing the buffer.
42. The method according to claim 41, wherein: The buffer is cleared on flush.
43. The method of claim 41, wherein: The buffer is cleared upon refresh, and then one or more pieces of default affine related information are placed into the buffer upon refresh.
44. The method of claim 43, wherein: The default affine related information is different for at least one of: different sequences, different pictures, different slices, different slices, different CTU lines, different CTUs.
45. The method of claim 43, wherein: The default affine related information is signaled in at least one of: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group header, a slice, a CTU, a CTU line.
46. The method of claim 41, wherein: The buffer is flushed when the first block of a picture starts to be encoded or decoded, or when the first block of a slice starts to be encoded or decoded, or when the first block of a slice starts to be encoded or decoded, or when the first block of a CTU line starts to be encoded or decoded, or when the first block of a CTU starts to be encoded or decoded, the buffer is flushed.
47. The method according to any one of claims 1 to 22, wherein: The affine-related information includes at least one of an affine parameter, a CPMV, a width, a height, or an upper left position of the first block.
48. The method of any one of claims 1-22, further comprising: Making a determination regarding a codec mode and / or other codec information of a codec unit (CU) of the first block; as well as A determination is made based on the determination whether to update the buffer.
49. The method of claim 48, further comprising: In response to a decision indicating that the CU is encoded and decoded in affine merge mode and merge candidates are derived from the buffer, the buffer is not updated after decoding the CU.
50. The method of claim 48, further comprising: In response to a decision indicating that the CU is encoded in affine merge mode and a merge candidate is derived from the buffer, the buffer is updated by moving associated affine parameters to a last entry of the buffer.
51. The method of claim 48, further comprising: The buffer is updated each time a block is encoded or decoded in affine mode.
52. The method of claim 48, further comprising: When a block is encoded or decoded in affine merge mode and the block uses a shared merge list, the update of the affine HMVP buffer is skipped.
53. The method according to any one of claims 1 to 22, wherein: The maximum allowed size of the buffer is equal to 1.
54. The method of claim 53, wherein: No counter is used to record the number of sets of affine parameters stored in the buffer.
55. The method according to any one of claims 1 to 22, wherein: The buffer is at least one of: a table, or a lookup table, or a first-in-first-out (FIFO) table, or a stack, or a queue, or a list, or a linked list, or an array, or any other storage having any data structure.
56. The method of claim 3, wherein: For a 4-parameter affine model, the first set of affine parameters includes at least one variable among a, b, c, d, Where (mv h 0,mv v 0) is the motion vector of the upper left control point, (mv h 1, mv v 1) is the motion vector of the upper right control point, and (mv h 2.mv v 2) is the motion vector of the lower left control point, w and h are the width and height of the first block.
57. The method of claim 3, wherein: For a 6-parameter affine model, the first set of affine parameters includes at least one variable among a, b, c, d, e, and f. or (e, f) = (mv x , mv y ), where (mv x , mv y ) is any MV used to encode or decode the first block, where (mv h 0,mv v 0) is the motion vector of the upper left control point, (mv h 1, mv v 1) is the motion vector of the upper right control point, and (mv h 2.mv v 2) is the motion vector of the lower left control point, w and h are the width and height of the first block.
58. The method of claim 1, further comprising: Calculating a first set of affine parameters associated with the affine model of the first block based on a motion vector (MV) associated with the affine model of the first block of the affine codec, a width of the first block, and a height of the first block; and Clip the first set of computed affine parameters.
59. The method of claim 58, wherein: Calculating the first set of affine parameters includes: Without using a division operation, a first set of affine parameters associated with the affine model of the first block is calculated based on a motion vector (MV) associated with the affine model of the first block, a width of the first block, a height of the first block, and an integer defining a calculation precision.
60. The method of claim 58 or 59, further comprising: The first set of affine parameters is clipped before storing the affine parameters in the buffer.
61. The method of claim 60, wherein: The first set of affine parameters is clipped based on a bit depth of the affine parameters.
62. The method of claim 60, wherein: The affine parameters in the first set of affine parameters are clipped to [-2 K-1 , 2 K-1 -1], where K is an integer.
63. The method of claim 62, wherein: K is equal to 7.
64. The method of claim 58 or 59, further comprising: The first set of affine parameters is clipped prior to deriving a motion vector (MV) for a sub-block of the first block.
65. The method of claim 63, wherein: The first set of affine parameters is clipped based on a set of clipping boundaries associated with the affine parameters.
66. The method of claim 64, wherein: The cropping boundary depends on at least one of: a precision of the affine parameters determined by a bit depth of the affine parameters, a width and a height of the first block, a profile of a standard, or a level of a standard.
67. The method of claim 65, wherein: The cropping boundary is signaled in at least one of the following: a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a slice group header.
68. The method of claim 1, further comprising: For conversion between a current block of a video and a bitstream of the current block, deriving a virtual affine model of the current block from a plurality of affine models stored in a buffer for storing affine models; The transformation is performed by using the virtual affine model.
69. The method of claim 68, wherein: By acquiring one or more parameters from a first affine model among the multiple affine models and acquiring remaining parameters from a second affine model among the multiple affine models, the parameters of the first affine model and the second affine model are combined to form the virtual affine model.
70. The method of claim 68, wherein: Parameters of a first affine model and a second affine model among the plurality of affine models are jointly used to generate the virtual affine model by using a function, wherein the function is an average.
71. The method of any one of claims 68 to 70, wherein: The virtual affine model of the current block includes at least one of a 4-parameter affine model and a 6-parameter affine model.
72. The method according to any one of claims 1 to 22, wherein: The conversion generates the current block of video from the bitstream.
73. The method according to any one of claims 1 to 22, wherein: The conversion generates the bitstream from the current block of video.
74. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1 to 73.
75. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for executing the method according to any one of claims 1 to 73.
Citation Information
Patent Citations
Video image coding method, video image decoding method, coding device and decoding device
CN106303543A
Method and apparatus for affine inter prediction for video coding system
CN108432250A
Method and apparatus of video coding with affine motion compensation
CN108781284A
Affine motion prediction for video coding
US20170332095A1
Adaptive Affine Motion Compensation Unit Determing in Video Picture Coding Method, Video Picture Decoding Method, Coding Device, and Decoding Device
US20180070102A1
