Motion vector prediction in video encoding and decoding

Abstract

A video codec may involve encoding and decoding picture information and a first flag and a second flag, wherein the encoding or decoding of the picture information is based on a decoding mode indicated by the first flag or the second flag, and the first flag indicates a sub-block merging mode and the second flag indicates an inter-frame affine prediction mode, and the encoding or decoding of the first flag uses CABAC decoding based on a first probability model, and the encoding or decoding of the second flag uses CABAC decoding based on a second probability model.

Description

Technical Field

[0001] The present disclosure relates to video encoding and decoding. Background Art

[0002] In order to achieve high compression efficiency, image and video coding schemes generally use prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original picture block and the predicted picture block (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy decoded. In order to reconstruct the video, the compressed data is decoded by an inverse process corresponding to the prediction, transformation, quantization, and entropy decoding. As explained below, various modifications and embodiments are contemplated that can provide improvements to video encoding and / or decoding systems, including but not limited to one or both of increased compression or coding efficiency and reduced complexity. Summary of the invention

[0003] Generally, an example of an embodiment may relate to a method comprising: decoding a first flag included in a decoded bitstream, the first flag having been decoded using CABAC decoding based on a first probability model; decoding a second flag included in the decoded bitstream, the second flag having been decoded using CABAC decoding based on a second probability model; and decoding picture information included in the decoded bitstream based on a decoding mode indicated by the first flag or the second flag; wherein the first flag indicates a sub-block merge prediction mode and the second flag indicates an inter affine prediction mode.

[0004] Generally, an example of an embodiment may relate to a method comprising: determining a value of a first flag indicating a first prediction mode associated with encoded picture information; determining a value of a second flag indicating a second prediction mode associated with the encoded picture information; and encoding at least a portion of the picture information and the first and second flags to form an encoded bitstream, wherein the first flag indicates a sub-block merge prediction mode and is decoded using CABAC decoding based on a first probability model, and the second flag indicates an inter-frame affine prediction mode and is decoded using CABAC decoding based on a second probability model.

[0005] Generally, another example of an embodiment may relate to a device comprising one or more processors configured to: decode a first flag included in a decoded bitstream, the first flag having been decoded using CABAC decoding based on a first probability model; decode a second flag included in the decoded bitstream, the second flag having been decoded using CABAC decoding based on a second probability model; and decode encoded picture information included in the decoded bitstream based on a decoding mode indicated by the first flag or the second flag; wherein the first flag indicates a sub-block merge prediction mode and the second flag indicates an inter-frame affine prediction mode.

[0006] Generally, another example of an embodiment may involve a device comprising: one or more processors configured to: determine a value of a first flag indicating a first prediction mode associated with encoded picture information; determine a value of a second flag indicating a second prediction mode associated with the encoded picture information; and encode at least a portion of the picture information and the first and second flags to form an encoded bitstream, wherein the first flag indicates a sub-block merge prediction mode and is decoded using CABAC decoding based on a first probability model, and the second flag indicates an inter-frame affine prediction mode and is decoded using CABAC decoding based on a second probability model.

[0007] In general, another example of an embodiment may involve a bitstream formatted to include encoded picture information, wherein the encoded picture information is encoded by processing the picture information based on any one or more of the examples of embodiments of the method according to the invention.

[0008] In general, one or more other examples of embodiments may also provide a computer-readable storage medium, for example, a non-volatile computer-readable storage medium, on which instructions for encoding or decoding picture information such as video data according to the method or apparatus described herein are stored. One or more embodiments may also provide a computer-readable storage medium on which a bitstream generated according to the method or apparatus described herein is stored. One or more embodiments may also provide methods and apparatus for sending or receiving a bitstream generated according to the method or apparatus described herein.

[0009] As explained below, various modifications and embodiments are contemplated which may provide improvements to video encoding and / or decoding systems including, but not limited to, one or more of: increasing compression efficiency and / or coding efficiency and / or processing efficiency and / or reducing complexity.

[0010] A simplified overview of the subject matter is provided above in order to provide a basic understanding of some aspects of the present disclosure. This overview is not an exhaustive review of the subject matter. It is not intended to identify the key / important elements of each embodiment or to describe the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a preface to the more detailed description provided below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure may be better understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0012] Figure 1 A block diagram depicting an example of an embodiment of a video encoder is provided;

[0013] Figure 2 A block diagram depicting an example of an embodiment of a video decoder is provided;

[0014] Figure 3 Aspects of the present disclosure relating to coding tree units (CTUs) are shown;

[0015] Figure 4 Aspects of the present disclosure relating to CTUs and coding units (CUs) are shown;

[0016] Figure 5 A flow chart is provided which illustrates an example of an inter-frame mode, such as in VTM-5;

[0017] Figure 6 A flow chart is provided that illustrates an example of signaling inter-frame prediction information, such as according to HEVC;

[0018] Figure 7 A block diagram is provided showing an example of the location of spatial and temporal motion vector predictors used in merge mode (e.g., in HEVC) (left side: spatial merge candidates; right side: temporal merge candidates);

[0019] Figure 8 A flow chart is provided which shows an example of the construction of a list of merged motion vector predictor candidates, for example in HEVC;

[0020] Fig. 9 A flow chart is provided which shows another example of the construction of a list of merged motion vector predictor candidates, for example in HEVC;

[0021] Fig.10 An example illustrating SbTMVP motion prediction of a CU in, for example, JEM is shown;

[0022] Fig.11 An example of inter-mode signaling using a temporal merging list is shown;

[0023] Fig.12 Examples of the positions of motion vector predictors are shown (left: spatial predictor; right: temporal predictor);

[0024] Fig.13 An example of an embodiment of construction of a list of motion vector predictor candidates based on sub-blocks is shown;

[0025] Fig.14 A flow chart is provided that illustrates an example of an implementation of an inter-frame decoding process utilizing a temporal merge list;

[0026] Fig.15 A block diagram is provided that illustrates an example of an embodiment of an apparatus or system according to various aspects described herein; and

[0027] Fig.16 and Fig.17 An example of the embodiments described herein is shown.

[0028] It should be understood that the drawings are for purposes of illustrating examples of various aspects and embodiments and are not necessarily the only possible configurations. In all drawings, the same reference numerals represent the same or similar features. DETAILED DESCRIPTION

[0029] Now turning to the attached figure, Figure 1 An example of a video encoder 100 is shown, such as a HEVC encoder. HEVC is a video Joint Collaboration Team on Coding (JCT-VC) A compression standard being developed (see, for example, "ITU-T H.265 ITU Telecommunication Standard Part (10 / 2014), SERIES H: AUDIOVISUAL AND MULTIMEDIASYSTEMS, Infrastructure of audiovisual services-Coding of moving video, High efficiency video coding, Recommendation ITU-T H.265"). Figure 1 It is also possible to describe an encoder in which improvements are made to the HEVC standard or an encoder that adopts a technique similar to HEVC, such as an encoder based on or improved upon the JEM (Joint Exploration Model) under development by JVET.

[0030] In this application, the terms "reconstruction" and "decoding" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "picture" and "frame" may be used interchangeably.

[0031] The HEVC specification distinguishes between "blocks" and "units", where a "block" addresses a specific area in the sample array (e.g., luma Y), and a "unit" includes a concatenated block of all encoded color components (Y, Cb, Cr, or monochrome) associated with the block, syntax elements, and prediction data (e.g., motion vectors).

[0032] For decoding, the picture is partitioned into coding tree blocks (CTBs) of a configurable square size, and a continuous set of coding tree blocks is grouped into slices. A coding tree unit (CTU) includes a CTB of a coded color component. The CTB is the root of a quadtree partitioned into coding blocks (CBs), and a coding block can be partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree partitioned into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a tree structure set of prediction units (PUs) and transform units (TUs), the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The sizes of the CB, PB, and TB of the luminance component are applicable to the corresponding CU, PU, ​​and TU. In the present application, the term "block" may be used to represent any of CTU, CU, PU, ​​TU, CB, PB, and TB. Additionally, "block" may also be used to refer to macroblocks and partitions as specified in H.264 / AVC or other video coding standards, and more generally to data arrays of various sizes.

[0033] exist Figure 1 In an encoder 100, as described below, a picture is encoded by an encoder element. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the CU, and indicates the intra / inter decision by a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0034] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients as well as motion vectors and other syntax elements are entropy decoded (145) to output a bitstream. The encoder may also skip the transform and apply quantization directly to the untransformed residual signal on a 4×4TU basis. The encoder may also bypass the transform and quantization, i.e., directly decode the residual without applying the transform or quantization process. In direct PCM decoding, no prediction is applied and the coding unit samples are decoded directly into the bitstream.

[0035] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0036] Figure 2 1 shows a block diagram of an example of a video decoder 200, such as a HEVC decoder. In the example decoder 200, a signal or bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described above. Figure 1 The encoding pass described in is reciprocal to the decoding pass, which performs video decoding as part of encoding the video data. Figure 2 Decoders in which improvements are made to the HEVC standard or decoders that employ techniques similar to HEVC may also be described, such as decoders based on or improved upon JEM.

[0037] Specifically, the input of the decoder includes a video signal or a bit stream, which can be obtained by, for example, Figure 1 The video encoder 100 of the video encoder generates. The signal or bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors and other decoding information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined with the prediction block (255) to reconstruct the image block. The prediction block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). Advanced motion vector prediction (AMVP) and merge mode techniques can be used to derive motion vectors for motion compensation, which can use interpolation filters to calculate interpolated values ​​of sub-integer samples of reference blocks. An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0038] In the HEVC video compression standard, motion compensated temporal prediction is used to exploit the redundancy that exists between consecutive pictures of a video. To this end, a motion vector is associated with each prediction unit (PU). Each coding tree unit (CTU) is represented by a coding tree (CT) in the compressed domain. This is a quadtree partitioning of the CTU, where each leaf is called a coding unit (CU), such as Figure 3 as shown in .

[0039] Each CU is then given some intra or inter prediction parameters (prediction information). To this end, it is spatially partitioned into one or more prediction units (PUs), each of which is assigned some prediction information. Figure 4 As shown, the intra-frame or inter-frame decoding mode is assigned at the CU level, which shows an example of dividing the decoding tree unit into a decoding unit, a prediction unit and a transform unit

[0040] Specifically, in HEVC, one motion vector is assigned to each PU. This motion vector is used for motion compensated temporal prediction of the considered PU. Therefore, in HEVC, the motion model of linking a predicted block with its reference block involves translation.

[0041] Two modes are used in HEVC to encode the motion data. They are called AMVP (Adaptive Motion Vector Prediction) and Merge. AMVP involves signaling the reference picture (one or more) used to predict the current PU, the motion vector predictor index (taken from a list with two predictors), and the motion vector difference. In general, at least one embodiment described herein involves a merge mode.

[0042] The merge mode consists of signaling and decoding the index of some motion data collected in a list of motion data predictors. The list consists of five candidates and is constructed in the same way on the decoder and encoder side. Therefore, the merge mode aims to derive some motion information taken from the merge list. The merge list usually contains motion information associated with some spatial and temporal surrounding blocks, which are available in the decoding state of these blocks when the current PU is being processed.

[0043] In the Joint Exploration Model (JEM) developed by the JVET (Joint Video Exploration Team) group, some additional temporal prediction tools with associated parameters determined on the decoder side include: Sub-block Temporal Motion Vector Prediction (SbTMVP), which is sometimes also called ATMVP (Alternative Temporal Motion Vector Prediction). The basic principle of SbTMVP is to derive some motion information of the current CU from some motion information in the reference pictures contained in the current picture. To this end, a so-called temporal vector is first obtained, which is the motion vector of the first merge candidate for the current CU. This temporal vector with its associated reference picture index can be used to retrieve the motion vector from the reference picture under consideration. To this end, the current CU is split into NxN sub-CUs, and the motion data pointed to by time from the center position of the sub-CU under consideration are regarded as sub-CU MV SbTMVP predictors.

[0044] The SbTMVP motion prediction mode may be included in the classic merge list as an additional candidate. Another approach may be to insert the SbTMVP motion prediction mode into an affine merge list that was originally dedicated to building a merge candidate set to predict the affine motion model of the current CU in case affine motion compensation is used for the current CU. The use of affine motion compensation for the current CU is signaled by a so-called affine flag at the CU level. It appears that moving the SbTMVP motion predictor into the affine merge list may result in increased compression efficiency compared to other approaches (e.g., when the SbTMVP candidate is part of the classic merge candidate list).

[0045] Figure 5 An example of inter-mode decoding, e.g. in VTM-5, is shown:

[0046] – First, decode a skip flag which indicates whether the CU contains any residual.

[0047] – The regular_merge flag is then decoded, which indicates to use regular merging including TMVP (Temporal Motion Vector Predictor) candidates.

[0048] – If not, the mmvd flag is decoded, which indicates that merge motion vector difference mode is used. This mode can also contain candidates constructed from TMVP candidates.

[0049] – If not MMDV, decode the sub-block flag. The sub-block candidate can be a SbTMVP of an affine candidate.

[0050] – If not a subblock, decode the CIIP flag (Combined Intra-Inter Prediction) in case of merging (not skipping). This mode may also contain candidates constructed from TMVP candidates.

[0051] – Finally, if none of the above modes are selected, a triangle mode is inferred. This mode can also contain candidates built from TMVP candidates.

[0052] In AMVP mode (Advanced Motion Vector Prediction), the block can be translational or affine. Note that the affine flag in AMVP is coded using the same CABAC context as the sub-block flag in skip or merge mode.

[0053] The merge mode in the HEVC standard involves deriving inter prediction information (hereinafter also referred to as motion information) for a given prediction unit from the selected motion information predictor candidate. The motion information includes all inter prediction parameters of the PU, that is:

[0054] – One-way or two-way time prediction type

[0055] – Reference picture index within each reference picture list

[0056] – Motion vector(s)

[0057] The decoding and decoding of inter-frame prediction information in HEVC Figure 6 , which shows the signaling of the inter prediction information. It can be seen that motion information coding / decoding according to merge mode occurs in two modes: skip mode and merge mode. In both modes, a single field is signaled to enable the decoder to retrieve the motion information of the PU: the so-called merge index. The merge index indicates which motion vector predictor in the merge motion information predictor list is used to derive the motion information of the current PU. In the following, the list of motion information predictors is called merge list or merge candidate list. Furthermore, candidate motion information predictors are called merge candidates.

[0058] In HEVC, the merge candidate list is systematically composed of five merge candidates. The merge list is constructed as follows on both the encoder and decoder sides. Figure 7 The positions of the spatial and temporal motion vector predictors used in merge mode are shown in Figure 2. The spatial merge candidates are Figure 7 The temporal merge candidates are shown on the left side of Figure 7 As shown on the right side of Figure 7 As can be seen in , up to five spatial locations can be considered to retrieve some potential candidates. They are visited according to the following order:

[0059] 1-Left side (A1)

[0060] 2-Above (B1)

[0061] 3-Top right (B0)

[0062] 4- Bottom left (A0)

[0063] 5-Top left (B2)

[0064] The symbols A0, A1, B0, B1, and B2 represent Figure 7 The spatial positions shown on the left. Four candidate spaces that are different from each other are selected. Then, by considering the temporal motion information at position H, a temporal predictor denoted as TMVP is selected, and then the "center" is the unavailable candidate at position H in the considered reference picture. Then, as Figure 8 As shown in , a list of merged motion vector predictor candidates is constructed, which shows the final pruning process that occurs to ensure that the selected set of spatial and temporal candidates does not contain redundant candidates.

[0065] Next, in the case of B slices, if the merge list is not full, another type of candidate is pushed to the merge list: the so-called combined candidate. This may involve forming a candidate consisting of motion information associated with one reference picture list (L0) from one candidate already in the list and motion associated with another reference picture list (L1) from another candidate already in the merge list. If the merge list is still not full (five elements), zero motion vectors are pushed to the back of the merge list until it is full. The entire process of merge list construction in HEVC is described in Fig. 9 Detailed description in the figure.

[0066] In the sub-block temporal motion vector prediction method (SbTMVP; also called ATMVP), one or more temporal motion vector predictors of the current CU are retrieved from the reference picture(s) of the current CU. First, the so-called temporal motion vector and the associated reference picture index are obtained as motion data associated with the first candidate in the normal merge list candidates of the current CU. Next, the current CU is partitioned into N×N sub-CUs, where N is usually equal to 4. This is as follows: Fig.10 As shown. For each N×N sub-block, the motion vector (one or more) and the reference picture flag (one or more) are identified in the reference picture associated with the temporal MV with the help of the temporal motion vector. The NxN sub-block in the reference picture pointed to by the temporal MV from the current sub-CU position is considered. Its motion data is regarded as the SbTMVP motion data prediction of the current sub-CU. It is then converted into the motion vector and reference picture index of the current sub-CU by appropriate motion vector scaling.

[0067] In general, an aspect of at least one example of an embodiment relates to organizing the number and content of the various lists of motion data predictors in a manner that improves coding efficiency. In general, another aspect of at least one embodiment relates to reducing complexity to improve the worst case on the decoder side, for example, this can be done by reducing the number of merge candidates constructed for a given mode. In general, another aspect of at least one embodiment relates to making affine mode coding more consistent by not mixing subblock flag coding with affine flag coding.

[0068] At least one example of an embodiment provides a temporal merge list containing one or more temporal predictors, such as SbTMVP, TMVP, TMVP-based MMVD, TMVP-based CIIP, TMVP-based triangle, where these candidates are removed from the traditional merge list, thereby reducing the worst-case complexity at the decoder.

[0069] At least one example of an embodiment relates to temporal patterns not included in the temporal merge list being retained in their respective lists (eg, MMVD, CIIP, Triangle).

[0070] At least one example of an embodiment involves completely removing temporal patterns not included in the temporal merge lists in order to keep each list free of temporal predictors.

[0071] At least one example of an embodiment is directed to extracting SbTMVP from a sub-block and providing for making the coding of affine flags consistent between merge / skip and AMVP modes.

[0072] At least one example of an embodiment involves separating sub-block and affine flag decoding.

[0073] Fig.11 An example of a classic (as opposed to sub-block) merge list construction is shown. Classic here means a merge list for translational motion compensated temporal prediction, where one motion vector is associated with each CU of a reference picture list. The method comprises:

[0074] – Add spatial candidates A1, B1, B0 A0 B2 with some pruning between them (for predictor positions, see Fig.13 ).

[0075] – Added Time MVP (H or Center)

[0076] – Add HMVP (history-based motion vector predictor) candidates, for the first added candidate, there is some pruning on at most A1 and B1

[0077] – If there are at least 2 candidates in the list, create pairwise (average) candidates

[0078] – Fill the list with empty motion vectors if necessary

[0079] An example of the construction of a list of sub-block based motion prediction sub-candidates is now provided. The affine merge list collects all merged MV predictors involved in sub-block based motion compensated temporal prediction. Therefore, this includes the SbTMVP merge mode and the affine merge mode mentioned above. The affine merge candidate represents a merge candidate from which a 4×4 block based affine motion field can be derived for the current CU and used in the temporal prediction of the current CU. Affine motion compensation is not described in detail here. One aspect to note here is that in at least one example, the SbTMVP candidate is placed in front of the affine merge list.

[0080] Other examples of merged lists may include the following:

[0081] – The MMVD merge list consists of the first two classic merge candidates, which are shifted by a given motion vector difference signaled in the bitstream. Thus, this list may contain candidates based on the temporal candidate (the shifted TMVP candidate).

[0082] – The CIIP merge list is the same as the classic merge list. Therefore, the list can contain candidates based on temporal candidates. The final predictor is a hybrid between motion based predictor and intra prediction.

[0083] – The triangle merge list consists of candidates created from the candidates of the classic merge list. Thus, this list may contain candidates based on temporal candidates.

[0084] Generally speaking, at least one aspect of at least one example of the embodiments described herein involves creating a merged list containing the temporal candidates.

[0085] Fig.14 An example of an embodiment of a process for decoding a modification of the temporal merging list is shown, the process comprising at least the following:

[0086] – After the mmvd mode flag is decoded, if the mmvd flag is false, decode the temporal mode flag.

[0087] – If temporal mode is true, the temporal candidate index is decoded similarly to the classic merge candidate index

[0088] – If not, decode the affine mode flag. Note that the affine flag replaces the sub-block mode flag when the SbTMVP candidate has been removed from the list. Due to this split, the affine flag encoding in merge mode and AMVP mode uses the same CABAC bin in a consistent way (in contrast to the method where the sub-block flag indicates both SbTMVP or affine mode).

[0089] Generally, at least one aspect of at least one example of an embodiment may involve providing various placements of a time merge list. For example, the time merge list may also be inserted into:

[0090] – Inserted as the first merge list

[0091] – is inserted after the regular merge list

[0092] – is inserted after the affine merge list

[0093] – is inserted after the ciip merge list

[0094] – is inserted after the triangle merge list. In this case, if triangle mode is false,

[0095] Then the inferred temporal pattern is true.

[0096] In general, a list of temporal candidates can be created in the same way as a merge list is created by adding potential candidates (when they exist) in a fixed order up to a maximum number of merge candidates (usually 2 or 3). Then, an index indicating the position in the temporal candidate list to be used is sent to the decoder. An example of a temporal candidate list is:

[0097] –SbTMVP candidate

[0098] –TMVP Candidate

[0099] – Time CIIP Candidate

[0100] Variations may include one or more of the following:

[0101] - Through additional signaling, TMVP candidates for MMVD and delta mode can also be added to the list. For this, additional information (usually mmvd flag and mmvd offset) is required.

[0102] – TMVP candidates of MMVD and / or triangle and / or CIIP are kept in their original lists.

[0103] - TMVP candidates for MMVD and / or triangles and / or CIIP are completely removed, preventing all other lists from having temporal candidates. Optionally, in this variant, the reconstructed affine candidates do not use temporal motion vectors in order to keep the affine merge lists unaffected by temporal motion vectors.

[0104] - Affine candidates for purely temporal reconstructions are also added to the temporal list.

[0105] Generally, one or more of the described embodiments may provide:

[0106] – Reduced complexity of decoder processing, where a given merge list (regular, mmvd, temporal, affine, ciip, or triangle) has reduced worst-case complexity (e.g., the length(s) of the list(s) are reduced).

[0107] – Since all spatial candidates are in a separate list, the bandwidth required to construct each candidate of the list is reduced, without having to access the temporal storage buffer. Typically, in all lists except the temporal list, only the spatial information around the current block needs to be put into the cache.

[0108] Table 1 attached to this disclosure provides an example of an embodiment of a syntax for decoding of a temporal merge list flag. The example shown in Table 1 relates to providing syntax for decoding of a temporal merge list flag after an MMVD flag. Note that the sub-block syntax is replaced by an affine syntax, as in the AMVP derivation (grey shading).

[0109] In the case of only SbTMVP and TMVP temporal merge lists, when both candidates are available, the variable MaxNumTemporalMergeCand is at most equal to two. Candidate availability is derived as described above, where

[0110] - If the SbTMVP SPS level flag is true and the current block size is greater than or equal to 8 for width and height, then SbTMVP is available – If the SPS level time flag is true and width+height is greater than 12, then TMVP is available.

[0111] Another aspect of at least one example of an embodiment may involve affine and sub-block flag decoding. In some systems, the decoding of merge_subblock_flag (in merge / skip mode) and inter_affine_flag (in AMVP) may involve sharing the same CABAC decoding. In the case of a separate temporal list such as in at least one embodiment described herein (i.e., SbTMVP is no longer in the sub-block list), merge_subblock_flag actually becomes merge_affine_flag, making the CABAC decoding more consistent. Another variant that does not use a separate temporal merge list is to use two separate CABAC decodings: one for merge_subblock_flag and one for inter_affine_flag.

[0112] generally, Fig.16 Another example of an embodiment is shown in FIG. Fig.16 In the embodiment of the present invention, a bitstream including the encoded picture information also includes encoded control information, such as a flag. At 1610, a first flag included in the decoded bitstream is decoded using CABAC decoding based on a first probability model. At 1620, a second flag included in the decoded bitstream is decoded using CABAC decoding based on a second probability model. That is, two separate CABAC decodings are used to decode the first and second flags. Then, at 1630, the encoded picture information included in the bitstream is decoded based on a decoding mode indicated by the first flag or the second flag to generate decoded picture information. For example, the first flag may indicate, correspond to, or be associated with a decoding mode such as a sub-block merge prediction mode, and the second flag may indicate, correspond to, or be associated with a decoding mode such as an inter-frame affine prediction mode.

[0113] generally, Fig.17 Another example of an embodiment is shown in FIG. Fig.17 In , at 1710, a value of a first flag is determined, wherein the first flag indicates a first prediction mode associated with encoding picture information included in the input. At 1720, a value of a second flag is determined, wherein the second flag indicates a second prediction mode associated with encoding the picture information. At 1730, at least a portion of the picture information and the first and second flags are encoded to form an encoded bitstream, wherein the first flag indicates, for example, a sub-block merge prediction mode and is decoded using CABAC decoding based on a first probability model, and the second flag indicates, for example, an inter-frame affine prediction mode and is decoded using CABAC decoding based on a second probability model. That is, two separate CABAC decodings are used to encode the first and second flags.

[0114] This document describes various examples of embodiments, features, models, methods, etc. Many such examples are described as specific, and at least to illustrate individual characteristics, are usually described in a manner that may appear to be restrictive. However, this is for the purpose of describing clearly and does not limit the application or scope. In fact, various examples of the embodiments, features, etc. described herein can be combined and interchanged in various ways to provide additional examples of embodiments. Examples according to embodiments of the present disclosure include, but are not limited to, the following.

[0115] Generally, an example of an embodiment may relate to a method comprising: decoding a first flag included in a decoded bitstream, the first flag having been decoded using CABAC decoding based on a first probability model; decoding a second flag included in the decoded bitstream, the second flag having been decoded using CABAC decoding based on a second probability model; and decoding picture information included in the decoded bitstream based on a decoding mode indicated by the first flag or the second flag; wherein the first flag indicates a sub-block merge prediction mode and the second flag indicates an inter-frame affine prediction mode.

[0116] Generally, another example of an embodiment may relate to a method comprising: determining a value of a first flag indicating a first prediction mode associated with encoded picture information; determining a value of a second flag indicating a second prediction mode associated with the encoded picture information; and encoding at least a portion of the picture information and the first and second flags to form an encoded bitstream, wherein the first flag indicates a sub-block merge prediction mode and is decoded using CABAC decoding based on a first probability model, and the second flag indicates an inter-frame affine prediction mode and is decoded using CABAC decoding based on a second probability model.

[0117] Generally, another example of an embodiment may relate to a device comprising one or more processors configured to: decode a first flag included in a decoded bitstream, the first flag having been decoded using CABAC decoding based on a first probability model; decode a second flag included in the decoded bitstream, the second flag having been decoded using CABAC decoding based on a second probability model; and decode encoded picture information included in the decoded bitstream based on a decoding mode indicated by the first flag or the second flag; wherein the first flag indicates a sub-block merge prediction mode and the second flag indicates an inter-frame affine prediction mode.

[0118] Generally, another example of an embodiment may relate to a device comprising one or more processors configured to: determine a value of a first flag indicating a first prediction mode associated with encoded picture information; determine a value of a second flag indicating a second prediction mode associated with the encoded picture information; and encode at least a portion of the picture information and the first and second flags to form an encoded bitstream, wherein the first flag indicates a sub-block merge prediction mode and is decoded using CABAC decoding based on a first probability model, and the second flag indicates an inter-frame affine prediction mode and is decoded using CABAC decoding based on a second probability model.

[0119] In general, another example of an embodiment may involve a bitstream formatted to include encoded picture information, wherein the encoded picture information is encoded by processing the picture information based on any one or more of the examples of embodiments of the method according to the invention.

[0120] Generally, one or more other examples of embodiments may also provide a computer-readable storage medium, such as a non-volatile computer-readable storage medium, on which instructions are stored for encoding or decoding picture information such as video data according to the methods or apparatus described herein.

[0121] Generally, at least one example of the embodiments may involve a computer program product comprising instructions which, when executed by a computer, cause the computer to perform a method according to one or more examples of the embodiments described herein.

[0122] Generally, at least one example of the embodiments may involve a non-transitory computer-readable medium storing executable program instructions that cause a computer executing the instructions to perform a method according to one or more examples of the embodiments described herein.

[0123] In general, at least one example of the embodiments may involve a signal including data generated according to any one or more examples of the embodiments described herein.

[0124] In general, at least one example of an embodiment may involve a bitstream formatted to include syntax elements and coded image information generated according to any one or more of the examples of the embodiments described herein.

[0125] In general, at least one example of the embodiments may involve a computer-readable storage medium having stored thereon a bitstream generated according to a method or apparatus described herein.

[0126] In general, at least one example of an embodiment may involve sending or receiving a bitstream or signal generated according to a method or apparatus described herein.

[0127] Generally, at least one example of an embodiment may involve a device comprising an apparatus according to any one or more of the examples of the embodiments described herein; and at least one of: (i) an antenna configured to receive a signal, the signal comprising data representing the image information, (ii) a band limiter configured to limit the received signal to a frequency band comprising the data representing the image information, and (iii) a display configured to display an image from the image information.

[0128] In general, at least one example of an embodiment may be directed to a device as described herein, wherein the device includes one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cellular phone, a tablet computer, or other electronic devices.

[0129] Generally, another example of an embodiment may involve a device comprising one or more processors configured to determine a value of a first flag indicating a first prediction mode associated with encoded picture information; determine a value of a second flag indicating a second prediction mode associated with the encoded picture information; and encode at least a portion of the picture information and the first and second flags to form an encoded bitstream, wherein the first flag indicates a sub-block merge prediction mode and is decoded using CABAC decoding based on a first probability model, and the second flag indicates an inter-frame affine prediction mode and is decoded using CABAC decoding based on a second probability model.

[0130] In general, the examples of the embodiments described and contemplated in this document may be implemented in many different forms. Figure 1 and 2 And the following Fig.15 Some embodiments are provided, but other embodiments are contemplated and are not intended to be construed in detail. Figure 1 , 2The discussion of 15 does not limit the breadth of implementations. At least one embodiment provides examples primarily related to video encoding and / or decoding, and at least one other embodiment is primarily directed to transmitting a generated or encoded bitstream or signal. These and other embodiments may be implemented as methods, apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream or signal generated according to any of the described methods.

[0131] The terms HDR (high dynamic range) and SDR (standard dynamic range) are used in this disclosure. Those terms generally convey a specific value of dynamic range to one of ordinary skill in the art. However, in additional embodiments, it is also intended that references to HDR be understood to mean "higher dynamic range" and references to SDR be understood to mean "lower dynamic range". Such additional embodiments are not constrained by any specific value of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range".

[0132] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0133] Various methods and other aspects described in this document may be used to modify modules, such as motion estimation and / or compensation modules, such as Figure 1 Modules 170 and / or 175 and / or Figure 2 275 of the video decoder 200 shown in FIG. In addition, aspects of the present invention are not limited to VVC or HEVC, and may be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described herein may be used alone or in combination.

[0134] For example, various numerical values ​​are used in this document. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0135] Fig.15A block diagram of an example of a system in which various aspects and embodiments can be implemented is shown. System 1000 can be implemented as a device including various components described below, and is configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1000 can be implemented in a single integrated circuit, multiple ICs and / or discrete components individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed on multiple ICs and / or discrete components. In various embodiments, the system 1000 is communicatively coupled to other similar systems or coupled to other electronic devices via, for example, a communication bus or by dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more aspects described herein.

[0136] The system 1000 includes at least one processor 1010, which is configured to execute instructions loaded therein, for implementing various aspects described herein, for example. The processor 1010 may include embedded memory, input-output interface, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., volatile memory device and / or non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical disk drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device, and / or a network-accessible storage device.

[0137] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding module and the decoding module. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated into the processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0138] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of the various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of the decoded video, bitstreams or signals, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0139] In some embodiments, memory within the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be a memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as working memory for MPEG-2, HEVC, or VVC (universal video coding).

[0140] As shown in block 1130, input to the elements of system 1000 may be provided through various input devices. Such input devices include, but are not limited to: (i) receiving an RF portion of an (RF) signal transmitted, for example, over the air by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0141] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band limiting the frequency band to a narrower frequency band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its relevant input processing element receive the RF signal that sends by wired (for example, cable) medium, and carry out the frequency selection to the desired frequency band by filtering, down-conversion and filtering again.Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example insert amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0142] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Solomon error correction) may be implemented, for example, within a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 1010. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams for presentation on output devices.

[0143] The various components of the system 1000 may be disposed in an integrated housing. In the integrated housing, the various components may be interconnected and transmit data therebetween using a suitable connection arrangement 1140 (eg, an internal bus known in the art, including an I2C bus, wiring, and a printed circuit board).

[0144] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0145] In various embodiments, data is streamed to the system 1000 using a Wi-Fi network (e.g., IEEE 802.11). The Wi-Fi signals of these embodiments are received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communications. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to an external network including the Internet to allow streaming applications and other cloud communications. Other embodiments use a set-top box that transfers data via an HDMI connection of an input box 1130 to provide streaming data to the system 1000. Still other embodiments use an RF connection of an input box 1130 to provide streaming data to the system 1000.

[0146] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripherals 1120. In various examples of various embodiments, the other peripherals 1120 include one or more of the following: a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 1000. In various embodiments, control signals are transmitted between the system 1000 and the display 1100, the speaker 1110, or other peripherals 1120 using signaling such as AV.Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 1000 via dedicated connections through the respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and the speaker 1110 may be integrated into a single unit in an electronic device (eg, a television) along with other components of the system 1000. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0147] For example, if the RF portion of the input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more of the other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0148] These embodiments may be implemented by the processor 1010 or by computer software implemented by hardware or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, as a non-limiting example, one or more of the following: a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0149] In the present disclosure, various implementations involve decoding. As used in the present application, "decoding" may include, for example, all or part of the processes performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by decoders of various implementations described in the present application, such as extracting pictures from tiled (or packed) pictures, determining the upsampling filter to be used and then upsampling the pictures, and flipping the pictures back to their intended orientation.

[0150] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0151] In addition, various implementations involve encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream or signal. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also or alternatively include processes performed by encoders of various implementations described in this application.

[0152] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or generally to a broader encoding process will become clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0153] Note that the syntax elements as used herein are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0154] When the figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when the figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0155] Various embodiments relate to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, usually given a constraint on computational complexity. The rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options, including all considered modes or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. It is also possible to use faster methods to save coding complexity, in particular to calculate approximate distortion based on prediction or prediction residual signals rather than reconstructed signals. A mixture of these two methods can also be used, for example by using approximate distortion only for some possible coding options and full distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete evaluation of both decoding costs and associated distortions.

[0156] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). For example, the device can be implemented with appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0157] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations in various places in this document do not necessarily all refer to the same embodiment.

[0158] Additionally, this document may refer to “obtaining” various information. Obtaining the information may include, for example, one or more of: determining the information, estimating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0159] In addition, this document may refer to "accessing" various information. Accessing the information may include, for example, one or more of: receiving the information, retrieving the information (e.g., retrieving the information from a memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0160] Additionally, this document may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving the information may include, for example, one or more of: accessing the information or retrieving the information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is often referred to in one way or another.

[0161] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to multiple items listed, as will be apparent to one of ordinary skill in this and related arts.

[0162] In addition, as used herein, the word "signal" especially refers to a corresponding decoder indicating something. For example, in some embodiments, the encoder sends a specific one of the multiple parameters for refinement with a signal. Thus, in one embodiment, the same transformation, parameter or mode is used on the encoder side and the decoder side. Therefore, for example, the encoder can send (explicitly send with a signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. On the contrary, if the decoder already has the specific parameter and other parameters, signaling can be used without sending (implicitly sending with a signal) to simply allow the decoder to know and select the specific parameter. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to send information to the corresponding decoder with a signal. Although the foregoing relates to the verb form of the word "signal", the word "signal" can also be used as a noun in this article.

[0163] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream or signal of the described embodiment. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0164] Various embodiments have been described. Embodiments may include any of the following features or entities, alone or in any combination, across various claim categories and types:

[0165] In an encoder and / or a decoder for processing a video, providing at least one merge mode based on a merge list containing the temporal candidates.

[0166] In an encoder and / or decoder for processing video, at least one merging mode is provided based on the number and content of the various lists organizing motion data predictors in a manner that improves decoding efficiency.

[0167] • In an encoder and / or decoder for processing video, provisioning is made based on complexity reduction to improve the worst case at the decoder side, which may be done, for example, by reducing the number of merging candidates constructed for a given mode.

[0168] In an encoder and / or decoder for processing a video, based on not mixing sub-block flag decoding and affine flag decoding, a coding consistency with respect to an affine mode is provided.

[0169] In an encoder and / or decoder for processing a video, provisioning is performed based on a temporal merge list comprising one or more temporal predictors, wherein the temporal predictors include one or more of the following: SbTMVP, or TMVP, or MMVD based on TMVP, or CIIP based on TMVP, or triangle based on TMVP, wherein these candidates are removed from the conventional merge list, thereby reducing the worst case complexity at the decoder.

[0170] In an encoder and / or a decoder for processing a video, provisioning is performed based on at least one temporal mode that is not contained in a temporal merging list and is maintained in a corresponding list (eg, MMVD, CIIP, triangle).

[0171] In an encoder and / or decoder for processing a video, provisioning is performed based on at least one temporal mode not included in the temporal merge list and completely removed so as to keep each list without a temporal predictor. In an encoder and / or decoder for processing a video, provisioning is performed based on extracting the SbTMVP from the sub-block, and provisioning is performed such that the decoding of the affine flag is performed in the merge / skip mode.

[0172] mode and AMVP mode.

[0173] In an encoder and / or decoder for processing a video, provision is made based on separate sub-block decoding and affine flag decoding.

[0174] In an encoder and / or decoder for processing video, provision is made based on a modified process for decoding a temporal merging list, the process comprising:

[0175] o After the mmvd mode flag is decoded, if the mmvd flag is false, then the temporal mode flag is decoded,

[0176] o If temporal mode is true, decode the temporal candidate index,

[0177] o If temporal mode is not true, decoding the affine mode flag, wherein the affine flag replaces the sub-block mode flag based on having removed the SbTMVP candidate from the merge list, and wherein the affine flag decoding in merge mode and in AMVP mode uses the same CABAC bins in a consistent manner.

[0178] In an encoder and / or decoder for processing a video, provision is made based on various placements of providing a temporal merge list, wherein the temporal merge list may also be inserted into:

[0179] o is inserted as the first merged list, or

[0180] o is inserted after the regular merge list, or

[0181] o is inserted after the affine merge list, or

[0182] o is inserted after the CIIP Consolidated List, or

[0183] o is inserted after the triangle merge list, where if triangle mode is false, the temporal mode is inferred to be true.

[0184] # In an encoder and / or decoder for processing video, provisioning is performed based on a list of temporal candidates created by adding potential candidates in a fixed order until a maximum number of merge candidates is reached, wherein an index indicating the position in the list of the temporal candidate to be used is then transmitted

[0185] to a decoder, and wherein the potential candidates include one or more of:

[0186] o SbTMVP candidate, or

[0187] o TMVP candidate, or

[0188] o Time CIIP candidate.

[0189] In an encoder and / or decoder for processing a video, provisioning is performed based on a merge list created comprising at least one of:

[0190] o signalling indicating that a TMVP candidate or triangle mode candidate for MMVD may be added to the merge list, or

[0191] o TMVP candidates for MMVD and / or Triangle and / or CIIP are kept in their respective original lists, or

[0192] o TMVP candidates for MMVD and / or Triangle and / or CIIP are completely removed, thus preventing all other lists from having temporal candidates, or

[0193] o TMVP candidates for MMVD and / or triangles and / or CIIP are completely removed, thereby preventing all other lists from having temporal candidates, where the reconstructed affine candidates do not use temporal motion vectors in order to keep the affine merge lists unaffected by temporal motion vectors, or

[0194] o Affine candidates of purely temporal reconstruction are also added to the temporal list. o In an encoder and / or decoder for processing a video, provision is made based on reducing the complexity of the decoder process, where a given merge list has a reduced worst case complexity.

[0195] In an encoder and / or decoder for processing video, provision is made based on reducing the complexity of a decoder process, wherein a given merge list has a reduced worst case complexity, including that the length of one or more merge lists is reduced.

[0196] # In an encoder and / or decoder for processing video, provisioning is performed based on reducing the complexity of a decoder process, wherein a given merge list has a reduced worst-case complexity, including reducing the length of one or more merge lists, wherein the given merge list

[0197] The list includes at least one of: normal, mmvd, temporal, affine, ciip, or triangle. In an encoder and / or decoder for processing a video, based on each of the

[0198] The bandwidth associated with each candidate is reduced for provisioning, where all spatial candidates are in a single list and no access to the temporal memory buffer is required.

[0199] • Inserting in the signaling a syntax element enabling the encoder to provide at least one merge mode based on a merge candidate list as described herein.

[0200] # Based on these syntax elements, a merge mode is selected to apply at the decoder.

[0201] • A bitstream or signal comprising one or more of the described syntax elements or variants thereof.

[0202] • Inserting syntax elements in the signaling that enable the decoder to provide the merge mode in a way that corresponds to the way used by the encoder.

[0203] • Creating and / or sending and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof.

[0204] A TV, set-top box, cell phone, tablet or other electronic device that performs video encoding and / or decoding according to any of the described embodiments and displays (e.g., using a monitor, screen or other type of display) the resulting images.

[0205] A TV, set-top box, cellular phone, tablet or other electronic device that tunes (e.g., using a tuner) a channel to receive a signal including an encoded image, and performs video encoding and / or decoding according to any of the described embodiments.

[0206] A TV, set-top box, cellular phone, tablet or other electronic device that receives over the air (eg, using an antenna) a signal comprising an encoded image and performs video encoding and / or decoding according to any of the described embodiments.

[0207] #A computer program product storing a program code which, when executed by a computer, implements video encoding and / or decoding according to any of the described embodiments.

[0208] • A non-transitory computer-readable medium comprising executable program instructions causing a computer to execute the instructions to implement video encoding and / or decoding according to any of the described embodiments.

[0209] Various other generalized and specialized embodiments are also supported and contemplated in the present disclosure.

[0210] Table 1 Examples of embodiments of grammar according to the present disclosure

[0211]

Claims

1. A method comprising: decoding a first flag included in a decoded bitstream, the first flag having been decoded using a first CABAC decoding; decoding a second flag included in the decoded bitstream, the second flag having been decoded using a second CABAC decoding separate from the first CABAC decoding; as well as decoding, based on the decoding mode indicated by the first flag or the second flag, coded picture information included in the decoded bitstream; in The first flag indicates a temporal mode, the temporal mode comprising at least one sub-block temporal motion vector prediction mode, and The second flag indicates an affine prediction mode, wherein the affine prediction mode indicates an inter-frame affine prediction mode in an AMVP mode or an affine merge prediction mode in a merge mode.

2. The method according to claim 1, wherein: - the affine prediction mode indicates the use of affine motion compensation, - the sub-block temporal motion vector prediction mode derives motion information of sub-blocks of the block from motion information in a reference picture of the picture to which the block belongs.

3. A method comprising: determining a value of a first flag indicating a first prediction mode associated with the encoded picture information; determining a value of a second flag indicating a second prediction mode associated with the encoded picture information; as well as At least a portion of the picture information and the first flag and the second flag are encoded to form an encoded bit stream, wherein The first flag indicates a temporal mode, the temporal mode comprising at least one sub-block temporal motion vector prediction mode, and the first flag is encoded using a first CABAC decoding, and The second flag indicates an affine prediction mode, wherein the affine prediction mode indicates an inter-frame affine prediction mode in an AMVP mode or an affine merge prediction mode in a merge mode, and the second flag is encoded using a second CABAC decoding separate from the first CABAC decoding.

4. The method according to claim 3, wherein: - the affine prediction mode indicates the use of affine motion compensation, - the sub-block temporal motion vector prediction mode derives motion information of sub-blocks of the block from motion information in a reference picture of the picture to which the block belongs.

5. A device comprising: One or more processors, configured to decoding a first flag included in a decoded bitstream, the first flag having been decoded using a first CABAC decoding; decoding a second flag included in the decoded bitstream, the second flag having been decoded using a second CABAC decoding separate from the first CABAC decoding; as well as decoding coded picture information included in the decoded bitstream based on the decoding mode indicated by the first flag or the second flag; in The first flag indicates a temporal mode, the temporal mode comprising at least one sub-block temporal motion vector prediction mode, and The second flag indicates an affine prediction mode, wherein the affine prediction mode indicates an inter-frame affine prediction mode in an AMVP mode or an affine merge prediction mode in a merge mode.

6. The device according to claim 5, wherein: - the affine prediction mode indicates the use of affine motion compensation, - the sub-block temporal motion vector prediction mode derives motion information of sub-blocks of the block from motion information in a reference picture of the picture to which the block belongs.

7. A device comprising: One or more processors, configured to determining a value of a first flag indicating a first prediction mode associated with the encoded picture information; determining a value of a second flag indicating a second prediction mode associated with the encoded picture information; as well as At least a portion of the picture information and the first flag and the second flag are encoded to form an encoded bit stream, wherein The first flag indicates a temporal mode, the temporal mode comprising at least one sub-block temporal motion vector prediction mode, and the first flag is encoded using a first CABAC decoding, and The second flag indicates an affine prediction mode, wherein the affine prediction mode indicates an inter-frame affine prediction mode in an AMVP mode or an affine merge prediction mode in a merge mode, and the second flag is encoded using a second CABAC decoding separate from the first CABAC decoding.

8. The device according to claim 7, wherein: - the affine prediction mode indicates the use of affine motion compensation, - the sub-block temporal motion vector prediction mode derives motion information of sub-blocks of the block from motion information in a reference picture of the picture to which the block belongs.

9. A non-transitory computer-readable medium storing executable program instructions, the executable program instructions causing a computer executing the instructions to perform the method according to claim 1 or 3.

10. The non-transitory computer readable medium of claim 9, wherein: - the affine prediction mode indicates the use of affine motion compensation, - the sub-block temporal motion vector prediction mode derives motion information of sub-blocks of the block from motion information in a reference picture of the picture to which the block belongs.

Images