Method and apparatus for video coding using triangular partitioning

Abstract

Methods and apparatus are provided for video coding using triangular partitions. The method comprises the following steps: segmenting a video picture into a plurality of CUs, wherein at least one of the plurality of CUs is further segmented into two PUs comprising at least one geometrical PU; constructing a first merge list comprising a plurality of candidates, each candidate being an MV comprising a List 0MV and / or a List 1MV; locating a first candidate of the first PU and a second candidate of the second PU according to the first index and the second index respectively; obtaining a first unidirectional prediction MV for the first PU by selecting a list X1MV of first candidates according to a first binary reference list indication flag; and obtaining a second one-way prediction MV for a second PU by selecting a list X2MV of a second candidate according to a second binary reference list indication flag.

Description

[0001] This application is a divisional application of a patent application with an application date of December 31, 2020, application number 202080089982.X, and invention name “Method and device for video encoding and decoding using triangular partitioning”. Technical Field

[0002] The present application relates generally to video coding and compression, and in particular, but not limited to, methods and apparatus for motion compensated prediction using triangle prediction units (ie, a special case of geometry partitioned prediction units) in video coding. Background Art

[0003] Various electronic devices (such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices send, receive, encode, decode and / or store digital video data by implementing video compression / decompression. Digital video devices implement video codec technologies, such as those described in the standards defined by Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-10, Advanced Video Coding (AVC), ITU-TH.265 / High Efficiency Video Coding (HEVC), and extensions of such standards.

[0004] Video codecs typically use prediction methods (e.g., inter-frame prediction, intra-frame prediction) that exploit redundancy present in video images or sequences. An important goal of video codec technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality. As evolving video services become available, codecs with better coding efficiency are needed. The block segmentation schemes in each standard are also evolving.

[0005] Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove the redundancy inherent in video data. For block-based video codecs, a video frame is divided into one or more slices, each slice having multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or may be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also referred to as a leaf CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU may be encoded in intra-frame mode, inter-frame mode, or IBC mode. Video blocks in intra-frame coded (I) slices of a video frame are encoded using spatial predictions of reference samples in neighboring blocks within the same video frame. Video blocks in inter-frame coded (P or B) slices of a video frame may use spatial predictions of reference samples in neighboring blocks within the same video frame or temporal predictions of reference samples in other previous reference video frames and / or future reference video frames.

[0006] A prediction block for the current video block to be encoded is derived based on spatial or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished by a block matching algorithm. The residual data representing the pixel differences between the current block to be encoded and the prediction block is called a residual block or prediction error. Inter-coded blocks are encoded based on the residual block and a motion vector pointing to a reference block in a reference frame that forms the prediction block. The process of determining the motion vector is generally referred to as motion estimation. Intra-coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain (e.g., the frequency domain) to derive residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which are then entropy encoded into a video bitstream to achieve even greater compression.

[0007] The coded video bitstream is then stored in a computer-readable storage medium (e.g., a flash memory) to be accessed by another electronic device with digital video capabilities or sent directly to the electronic device by wire or wirelessly. The electronic device then performs video decompression (which is the reverse process of the above-mentioned video compression), for example, by parsing the coded video bitstream to obtain syntax elements from the bitstream, and reconstructing digital video data from the coded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and the electronic device presents the reconstructed digital video data on a display of the electronic device.

[0008] As digital video quality changes from HD to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. How to encode / decode video data more efficiently while maintaining the image quality of the decoded video data is a long-standing challenge.

[0009] In the Joint Video Experts Group (JVET) meeting, JVET defined the first proposal for Versatile Video Coding (VVC) and the VVC Test Model 1 (VTM1) coding method. It was decided to include a quadtree with nested multi-type trees using binary and ternary partitioning coding block structures as the initial new coding features of VVC. Since then, the reference software VTM for implementing the coding method and the proposed VVC decoding process have been developed during the JVET meeting. Summary of the invention

[0010] In general, this disclosure describes examples of techniques related to motion compensated prediction using geometric prediction units in video coding.

[0011] According to a first aspect of the present disclosure, a method for video encoding and decoding using geometric partitioning is provided, comprising: partitioning a video picture into a plurality of coding units CU, wherein at least one CU among the plurality of CUs is further partitioned into two prediction units PU, and the two PUs include at least one geometric shape PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each candidate among the plurality of candidates is a motion vector MV, and the MV includes list 0MV or list 1MV or both list 0MV and list 1MV; obtaining a unidirectional prediction MV for the PU by selecting list 0MV or list 1MV of the plurality of candidates; constructing a unidirectional prediction merge list based on the unidirectional prediction MV; and performing a pruning operation on the unidirectional prediction MV when constructing the unidirectional prediction merge list.

[0012] According to a second aspect of the present disclosure, a device for video encoding and decoding using geometric partitioning is provided, comprising: one or more processors; and a memory configured to store instructions executable by the one or more processors; wherein the one or more processors are configured to: partition a video picture into multiple coding units CU, wherein at least one CU of the multiple CUs is further partitioned into two prediction units PU, and the two PUs include at least one geometric shape PU; construct a first merge list including multiple candidates based on a merge list construction process for conventional merge prediction, wherein each of the multiple candidates is a motion vector MV, and the MV includes list 0MV or list 1MV or both list 0MV and list 1MV; obtain a unidirectional prediction MV for the PU by selecting list 0MV or list 1MV of the multiple candidates; construct a unidirectional prediction merge list based on the unidirectional prediction MV; and perform a pruning operation on the unidirectional prediction MV when constructing the unidirectional prediction merge list.

[0013] According to a third aspect of the present disclosure, a non-volatile computer-readable storage medium for video encoding and decoding using geometric partitioning is provided, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform actions, including: partitioning a video picture into multiple coding units CU, wherein at least one CU of the multiple CUs is further partitioned into two prediction units PU, and the two PUs include at least one geometric shape PU; constructing a first merge list including multiple candidates based on a merge list construction process for conventional merge prediction, wherein each of the multiple candidates is a motion vector MV, and the MV includes list 0MV or list 1MV or both list 0MV and list 1MV; obtaining a unidirectional prediction MV for the PU by selecting list 0MV or list 1MV of the multiple candidates; constructing a unidirectional prediction merge list based on the unidirectional prediction MV; and performing a pruning operation on the unidirectional prediction MV when constructing the unidirectional prediction merge list. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A more particular description of examples of the present disclosure will be presented by reference to specific examples shown in the accompanying drawings. Given that these drawings depict only some examples and are not therefore considered limiting in scope, the examples will be described and explained with additional specificity and detail through use of the accompanying drawings.

[0015] Figure 1 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0016] Figure 2is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0017] Figure 3 is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to some embodiments of the present disclosure.

[0018] Figure 4 is a diagram showing an example of a picture divided into CTUs according to some embodiments of the present disclosure.

[0019] Figure 5 is a schematic diagram illustrating a multi-type tree partitioning mode according to some embodiments of the present disclosure.

[0020] Figure 6 is a schematic diagram showing how a CU is divided into triangular prediction units according to some embodiments of the present disclosure.

[0021] Figure 7 is a schematic diagram illustrating the locations of neighboring blocks according to some embodiments of the present disclosure.

[0022] Figure 8 is a schematic diagram illustrating locations of spatial merging candidates according to some embodiments of the present disclosure.

[0023] Fig. 9 is a schematic diagram illustrating motion vector scaling of temporal merging candidates according to some embodiments of the present disclosure.

[0024] Fig.10 is a schematic diagram illustrating candidate positions of temporal merging candidates according to some embodiments of the present disclosure.

[0025] FIG. 11A to FIG. 11B is a diagram illustrating an example of unidirectional prediction motion vector (MV) selection for triangular prediction mode according to some embodiments of the present disclosure.

[0026] FIG. 12A to FIG. 12D is a schematic diagram illustrating an example of unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure.

[0027] Fig.13 is a schematic diagram illustrating an example of flexible unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure.

[0028] Fig.14 is a schematic diagram showing an example of dividing a CU into geometric partitions according to some embodiments of the present disclosure.

[0029] Fig.15 is a schematic diagram illustrating an example of a geometric partitioning representation according to some embodiments of the present disclosure.

[0030] FIG. 16A to FIG. 16B is a schematic diagram illustrating an example of unidirectional prediction MV selection for triangular prediction / geometric prediction mode according to some embodiments of the present disclosure.

[0031] Fig.17 is a diagram illustrating an example of unidirectional prediction MV selection for triangle prediction / geometry prediction with MV pruning operations performed according to some embodiments of the present disclosure.

[0032] Fig.18 is a diagram illustrating an example of unidirectional prediction MV selection for triangle prediction / geometry prediction in which an MV pruning operation is performed once when selecting each MV candidate according to some embodiments of the present disclosure.

[0033] Fig.19 is a block diagram illustrating an exemplary apparatus for video encoding and decoding according to some embodiments of the present disclosure.

[0034] Fig. 20 is a flow chart illustrating an exemplary process for video encoding and decoding using motion compensated prediction of geometric prediction units according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0035] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, many non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0036] References throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language indicate that the particular features, structures, or characteristics being described are included in at least one embodiment or example. Unless explicitly stated otherwise, features, structures, elements, or characteristics described in conjunction with one or some embodiments may also be applied to other embodiments.

[0037] Throughout the disclosure, unless otherwise explicitly stated, the terms "first", "second", "third", etc. are used as names used only to refer to related elements (e.g., devices, components, compositions, steps, etc.) without indicating any spatial or temporal order. For example, "first device" and "second device" may refer to two separately formed devices, or two parts, components, or operating states of the same device, and may be named arbitrarily.

[0038] As used herein, the terms “if” or “when” may be understood to mean “when” or “in response to” depending on the context. These terms, if appearing in a claim, may not indicate that the associated limitation or feature is conditional or optional.

[0039] The terms "module", "sub-module", "circuit", "sub-circuit", "circuitry", "sub-circuitry", "unit", or "sub-unit" may include memory (shared, dedicated, or combined) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components connected directly or indirectly. These components may or may not be physically attached to each other or located adjacent to each other.

[0040] A unit or module may be implemented purely by software, purely by hardware, or by a combination of hardware and software. In a pure software implementation, for example, a unit or module may include functionally related code blocks or software components that are linked together directly or indirectly to perform a specific function.

[0041] Figure 1 A block diagram showing an exemplary block-based hybrid video encoder 100 that can be used in conjunction with many video codec standards that use block-based processing is shown. In the encoder 100, a video frame is segmented into multiple video blocks for processing. For each given video block, a prediction is formed based on an inter-frame prediction method or an intra-frame prediction method. In inter-frame prediction, one or more prediction values ​​are formed based on pixels from a previously reconstructed frame through motion estimation and motion compensation. In intra-frame prediction, prediction values ​​are formed based on reconstructed pixels in the current frame. Through mode decision, the best prediction value can be selected to predict the current block.

[0042] The prediction residual representing the difference between the current video block and its predicted value is sent to the transform circuit 102. The transform coefficients are then sent from the transform circuit 102 to the quantization circuit 104 for entropy reduction. The quantized coefficients are then fed to the entropy coding circuit 106 to generate a compressed video bitstream. Figure 1 As shown, prediction related information 110 (such as video block segmentation information, motion vector, reference picture index and intra prediction mode) from the inter-frame prediction circuit and / or the intra-frame prediction circuit 112 is also fed through the entropy encoding circuit 106 and saved into the compressed video bitstream 114.

[0043] In the encoder 100, decoder-related circuits are also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by inverse quantization 116 and inverse transform circuit 118. The reconstructed prediction residual is combined with the block prediction value 120 to generate the unfiltered reconstructed pixel points of the current video block.

[0044] Spatial prediction (or "intra-prediction") uses pixels from samples of already encoded neighboring blocks in the same video frame as the current video block (called reference samples) to predict the current video block.

[0045] Temporal prediction (also known as "inter prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in video signals. The temporal prediction signal for a given coding unit (CU) or coding block is typically signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. In addition, if multiple reference pictures are supported, a reference picture index is additionally sent, where the reference picture index is used to identify which reference picture in the reference picture memory the temporal prediction signal comes from.

[0046] After performing spatial and / or temporal prediction, the intra / inter mode decision circuit 121 in the encoder 100 selects the best prediction mode, for example based on a rate-distortion optimization method. The block prediction value 120 is then subtracted from the current video block; and the resulting prediction residual is decorrelated using the transform circuit 102 and the quantization circuit 104. The resulting quantized residual coefficients are dequantized by the dequantization circuit 116 and inversely transformed by the inverse transform circuit 118 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Before the reconstructed CU is placed in the reference picture memory of the picture buffer 117 and used to encode and decode the subsequent video blocks, loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO) and / or adaptive loop filter (ALF), may be further applied to the reconstructed CU. In order to form an output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit 106 for further compression and packaging to form a bitstream.

[0047] For example, deblocking filters are available in current versions of AVC, HEVC, and VVC. In HEVC, an additional loop filter called sample adaptive offset (SAO) is defined to further improve coding efficiency. In the current version of the VVC standard, an additional loop filter called adaptive loop filter (ALF) is being actively studied and is likely to be included in the final standard.

[0048] These loop filter operations are optional. Performing these operations helps improve coding efficiency and visual quality. They can also be turned off according to the decision presented by the encoder 100 to save computational complexity.

[0049] It should be noted that intra prediction is usually based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels (if the encoder 100 turns on these filter options).

[0050] Figure 2 is a block diagram illustrating an exemplary block-based video decoder 200 that can be used in conjunction with many video codec standards. The decoder 200 is similar to the one residing in Figure 1 The reconstruction-related part in the encoder 100 of FIG. 200. In the decoder 200, the input video bitstream 201 is first decoded by entropy decoding 202 to obtain quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed by inverse quantization 204 and inverse transform 206 to obtain reconstructed prediction residuals. The block prediction value mechanism implemented in the intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixel points is obtained by summing the reconstructed prediction residuals from the inverse transform 206 and the prediction output generated by the block prediction value mechanism using a summer 214.

[0051] The reconstructed block may be further passed through a loop filter 209 before being stored in a picture buffer 213 used as a reference picture memory. The reconstructed video in the picture buffer 213 may be sent to drive a display device and used to predict subsequent video blocks. When the loop filter 209 is turned on, filtering operations are performed on these reconstructed pixels to obtain the final reconstructed video output 222.

[0052] The video encoding / decoding standards mentioned above (such as VVC, JEM, HEVC, MPEG-4, Part 10) are similar in concept. For example, they all use block-based processing. The following describes the block segmentation schemes in some standards.

[0053] High Efficiency Video Coding (HEVC)

[0054] HEVC is based on a hybrid block-based motion compensated transform coding architecture. The basic unit for compression is called a coding tree unit (CTU). For the 4:2:0 chroma format, the maximum CTU size is defined as up to 64 by 64 luma pixels and two blocks of 32 by 32 chroma pixels. Each CTU can contain one coding unit (CU) or be recursively divided into four smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more prediction units (PUs) and a transform unit (TU) tree.

[0055] In general, except for monochrome content, a CTU may include one luma coding tree block (CTB) and two corresponding chroma CTBs; a CU may include one luma coding block (CB) and two corresponding chroma CBs; a PU may include one luma prediction block (PB) and two corresponding chroma PBs; and a TU may include one luma transform block (TB) and two corresponding chroma TBs. However, exceptions may occur because the minimum TB size is 4×4 for both luma and chroma (i.e., 2×2 chroma TBs are not supported for 4:2:0 color format) and each intra chroma CB always has only one intra chroma PB, regardless of the number of intra luma PBs in the corresponding intra luma CB.

[0056] For an intra CU, the luma CB can be predicted by one or four luma PBs, and each of the two chroma CBs is always predicted by one chroma PB, where each luma PB has one intra luma prediction mode and the two chroma PBs share one intra chroma prediction mode. In addition, for an intra CU, the TB size cannot be larger than the PB size. In each PB, intra prediction is applied to predict the samples of each TB inside the PB from the adjacent reconstructed samples of the TB. For each PB, in addition to 33 directional intra prediction modes, DC mode and planar mode are also supported to predict flat areas and gradually changing areas, respectively.

[0057] For each inter PU, one of three prediction modes including inter, skip and merge can be selected. In general, a motion vector competition (MVC) scheme is introduced to select a motion candidate from a given candidate set including spatial motion candidates and temporal motion candidates. Multiple references for motion estimation enable the best reference to be found in 2 possible reconstructed reference picture lists (i.e., List 0 and List 1). For inter mode (called AMVP mode, where AMVP stands for advanced motion vector prediction), an inter prediction indicator (list 0, list 1 or bidirectional prediction), a reference index, a motion candidate index, a motion vector difference (MVD), and a prediction residual are sent. For skip mode and merge mode, only the merge index is sent, and the current PU inherits the inter prediction indicator, reference index, and motion vector from the neighboring PU referenced by the encoded merge index. In the case of a skip-coded CU, the residual signal is also omitted.

[0058] Joint Exploration Test Model (JEM)

[0059] The Joint Exploration Test Model (JEM) is built on top of the HEVC test model. The basic encoding and decoding flow of HEVC remains unchanged in JEM; however, the design elements of the most important modules (including the modules for block structure, intra-frame and inter-frame prediction, residual transform, loop filter, and entropy coding and decoding) are slightly modified, and additional coding tools are added. The following new coding features are included in JEM.

[0060] In HEVC, a CTU is divided into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics. The decision to encode a picture area using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs according to the PU partition type. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining the residual block by applying the prediction process based on the PU partition type, the CU can be partitioned into transform units (TUs) according to another quadtree structure of the CU similar to the coding tree. One of the key features of the HEVC structure is that it has multiple partition concepts including CU, PU, ​​and TU.

[0061] Figure 3 is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to some embodiments of the present disclosure.

[0062] The QTBT structure removes the concept of multiple partition types, that is, it removes the distinction between the concepts of CU, PU, ​​and TU, and supports greater flexibility for the CU partition shape. In the QTBT block structure, a CU can have a square or rectangular shape. Figure 3As shown in , the coding tree unit (CTU) is first partitioned by a quadtree (i.e., quadtree) structure. The quadtree leaf nodes can be further partitioned by a binary tree structure. There are two types of partitions in the binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. The binary tree leaf nodes are called coding units (CUs), and such partitions are used for prediction and transform processing without any further partitioning. This means that CU, PU, ​​and TU have the same block size in the QTBT coding block structure. In JEM, a CU sometimes consists of coding blocks (CBs) of different color components, for example, in the case of P and B slices in a 4:2:0 chroma format, a CU contains one luminance CB and two chroma CBs, and a CU sometimes consists of CBs of one component, for example, in the case of an I slice, a CU contains only one luminance CB or only two chroma CBs.

[0063] The following parameters are defined for the QTBT segmentation scheme.

[0064] -CTU size: the root node size of the quadtree, which is the same as the concept in HEVC;

[0065] -MinQTSize: The minimum allowed quad leaf node size;

[0066] -MaxBTSize: The maximum allowed binary tree root node size;

[0067] -MaxBTDepth: Maximum allowed binary tree depth;

[0068] -MinBTSize: The minimum allowed binary tree leaf node size.

[0069] In one example of a QTBT partitioning structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 chroma sample blocks (with a 4:2:0 chroma format), MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for both width and height) is set to 4×4, and MaxBTDepth is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. The quadtree leaf node may have a size from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the quadtree leaf node is 128×128, it will not be further divided by the binary tree because the size exceeds MaxBTSize (i.e., 64×64). Otherwise, the quadtree leaf node can be further divided by the binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, and its binary tree depth is 0. When the binary tree depth reaches MaxBTDepth (i.e., 4), no further splits are considered. When the binary tree node has a width equal to MinBTSize (i.e., 4), no further horizontal splits are considered. Similarly, when the binary tree node has a height equal to MinBTSize, no further vertical splits are considered. The leaf nodes of the binary tree are further processed by prediction and transform processing without any further splitting. In JEM, the maximum CTU size is 256×256 luma samples.

[0070] exist Figure 3 An example of block partitioning using the QTBT scheme and the corresponding tree representation are shown in FIG. The solid line indicates a quadtree partition, and the dashed line indicates a binary tree partition. Figure 3 As shown in , the coding tree unit (CTU) 300 is first partitioned by a quadtree structure, and three of the four quadtree leaf nodes 302, 304, 306, and 308 are further partitioned by a quadtree structure or a binary tree structure. For example, the quadtree leaf node 306 is further partitioned by quadtree partitioning; the quadtree leaf node 304 is further partitioned into two leaf nodes 304a and 304b by binary tree partitioning; and the quadtree leaf node 302 is further partitioned by binary tree partitioning. In each partition (i.e., non-leaf) node of the binary tree, a flag is indicated to indicate which partition type (i.e., horizontal or vertical) is used, where 0 indicates horizontal partitioning and 1 indicates vertical partitioning. For example, for the quadtree leaf node 304, it is marked as 0 to indicate horizontal partitioning, and for the quadtree leaf node 302, it is marked as 1 to indicate vertical partitioning. For quadtree partitioning, there is no need to indicate the partitioning type, because quadtree partitioning always partitions the block horizontally and vertically to produce 4 sub-blocks of equal size.

[0071] In addition, the QTBT scheme supports the ability of luma and chroma to have separate QTBT structures. Currently, for P and B slices, the luma and chroma CTBs in one CTU share the same QTBT structure. However, for I slices, the luma CTB is partitioned into CUs by the QTBT structure, and the chroma CTB is partitioned into chroma CUs by another QTBT structure. This means that the CU in the I slice consists of a coding block of the luma component or a coding block of two chroma components, and the CU in the P or B slice consists of coding blocks of all three color components.

[0072] Versatile Video Coding (VVC)

[0073] In the Joint Video Experts Group (JVET) meeting, JVET defined the first draft of Versatile Video Coding (VVC) and the VVC Test Model 1 (VTM1) coding method. It was decided to include quadtrees with nested multi-type trees using two-partition and three-partition coding block structures as the initial new coding features of VVC.

[0074] In VVC, the picture partitioning structure divides the input video into blocks called coding tree units (CTUs). A quadtree with a nested multi-type tree structure is used to divide CTUs into coding units (CUs), where leaf coding units (CUs) define areas that share the same prediction mode (e.g., intra or inter). Here, the term "unit" defines an area of ​​an image that covers all components; the term "block" is used to define an area that covers a specific component (e.g., luminance) and can be different in spatial position when considering chroma sampling formats (such as 4:2:0).

[0075] Split the image into CTUs

[0076] Figure 4 is a diagram showing an example of a picture divided into CTUs according to some embodiments of the present disclosure.

[0077] In VVC, a picture is divided into a CTU sequence, and the CTU concept is the same as that of HEVC. For a picture with three sample arrays, a CTU consists of an N×N luma sample block and two corresponding chroma sample blocks. Figure 4 An example of a picture 400 partitioned into CTUs 402 is shown.

[0078] The maximum allowed size of a luma block in a CTU is specified as 128x128 (although the maximum size of a luma transform block is 64x64).

[0079] Use tree structure to split CTU

[0080] Figure 5 is a schematic diagram illustrating a multi-type tree partitioning mode according to some embodiments of the present disclosure.

[0081] In HEVC, a CTU is divided into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics. The decision to encode a picture area using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the leaf-CU level. Each leaf-CU can be further divided into one, two, or four PUs according to the PU partition type. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining the residual block by applying the prediction process based on the PU partition type, the leaf CU can be divided into transform units (TUs) according to another quadtree structure of the CU similar to the coding tree. One of the key features of the HEVC structure is that it has multiple partitioning concepts including CU, PU, ​​and TU.

[0082] In VVC, a quadtree with nested multi-type trees of two-partition and three-partition partition structures is used to replace the concept of multiple partition unit types, that is, it removes the distinction between the concepts of CU, PU, ​​and TU (except that a CU with a size that is too large for the maximum transform length requires such a distinction), and supports greater flexibility in the CU partition shape. In the coding tree structure, the CU can have a square or rectangular shape. The coding tree unit (CTU) is first partitioned by a quadtree (i.e., quadtree) structure. Then, the quadtree leaf nodes can be further partitioned by a multi-type tree structure. As Figure 5 As shown, there are four types of partitions in the multi-type tree structure: vertical two-partition 502 (SPLIT_BT_VER), horizontal two-partition 504 (SPLIT_BT_HOR), vertical three-partition 506 (SPLIT_TT_VER) and horizontal three-partition 508 (SPLIT_TT_HOR). The multi-type tree leaf node is called a coding unit (CU), and unless the CU is too large for the maximum transform length, this partition is used for prediction and transform processing without any further segmentation. This means that in most cases, CU, PU and TU have the same block size in a quadtree with a nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the color component of the CU. In VTM1, a CU consists of coding blocks (CBs) of different color components, for example, a CU contains one luminance CB and two chrominance CBs (unless the video is monochrome, i.e., has only one color component).

[0083] Split the CU into multiple prediction units

[0084] In VVC, for each CU partitioned based on the structure described above, prediction of block content can be performed on the entire CU block or in a sub-block manner as explained in the following paragraphs. This prediction operation unit is called a prediction unit (or PU).

[0085] In the case of intra prediction (or intra-frame prediction), the size of the PU is generally equal to the size of the CU. In other words, prediction is performed on the entire CU block. For inter prediction (or inter-frame prediction), the size of the PU may be equal to or smaller than the size of the CU. In other words, there is a case where a CU can be divided into multiple PUs for prediction.

[0086] Some examples where the PU size is smaller than the CU size include affine prediction mode, advanced temporal motion vector prediction (ATMVP) mode, triangle prediction mode, and the like.

[0087] In affine prediction mode, the CU may be divided into multiple 4×4 PUs for prediction. A motion vector may be obtained for each 4×4 PU and motion compensation may be performed on the 4×4 PU accordingly. In ATMVP mode, the CU may be divided into one or more 8×8 PUs for prediction. A motion vector may be obtained for each 8×8 PU and motion compensation may be performed on the 8×8 PU accordingly. In triangle prediction mode, the CU may be divided into two triangle shaped prediction units. A motion vector may be obtained for each PU and motion compensation may be performed accordingly. For inter prediction, triangle prediction mode is supported. More details of the triangle prediction mode are described below.

[0088] Triangle prediction mode

[0089] Figure 6 is a schematic diagram showing how a CU is divided into triangular prediction units according to some embodiments of the present disclosure.

[0090] The concept of triangle prediction mode is to introduce triangle partitioning for motion compensation prediction. Triangle prediction mode can also be named triangle prediction unit mode. Figure 6 As shown, CU 602 or 604 is divided into two triangle prediction units PU1 and PU2 in a diagonal or anti-diagonal direction (i.e., from the upper left corner to the lower right corner as shown in CU 602 or from the upper right corner to the lower left corner as shown in CU 604). Each triangle prediction unit in the CU uses its own unidirectional prediction motion vector and reference frame index obtained from the unidirectional prediction candidate list for inter-frame prediction. After predicting the triangle prediction unit, an adaptive weighting process is performed on the diagonal edges. Then, the transformation and quantization process is applied to the entire CU. Note that this mode is only applied to the skip and merge modes in the current VVC. Although in Figure 6 , the CU is shown as a square block, but the triangle prediction mode may also be applied to non-square (ie, rectangular) shaped CUs.

[0091] The unidirectional prediction candidate list may include one or more candidates, and each candidate may be a motion vector. Therefore, throughout this disclosure, the terms "unidirectional prediction candidate list", "unidirectional prediction motion vector candidate list" and "unidirectional prediction merge list" may be used interchangeably; and the terms "unidirectional prediction merge candidate" and "unidirectional prediction motion vector" may also be used interchangeably.

[0092] Unidirectional motion vector prediction candidate list

[0093] Figure 7 is a schematic diagram illustrating the locations of neighboring blocks according to some embodiments of the present disclosure.

[0094] In some examples, the unidirectional prediction motion vector candidate list may include two to five unidirectional prediction motion vector candidates. In some other examples, there may be other numbers. It is obtained from neighboring blocks. Figure 7 As shown in , the candidate list of unidirectional prediction motion vectors is obtained from seven neighboring blocks including five spatial neighboring blocks (1 to 5) and two temporal co-located blocks (6 to 7). The motion vectors of the seven neighboring blocks are collected into a first merge list. Then, a unidirectional prediction candidate list is formed based on the first merge list motion vectors according to a specific order. Based on the order, the unidirectional prediction motion vector from the first merge list is first placed in the candidate list of unidirectional prediction motion vectors, followed by the reference picture list 0 or L0 motion vector of the bidirectional prediction motion vector, then the reference picture list 1 or L1 motion vector of the bidirectional prediction motion vector, and then the average motion vector of the L0 and L1 motion vectors of the bidirectional prediction motion vector. At this point, if the number of candidates is still less than the target number (which can be 5), a zero motion vector is added to the list to meet the target number.

[0095] A prediction factor is obtained for each triangle PU based on its motion vector. It is worth noting that the obtained prediction factor covers a larger area than the actual triangle PU, so that there is an overlapping area of ​​the two prediction factors along the shared diagonal edge of the two triangle PUs. A weighting process is applied to the diagonal edge area between the two prediction factors to obtain the final prediction of the CU. The current weighting factors for luma samples and chroma samples are {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} respectively.

[0096] Triangle prediction mode syntax and signaling

[0097] The triangle prediction mode is signaled using the triangle prediction flag. When a CU is encoded in skip mode or merge mode, the triangle prediction flag is signaled. For a given CU, if the triangle prediction flag has a value of 1, it indicates that the corresponding CU is encoded using the triangle prediction mode. Otherwise, the CU is encoded using a prediction mode other than the triangle prediction mode.

[0098] For example, the triangle prediction flag is conditionally signaled in skip mode or merge mode. First, the triangle prediction tool enable / disable flag is signaled in the sequence parameter set (or SPS). The triangle prediction flag is signaled at the CU level only when the triangle prediction tool enable / disable flag is true. Secondly, the triangle prediction tool is allowed only in B slices. Therefore, the triangle prediction flag is signaled at the CU level only in B slices. Third, the triangle prediction mode is signaled only for CUs with a size equal to or greater than a certain threshold. If the CU has a size smaller than the threshold, the triangle prediction flag is not signaled. Fourth, the triangle prediction mode can be signaled only for the CU as long as the CU is not encoded in a sub-block merge mode including both affine mode and ATMVP mode. In the four cases listed above, when the triangle prediction flag is not signaled, it is inferred to be 0 on the decoder side.

[0099] When the triangle prediction flag is signaled, the triangle prediction flag is signaled using a context adaptive binary arithmetic coding (CABAC) entropy encoder with a specific context. The context is formed based on the triangle prediction flag values ​​of the upper and left blocks of the current CU.

[0100] In order to encode or decode the triangle prediction flag of the current block (or current CU), the triangle prediction flags from both the upper block (or CU) and the left block (or CU) are obtained and their values ​​are summed. This leads to three possible contexts corresponding to the following situations:

[0101] 1) Both the left block and the upper block have triangle prediction flag 0;

[0102] 2) Both the left block and the upper block have triangle prediction flag 1;

[0103] 3)Others.

[0104] Each of the three contexts maintains a separate probability.Once the context value of the current block is determined, the triangle prediction flag of the current block is encoded and decoded using the CABAC probability model corresponding to the context value.

[0105] If the triangle prediction flag is true, the triangle split direction flag is signaled to indicate whether the split is from the top left corner towards the bottom right corner or from the top right corner towards the bottom left corner.

[0106] In the case where the triangle prediction mode is used for the CU, two index values ​​are signaled to indicate the index values ​​of the first unidirectional prediction merge candidate and the second unidirectional prediction merge candidate for triangle prediction, respectively. The two index values ​​are used to locate two merge candidates from the above-mentioned unidirectional prediction motion vector candidate list for the first partition and the second partition, respectively. For triangle prediction, the two index values ​​need to be different so that the two prediction values ​​of the two triangle partitions can be different from each other. Therefore, the first index value is directly signaled. In order to signal the second index value, if it is less than the first index value, its value is directly signaled. Otherwise, before the value of the second index value is signaled to the decoder, its value is subtracted by 1. On the decoder side, the first index is decoded and used directly. In order to decode the second index value, the value represented as "idx" is first decoded by the CABAC engine. If idx is less than the first index value, the second index value will be equal to the value of idx. Otherwise, the second index value will be equal to (idx+1).

[0107] Geometry Merge Mode

[0108] In geometry merge mode (ie, GEO), a CU may be partitioned into two prediction units (ie, geometry PUs) according to geometry partitioning. Fig.14 Some examples of geometric partitioning according to some embodiments of the present disclosure are shown. Fig.14 As shown in , a CU may be partitioned into two PUs along a non-completely diagonal line, and the two PUs may have a geometric shape, such as a triangle, a wedge, or a trapezoid. The geometric merge mode may be considered an extension of the triangle prediction mode, or a superset of the triangle prediction mode.

[0109] Fig.15 is a schematic diagram showing an example of a geometric partition representation according to some embodiments of the present disclosure. Fig.15 As shown, given a GEO partition, the signaling of the partition requires the signaling of an angle α and a separation line displacement denoted as p relative to the center of the block. Here, α represents a quantized angle between 0 and 360 degrees.

[0110] Similar to the triangle prediction mode, under GEO partitioning, only unidirectional prediction is allowed for each of the two GEO partitions, so that the memory bandwidth requirement in this mode on the decoder side does not exceed the memory bandwidth requirement of the bidirectional prediction block. The motion vector prediction of GEO partitioning is consistent with the triangle prediction mode. Also similar to the triangle prediction mode, a weighted average-based blending operation can be applied to the inner boundary between the two partitions in the geometric merge mode. Therefore, the techniques related to motion compensated prediction using triangle prediction units can also be applied to motion compensated prediction using geometric shape prediction units.

[0111] Normal merge mode motion vector candidate list

[0112] According to the current VVC, in the normal merge mode where the entire CU is predicted without being divided into more than one PU, a motion vector candidate list or a merge candidate list is constructed using a process different from that for the triangle prediction mode.

[0113] Figure 8 is a schematic diagram showing the positions of spatial merging candidates according to some embodiments of the present disclosure, such as Figure 8 As shown, first, a spatial motion vector candidate is selected based on the motion vector from the adjacent block. Figure 8 Up to four merge candidates are selected from the candidates in the positions depicted in . Candidates are selected according to a specific order. An example resulting order is A1→B1→B0→A0→(B2). Position B2 is considered only if any PU at position A1, B1, B0, A0 is unavailable or intra-coded. It should be noted that other different orders may also be used.

[0114] Next, a temporal merge candidate is obtained. In obtaining a temporal merge candidate, a scaled motion vector is obtained based on the co-located PU of a picture with the smallest picture order count (POC) difference with the current picture in a given reference picture list. The reference picture list to be used to obtain the co-located PU is explicitly indicated in the slice header. Fig. 9 The motion vector scaling of the temporal merge candidate according to some embodiments of the present disclosure is shown, as Fig. 9As shown by the dotted line in , the scaled motion vector of the temporal merge candidate is obtained. The scaled motion vector of the temporal merge candidate is scaled from the motion vector of the co-located PU col_PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture curr_ref of the current picture curr_pic and the current picture curr_pic, and td is defined as the POC difference between the reference picture col_ref of the co-located picture col_pic and the co-located picture col_pic. The reference picture index of the temporal merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC draft specification. For B slices, two motion vectors (one for reference picture list 0 and the other for reference picture list 1) are obtained and combined to form a bidirectional prediction merge candidate.

[0115] Fig.10 is a schematic diagram illustrating candidate positions of temporal merging candidates according to some embodiments of the present disclosure.

[0116] like Fig.10 As depicted in , the position of the co-located PU is selected between two candidate positions C3 and H. If the PU at position H is not available, or is intra-coded, or is outside the current CTU, then position C3 is used to obtain the temporal merge candidate. Otherwise, position H is used to obtain the temporal merge candidate.

[0117] After inserting both spatial and temporal motion vectors into the merge candidate list as described above, history-based merge candidates are added. The so-called history-based merge candidates include those motion vectors from previously coded CUs, which are maintained in a separate motion vector list and managed based on specific rules.

[0118] After inserting the history-based candidates, if the merge candidate list is not full, the pairwise average motion vector candidates are further added to the list. As its name suggests, this type of candidate is constructed by averaging the candidates that are already in the current list. More specifically, based on a specific order or rule, two candidates in the merge candidate list are taken at a time, and the average motion vector of the two candidates is appended to the current list.

[0119] After inserting the pairwise average motion vectors, if the merge candidate list is still not full, zero motion vectors will be added to fill up the list.

[0120] The first merge list for triangle prediction is constructed using the normal merge list construction process.

[0121] The triangle prediction mode in the current VVC shares some similarities with the conventional merge prediction mode in the whole process of forming the prediction value. For example, in both prediction modes, it is necessary to build a merge list based on at least the neighboring spatial motion vector and the co-located motion vector of the current CU. At the same time, the triangle prediction mode also has some aspects that are different from the conventional merge prediction mode.

[0122] For example, although it is necessary to construct a merge list in the triangle prediction mode and the normal merge prediction mode, the detailed process of constructing such a list is different.

[0123] These differences result in additional cost to the codec implementation because additional logic is required.The process and logic for building merge lists may be unified and shared between triangle prediction mode and regular merge prediction mode.

[0124] In some examples, when forming a uni-prediction (also referred to as uni-prediction) merge list for triangular prediction mode, before adding the new motion vector to the merge list, the new motion vector is completely pruned against those motion vectors already in the list. In other words, the new motion vector is compared to each motion vector already in the uni-prediction merge list, and the new motion vector is added to the list only if the new motion vector is different from each motion vector in the merge list. Otherwise, the new motion vector is not added to the list.

[0125] According to some examples of the present disclosure, in triangle prediction mode, a uni-directional prediction merge list is constructed from a normal merge mode motion vector candidate list (which may be referred to as a normal merge list).

[0126] More specifically, in order to construct a merge candidate list for triangle prediction mode, a first merge list is first constructed based on the merge list construction process for conventional merge prediction. The first merge list includes multiple candidates, each candidate is a motion vector. Then, the motion vectors in the first merge list are used to further construct a unidirectional prediction merge list for triangle prediction mode.

[0127] It should be noted that the first merged list constructed in this case may select a list size different from the list size of the general merge mode or the regular merge mode. In one example of the present disclosure, the first merged list has the same size as the list of the general merge mode. In another example of the present disclosure, the constructed first merged list has a list size different from the list size of the general merge mode.

[0128] Construct a one-way lookahead merge list from the first merge list

[0129] According to some examples of the present disclosure, a unidirectional prediction merge list for the triangle prediction mode may be constructed from the first merge list based on one of the following methods.

[0130] In an example of the present disclosure, to construct a unidirectional prediction merge list, the prediction list 0 motion vectors of the candidates in the first merge list are first checked and selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is not full after this process (for example, the number of candidates in this list is still less than the target number), the prediction list 1 motion vectors of the candidates in the first merge list are checked and selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is still not full, the prediction list 0 zero vector is added to the unidirectional prediction merge list. If the unidirectional prediction merge list is still not full, the prediction list 1 zero vector is added to the unidirectional prediction merge list.

[0131] In another example of the present disclosure, for each candidate in the first merge list, its prediction list 0 motion vector and prediction list 1 motion vector are added to the unidirectional prediction merge list in an interleaved manner. More specifically, for each candidate in the first merge list, if the candidate is a unidirectional prediction motion vector, it is directly added to the unidirectional prediction merge list. Otherwise, if the candidate is a bidirectional prediction motion vector in the first merge list, its prediction list 0 motion vector is first added to the unidirectional prediction merge list, followed by its prediction list 1 motion vector. Once all motion vector candidates in the first merge list have been checked and added, and the unidirectional prediction merge list is still not full, a unidirectional prediction zero motion vector can be added. For example, for each reference frame index, a prediction list 0 zero motion vector and a prediction list 1 zero motion vector can be added separately to the unidirectional prediction merge list until the list is full.

[0132] In yet another example of the present disclosure, the unidirectional prediction motion vectors from the first merge list are first selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is not full after this process, for each bidirectional prediction motion vector in the first merge list, first its prediction list 0 motion vector is added to the unidirectional prediction merge list, followed by its prediction list 1 motion vector. After this process, if the unidirectional prediction merge list is still not full, a unidirectional prediction zero motion vector may be added. For example, for each reference frame index, a prediction list 0 zero motion vector and a prediction list 1 zero motion vector may be added to the unidirectional prediction merge list individually until the list is full.

[0133] In the above description, when a unidirectional prediction motion vector is added to a unidirectional prediction merge list, a motion vector pruning process may be performed to ensure that the new motion vector to be added is different from those motion vectors already in the unidirectional prediction merge list. Such a motion vector pruning process may also be performed in a partial manner to obtain lower complexity, for example, the new motion vector to be added is only checked against some but not all motion vectors already in the unidirectional prediction merge list. In an extreme case, motion vector pruning (i.e., motion vector comparison operation) is not performed in the process.

[0134] Construct a unidirectional prediction merge list from the first merge list based on the picture prediction configuration

[0135] In some examples of the present disclosure, a unidirectional prediction merge list may be constructed in an adaptive manner based on whether the current picture uses backward prediction. For example, different methods may be used to construct a unidirectional prediction merge list depending on whether the current picture uses backward prediction. If the picture order count (POC) values ​​of all reference pictures are not greater than the POC value of the current picture, it indicates that the current picture does not use backward prediction.

[0136] In an example of the present disclosure, when the current picture does not use backward prediction, or after determining that the current picture does not use backward prediction, the candidate prediction list 0 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list, followed by those candidate prediction list 1 motion vectors; if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector may be added. Otherwise, if the current picture uses backward prediction, each candidate prediction list 0 motion vector and prediction list 1 motion vector in the first merge list may be checked and selected into the unidirectional prediction merge list in an interlaced manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, followed by the first candidate prediction list 1 motion vector, then the second candidate prediction list 0 motion vector is added, followed by the second candidate prediction list 1 motion vector, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector may be added.

[0137] In another example of the present disclosure, if the current picture does not use backward prediction, the candidate prediction list 1 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list, followed by those candidate prediction list 0 motion vectors; if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector may be added. Otherwise, if the current picture uses backward prediction, each candidate prediction list 0 motion vector and prediction list 1 motion vector in the first merge list may be checked and selected into the unidirectional prediction merge list in an interlaced manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, followed by the first candidate prediction list 1 motion vector, then the second candidate prediction list 0 motion vector is added, followed by the second candidate prediction list 1 motion vector, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector may be added.

[0138] In another example of the present disclosure, if the current picture does not use backward prediction, only the candidate prediction list 0 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list, and if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector can be added. Otherwise, if the current picture uses backward prediction, each candidate prediction list 0 motion vector and prediction list 1 motion vector in the first merge list can be checked and selected into the unidirectional prediction merge list in an interlaced manner as described above, that is, the prediction list 0 motion vector of the first candidate in the first merge list is added, followed by the prediction list 1 motion vector of the first candidate, and then the prediction list 0 motion vector of the second candidate is added, followed by the prediction list 1 motion vector of the second candidate, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector can be added.

[0139] In another example of the present disclosure, if the current picture does not use backward prediction, only the candidate prediction list 1 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list, and if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector may be added. Otherwise, if the current picture uses backward prediction, each candidate prediction list 0 motion vector and prediction list 1 motion vector in the first merge list may be checked and selected into the unidirectional prediction merge list in an interlaced manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, followed by the first candidate prediction list 1 motion vector, then the second candidate prediction list 0 motion vector is added, followed by the second candidate prediction list 1 motion vector, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector may be added.

[0140] In the above example, a unidirectional prediction merge list for triangle prediction is constructed by selecting motion vectors from the first merge list into the unidirectional prediction merge list. However, in practice, the method can be implemented in different ways (physically forming a unidirectional prediction (or unidirectional prediction) merge list or not physically forming a unidirectional prediction (or unidirectional prediction) merge list). In some examples, the first merge list can be used directly without physically creating a unidirectional prediction merge list. For example, an index can be set based on a specific order for only the list 0 motion vector and / or list 1 motion vector of each candidate in the first merge list and accessed directly from the first merge list. That is, after constructing a first merge list including multiple candidates (each candidate is one or more motion vectors) based on a merge list construction process for conventional merge prediction, a unidirectional prediction merge list is not constructed, but an index list including multiple reference indexes (each reference index is a reference to the motion vector of a candidate in the first merge list) is constructed. It should be noted that the order of the indexes can follow any selection order described in the example of constructing a unidirectional prediction merge list.

[0141] In one example of the present disclosure, when the current picture does not use backward prediction, the prediction list 0 motion vectors of the candidates in the first merge list are used as unidirectional prediction merge candidates and are indexed according to the same index order as they are in the first merge list. That is, after determining that the POC of the current picture is greater than each of the POCs of the reference picture, the reference indexes are arranged according to the same order of the list 0 motion vectors of the candidates in the first merge list. Otherwise, if the current picture uses backward prediction, the list 0 motion vector and list 1 motion vector of each candidate in the first merge list are used as unidirectional prediction merge candidates and are indexed based on an interlaced manner, that is, the list 0 motion vector of the first candidate in the first merge list is first, followed by the list 1 motion vector of the first candidate, then the list 0 motion vector of the second candidate, followed by the list 1 motion vector of the second candidate, and so on. That is, after determining that the POC of the current picture is less than at least one of the POCs of the reference picture, in the case where each candidate in the first merge list is a bidirectional prediction motion vector, the reference indexes are arranged according to the interlaced manner of the list 0 motion vector and the list 1 motion vector of each candidate in the first merge list. In the case where the candidate in the first merge list is a unidirectional motion vector, the zero motion vector is set to be indexed as the unidirectional prediction merge candidate after the motion vector of the candidate. This ensures that for the case where the current picture uses backward prediction, each candidate in the first merge list (regardless of whether it is a bidirectional prediction motion vector or a unidirectional prediction motion vector) provides two unidirectional motion vectors as unidirectional prediction merge candidates.

[0142] In another example of the present disclosure, when the current picture does not use backward prediction, the prediction list 0 motion vectors of the candidates in the first merge list are used as unidirectional prediction merge candidates and are indexed according to the same index order as they are in the first merge list. Otherwise, if the current picture uses backward prediction, each candidate list 0 motion vector and list 1 motion vector in the first merge list are used as unidirectional prediction merge candidates and are indexed based on the staggered manner as described above, that is, the list 0 motion vector of the first candidate in the first merge list is first, followed by the list 1 motion vector of the first candidate, then the list 0 motion vector of the second candidate, followed by the list 1 motion vector of the second candidate, and so on. In the case where the candidate in the first merge list is a unidirectional motion vector, the motion vector plus a specific motion offset is indexed as a unidirectional prediction merge candidate after the motion vector of the candidate.

[0143] In the above process, when checking for a new motion vector to be added to the uni-directional prediction merge list, pruning may be performed completely or partially. When performed partially, this means that the new motion vector is compared with some, but not all, of the motion vectors already in the uni-directional prediction merge list. In the extreme case, motion vector pruning (i.e., motion vector comparison operations) is not performed in the process.

[0144] Motion vector pruning may also be adaptively performed when forming a unidirectional prediction merge list based on whether the current picture uses backward prediction. For example, in the example disclosed herein related to index list construction based on picture prediction configuration, when the current picture does not use backward prediction, a motion vector pruning operation is performed completely or partially. When the current picture uses backward prediction, a motion vector pruning operation is not performed.

[0145] Select unidirectional prediction merge candidates for triangle prediction mode

[0146] In addition to the above examples, other methods of constructing a unidirectional prediction merging list or selecting unidirectional prediction merging candidates are also disclosed.

[0147] In one example of the present disclosure, once the first merge list for the normal merge mode is constructed, the unidirectional prediction merge candidates may be selected for triangle prediction according to the following rules:

[0148] For the motion vector candidate in the first merge list, one and only one of the list 0 motion vector or the list 1 motion vector is used for triangle prediction;

[0149] for a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is even, its list 0 motion vector is used for triangle prediction if available, and if this motion vector candidate does not have a list 0 motion vector, its list 1 motion vector is used for triangle prediction; and

[0150] For a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is an odd number, its list 1 motion vector is used for triangle prediction if available, and if this motion vector candidate does not have a list 1 motion vector, its list 0 motion vector is used for triangle prediction.

[0151] Fig.11A An example of unidirectional prediction motion vector (MV) selection (or unidirectional prediction merge candidate selection) for triangular prediction mode is shown. In the example, the first N (for example, N is equal to 5) merged MV candidates obtained in the first merge list are indexed from 0 to (N-1); and each row has two columns, representing the list 0 motion vector and the list 1 motion vector of the candidates in the first merge list, respectively. Each candidate in the list can be unidirectionally predicted or bidirectionally predicted. For unidirectional prediction candidates, it only has list 0 motion vector or list 1 motion vector, but not both. For bidirectional prediction candidates, it has both list 0 motion vector and list 1 motion vector. In Fig.11A In , for each merge index, the motion vectors marked with "x" are those that are first used for triangle prediction if they are available. If the motion vectors marked with "x" are not available, the unmarked motion vectors corresponding to the same merge index will be used for triangle prediction.

[0152] The above concepts can be extended to other examples. Fig. 11B Another example of unidirectional prediction motion vector (MV) selection for triangular prediction mode is shown. Fig. 11B , the rules for selecting unidirectional prediction merge candidates for triangle prediction are as follows:

[0153] For the motion vector candidate in the first merge list, one and only one of the list 0 motion vector or the list 1 motion vector is used for triangle prediction;

[0154] for a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is even, its list 1 motion vector is used for triangle prediction if available, and if this motion vector candidate does not have a list 1 motion vector, its list 0 motion vector is used for triangle prediction; and

[0155] For a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is odd, its list 0 motion vector is used for triangle prediction if available, and if this motion vector candidate does not have a list 0 motion vector, its list 1 motion vector is used for triangle prediction.

[0156] In some examples, other different orders may be defined and used to select unidirectional prediction merge candidates for triangle prediction from those motion vector candidates in the first merge list. More specifically, for a given motion vector candidate in the first merge list, the decision of whether to use its list 0 motion vector or list 1 motion vector first when the motion vector candidate can be used for triangle prediction does not have to depend on the parity of the index values ​​of the candidates in the first merge list as described above. For example, the following rule may also be used:

[0157] For the motion vector candidate in the first merge list, one and only one of the list 0 motion vector or the list 1 motion vector is used for triangle prediction;

[0158] Based on a certain predefined pattern, for several motion vector candidates in the first merge list, their list 0 motion vectors are used for triangle prediction if available, and if there is no list 0 motion vector, the corresponding list 1 motion vector is used for triangle prediction; and

[0159] Based on the same predefined pattern, for the remaining motion vector candidates in the first merge list, their list 1 motion vectors are used for triangle prediction if available, and in the absence of a list 1 motion vector, the corresponding list 0 motion vector is used for triangle prediction.

[0160] FIG. 12A to FIG. 12D Some examples of predefined patterns in unidirectional prediction motion vector (MV) selection for triangle prediction mode are shown. For each merge index, the motion vectors marked with "x" are those that are first used for triangle prediction if they are available. If a motion vector marked with "x" is not available, the unmarked motion vector corresponding to the same merge index will be used for triangle prediction.

[0161] exist Fig. 12A In the example above, for the first three motion vector candidates in the first merge list, their list 0 motion vectors are checked first. Only when the list 0 motion vector is not available, the corresponding list 1 motion vector is used for triangle prediction. For the fourth and fifth motion vector candidates in the first merge list, their list 1 motion vectors are checked first. Only when the list 1 motion vector is not available, the corresponding list 0 motion vector is used for triangle prediction. FIG. 12B to FIG. 12DThree other modes of selecting unidirectional prediction merge candidates from the first merge list are shown. The examples shown in the figures are not limiting, and there are other examples. For example, FIG. 12A to FIG. 12D Horizontally and / or vertically mirrored versions of those patterns shown in .

[0162] The concepts illustrated by some examples may be used in combination with the concepts illustrated by some other examples in the present disclosure. According to one example of the present disclosure, for a given motion vector candidate in the first merge list, the decision of whether to use its list 0 motion vector or list 1 motion vector first when both its list 0 motion vector and list 1 motion vector are available for triangle and / or geometry prediction may additionally depend on whether the current picture and / or slice uses backward prediction. FIG. 16A to FIG. 16B is a schematic diagram showing an example of unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure. Fig.16A As shown in , if the current picture and / or slice uses backward prediction, the selection order between the candidate list 0 motion vector and the list 1 motion vector is based on the parity of the index value of the candidate in the first merge list. If the current picture and / or slice does not use backward prediction, the candidate list x (where x is 0 or 1) motion vector (if available) is preferred and is selected first, and the example where x is equal to 0 is shown in Fig. 16B . Similar to the other examples explained previously, in case a unidirectional prediction motion vector is not available for a particular candidate in the first merge list, the corresponding unidirectional prediction motion vector of the same candidate from the other reference lists is selected. Thus, for each candidate in the first merge list, a unidirectional prediction motion vector may be obtained and used for triangular and / or geometric prediction mode. The obtained unidirectional prediction motion vector shares the same index value as the merge candidate in the first merge list.

[0163] Flexible merge list construction and candidate index signaling for triangle prediction modes

[0164] In one example of the present disclosure, once the first merge list for the normal merge mode is constructed, the unidirectional prediction motion vector for triangle prediction is directly selected from the list. In order to indicate a specific list 0 motion vector or list 1 motion vector used for triangle prediction, first, an index value is signaled to indicate which candidate from the first merge list is selected. Next, a binary reference list indication flag (referred to as L0L1_flag) is signaled to indicate whether the list 0 motion vector or list 1 motion vector of the selected candidate from the first merge list is selected for the first partition of triangle prediction. The same signaling method is used to indicate the second list 0 motion vector or list 1 motion vector of the second partition to be used for triangle prediction. For example, the syntax signaled for the triangle mode coded CU may include index1, L0L1_flag1, index2, L0L1_flag2. Here, index1 and index2 are merge index values ​​of two candidates selected from the first merge list for the first partition and the second partition, respectively. Index1 and index2 are used to locate candidates for the first partition and the second partition (i.e., the first candidate and the second candidate), respectively. L0L1_flag1 is a binary flag for the first partition to indicate whether to select the list 0 motion vector or the list 1 motion vector of the selected candidate based on index1 from the first merge list. L0L1_flag2 is a binary flag for the second partition to indicate whether to select the list 0 motion vector or the list 1 motion vector of the selected candidate based on index2 from the first merge list. The unidirectional prediction MV for the first partition is obtained by selecting the list X1 MV of the first candidate according to the first binary reference list indication flag L0L1_flag1, where X1 takes a value of 0 or 1. The unidirectional prediction MV for the second partition is obtained by selecting the list X2 MV of the second candidate according to the second binary reference list indication flag L0L1_flag2, where X2 takes a value of 0 or 1.

[0165] Fig.13 is a schematic diagram showing an example of flexible unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure. Fig.13As shown, in the triangle prediction mode, each list 0 motion vector and / or list 1 motion vector indicated by the symbol "x" in the rectangular box can be indicated / sent to the decoder with a signal for obtaining a prediction of the first partition, and each list 0 motion vector and / or list 1 motion vector indicated by the symbol "x" in the rectangular box can be indicated / sent to the decoder with a signal for obtaining a prediction of the second partition. Therefore, the selection of a unidirectional prediction motion vector from the first merge list becomes very flexible. Given a first merge list with a size of N candidates, up to 2N unidirectional prediction motion vectors can be used for each of the two triangle partitions. The two merge index values ​​of the two partitions in the triangle prediction mode do not have to be different from each other. In other words, they can take the same value. The index value is directly signaled without adjustment before being signaled. More specifically, unlike what is defined in the current VVC, the second index value is directly signaled to the decoder without performing any adjustment on the value before being signaled.

[0166] In another example of the present disclosure, when the two index values ​​are the same, it is not necessary to signal the binary flag L0L1_flag2 for the second partition. Instead, it is inferred to have an opposite value relative to the binary flag L0L1_flag1 for the first partition. In other words, in this case, L0L1_flag2 can take the value (1-L0L1_flag1).

[0167] In another example of the present disclosure, binary reference list indication flags (i.e., L0L1_flag1 and L0L1_flag2) may be encoded as CABAC context bits. The context for L0L1_flag1 may be separate from the context for L0L1_flag2. The CABAC probability under each context may be initialized at the beginning of the video sequence and / or at the beginning of the picture and / or at the beginning of the tile group. In this example, different context modeling methods may be used to encode the two flags. In one method, the context model selection of the second flag may depend on the value of the first flag. In other words, at least two context models may be used to encode the second flag, wherein the context model is selected based on the value of the first flag sent by the signal. In another method, the CABAC probability under each context model may be initialized differently depending on whether the current picture uses backward prediction. In the case where the picture uses backward prediction, the probability for the flag may be initialized higher to indicate a specific list, such as list 0. In another method, the first flag may be encoded as a bypass bit, and only the second flag may be encoded as a context bit. The methods mentioned here can be used individually or in combination.

[0168] In yet another example of the present disclosure, when the motion vector indicated by the merge index value and the associated binary reference list indication flag (i.e., L0L1_flag) does not exist, a unidirectional prediction zero motion vector may be used instead. That is, when it is determined that the list X1 MV of the first candidate does not exist, the unidirectional prediction zero MV is selected as the unidirectional prediction MV for the first partition; and / or when it is determined that the list X2 MV of the second candidate does not exist, the unidirectional prediction zero MV is selected as the unidirectional prediction MV for the second partition.

[0169] In yet another example of the present disclosure, when a motion vector indicated by a merge index value and an associated L0L1_flag does not exist, a corresponding motion vector indicated by the same merge index value but from another list (i.e., list (1-L0L1_flag)) may be used instead. That is, a unidirectional prediction MV for a first partition is obtained by selecting a first candidate list (1-X1) MV when it is determined that the first candidate list X1 MV does not exist; and / or a unidirectional prediction MV for a second partition is obtained by selecting a second candidate list (1-X2) MV when it is determined that the second candidate list X2 MV does not exist.

[0170] In another example of the present disclosure, for a triangle mode coded CU, a second L0L1_flag (i.e., L0L1_flag2) associated with a second index (i.e., index2) is not signaled but is always inferred. In this case, the index1, L0L1_flag1, and index2 syntax still need to be signaled. In one method, L0L1_flag2 is inferred based on the value of L0L1_flag1 and whether the current picture uses backward prediction. More specifically, for a triangle mode coded CU, if the current picture uses backward prediction, the value of L0L1_flag2 is inferred to be the opposite binary value of L0L1_flag1 (i.e., 1-L0L1_flag1); if the current picture does not use backward prediction, the value of L0L1_flag2 is inferred to be the same as L0L1_flag1. In addition, if the current picture does not use backward prediction, the value of index2 can be further implemented as a different value from index1 because both motion vectors (one for each triangle partition) come from the same prediction list. If index2 has a value equal to index1, this means that the same motion vector will be used for both triangle partitions, which is useless from a coding efficiency perspective. In this case, when the value of index2 is signaled, a corresponding adjustment to the value of index2 may be performed before index binarization, which is the same as the adjustment for signaling index2 in the current VVC design. For example, when the actual value of index1 is less than the actual value of index2, the CABAC binarization codeword corresponding to (index2-1) is used to signal the value of index2; otherwise, the CABAC binarization codeword corresponding to index2 is used to signal the value of index2. Based on this example of the present disclosure, optionally, together with the same Index2 value adjustment for CABAC binarization, an implementation of Index2 with a different value from index1 may also be applied to the case when the current picture uses backward prediction.

[0171] In yet another example of the present disclosure, for a triangle mode coded CU, none of the L0L1_flags are signaled. Instead, they are all inferred. In this case, it is still necessary to signal the index1 and index2 syntaxes, representing the merge index values ​​of the two candidates selected from the first merge list for the first partition and the second partition, respectively. Given a merge candidate index value, a specific method may be defined or used to determine whether the list 0 motion vector or the list 1 motion vector of the corresponding merge candidate from the first list is selected for triangle mode prediction. In one method, for index1, Fig.11A The mode shown in is used to determine from which prediction list to select the motion vector of the merge candidate for triangle mode prediction; and for index2, Fig. 11B The pattern shown in is used to determine from which prediction list to select the motion vector of the merge candidate for triangle mode prediction. In other words, if index1 is an even value, the list 0 motion vector of the candidate indicated by index1 is selected, and if index1 is an odd value, the list 1 motion vector of the candidate indicated by index1 is selected. For index2, if it is an even value, the list 1 motion vector is selected, and if it is an odd value, the list 0 motion vector is selected. In the event that there is no motion vector corresponding to a particular prediction list, a specific default motion vector, such as a zero motion vector, or a corresponding motion vector from another prediction list, etc., may be used instead. When determining from which prediction list to select the motion vector of the merge candidate for triangle mode prediction, Fig. 11B The pattern shown in is used for index1 and Fig.11A It is also possible that the mode shown in is used for index2. That is, L0L1_flag can be determined based on the values ​​of index1 and index2.

[0172] Although the methods in the present disclosure are described using the triangle prediction mode as an example, assuming that the geometry merge mode can be considered as an extension or superset of the triangle prediction mode, the methods shown in the present disclosure can naturally be applied to the geometry merge mode as well. All merge list construction methods shown in the present disclosure can naturally be applied to the geometry merge mode without any modification. Similarly, all merge index signaling methods shown in the present disclosure can naturally be applied to the geometry merge mode without any modification. In one example, according to Fig.11A The same merge list construction process is used for the geometric merge mode. Fig. 11B The same merge list construction process is used for the geometric merge mode. In addition, it is worth mentioning that if both triangle prediction mode and geometric merge mode exist in VVC, the same merge list construction process can be shared and used for these two modes.

[0173] It should be mentioned that although a first merge list containing 5 merge candidates is used in some examples of the present disclosure, the size of the first merge list may be defined differently, for example, 6 or 4, or some other value. The method shown in the example is applicable to the case when the first merge list has a size other than 5.

[0174] In the above example, motion vector pruning may also be performed. This pruning may be done completely or partially. When performed partially, this means that the new motion vector is compared to some, but not all, of the motion vectors already in the uni-prediction merge list. This may also mean that only some, but not all, of the new motion vectors need to be checked for pruning before being used as merge candidates for triangle prediction. A specific example is to check only the second motion vector against the first motion vector for pruning before using it as a merge candidate for triangle prediction, rather than checking all other motion vectors for pruning. In extreme cases, motion vector pruning (i.e., motion vector comparison operations) is not performed in the process.

[0175] Another example of performing partial motion vector pruning is to control the total number of motion vector pruning operations performed in generating the unidirectional prediction merge list to be no more than N times, where N is a positive integer. As an exemplary embodiment, a counter may be used during this process; each time a motion vector pruning operation is performed, the counter is incremented by 1 regardless of the result of the pruning operation; once the counter has a value equal to N, no motion vector pruning operations are performed when selecting the remaining unidirectional prediction motion vectors into the unidirectional prediction merge list. Fig.17 Take the case shown as an example. For example, the arrow indicates the order according to which the unidirectional prediction motion vectors from the first merge list are checked. For each candidate in the first merge list, its prediction list 0 motion vector and prediction list 1 motion vector are checked in an interlaced order and added to the unidirectional prediction merge list. More specifically, for each candidate in the first merge list, if it is a unidirectional prediction motion vector, it is directly added to the unidirectional prediction merge list. Otherwise, for the bidirectional prediction motion vector in the first merge list, its list 0 motion vector is first checked and added to the unidirectional prediction merge list, and then its list 1 motion vector is checked. Once all motion vector candidates in the first merge list are checked and the unidirectional prediction merge list is not full, a unidirectional prediction zero motion vector can be added. For example, for each reference frame index, a list-0 zero vector and a list-1 zero vector can be added to the unidirectional prediction merge list respectively until the list is full. During the process, each time a motion vector is checked to be added to the unidirectional prediction merge list, a motion vector pruning operation is performed to see if the motion vector is the same as any of the motion vectors selected in the unidirectional prediction merge list. If the pruning operation concludes that the motion vector is different from those motion vectors compared to it, the motion vector is added to the uni-directional prediction merge list. Otherwise, the motion vector is not selected or added to the list. If the total number of motion vector pruning operations performed reaches N, no more pruning operations are performed when selecting remaining motion vectors into the list.

[0176] Another example of performing partial motion vector pruning is to perform motion vector pruning operations only during the generation of the first M candidates in the unidirectional prediction merge list, where M is a positive integer. For example, M can take values ​​of 2, 3, etc. Once M motion vectors are selected into the unidirectional prediction merge list, motion vector pruning operations will no longer be performed when the remaining unidirectional prediction motion vectors are selected into the list.

[0177] In yet another example of partial motion vector pruning, motion vector pruning operations are performed only among the first K candidates from the first merge list, where K is a positive integer. Fig.17 In the example shown, a motion vector pruning operation is performed when checking the first K motion vectors in the order indicated by those arrows. Starting from the (K+1)th candidate, no motion vector pruning operation is performed when selecting the remaining unidirectional prediction motion vectors into the unidirectional prediction merge list.

[0178] Another example of partial motion vector pruning is to allow a fixed number of L pruning operations to be performed to select each motion vector, where L is a positive integer. If the value of L is 1, it means that only one motion vector pruning operation is allowed when selecting each motion vector. Based on this method, Fig.18 A specific example is provided in , where the arrows indicate the order in which each unidirectionally predicted motion vector position from the first merge list is checked. In this figure, each row has two blocks (or columns), representing the list 0 motion vector and the list 1 motion vector of the candidates in the first merge list. Each candidate in the list can be a motion vector that is unidirectionally predicted or bidirectionally predicted. For a unidirectionally predicted candidate, it only has a list 0 motion vector or a list 1 motion vector, but not both. For a bidirectionally predicted candidate, it has both a list 0 motion vector and a list 1 motion vector. In Fig.18In , for each merge index, the motion vectors marked with "x" (if they are available) are the first ones to be checked for triangle prediction / geometry prediction. For the first motion vector marked with "x" with index 0, if it is available, it is used for triangle prediction / geometry prediction. Otherwise, the unmarked motion vector corresponding to the same merge index will be used for triangle prediction / geometry prediction mode. Starting from the second motion vector (with index value 1 or greater), if the motion vector marked with "x" is available and it is different from its immediately previous motion vector marked with "x", or if the unmarked motion vector corresponding to the same merge index is not available, the motion vector marked with "x" is used for triangle prediction / geometry prediction mode. Otherwise, the unmarked motion vector corresponding to the same merge index will be used for triangle prediction / geometry prediction. In this case, except for the first candidate, only one motion vector pruning operation is performed when selecting each motion vector candidate. More importantly, in triangle prediction / geometry prediction mode, this design has very low decoder complexity when locating the corresponding unidirectional prediction motion vector from the first merge list given a merge index. On the decoder side, if the decoded merge index value is 0, the first motion vector marked with "x" in the first merge list is selected (if it is available). Otherwise, the unmarked motion vector corresponding to the same merge index is selected. If the decoded merge index has a value greater than 0, the motion vector marked with "x" with the same merge index is checked. If it is available and different from its immediately previous motion vector marked with "x" in the first merge list, or if the unmarked motion vector corresponding to the same merge index in the first merge list is not available (that is, the original candidate with the same merge index in the first merge list is not a bidirectional prediction motion vector), the motion vector marked with "x" is selected. Otherwise, the unmarked unidirectional motion vector corresponding to the same merge index in the first merge list is selected.

[0179] Fig.19 1900 is a block diagram showing an apparatus for video encoding and decoding according to some embodiments of the present disclosure. Apparatus 1900 may be a terminal such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

[0180] like Fig.19 As shown, device 1900 may include one or more of the following components: a processing component 1902 , a memory 1904 , a power component 1906 , a multimedia component 1908 , an audio component 1910 , an input / output (I / O) interface 1912 , a sensor component 1914 , and a communication component 1916 .

[0181] The processing component 1902 generally controls the overall operation of the device 1900, such as operations related to display, phone calls, data communications, camera operations, and recording operations. The processing component 1902 may include one or more processors 1920 for executing instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1902 may include one or more modules for facilitating the interaction between the processing component 1902 and other components. For example, the processing component 1902 may include a multimedia module for facilitating the interaction between the multimedia component 1908 and the processing component 1902.

[0182] The memory 1904 is configured to store different types of data to support the operation of the device 1900. Examples of such data include instructions for any application or method operating on the device 1900, contact data, phone book data, messages, pictures, videos, etc. The memory 1904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the memory 1904 can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0183] The power supply component 1906 provides power to the various components of the device 1900. The power supply component 1906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1900.

[0184] The multimedia component 1908 includes a screen that provides an output interface between the device 1900 and the user. In some examples, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen that receives input signals from the user. The touch panel may include one or more touch sensors for sensing touch, sliding, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or sliding action, but also detect the duration and pressure associated with the touch or sliding operation. In some examples, the multimedia component 1908 may include a front camera and / or a rear camera. When the device 1900 is in an operating mode such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data.

[0185] The audio component 1910 is configured to output and / or input audio signals. For example, the audio component 1910 includes a microphone (MIC). When the device 1900 is in an operating mode (such as a call mode, a recording mode, and a speech recognition mode), the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 1904 or sent via the communication component 1916. In some examples, the audio component 1910 also includes a speaker for outputting an audio signal.

[0186] I / O interface 1912 provides an interface between processing component 1902 and peripheral interface modules. The above peripheral interface modules may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to home buttons, volume buttons, start buttons, and lock buttons.

[0187] Sensor assembly 1914 includes one or more sensors for providing status evaluation in different aspects of device 1900. For example, sensor assembly 1914 can detect the on / off state of device 1900 and the relative position of components. For example, the components are the display and keyboard of device 1900. Sensor assembly 1914 can also detect position changes of device 1900 or components of device 1900, the presence or departure of user contact on device 1900, the direction or acceleration / deceleration of device 1900, and temperature changes of device 1900. Sensor assembly 1914 may include a proximity sensor configured to detect the presence of nearby objects without any physical touch. Sensor component 1914 may also include an optical sensor, such as a CMOS or CCD image sensor used in imaging applications. In some examples, sensor assembly 1914 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0188] The communication component 1916 is configured to facilitate wired or wireless communication between the device 1900 and other devices. The device 1900 can access the wireless network based on communication standards such as WiFi, 4G or a combination thereof. In an example, the communication component 1916 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an example, the communication component 1916 may also include a near field communication (NFC) module for facilitating short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0189] In an example, device 1900 may be implemented by one or more of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components to perform the above method.

[0190] The non-volatile computer-readable storage medium may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a flash memory, a hybrid drive or a solid-state hybrid drive (SSHD), a read-only memory (ROM), a compact disk read-only memory (CD-ROM), a magnetic tape, a floppy disk, etc.

[0191] Fig. 20 is a flow chart illustrating an exemplary process for video encoding and decoding using motion compensated prediction of geometric prediction units according to some embodiments of the present disclosure.

[0192] In step 2002, the processor 1920 divides the video picture into a plurality of coding units (CUs), wherein at least one CU of the plurality of CUs is further divided into two prediction units (PUs). The two PUs may include at least one geometric PU. For example, the geometric PU may include a pair of triangular PUs, a pair of wedge PUs, or other geometric PUs.

[0193] In step 2004, the processor 1920 constructs a first merge list including multiple candidates, each candidate being a motion vector including a list 0 motion vector and / or a list 1 motion vector. For example, the processor 1920 may construct the first merge list based on a merge list construction process for conventional merge prediction. The processor 1920 may also obtain the first merge list from other electronic devices or memory.

[0194] In step 2006, the processor 1920 obtains a unidirectional prediction MV for the PU by selecting a plurality of candidate list 0 MVs or list 1 MVs.

[0195] In step 2008, the processor 1920 constructs a unidirectional prediction merge list based on the unidirectional prediction MV.

[0196] In step 2010, the processor 1920 performs a pruning operation on the unidirectional prediction MV when constructing the unidirectional prediction merge list.

[0197] In some examples, a device for video encoding and decoding is provided. The device includes a processor 1920; and a memory 1904, which is configured to store instructions executable by the processor; wherein the processor is configured to perform the following when executing the instructions: Fig. 20 The method shown.

[0198] In some other examples, a non-volatile computer-readable storage medium 1904 is provided having instructions stored therein. When the instructions are executed by the processor 1920, the instructions cause the processor to perform the following steps: Fig. 20 The method shown.

[0199] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0200] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A method for video decoding, comprising: Obtaining, from a video bitstream, a plurality of coding units CUs partitioned from a video picture, wherein at least one CU of the plurality of CUs is further partitioned into two prediction units PUs, the two PUs including at least one geometry PU; Obtain a plurality of candidates, wherein each candidate of the plurality of candidates comprises a motion vector MV, and the MV comprises list 0 MV or list 1 MV or both list 0 MV and list 1 MV; Obtaining a unidirectional prediction MV candidate for the PU by selecting one MV of each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates, The step of obtaining a unidirectional prediction MV candidate for the PU by selecting an MV of each of the multiple candidates includes performing the following operations on a first candidate of the multiple candidates: If the first candidate corresponds to the bi-directional prediction mode, the list 0 MV of the first candidate is selected as the uni-directional prediction MV candidate for the PU.

2. The video decoding method according to claim 1, wherein: The steps to construct a one-way prediction merge list include: Performing the pruning operation a total number of times less than or equal to N, where N is a positive integer; A counter is used to record the total number of times the pruning operation is performed, wherein each time the pruning operation is performed on the MV, the counter is incremented by 1 regardless of the result of the pruning operation; and When the counter reaches a value equal to N, the pruning operation on the remaining uni-directionally predicted MV candidates is stopped.

3. The video decoding method according to claim 1, wherein: The steps to construct a one-way prediction merge list include: For each candidate of the plurality of candidates, the pruning operation is performed on the list 0 MV or the list 1 MV to be added to the unidirectional prediction merge list by checking the list 0 MV or the list 1 MV for MVs existing in the unidirectional prediction merge list.

4. The video decoding method according to claim 1, wherein: The steps to construct a one-way prediction merge list include: The pruning operation is performed when first M candidates in the unidirectional prediction merge list are obtained, where M is a positive integer; and When M unidirectional prediction MV candidates are obtained in the unidirectional prediction merge list, the pruning operation on the remaining unidirectional prediction MV candidates is stopped.

5. The video decoding method according to claim 1, wherein: The steps to construct a one-way prediction merge list include: Performing the pruning operation on first K candidates among the multiple candidates, where K is a positive integer; and When K candidates among the plurality of candidates are checked, the pruning operation to obtain the remaining uni-directional prediction MV candidates is stopped.

6. The video decoding method according to claim 1, wherein: The steps to construct a one-way prediction merge list include: A pruning operation is performed at most once on each of the plurality of candidates to obtain only one unidirectionally predicted MV candidate from each of the plurality of candidates.

7. The video decoding method according to claim 1, wherein: The step of obtaining a unidirectional prediction MV candidate for the PU by selecting an MV of each of the plurality of candidates comprises performing the following operations on a first candidate of the plurality of candidates: If the first candidate corresponds to the uni-directional prediction mode, the first candidate is selected as the uni-directional prediction MV candidate for the PU.

8. The video decoding method according to claim 1, wherein: The step of obtaining a unidirectional prediction MV candidate for the PU by selecting an MV of each candidate of the plurality of candidates comprises performing the following operations on a candidate of the plurality of candidates corresponding to the bidirectional prediction mode: If the candidate has an odd merge index number, selecting the list 1 MV of the candidate as the unidirectional prediction MV candidate for the PU; as well as If the candidate has an even merge index number, the list 0 MV of the candidate is selected as the uni-directional prediction MV candidate for the PU.

9. A video decoding device, comprising: one or more processors; as well as A memory configured to store instructions executable by the one or more processors; wherein the one or more processors, when executing the instructions, are configured to: Obtaining, from a video bitstream, a plurality of coding units CUs partitioned from a video picture, wherein at least one CU of the plurality of CUs is further partitioned into two prediction units PUs, the two PUs including at least one geometry PU; Obtain a plurality of candidates, wherein each candidate of the plurality of candidates comprises a motion vector MV, and the MV comprises list 0 MV or list 1 MV or both list 0 MV and list 1 MV; Obtaining a unidirectional prediction MV candidate for the PU by selecting one MV of each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates, The one or more processors are further configured to perform the following operations on a first candidate among the multiple candidates: If the first candidate corresponds to the bi-directional prediction mode, the list 0 MV of the first candidate is selected as the uni-directional prediction MV candidate for the PU.

10. The apparatus for video decoding according to claim 9, wherein the one or more processors are further configured to: Performing the pruning operation a total number of times less than or equal to N, where N is a positive integer; The total number of times the pruning operation is performed is recorded by a counter, where: Each time a pruning operation is performed on the MV, the counter is incremented by 1 regardless of the result of the pruning operation; as well as When the counter reaches a value equal to N, the pruning operation on the remaining uni-directionally predicted MV candidates is stopped.

11. The apparatus for video decoding according to claim 9, wherein the one or more processors are further configured to: For each candidate of the plurality of candidates, the pruning operation is performed on the list 0 MV or the list 1 MV to be added to the unidirectional prediction merge list by checking the list 0 MV or the list 1 MV for MVs existing in the unidirectional prediction merge list.

12. The apparatus for video decoding according to claim 9, wherein the one or more processors are further configured to: The pruning operation is performed when first M candidates in the unidirectional prediction merge list are obtained, where M is a positive integer; and When M unidirectional prediction MV candidates are obtained in the unidirectional prediction merge list, the pruning operation on the remaining unidirectional prediction MV candidates is stopped.

13. The apparatus for video decoding according to claim 9, wherein the one or more processors are further configured to: The pruning operation is performed on the first K candidates among the multiple candidates, wherein: K is a positive integer; as well as When K candidates among the plurality of candidates are checked, the pruning operation to obtain the remaining uni-directional prediction MV candidates is stopped.

14. The apparatus for video decoding according to claim 9, wherein the one or more processors are further configured to: A pruning operation is performed at most once on each of the plurality of candidates to obtain only one unidirectionally predicted MV candidate from each of the plurality of candidates.

15. The apparatus for video decoding according to claim 9, wherein the one or more processors are further configured to perform the following operations on a first candidate among the plurality of candidates: If the first candidate corresponds to the uni-directional prediction mode, the first candidate is selected as the uni-directional prediction MV candidate for the PU.

16. The video decoding apparatus according to claim 9, wherein the one or more processors are further configured to perform the following operations on a candidate corresponding to the bidirectional prediction mode among the multiple candidates: If the candidate has an odd merge index number, selecting the list 1 MV of the candidate as a uni-directional prediction MV candidate for the PU; and If the candidate has an even merge index number, the list 0 MV of the candidate is selected as the uni-directional prediction MV candidate for the PU.

17. A non-volatile computer-readable storage medium for video decoding, the non-volatile computer-readable storage medium storing computer-executable instructions, the computer-executable instructions, when executed by one or more computer processors, causing the one or more computer processors to perform actions, the actions comprising: Obtaining, from a video bitstream, a plurality of coding units CUs partitioned from a video picture, wherein at least one CU of the plurality of CUs is further partitioned into two prediction units PUs, the two PUs including at least one geometry PU; Obtain a plurality of candidates, wherein each candidate of the plurality of candidates comprises a motion vector MV, and the MV comprises list 0 MV or list 1 MV or both list 0 MV and list 1 MV; Obtaining a unidirectional prediction MV candidate for the PU by selecting one MV of each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates, Wherein, obtaining a unidirectional prediction MV candidate for the PU by selecting an MV of each candidate of the multiple candidates further includes performing the following operations on a first candidate of the multiple candidates: If the first candidate corresponds to the bi-directional prediction mode, the list 0 MV of the first candidate is selected as the uni-directional prediction MV candidate for the PU.

18. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The actions also include: Performing the pruning operation a total number of times less than or equal to N, where N is a positive integer; A counter is used to record the total number of times the pruning operation is performed, wherein each time the pruning operation is performed on the MV, the counter is incremented by 1 regardless of the result of the pruning operation; and When the counter reaches a value equal to N, the pruning operation on the remaining uni-directionally predicted MV candidates is stopped.

19. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The actions also include: For each candidate of the plurality of candidates, the pruning operation is performed on the list 0 MV or the list 1 MV to be added to the unidirectional prediction merge list by checking the list 0 MV or the list 1 MV for MVs existing in the unidirectional prediction merge list.

20. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The actions also include: The pruning operation is performed when first M candidates in the unidirectional prediction merge list are obtained, where M is a positive integer; and When M unidirectional prediction MV candidates are obtained in the unidirectional prediction merge list, the pruning operation on the remaining unidirectional prediction MV candidates is stopped.

21. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The actions also include: Performing the pruning operation on first K candidates among the multiple candidates, where K is a positive integer; and When K candidates among the plurality of candidates are checked, the pruning operation to obtain the remaining uni-directional prediction MV candidates is stopped.

22. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The actions also include: A pruning operation is performed at most once on each of the plurality of candidates to obtain only one unidirectionally predicted MV candidate from each of the plurality of candidates.

23. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The action further includes performing the following operation on a first candidate among the multiple candidates: If the first candidate corresponds to the uni-directional prediction mode, the first candidate is selected as the uni-directional prediction MV candidate for the PU.

24. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein: The action further includes performing the following operation on a candidate corresponding to the bidirectional prediction mode among the multiple candidates: If the candidate has an odd merge index number, selecting the list 1 MV of the candidate as the unidirectional prediction MV candidate for the PU; as well as If the candidate has an even merge index number, the list 0 MV of the candidate is selected as the uni-directional prediction MV candidate for the PU.

25. A method for sending a bit stream, comprising: performing an encoding method to generate a bitstream; as well as sending the bit stream to a decoding device, Wherein, the encoding method includes: Divide the video picture into a plurality of coding units CU, wherein at least one CU of the plurality of CUs is further divided into two prediction units PU, and the two PUs include at least one geometric shape PU; Obtain a plurality of candidates, wherein each candidate of the plurality of candidates comprises a motion vector MV, and the MV comprises list 0 MV or list 1 MV or both list 0 MV and list 1 MV; Obtaining a unidirectional prediction MV candidate for the PU by selecting one MV of each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates, The step of obtaining a unidirectional prediction MV candidate for the PU by selecting an MV of each of the multiple candidates includes performing the following operations on a first candidate of the multiple candidates: If the first candidate corresponds to the bi-directional prediction mode, the list 0 MV of the first candidate is selected as the uni-directional prediction MV candidate for the PU.

Images