Video encoding and decoding method and device using triangular partition

Abstract

Methods and apparatus for video encoding are provided. The method comprises the following steps: at least one coding unit in the plurality of coding units is further divided into two prediction units (PU), a first PU and a second PU, and the two prediction units comprise at least one geometrical PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0MV and / or a list 1MV; locating a first candidate for the first PU and a second candidate for the second PU according to a first index and a second index, respectively; obtaining a first unidirectional prediction MV for the first PU by selecting a list X1MV of the first candidate according to a first binary reference list indication flag; and obtaining a second unidirectional prediction MV for the second PU by selecting a list X2MV of the second candidate according to a second binary reference list indication flag.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application number "202080085641.5", application date "October 12, 2020", and invention name "Video encoding and decoding method and device using triangular partitioning". Technical Field

[0002] The present application relates generally to video coding and compression, and particularly, but not limited to, methods and apparatus for motion compensated prediction using triangular prediction units (ie, a special case of geometrically partitioned prediction units) in video coding. Background Art

[0003] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression. Digital video devices implement video codec technologies such as those described in the Versatile Video Codec (VVC), Joint Exploration Test Model (JEM), MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC) standards, and extensions of such standards.

[0004] Video codecs typically use prediction methods (e.g., inter-frame prediction, intra-frame prediction) that exploit the redundancy present in video images or sequences. An important goal of video codec technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing degradation in video quality. As ever-evolving video services become available, coding techniques with better codec efficiency are needed.

[0005] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video codecs, a video frame is partitioned into one or more slices, each slice having multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU), or may be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also referred to as a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU may be coded or decoded in intra, inter, or IBC mode. Video blocks in an intra-coded (I) slice of a video frame are coded using spatial prediction relative to reference samples in neighboring blocks within the same video frame. Video blocks in an inter-coded (P or B) slice of a video frame may use spatial prediction relative to reference samples in neighboring blocks within the same video frame, or use temporal prediction relative to reference samples in other previous and / or future reference video frames.

[0006] A prediction block for the current video block to be encoded is generated based on spatial or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding a reference block can be accomplished using a block matching algorithm. The residual data representing the pixel differences between the current block to be encoded and the prediction block is called a residual block or prediction error. Inter-coded blocks are encoded based on motion vectors pointing to reference blocks in the reference frames that form the prediction block and the residual block. The process of determining the motion vector is often referred to as motion estimation. Intra-coded blocks are encoded based on the intra-prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain (e.g., the frequency domain), generating residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which can then be entropy coded into the video bitstream to achieve even greater compression.

[0007] The coded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or directly transmitted to the electronic device in a wired or wireless manner. The electronic device then performs video decompression (which is the reverse process of the above-mentioned video compression) by, for example, parsing the coded video bitstream to obtain syntax elements from the bitstream and reconstructing digital video data from the coded video bitstream into its original format based at least in part on the syntax elements obtained from the bitstream, and presents the reconstructed digital video data on a display of the electronic device.

[0008] As digital video quality increases from HD to 4Kx2K or even 8Kx4K, the amount of video data to be encoded / decoded increases exponentially. This is an ongoing challenge in terms of how to encode / decode video data more efficiently while maintaining the image quality of the decoded video data.

[0009] In a Joint Video Experts Team (JVET) meeting, JVET defined the first draft of the Universal Video Codec (VVC) and VVC Test Model 1 (VTM1) coding methods. It was decided to include a quadtree with nested multi-type trees using binary and ternary split coding block structures as an initial new codec feature for VVC. Subsequently, the reference software VTM for implementing the coding method and the draft VVC decoding process has been developed during the JVET meeting. Summary of the Invention

[0010] This disclosure generally describes examples of techniques related to motion compensated prediction with geometric prediction units in video codecs.

[0011] According to a first aspect of the present disclosure, a video encoding method using geometric partitioning is provided, comprising: partitioning a video picture into a plurality of coding units (CUs), at least one of the plurality of coding units being further partitioned into two prediction units (PUs), a first PU and a second PU, the two prediction units including at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0MV and / or a list 1MV; locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; obtaining a first unidirectional prediction MV for the first PU by selecting a list X1MV of the first candidate according to a first binary reference list indication flag, wherein X1 takes a value of 0 or 1 and is indicated by the first binary reference list indication flag; and obtaining a second unidirectional prediction MV for the second PU by selecting a list X2MV of the second candidate according to a second binary reference list indication flag, wherein X2 takes a value of 0 or 1 and is indicated by the second binary reference list indication flag.

[0012] According to a second aspect of the present disclosure, a video encoding apparatus using geometric partitioning is provided, comprising: one or more processors; and a memory configured to store instructions executable by the one or more processors; wherein the one or more processors, when executing the instructions, are configured to: partition a video picture into a plurality of coding units (CUs), wherein at least one of the plurality of coding units is further partitioned into two prediction units (PUs), a first PU and a second PU, wherein the two prediction units include at least one geometrically shaped PU; construct a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0 MV and / or a list 1 MV; locate a first candidate for the first PU according to a first index; locate a second candidate for the second PU according to a second index; and select a list X of the first candidate according to a first binary reference list indication flag. 1MV to obtain a first unidirectional prediction MV for the first PU, where X1 takes a value of 0 or 1 and is indicated by the first binary reference list indication flag; and to obtain a second unidirectional prediction MV for the second PU by selecting the second candidate list X2MV according to the second binary reference list indication flag, where X2 takes a value of 0 or 1 and is indicated by the second binary reference list indication flag.

[0013] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium for video coding using geometric partitioning is provided, wherein the non-transitory computer-readable storage medium stores computer-executable instructions. When executed by one or more computer processors, the instructions cause the one or more computer processors to perform the following actions: partitioning a video picture into a plurality of coding units (CUs), wherein at least one coding unit of the plurality of coding units is further partitioned into two prediction units (PUs), a first PU and a second PU, wherein the two prediction units include at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein the plurality of candidates are Each of which is a motion vector (MV) including list 0MV and / or list 1MV; locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; obtaining a first unidirectional prediction MV for the first PU by selecting the list X1MV of the first candidate according to a first binary reference list indication flag, where X1 takes a value of 0 or 1 and is indicated by the first binary reference list indication flag; and obtaining a second unidirectional prediction MV for the second PU by selecting the list X2MV of the second candidate according to a second binary reference list indication flag, where X2 takes a value of 0 or 1 and is indicated by the second binary reference list indication flag. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A more particular description of examples of the present disclosure will be presented by reference to specific examples shown in the accompanying drawings. Given that these drawings depict only some examples and are therefore not to be considered limiting of scope, these examples will be described and explained with additional specificity and detail through use of the accompanying drawings.

[0015] Figure 1 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0016] Figure 2 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0017] Figure 3 is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to some embodiments of the present disclosure.

[0018] Figure 4 is a diagram illustrating an example of a picture divided into CTUs according to some embodiments of the present disclosure.

[0019] Figure 5 is a schematic diagram illustrating a multi-type tree splitting mode according to some embodiments of the present disclosure.

[0020] Figure 6 FIG. 4 is a schematic diagram illustrating splitting a CU into triangular prediction units according to some embodiments of the present disclosure.

[0021] Figure 7 is a schematic diagram illustrating positions of adjacent blocks according to some embodiments of the present disclosure.

[0022] Figure 8 is a schematic diagram illustrating an example of geometric partitioning according to some embodiments of the present disclosure.

[0023] Figure 9 is a schematic diagram illustrating an example of a geometric partitioning representation according to some embodiments of the present disclosure.

[0024] Figure 10 is a schematic diagram illustrating positions of spatial merging candidates according to some embodiments of the present disclosure.

[0025] Figure 11 is a schematic diagram illustrating motion vector scaling of temporal merging candidates according to some embodiments of the present disclosure.

[0026] Figure 12 is a schematic diagram illustrating candidate positions of temporal merging candidates according to some embodiments of the present disclosure.

[0027] 13A to 13B is a diagram illustrating an example of unidirectional prediction motion vector (MV) selection for triangular prediction mode according to some embodiments of the present disclosure.

[0028] 14A to 14D is a diagram illustrating an example of unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure.

[0029] FIG. 15A to FIG. 15B is a diagram illustrating an example of unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure.

[0030] Figure 16 is a diagram illustrating an example of flexible unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure.

[0031] Figure 17 is a block diagram illustrating an exemplary apparatus for video encoding and decoding according to some embodiments of the present disclosure.

[0032] Figure 18 is a flowchart illustrating an exemplary process for video encoding and decoding for motion compensated prediction using a geometric prediction unit according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0033] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to facilitate understanding of the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be employed without departing from the scope of the claims. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices having digital video capabilities.

[0034] Reference throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language indicates that a particular feature, structure, or characteristic being described is included in at least one embodiment or example. Examples Unless expressly stated otherwise, a feature, structure, element, or characteristic described in conjunction with one or some embodiments may also be applicable to other embodiments.

[0035] Throughout the disclosure, unless otherwise expressly stated, the terms "first," "second," "third," etc. are used merely to refer to related elements (e.g., devices, components, compositions, steps, etc.) without indicating any spatial or temporal order. For example, "first device" and "second device" may refer to two separately formed devices, or two parts, components, or operating states of the same device, and may be arbitrarily named.

[0036] As used herein, depending on the context, the terms "if" or "when" may be understood to mean "when" or "in response to..." These terms, if appearing in a claim, may not indicate that the associated limitation or feature is conditional or optional.

[0037] The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "sub-unit" may include memory (shared, dedicated, or combined) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. These components may or may not be physically attached to each other, or may or may not be located adjacent to each other.

[0038] A unit or module may be implemented purely by software, purely by hardware, or by a combination of hardware and software. In a pure software implementation, for example, a unit or module may include functionally related code blocks or software components that are linked together directly or indirectly to perform a specific function.

[0039] Figure 1 The block diagram illustrates an exemplary block-based hybrid video encoder 100 that can be used in conjunction with various video codec standards that utilize block-based processing. In encoder 100, a video frame is partitioned into multiple video blocks for processing. For each given video block, a prediction is formed based on either an inter-frame prediction method or an intra-frame prediction method. In inter-frame prediction, one or more prediction values ​​are formed based on pixels from a previously reconstructed frame through motion estimation and motion compensation. In intra-frame prediction, prediction values ​​are formed based on reconstructed pixels in the current frame. Through mode decision, the best prediction value is selected to predict the current block.

[0040] The prediction residual representing the difference between the current video block and its predicted value is sent to the transform circuit 102. The transform coefficients are then sent from the transform circuit 102 to the quantization circuit 104 for entropy reduction. The quantized coefficients are then fed to the entropy coding circuit 106 to generate a compressed video bitstream. Figure 1As shown, prediction-related information 110 from the inter-frame prediction circuit and / or the intra-frame prediction circuit 112, such as video block partition information, motion vector (MV), reference picture index and intra-frame prediction mode, is also fed through the entropy coding circuit 106 and saved into the compressed video bitstream 114.

[0041] In encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, a prediction residual is reconstructed via inverse quantization 116 and inverse transform circuitry 118. This reconstructed prediction residual is combined with a block prediction value 120 to generate unfiltered reconstructed pixels for the current video block.

[0042] Spatial prediction (or "intra-prediction") uses pixels from samples of already-encoded neighboring blocks in the same video frame as the current video block (called reference samples) to predict the current video block.

[0043] Temporal prediction (also known as "inter prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in video signals. The temporal prediction signal for a given coding unit (CU) or coding block is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference. In addition, if multiple reference pictures are supported, a reference picture index is additionally sent, which identifies which reference picture in the reference picture memory the temporal prediction signal comes from.

[0044] After performing spatial and / or temporal prediction, the intra / inter mode decision circuit 121 in the encoder 100 selects the optimal prediction mode based on, for example, a rate-distortion optimization method. The block prediction value 120 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using the transform circuit 102 and the quantization circuit 104. The resulting quantized residual coefficients are inversely quantized by the inverse quantization circuit 116 and inversely transformed by the inverse transform circuit 118 to form a reconstructed residual, which is then added back to the prediction block to form the reconstructed signal for that CU. In-loop filtering 115 (e.g., a deblocking filter, sample adaptive offset (SAO), and / or an adaptive in-loop filter (ALF)) may be further applied to the reconstructed CU before it is placed in the reference picture memory of the picture buffer 117 and used to encode and decode future video blocks. To form the output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and the quantized residual coefficients are sent to the entropy coding unit 106 for further compression and packetization to form the bitstream.

[0045] For example, deblocking filters are available in the current versions of AVC, HEVC, and VVC. In HEVC, an additional loop filter called SAO (Sample Adaptive Offset) is defined to further improve codec efficiency. In the latest version of the VVC standard, another loop filter called ALF (Adaptive Loop Filter) is being actively studied and may be included in the final standard.

[0046] These loop filter operations are optional. Performing these operations helps improve codec efficiency and visual quality. They can also be turned off as a decision made by encoder 100 to save computational complexity.

[0047] It should be noted that intra prediction is usually based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels if these filter options are turned on by the encoder 100 .

[0048] Figure 2 is a block diagram illustrating an exemplary fast-based video decoder 200 that can be used in conjunction with various video coding standards. The decoder 200 is similar to the one residing in Figure 1 The reconstruction-related portion of the encoder 100 is described. In the decoder 200, the input video bitstream 201 is first decoded by entropy decoding 202 to derive quantization coefficient levels and prediction-related information. These quantization coefficient levels are processed by inverse quantization 204 and inverse transformation 206 to obtain reconstructed prediction residuals. The block predictor mechanism implemented in the intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. The unfiltered reconstructed pixel set is obtained by adding the reconstructed prediction residual from the inverse transformation 206 and the prediction output generated by the block predictor mechanism using an adder 214.

[0049] The reconstructed block may also pass through an in-loop filter 209 before being stored in a picture buffer 213, which serves as a reference picture memory. The reconstructed video in the picture buffer 213 may then be sent to drive a display device and used to predict future video blocks. With the in-loop filter 209 turned on, filtering operations are performed on these reconstructed pixels to derive the final reconstructed video output 222.

[0050] The video coding / decoding standards mentioned above, such as VVC, JEM, HEVC, and MPEG-4, Part 10, are conceptually similar. For example, they all use block-based processing. The following details the block segmentation schemes used in some of these standards.

[0051] High Efficiency Video Coding (HEVC)

[0052] HEVC is based on a hybrid block-based motion compensated transform codec architecture. The basic unit for compression is called a coding tree unit (CTU). The maximum CTU size is defined as up to 64 by 64 luma pixels and two 32 by 32 chroma pixel blocks for the 4:2:0 chroma format. Each CTU can contain one coding unit (CU) or be recursively split into four smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more prediction units (PUs) and a transform unit (TU) tree.

[0053] In general, except for monochrome content, a CTU may include one luma coding tree block (CTB) and two corresponding chroma CTBs; a CU may include one luma coding block (CB) and two corresponding chroma CBs; a PU may include one luma prediction block (PB) and two corresponding chroma PBs; and a TU may include one luma transform block (TB) and two corresponding chroma TBs. However, exceptions may occur because the minimum TB size for both luma and chroma is 4×4 (i.e., for 4:2:0 color format, 2×2 chroma TBs are not supported), and each intra chroma CB always has only one intra chroma PB, regardless of the number of intra luma PBs in the corresponding intra luma CB.

[0054] For an intra CU, the luma CB can be predicted by one or four luma PBs, and each of the two chroma CBs is always predicted by one chroma PB, where each luma PB has one intra luma prediction mode and the two chroma prediction modes share one intra chroma prediction mode. Moreover, for an intra CU, the TB size cannot be larger than the PB size. In each PB, intra prediction is used to predict the samples of each TB within the PB from the adjacent reconstructed samples of the TB. For each PB, in addition to 33 directional intra prediction modes, DC and planar modes are also supported to predict flat areas and gradient areas, respectively.

[0055] For each inter PU, one of three prediction modes including inter, skip and merge can be selected. Generally speaking, a motion vector competition (MVC) scheme is introduced to select a motion candidate from a given candidate set including spatial and temporal motion candidates. Multiple references for motion estimation allow the best reference to be found in 2 possible reconstructed reference picture lists (i.e., list 0 and list 1). For this inter mode (called AMVP mode, where AMVP stands for advanced motion vector prediction), an inter prediction indicator (list 0, list 1 or bidirectional prediction), a reference index, a motion candidate index, a motion vector difference (MVD) and a prediction residual are generated. For skip mode and merge mode, only the merge index is sent, and the current PU inherits the inter prediction indicator, reference index and motion vector from the adjacent PU referenced by the encoded merge index. In the case of a skip-coded CU, the residual signal is also omitted.

[0056] Joint Exploration Test Model (JEM)

[0057] The Joint Exploratory Test Model (JEM) is built on the HEVC test model. The basic HEVC codec flow chart remains unchanged in JEM; however, the most important modules, including the block structure, intra-frame and inter-frame prediction, residual transform, loop filter, and entropy coding module design elements, have been modified, and additional coding tools have been added. The following new coding features are included in JEM.

[0058] In HEVC, a CTU is split into CUs using a quadtree structure represented as a coding tree to accommodate various local features. Whether to use inter-picture (temporal) prediction or intra-picture (spatial) prediction to encode a picture area is decided at the CU level. Depending on the PU split type, each CU can be further split into one, two, or four PUs. Within a PU, the same prediction process is applied, and relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU split type, the CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree for the CU. One of the key features of the HEVC structure is that it has multiple partition concepts, including CU, PU, ​​and TU.

[0059] Figure 3 is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to some embodiments of the present disclosure.

[0060] The QTBT structure removes the concept of multiple partition types, that is, the separation of the concepts of CU, PU, ​​and TU, and supports greater flexibility in the shape of CU partitions. In this QTBT block structure, a CU can have a square or rectangular shape. Figure 3As shown, the coding tree unit (CTU) is first partitioned by a quadtree structure. The quadtree tree leaf nodes can be further partitioned by a binary tree structure. There are two types of partitioning in the binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. The binary tree leaf nodes are called coding units (CUs), and the partitioning is used for prediction and transform processing without further sealing. This means that CU, PU and TU have the same block size in the QTBT coding block structure. In JEM, a CU sometimes consists of coding blocks (CBs) of different color components, for example, in the case of P and B slices of a 4:2:0 chroma format, a CU contains one luminance CB and two chroma CBs, and sometimes consists of CBs of a single component, for example, in the case of an I slice, a CU contains only one luminance CB or only two chroma CBs.

[0061] Define the following parameters for this QTBT partition scheme.

[0062] -CTUsize: The root node size of the quadtree, the same concept as in HEVC;

[0063] -MinQTSize: the minimum allowed quadtree leaf node size;

[0064] -MaxBTSize: The maximum allowed binary tree root node size;

[0065] -MaxBTDepth: Maximum allowed binary tree depth;

[0066] -MinBTSize: The minimum allowed binary tree node size.

[0067] In one example of a QTBT partitioning structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 chroma sample blocks (with a 4:2:0 chroma format), MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for width and height) is set to 4×4, and MaxBTDepth is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree child nodes. These quadtree child nodes can be sized from 16×16 (i.e., MinQTSize) to 128×128 (i.e., the CTU size). If the quadtree child node is 128×128, it cannot be further partitioned by the binary tree because the size exceeds MaxBTSize (i.e., 64×64). Otherwise, the quadtree child node can be further partitioned by the binary tree. Therefore, the quadtree child node is also the root node of the binary tree and has a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (i.e. 4), no further splitting is considered. When the width of the binary tree node is equal to MinBTSize (i.e. 4), no further horizontal splitting is considered. Similarly, when the height of the binary tree node is equal to MinBTSize, no further vertical splitting is considered. The leaf nodes of the binary tree are further predicted and transformed without further splitting. In JEM, the maximum CTU size is 256×256 luma samples.

[0068] An example of block partitioning using this QTBT scheme and the corresponding tree representation is given in Figure 3 As shown in the figure. The solid line represents the quadtree partition, and the dotted line represents the binary tree partition. Figure 3 As shown, a coding tree unit (CTU) 300 is first partitioned using a quadtree structure. Three of the four quadtree leaf nodes 302, 304, 306, and 308 are further partitioned using either a quadtree or binary tree structure. For example, quadtree leaf node 306 is further partitioned using quadtree partitioning; quadtree leaf node 304 is further partitioned into two leaf nodes 304a and 304b using binary tree partitioning; and quadtree leaf node 302 is also further partitioned using binary tree partitioning. In each partition (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which partition type (i.e., horizontal or vertical) is used, where 0 indicates horizontal partitioning and 1 indicates vertical partitioning. For example, for quadtree leaf node 304, 0 is signaled to indicate horizontal partitioning, and for quadtree leaf node 302, 1 is signaled to indicate vertical partitioning. For quadtree partitioning, specifying the partition type is not necessary, as quadtree partitioning always partitions the block horizontally and vertically to produce four equally sized sub-blocks.

[0069] In addition, the QTBT scheme supports the ability to have separate QTBT structures for luma and chroma. Currently, for P and B slices, the luma and chroma CTBs in a CTU share the same QTBT structure. However, for I slices, the luma CTBs are partitioned into CUs using the QTBT structure, and the chroma CTBs are partitioned into chroma CUs using another QTBT structure. This means that a CU in an I slice consists of coding blocks for the luma component or coding blocks for two chroma components, while a CU in a P or B slice consists of coding blocks for all three color components.

[0070] Versatile Video Codec (VVC)

[0071] At the Joint Video Experts Team (JVET) meeting, JVET defined the first draft of the Versatile Video Codec (VVC) and VVC Test Model 1 (VTM1) coding methods. It was decided to include a quadtree with a nested multi-type tree structure using binary and ternary partitioning of coding blocks as the initial new coding feature of VVC.

[0072] In VVC, the picture partitioning structure divides the input video into blocks called coding tree units (CTUs). A quadtree with a nested multi-type tree structure is used to split the CTU into coding units (CUs), which have leaf coding units (CUs) that define regions that share the same prediction mode (e.g., intra or inter). Here, the term "unit" defines an image region covering all components; the term "block" is used to define a region covering a specific component (e.g., luma), and may differ in spatial location when taking into account chroma sampling formats (e.g., 4:2:0).

[0073] Divide the image into CTUs

[0074] Figure 4 is a diagram illustrating an example of a picture divided into CTUs according to some embodiments of the present disclosure.

[0075] In VVC, a picture is divided into a series of CTUs, and the concept of CTU is the same as that of CTU in HEVC. For a picture with three sample arrays, a CTU consists of an N×N luma sample block and two corresponding chroma sample blocks. Figure 4 An example of a picture 400 divided into CTUs 402 is shown.

[0076] The maximum allowed size of a luma block in a CTU is specified as 128x128 (although the maximum size of a luma transform block is 64x64).

[0077] Use tree structure to split CTU

[0078] Figure 5is a schematic diagram illustrating a multi-type tree splitting mode according to some embodiments of the present disclosure.

[0079] In HEVC, a CTU is split into CUs using a quadtree structure represented as a coding tree to accommodate various local features. Whether to use inter-picture (temporal) prediction or intra-picture (spatial) prediction to encode a picture area is decided at the leaf CU level. Depending on the PU split type, each leaf CU can be further split into one, two, or four PUs. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU split type, the leaf CU can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree for CUs. One of the key features of the HEVC structure is that it has multiple partition concepts, including CU, PU, ​​and TU.

[0080] In VVC, the concept of multiple partition unit types is replaced by a quadtree with nested multi-type trees using binary and ternary split structures, that is, it removes the distinction between CU, PU, ​​and TU concepts, unless a CU with a size that is too large for the maximum transform length is required, and supports more flexibility for the CU partition shape. In the coding tree structure, the CU can have a square or rectangular shape. The coding tree unit (CTU) is first divided by a quadtree (i.e., quadtree) structure. The leaf nodes of the quadtree can then be further divided by a multi-type tree structure. As Figure 5 As shown, there are four types of splits in the multi-type tree structure: vertical binary split 502 (SPLIT_BT_VER), horizontal binary split 504 (SPLIT_BT_HOR), vertical ternary split 506 (SPLIT_TT_VER), and horizontal ternary split 508 (SPLIT_TT_HOR). The child nodes of the multi-type tree are called coding units (CUs), and this split is used for prediction and transform processing without further splitting unless the CU is too large for the maximum transform length. This means that in most cases, CUs, PUs, and TUs have the same block size in a quadtree with a nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the CU color component. In VTM1, a CU consists of coding blocks (CBs) of different color components, for example, a CU contains one luminance CB and two chrominance CBs (unless the video is monochrome, i.e., only one color component).

[0081] Split the CU into multiple prediction units

[0082] In VVC, for each CU partitioned based on the structure shown above, prediction of the block content can be performed on the entire CU block or in a sub-block manner as described in the following paragraphs. The operating unit of this prediction is called a prediction unit (or PU).

[0083] In the case of intra prediction, the size of the PU is usually equal to the size of the CU. In other words, the entire CU block is predicted. For inter prediction, the size of the PU can be equal to or smaller than the size of the CU. In other words, there are many cases where the CU is split into multiple PUs for prediction.

[0084] Some examples where the PU size is smaller than the CU size include affine prediction mode, advanced temporal motion vector prediction (ATMVP) mode, triangular prediction mode, and the like.

[0085] In affine prediction mode, a CU can be split into multiple 4×4 PUs for prediction. Motion vectors can be derived for each 4×4 PU and motion compensation can be performed accordingly. In ATMVP mode, a CU can be split into one or more 8×8 PUs for prediction. Motion vectors are derived for each 8×8 PU and motion compensation can be performed accordingly. In triangular prediction mode, a CU can be split into two triangular prediction units. Motion vectors are derived for each PU and motion compensation is performed accordingly. Inter-frame prediction supports triangular prediction mode. More details of triangular prediction mode are described below.

[0086] Triangular prediction model

[0087] Figure 6 FIG. 4 is a schematic diagram illustrating partitioning a CU into triangular prediction units according to some embodiments of the present disclosure.

[0088] The concept of triangular prediction mode is to introduce triangular partitioning for motion compensation prediction. Triangular prediction mode can also be called triangular prediction unit mode. Figure 6 As shown, CU 602 or 604 is partitioned into two triangular prediction units PU1 and PU2 in a diagonal or anti-diagonal direction (i.e., partitioned from the upper left corner to the lower right corner as shown in CU 602, or partitioned from the upper right corner to the lower left corner as shown in CU 604). Each triangular prediction unit in the CU uses its own unidirectional prediction motion vector and reference frame index derived from the unidirectional prediction candidate list to perform inter-frame prediction on the triangular prediction unit. After predicting these triangular prediction units, the diagonal edges are adaptively weighted. Then, the transformation and quantization process is applied to the entire CU. It should be noted that this mode is only applicable to the skip and merge modes in the current VVC. Although in Figure 4. Figure 6The CU is shown as a square block, but the triangular prediction mode can also be applied to non-square (ie rectangular) shaped CUs.

[0089] The unidirectional prediction candidate list may include one or more candidates, and each candidate may be a motion vector. Therefore, throughout this disclosure, the terms "unidirectional prediction candidate list," "unidirectional prediction motion vector candidate list," and "unidirectional prediction merge list" may be used interchangeably; and the terms "unidirectional prediction merge candidate" and "unidirectional prediction motion vector" may also be used interchangeably.

[0090] Unidirectional motion vector prediction candidate list

[0091] Figure 7 is a schematic diagram illustrating positions of adjacent blocks according to some embodiments of the present disclosure.

[0092] In some examples, the unidirectional prediction motion vector candidate list may include two to five unidirectional prediction motion vector candidates. In some other examples, it may be other numbers. It is derived from neighboring blocks. Figure 7 As shown, the unidirectional prediction motion vector candidate list is derived from seven neighboring blocks including five spatial neighboring blocks (1 to 5) and two temporal co-located blocks (6 to 7). The motion vectors of the seven neighboring blocks are collected into a first merge list. Then, a unidirectional prediction candidate list is formed based on the first merge list motion vectors according to a specific order. Based on the order, the unidirectional prediction motion vector from the first merge list is first placed in the unidirectional prediction motion vector candidate list, followed by the reference picture list 0 or L0 motion vector of the bidirectional prediction motion vector, then the reference picture list 1 or L1 motion vector of the bidirectional prediction motion vector, and then the average motion vector of the L0 and L1 motion vectors of the bidirectional prediction motion vector. At this point, if the number of candidates is still less than the target number (which can be 5), a zero motion vector is added to the list to meet the target number.

[0093] For each triangle PU, a prediction value is derived based on its motion vector. It is important to note that the derived prediction value covers a larger area than the actual triangle PU, so that there is an overlap of the two prediction values ​​along the shared diagonal edge of two triangle PUs. A weighting process is applied to the diagonal edge area of ​​these two predictor values ​​to derive the final prediction for that CU. The current weighting factors used for luma and chroma samples are {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} respectively.

[0094] Triangular prediction mode syntax and signaling

[0095] Triangular prediction mode is signaled using the triangular prediction flag. This flag is signaled when a CU is encoded or decoded in skip or merge mode. For a given CU, if the triangular prediction flag is set to 1, the CU is encoded or decoded using triangular prediction mode. Otherwise, the CU is encoded or decoded using a prediction mode other than triangular prediction mode.

[0096] For example, the triangular prediction flag is conditionally signaled in skip mode or merge mode. First, the triangular prediction tool enable / disable flag is signaled in the sequence parameter set (or SPS). The triangular prediction flag is signaled at the CU level only when this triangular prediction tool enable / disable flag is true. Second, the triangular prediction tool is allowed only in B slices. Therefore, the triangular prediction flag is signaled at the CU level only in B slices. Third, the triangular prediction mode is signaled only for CUs whose size is equal to or greater than a certain threshold. If the CU has a size smaller than the threshold, the triangular prediction flag is not signaled. Fourth, the triangular prediction flag is signaled only for the following CUs: this CU is not encoded and decoded in the sub-block merge mode including both affine prediction mode and ATMVP mode. In the four cases listed above, if the triangular prediction flag is not signaled, it is inferred to be 0 on the decoder side.

[0097] When the triangular prediction flag is signaled, the entropy codec using context-adaptive binary arithmetic coding (CABAC) uses a specific context to signal the triangular prediction flag. These contexts are formed based on the triangular prediction flag values ​​of the top block and the left block of the current CU.

[0098] In order to encode or decode the triangular prediction flag of the current block (or current CU), the triangular prediction flags from the top block and the left block (or the top CU and the left CU) are derived and their values ​​are added. This results in three possible contexts corresponding to the following situations:

[0099] 1) Both the left block and the top block have triangular prediction flag 0;

[0100] 2) Both the left block and the top block have triangular prediction flag 1;

[0101] 3)Others.

[0102] A separate probability is maintained for each of the three contexts.Once the context value is determined for the current block, the triangular prediction flag of the current block is encoded and decoded using the probability model of CABAC corresponding to the context value.

[0103] If the triangle prediction flag is true, the triangle partition orientation flag is signaled to indicate whether the partition is oriented from top left to bottom right or from top right to bottom left.

[0104] In the case where triangular prediction mode is used for a CU, two index values ​​are signaled to indicate the index values ​​of the first and second unidirectional prediction merge candidates for triangular prediction, respectively. These two index values ​​are used to locate the two merge candidates for the first and second partitions from the above-mentioned unidirectional prediction motion vector candidate list. For triangular prediction, these two index values ​​are required to be different so that the two prediction values ​​of the two triangular partitions are different. Therefore, the first index value is directly signaled. In order to signal the second index value, if it is less than the first index value, its value is directly signaled. Otherwise, its value is subtracted by 1 before being signaled to the decoder. On the decoder side, the first index is decoded and used directly. In order to decode the second index value, the value represented as "idx" is first decoded from the CABAC engine. If idx is less than the first index value, the second index value will be equal to the value of idx. Otherwise, the second index value will be equal to (idx+1).

[0105] Geometry Merge Mode

[0106] In the geometric merge mode (ie, GEO), the CU can be divided into two prediction units (ie, geometric PUs) according to the geometric partitioning. Figure 8 Some examples of geometric partitioning according to some embodiments of the present disclosure are illustrated. Figure 8 As shown, a CU can be partitioned into two PUs along a line that does not completely pass through a diagonal, and the two PUs can have geometric shapes such as triangles, wedges, or trapezoids. Geometric merge mode can be considered an extension of triangle prediction mode, or a superset of triangle prediction mode.

[0107] Figure 9 is a schematic diagram showing an example of a geometric partition representation according to some embodiments of the present disclosure. Figure 9 As shown in

[15] , given a GEO partition, the partition identification requires an angle α and a displacement of the partition line relative to the center of the block, denoted as ρ. Here, α represents a quantized angle and is between 0 and 360 degrees.

[0108] Similar to triangular prediction mode, in GEO partitioning, only unidirectional prediction is allowed for each of the two GEO partitions so that the memory bandwidth requirements in this mode do not exceed the memory bandwidth requirements of the bidirectional prediction block at the decoder side. The motion vector prediction of the GEO partition is aligned with the triangular prediction mode. Similarly, similar to triangular prediction mode, the weighted average-based blending operation can be applied to the inner boundary between the two partitions in geometric merge mode. Therefore, the techniques related to motion compensated prediction using triangular prediction units can also be applied to motion compensated prediction using geometric prediction units.

[0109] Normal merge mode motion vector candidate list

[0110] According to the current VVC, in a normal merge mode in which an entire CU is predicted without being divided into more than one PU, a motion vector candidate list or a merge candidate list is constructed using a different process from that of the triangular prediction mode.

[0111] First, based on the motion vectors from neighboring blocks, a spatial motion vector candidate is selected, such as Figure 10 As shown, Figure 10 Schematic diagram showing the location of spatial merging candidates according to some embodiments of the present disclosure. In the derivation of spatial merging candidates for the current block 1002, in the case where Figure 10 A maximum of four merge candidates are selected from the candidates at the indicated positions. These candidates are selected in a specific order. An exemplary derivation order is A1 → B1 → B0 → A0 → (B2). Position B2 is only considered if any PU at positions A1, B1, B0, A0 is unavailable or intra-coded. Note that other different orders can also be used.

[0112] Next, a temporal merge candidate is derived. In the derivation of the temporal merge candidate, a scaled motion vector is derived based on the co-located PU belonging to the picture with the smallest picture order count (POC) difference with the current picture in a given reference picture list. The reference picture list used to derive the co-located PU is explicitly signaled in the slice header. The scaled motion vector for the temporal merge candidate is obtained as Figure 11 As shown by the dotted line in Figure 11Motion vector scaling for a temporal merge candidate according to some implementations of the present disclosure is shown. The scaled motion vector for the temporal merge candidate is scaled from the motion vector of the co-located PU col_PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture curr_ref of the current picture and the current picture curr_pic, and td is defined as the POC difference between the reference picture col_ref of the co-located picture and the co-located picture col_pic. The reference picture index of the temporal merge candidate is set to zero. The actual implementation of this scaling process is described in the draft HEVC specification. For B slices, two motion vectors are obtained and combined, one for reference picture list 0 and the other for reference picture list 1, to form a bi-predictive merge candidate.

[0113] Figure 12 is a schematic diagram illustrating candidate positions of temporal merging candidates according to some embodiments of the present disclosure.

[0114] Select the same PU position between the two candidate positions C3 and H, such as Figure 12 If the PU at position H is not available, or is intra-coded, or is outside the current CTU, position C3 is used to derive the temporal merge candidate. Otherwise, position H is used to derive the temporal merge candidate.

[0115] After inserting the spatial and temporal motion vectors into the merge candidate list as described above, history-based merge candidates are added. These so-called history-based merge candidates include motion vectors from previously coded CUs, which are stored in a separate motion vector list and managed based on certain rules.

[0116] After inserting the history-based candidates, if the merge candidate list is not full, pairwise average motion vector candidates are further added to the list. As the name suggests, this type of candidate is constructed by averaging the candidates already in the current list. More specifically, based on a certain order or rule, two candidates are taken from the merge candidate list at a time and their average motion vector is appended to the current list.

[0117] After inserting the pairwise average motion vectors, if the merge candidate list is still not full, zero motion vectors will be added to make the list full.

[0118] The first merge list for triangular prediction is constructed using the conventional merge list construction process.

[0119] The triangular prediction mode in current VVC shares some similarities with the conventional merge prediction mode in the overall process of forming the prediction value. For example, in both prediction modes, it is necessary to construct a merge list based on at least the neighboring spatial motion vectors and the co-located motion vectors of the current CU. At the same time, triangular prediction mode also has some differences from the conventional merge prediction mode.

[0120] For example, although a merge list needs to be constructed in both triangular prediction mode and normal merge prediction mode, the detailed process of obtaining such a list is different.

[0121] These differences impose additional cost on the codec implementation due to the need for additional logic.The process and logic for building the merge list can be unified and shared between the triangular prediction mode and the regular merge prediction mode.

[0122] In some examples, when forming a uni-prediction merge list for triangular prediction mode, the new motion vector is completely pruned against those already in the merge list before being added to the merge list. In other words, the new motion vector is compared with every motion vector already in the uni-prediction merge list and is added to the list only if it is different from every motion vector in the merge list. Otherwise, the new motion vector is not added to the list.

[0123] According to some examples of the present disclosure, in triangular prediction mode, a uni-directional prediction merge list may be obtained or constructed from a normal merge mode motion vector candidate list (which may be referred to as a normal merge list).

[0124] More specifically, to construct a merge candidate list for triangular prediction mode, a first merge list is first constructed based on the merge list construction process for conventional merge prediction. The first merge list includes multiple candidates, each of which is a motion vector. The motion vectors in the first merge list are then used to further construct or derive a unidirectional prediction merge list for triangular prediction mode.

[0125] It should be noted that the first merge list constructed in this case may have a list size that is different from the list size of the general merge mode or the regular merge mode. In one example of the present disclosure, the first merge list has the same size as the size of the general merge mode. In another example of the present disclosure, the first merge list constructed has a list size that is different from the list size of the general merge mode.

[0126] Build a one-way prediction merge list from the first merge list

[0127] According to some examples of the present disclosure, a unidirectional prediction merge list for triangular prediction mode may be constructed from the first merge list based on one of the following methods.

[0128] In an example of the present disclosure, to construct a unidirectional prediction merge list, the candidate prediction list 0 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is not full after this process (for example, the number of candidates in the list is still less than the target number), the candidate prediction list 1 motion vectors in the first merge list are checked and selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is still not full, the prediction list 0 zero vector is added to the unidirectional prediction merge list. If the unidirectional prediction merge list is still not full, the prediction list 1 zero vector is added to the unidirectional prediction merge list.

[0129] In another example of the present disclosure, for each candidate in the first merge list, its prediction list 0 motion vector and prediction list 1 motion vector are added to the unidirectional prediction merge list in an interleaved manner. More specifically, for each candidate in the first merge list, if the candidate is a unidirectional prediction motion vector, it is added directly to the unidirectional prediction merge list. Otherwise, if the candidate is a bidirectional prediction motion vector in the first merge list, its prediction list 0 motion vector is first added to the unidirectional prediction merge list, followed by its prediction list 1 motion vector. Once all motion vector candidates in the first merge list have been checked and added, but the unidirectional prediction merge list is not yet full, a unidirectional prediction zero motion vector can be added. For example, for each reference frame index, a prediction list 0 zero motion vector and a prediction list 1 zero motion vector can be added to the unidirectional prediction merge list respectively until the list is full.

[0130] In another example of the present disclosure, the unidirectional prediction motion vectors from the first merge list are first selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is not full after this process, then for each bidirectional prediction motion vector in the first merge list, its prediction list 0 motion vector is first added to the unidirectional prediction merge list, followed by its prediction list 1 motion vector. After this process, if the unidirectional prediction merge list is not full, a unidirectional prediction zero motion vector can be added. For example, for each reference frame index, a prediction list 0 zero motion vector and a prediction list 1 zero motion vector can be added to the unidirectional prediction merge list respectively until the list is full.

[0131] In the above description, when a unidirectional prediction motion vector is added to a unidirectional prediction merge list, a motion vector pruning process may be performed to ensure that the new motion vector to be added is different from those already in the unidirectional prediction merge list. For lower complexity, such a motion vector pruning process may also be performed in a partial manner, for example, by only checking the new motion vector to be added against some but not all motion vectors already in the unidirectional prediction merge list. In an extreme case, motion vector pruning (i.e., motion vector comparison operations) is not performed in this process.

[0132] Construct a unidirectional prediction merge list from the first merge list based on the picture prediction configuration

[0133] In some examples of the present disclosure, a unidirectional prediction merge list can be adaptively constructed based on whether the current picture uses backward prediction. For example, different methods can be used to construct the unidirectional prediction merge list depending on whether the current picture uses backward prediction. If the picture order count (POC) values ​​of all reference pictures are not greater than the POC value of the current picture, it means that the current picture does not use backward prediction.

[0134] In an example of the present disclosure, when the current picture does not use backward prediction, or when it is determined that the current picture does not use backward prediction, the candidate prediction list 0 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list, followed by those candidate prediction list 1 motion vectors; and if the unidirectional prediction merge list is still not full, a unidirectional prediction zero motion vector can be added. Otherwise, if the current picture uses backward prediction, each candidate prediction list 0 and list 1 motion vector in the first merge list can be checked and selected into the unidirectional prediction merge list in an interleaved manner as described above, that is, the first candidate prediction list 0 motion vector in the first merge list is added, followed by the first candidate prediction list 1 motion vector, and then the second candidate prediction list 0 motion vector is added, followed by the second candidate prediction list 1 motion vector, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, a unidirectional prediction zero vector can be added.

[0135] In another example of the present disclosure, if the current picture does not use backward prediction, the candidate prediction list 1 motion vectors in the first merge list are first checked and selected into the unidirectional prediction merge list, followed by those candidate prediction list 0 motion vectors; and if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector can be added. Otherwise, if the current picture uses backward prediction, the prediction list 0 and list 1 motion vectors of each candidate in the first merge list can be checked and selected into the unidirectional prediction merge list in an interleaved manner as described above, that is, the prediction list 0 motion vector of the first candidate in the first merge list is added, followed by the prediction list 1 motion vector of the first candidate, and then the prediction list 0 motion vector of the second candidate is added, followed by the prediction list 1 motion vector of the second candidate, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector can be added.

[0136] In another example of the present disclosure, if the current picture does not use backward prediction, first only the prediction list 0 motion vectors of the candidates in the first merge list are checked and selected into the unidirectional prediction merge list, and if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector can be added. Otherwise, if the current picture uses backward prediction, the prediction list 0 and list 1 motion vectors of each candidate in the first merge list can be checked and selected into the unidirectional prediction merge list in an interlaced manner as described above, that is, the prediction list 0 motion vector of the first candidate in the first merge list is added, followed by the prediction list 1 motion vector of the first candidate, and then the prediction list 0 motion vector of the second candidate is added, followed by the prediction list 1 motion vector of the second candidate, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector can be added.

[0137] In another example of the present disclosure, if the current picture does not use backward prediction, first only the prediction list 1 motion vectors of the candidates in the first merge list are checked and selected into the unidirectional prediction merge list, and if the unidirectional prediction merge list is still not full, the unidirectional prediction zero motion vector can be added. Otherwise, if the current picture uses backward prediction, the prediction list 0 and list 1 motion vectors of each candidate in the first merge list can be checked and selected into the unidirectional prediction merge list in an interlaced manner as described above, that is, the prediction list 0 motion vector of the first candidate in the first merge list is added, followed by the prediction list 1 motion vector of the first candidate, and then the prediction list 0 motion vector of the second candidate is added, followed by the prediction list 1 motion vector of the second candidate, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, the unidirectional prediction zero vector can be added.

[0138] In the above description, a unidirectional prediction merge list for triangular prediction is constructed by selecting motion vectors from the first merge list into the unidirectional prediction merge list. However, in practice, the method can be implemented in different ways, where the unidirectional prediction merge list is physically formed or the unidirectional prediction (or uniprediction) merge list is non-physically formed. In some examples, the first merge list can be used directly instead of physically creating a unidirectional prediction merge list. For example, the list 0 and / or list 1 motion vectors of each candidate in the first merge list can be simply indexed based on a specific order and accessed directly from the first merge list. That is, after constructing a first merge list including multiple candidates (each candidate is one or more motion vectors) based on the merge list construction process for conventional merge prediction, a unidirectional prediction merge list is not constructed, but an index list including multiple reference indexes (each reference index is a reference to the candidate motion vector in the first merge list) is constructed. It should be noted that such an index order can follow the selection order described in the above examples of constructing unidirectional prediction merge lists.

[0139] In one example of the present disclosure, when the current picture does not use backward prediction, the prediction list 0 motion vectors of the candidates in the first merge list are used as unidirectional prediction merge candidates and are indexed in the same index order as their index order in the first merge list. That is, when it is determined that the POC of the current picture is greater than each POC of the reference picture, the reference indexes are arranged in the same order as the list 0 motion vectors of the candidates in the first merge list. Otherwise, if the current picture uses backward prediction, the list 0 and list 1 motion vectors of each candidate in the first merge list are used as unidirectional prediction merge candidates and are indexed based on an interleaved manner, that is, the list 0 motion vector is the first candidate in the first merge list, followed by the list 1 motion vector of the first candidate, then the list 0 motion vector of the second candidate, then the list 1 motion vector of the second candidate, and so on. That is, when it is determined that the POC of the current picture is less than the POC of at least one reference picture, the reference indexes are arranged in an interleaved manner according to the list 0 and list 1 motion vectors of each candidate in the first merge list, where the candidate is a bidirectional prediction motion vector. In the case where the candidate in the first merge list is a unidirectional motion vector, the zero motion vector is indexed as the unidirectional prediction merge candidate after the motion vector of the candidate. This ensures that for the case where backward prediction is used for the current picture, each candidate in the first merge list, whether it is a bidirectional prediction motion vector or a unidirectional prediction motion vector, provides two unidirectional motion vectors as unidirectional prediction merge candidates.

[0140] In another example of the present disclosure, when the current picture does not use backward prediction, the prediction list 0 motion vectors of the candidates in the first merge list are used as unidirectional prediction merge candidates and are indexed in the same index order as their index order in the first merge list. Otherwise, if the current picture uses backward prediction, the list 0 and list 1 motion vectors of each candidate in the first merge list are used as unidirectional prediction merge candidates and are indexed based on the staggered manner as described above, that is, the list 0 motion vector of the first candidate in the first merge list is followed by the list 1 motion vector of the first candidate, and then the list 0 motion vector of the second candidate is followed by the list 1 motion vector of the second candidate, and so on. In the case where the candidate in the first merge list is a unidirectional motion vector, the same motion vector plus a specific motion offset is indexed after the candidate in the unidirectional prediction merge list.

[0141] In the above process, when checking for new motion vectors to be added to the uni-predictive merge list, pruning can be performed completely or partially. When pruning is performed partially, it means that the new motion vector is compared with some, but not all, of the motion vectors already in the uni-predictive merge list. In extreme cases, motion vector pruning (i.e., motion vector comparison operations) are not performed in this process.

[0142] When forming the uni-prediction merge list, motion vector pruning may also be adaptively performed based on whether the current picture uses backward prediction. For example, with respect to the examples of this disclosure related to index list construction based on picture prediction configuration, when the current picture does not use backward prediction, motion vector pruning may be performed in whole or in part. When the current picture uses backward prediction, motion vector pruning may not be performed.

[0143] Select unidirectional prediction merge candidates for triangular prediction mode

[0144] In addition to the above examples, other methods of constructing a unidirectional prediction merging list or selecting unidirectional prediction merging candidates are also disclosed.

[0145] In one example of the present disclosure, once the first merge list for normal merge mode is constructed, unidirectional prediction merge candidates for triangular prediction may be selected according to the following rules:

[0146] For the motion vector candidate in the first merge list, one and only one of the list 0 motion vector or list 1 motion vector is used for triangular prediction;

[0147] for a given motion vector candidate in the first merge list, if its merge index value in the list is even, its list 0 motion vector is used for triangular prediction (if its list 0 motion vector is available), and if the motion vector candidate does not have a list 1 motion vector, its list 0 motion vector is used for triangular prediction; and

[0148] For a given motion vector candidate in the first merge list, if its merge index value in the list is odd, its list 1 motion vector is used for triangular prediction (if its list 1 motion vector is available), and if the motion vector candidate does not have a list 1 motion vector, its list 0 motion vector is used for triangular prediction.

[0149] Figure 13A is a schematic diagram showing an example of unidirectional prediction motion vector (MV) selection (or unidirectional prediction merge candidate selection) for triangular prediction mode according to some embodiments of the present disclosure. In this example, the first N (for example, N is equal to 5) merged MV candidates derived from the first merge list are indexed from 0 to (N-1); each row has two columns, representing the list 0 and list 1 motion vectors for the candidates in the first merge list. Each candidate in the list can be unidirectionally predicted or bidirectionally predicted. For a unidirectionally predicted candidate, it has only one list 0 or one list 1 motion vector, but not both. For a bidirectionally predicted candidate, it has both list 0 and list 1 motion vectors. In Figure 13A In , for each merge index, the motion vectors marked with "x" are the motion vectors used first for triangular prediction (if they are available). If the motion vectors marked with "x" are not available, the unmarked motion vectors corresponding to the same merge index are used for triangular prediction.

[0150] The above concepts can be extended to other examples. Figure 13B FIG. 1 is a diagram illustrating another example of unidirectional prediction motion vector (MV) selection for triangular prediction mode according to some embodiments of the present disclosure. Figure 13B , the rules for selecting unidirectional prediction merging candidates for triangular prediction are as follows:

[0151] For the motion vector candidate in the first merge list, one and only one of the list 0 motion vector or list 1 motion vector is used for triangular prediction;

[0152] for a given motion vector candidate in the first merge list, if its merge index value in the list is even, its list 1 motion vector is used for triangular prediction (if its list 1 motion vector is available), and if the motion vector candidate does not have a list 1 motion vector, its list 0 motion vector is used for triangular prediction; and

[0153] For a given motion vector candidate in the first merge list, if its merge index value in the list is odd, its list 0 motion vector is used for triangular prediction (if its list 0 motion vector is available), and if the motion vector candidate does not have a list 0 motion vector, its list 1 motion vector is used for triangular prediction.

[0154] In some examples, other different orders can be defined and used to select unidirectional prediction merge candidates for triangular prediction from those motion vector candidates in the first merge list. More specifically, for a given motion vector candidate in the first merge list, the decision of whether to use its list 0 motion vector or list 1 motion vector first (when available for triangular prediction) does not have to depend on the parity of the candidate's index value in the first merge list as described above. For example, the following rule can also be used:

[0155] For the motion vector candidate in the first merge list, one and only one of the list 0 motion vector or list 1 motion vector is used for triangular prediction;

[0156] Based on a predefined pattern, for multiple motion vector candidates in the first merge list, their list 0 motion vectors are used for triangular prediction (if available), and if the list 0 motion vector does not exist, the corresponding list 1 motion vector is used for triangular prediction; and

[0157] Based on the same predefined pattern, for the remaining motion vector candidates in the first merge list, their list 1 motion vectors are used for triangular prediction (if available), and in the absence of a list 1 motion vector, the corresponding list 0 motion vector is used for triangular prediction.

[0158] 14A to 14D Some examples of predefined modes in unidirectional prediction motion vector (MV) selection for triangular prediction mode according to some embodiments of the present disclosure are shown. For each merge index, the motion vectors marked with "x" are those that are first used for triangular prediction (if they are available). If the motion vector marked with "x" is not available, then the unmarked motion vector corresponding to the same merge index is used for triangular prediction.

[0159] exist Figure 14A In

[15] , for the first three motion vector candidates in the first merge list, their list 0 motion vectors are checked first. Only if the list 0 motion vector is not available, the corresponding list 1 motion vector is used for triangular prediction. For the fourth and fifth motion vector candidates in the first merge list, their list 1 motion vectors are checked first. Only if the list 1 motion vector is not available, the corresponding list 0 motion vector is used for triangular prediction. 14B to 14D Three other modes of selecting unidirectional prediction merge candidates from the first merge list are shown. The examples shown in the figure are not limiting, and further examples exist. For example, 14A to 14D Horizontally and / or vertically mirrored versions of those patterns shown in .

[0160] The concepts described in some examples may be used in combination with the concepts described in other examples of the present disclosure. According to one example of the present disclosure, for a given motion vector candidate in the first merge list, when its list 0 or list 1 motion vector is available for triangular and / or geometric prediction, the decision of whether to use its list 0 or list 1 motion vector first may also depend on whether the current picture and / or slice uses backward prediction. FIG. 15A to FIG. 15B FIG is a diagram illustrating an example of unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure. Figure 15A As shown, if the current picture and / or slice uses backward prediction, the selection order between the candidate list 0 and list 1 motion vectors is based on the parity of the candidate index values ​​in the first merge list. If the current picture and / or slice does not use backward prediction, the motion vector of the candidate list x (x is 0 or 1) is preferred, and if the motion vector is available, this motion vector is selected first. In the example where x is equal to 0, Figure 15B . Similar to the other examples explained earlier, if a unidirectional prediction motion vector is not available for a candidate in the first merge list, the corresponding unidirectional prediction motion vector of the same candidate is selected from another reference list. Thus, for each candidate in the first merge list, a unidirectional prediction motion vector can be obtained and used for triangular and / or geometric prediction mode. The obtained unidirectional prediction motion vector shares the same index value as the merge candidate in the first merge list.

[0161] Flexible merge list construction and candidate index signaling for triangular prediction mode

[0162] In one example of the present disclosure, once the first merge list for normal merge mode is constructed, the unidirectional prediction motion vector is directly selected from the list for triangular prediction. In order to indicate a certain list 0 or list 1 motion vector for triangular prediction, an index value is first signaled to indicate which candidate is selected from the first merge list. Then, a binary reference list indication flag (called L0L1_flag) is signaled to indicate whether the list 0 or list 1 motion vector of the candidate selected from the first merge list is selected for the first partition of triangular prediction. The same signaling method is used to indicate the second list 0 or list 1 motion vector of the second partition to be used for triangular prediction. For example, the syntax of the CU for triangular mode encoding and decoding signaled may include index1, L0L1_flag1, index2, L0L1_flag2. Here, index1 and index2 are the merge index values ​​of the two candidates selected from the first merge list, used for the first and second partitions, respectively. Index1 and index2 are used to locate the candidates for the first partition and the second partition, respectively (i.e., the first candidate and the second candidate). L0L1_flag1 is a binary flag for the first partition that indicates whether list 0 or list 1 motion vectors based on the selected candidate at index 1 from the first merge list are selected. L0L1_flag2 is a binary flag for the second partition that indicates whether list 0 or list 1 motion vectors based on the selected candidate at index 2 from the first merge list are selected. The unidirectional prediction MV for the first partition is obtained by selecting list X1MV of the first candidate according to the first binary reference list indication flag L0L1_flag1, where X1 takes the value 0 or 1. The unidirectional prediction MV for the second partition is obtained by selecting list X2MV of the second candidate according to the second binary reference list indication flag L0L1_flag2, where X2 takes the value 0 or 1.

[0163] Figure 16 FIG is a schematic diagram illustrating an example of flexible unidirectional prediction MV selection for triangular prediction mode according to some embodiments of the present disclosure. Figure 16As shown, each List 0 and / or List 1 motion vector indicated by the symbol "x" in the rectangular box can be indicated / signaled to the decoder for deriving the prediction of the first partition, and each List 0 and / or List 1 motion vector indicated by the symbol "x" in the rectangular box can be indicated / signaled to the decoder for deriving the prediction of the second partition in triangular prediction mode. Therefore, the selection of unidirectional prediction motion vectors from the first merge list becomes very flexible. Given a first merge list with N candidate sizes, up to 2N unidirectional prediction motion vectors can be used for each of the two triangular partitions. The two merge index values ​​of the two partitions in triangular prediction mode do not have to be different from each other. In other words, they may take the same value. These index values ​​can be directly signaled without any adjustment before signaling. More specifically, unlike what is defined in the current VVC, the second index value is directly signaled to the decoder without any adjustment to the value before signaling.

[0164] In another example of the present disclosure, when the two index values ​​are the same, the binary flag L0L1_flag2 of the second partition does not need to be signaled. Instead, it is inferred to have the opposite value relative to the binary flag L0L1_flag1 of the first partition. In other words, in this case, L0L1_flag2 can take the value (1-L0L1_flag1).

[0165] In another example of the present disclosure, these binary reference list indicator flags, namely L0L1_flag1 and L0L1_flag2, can be encoded and decoded as CABAC context bins. The context for L0L1_flag1 can be separate from the context for L0L1_flag2. The CABAC probability under each context can be initialized at the beginning of the video sequence and / or the beginning of the picture and / or the beginning of the tile group. In this example, different context modeling methods can be used to encode and decode these two flags. In one method, the context model selection for the second flag can depend on the value of the first flag. In other words, at least two context models can be used to encode and decode the second flag, and the context model is selected based on the signaled value of the first flag. In another method, the CABAC probability under each context model can be initialized differently depending on whether the current picture uses backward prediction. If the picture uses backward prediction, the probability for this flag can be initialized higher to indicate a certain list, such as list 0. In another method, the first flag can be encoded and decoded as a bypass bin, and only the second flag can be encoded and decoded as a context bin. The methods mentioned here can be used individually or in combination.

[0166] In yet another example of the present disclosure, when a motion vector indicated by a merge index value and an associated binary reference list indication flag (i.e., L0L1_flag) does not exist, a unidirectionally predicted zero motion vector may be used instead. That is, when it is determined that the first candidate list X1MV does not exist, the unidirectionally predicted zero MV is selected as the unidirectionally predicted MV for the first partition; and / or when it is determined that the second candidate list X2MV does not exist, the unidirectionally predicted zero MV is selected as the unidirectionally predicted MV for the second partition.

[0167] In another example of the present disclosure, when a motion vector indicated by a merge index value and an associated L0L1_flag does not exist, a corresponding motion vector indicated by the same merge index value from another list, namely, List(1-L0L1_flag), may be used instead. That is, when it is determined that the first candidate list X1MV does not exist, the unidirectional prediction MV for the first partition is obtained by selecting the first candidate list (1-X1)MV; and / or, when it is determined that the second candidate list X2MV does not exist, the unidirectional prediction MV for the second partition is obtained by selecting the second candidate list (1-X2)MV.

[0168] In another example of the present disclosure, for a CU encoded in triangle mode, the second L0L1_flag (i.e., L0L1_flag2) associated with the second index (i.e., index2) is not signaled, but is always inferred. In this case, the index1, L0L1_flag1, and index2 syntax still need to be signaled. In one approach, L0L1_flag2 is inferred based on the value of L0L1_flag1 and whether the current picture uses backward prediction. More specifically, for a CU encoded in triangle mode, if the current picture uses backward prediction, the value of L0L1_flag2 is inferred to be the opposite binary value of L0L1_flag1 (i.e., 1-L0L1_flag1); if the current picture does not use backward prediction, the value of L0L1_flag2 is inferred to be the same as L0L1_flag1. In addition, if the current picture does not use backward prediction, the value of index2 can be further forced to be different from the value of index1 because both motion vectors (one for each triangle partition) come from the same prediction list. If the value of index2 is equal to index1, it means that the same motion vector will be used for these two triangular partitions, which is not useful from the perspective of codec efficiency. In this case, when the value of index2 is signaled, the value of index2 can be adjusted accordingly before the index is binarized, which is the same as the adjustment for index2 signaling in the current VVC design. For example, when the actual value of index1 is less than the actual value of index2, the corresponding (index2-1) CABAC binarization codeword is used to indicate the value of index2; otherwise, the CABAC binarization codeword corresponding to index2 is used to signal the value of index2. Based on this example of the present disclosure, optionally, forcing index2 to have a value different from index1 can also be applied to the case where the current picture uses backward prediction, as well as the adjustment of the same index2 value for CABAC binarization.

[0169] In yet another example of the present disclosure, for a CU encoded in triangular mode, none of the L0L1_flags are signaled. Instead, they are both inferred. In this case, the index1 and index2 syntaxes still need to be signaled, representing the two candidate merge index values ​​selected from the first merge list for the first and second partitions, respectively. Given a merge candidate index value, some method can be defined or used to determine whether to select the list 0 motion vector or the list 1 motion vector of the corresponding merge candidate from the first list for triangular mode prediction. In one method, for index1, shown in Figure 13AThe mode is used to determine from which prediction list the motion vector of the merge candidate is selected for triangular mode prediction; and, for index2, Figure 13B The pattern shown in is used to determine from which prediction list the motion vector of the merged candidate is selected for triangular mode prediction. In other words, if index1 is an even value, the candidate list 0 motion vector indicated by index1 is selected, and if index1 is an odd value, the candidate list 1 motion vector indicated by index1 is selected. For index2, if it is an even value, the list 1 motion vector is selected, and if it is an odd value, the list 0 motion vector is selected. In the case where there is no motion vector corresponding to a certain prediction list, a default motion vector can be used instead, such as a zero motion vector or a corresponding motion vector from another prediction list, and so on. When determining from which prediction list the motion vector of the merged candidate is selected for triangular mode prediction, the pattern shown in Figure 13B The pattern in is available for index1, and Figure 13A The pattern shown in can be used for index2. That is, L0L1_flags can be determined based on the values ​​of index1 and index2.

[0170] Although the method in this disclosure is described using the triangular prediction mode as an example, since the geometric merge mode can be considered as an extension or superset of the triangular prediction mode, the method described in this disclosure is naturally applicable to the geometric merge mode. All the merge list construction methods described in this disclosure can be naturally used in the geometric merge mode without any modification. Similarly, all the merge index signaling methods described in this disclosure can be naturally used in the geometric merge mode without any modification. In one example, according to Figure 13A The same merge list construction process is used for the geometric merge mode. In another example, according to Figure 13B The same merge list construction process is used for the geometric merge mode. Moreover, it is worth mentioning that if there are triangle prediction mode and geometric merge mode in VVC, the same merge list construction process can be shared and used for both the triangle prediction mode and the geometric merge mode.

[0171] It should be noted that although a first merge list containing 5 merge candidates is used in some examples of the present disclosure, the size of the first merge list can actually be defined differently, such as 6 or 4, or some other value. The methods described in these examples are applicable to the case where the first merge list has a size other than 5.

[0172] In the above process, motion vector pruning may also be performed. This pruning may be done completely or partially. When it is performed partially, it means that the new motion vector is compared with some but not all motion vectors already in the uni-directional prediction merge list. This may also mean that only some but not all new motion vectors need to be checked for pruning before being used as merge candidates for triangular prediction. A specific example is that before the second motion vector is used as a merge candidate for triangular prediction, only the second motion vector is checked against the first motion vector for pruning, while all other motion vectors are not checked for pruning. In extreme cases, motion vector pruning (i.e., motion vector comparison operations) is not performed in this process.

[0173] Figure 17 1700 is a block diagram illustrating an apparatus for video encoding and decoding according to some embodiments of the present disclosure. Apparatus 1700 may be a terminal, such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

[0174] like Figure 17 As shown in , device 1700 may include one or more of the following components: a processing component 1702, a memory 1704, a power component 1706, a multimedia component 1708, an audio component 1710, an input / output (I / O) interface 1712, a sensor component 1714, and a communication component 1716.

[0175] Processing component 1702 generally controls the overall operation of device 1700, such as operations related to display, phone calls, data communications, camera operation, and recording operation. Processing component 1702 may include one or more processors 1720 for executing instructions to perform all or part of the steps of the above-described method. Furthermore, processing component 1702 may include one or more modules to facilitate interaction between processing component 1702 and other components. For example, processing component 1702 may include a multimedia module to facilitate interaction between multimedia component 1708 and processing component 1702.

[0176] The memory 1704 is configured to store different types of data to support the operation of the device 1700. Examples of such data include instructions for any application or method operating on the device 1700, contact data, phone book data, messages, pictures, videos, etc. The memory 1704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the memory 1704 can be static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0177] The power supply component 1706 provides power to the various components of the device 1700. The power supply component 1706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1700.

[0178] Multimedia component 1708 includes a screen that provides an output interface between device 1700 and a user. In some examples, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen that receives input signals from the user. The touch panel may include one or more touch sensors for sensing touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch action or a slide action, but can also detect the duration and pressure associated with the touch or slide operation. In some examples, multimedia component 1708 may include a front camera and / or a rear camera. When device 1700 is in an operating mode (such as a shooting mode or a video mode), the front camera and / or the rear camera can receive external multimedia data.

[0179] The audio component 1710 is configured to output and / or input audio signals. For example, the audio component 1710 includes a microphone (MIC). When the device 1700 is in an operating mode (such as a call mode, a recording mode, and a speech recognition mode), the microphone is configured to receive external audio signals. The received audio signal can be further stored in the memory 1704 or sent via the communication component 1716. In some examples, the audio component 1710 further includes a speaker for outputting audio signals.

[0180] I / O interface 1712 provides an interface between processing component 1702 and peripheral interface modules. The peripheral interface modules may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0181] Sensor assembly 1714 includes one or more sensors for providing various aspects of the status assessment of device 1700. For example, sensor assembly 1714 can detect the on / off state of device 1700 and the relative positions of components. Examples of these components are the display and keyboard of device 1700. Sensor assembly 1714 can also detect changes in the position of device 1700 or a component of device 1700, the presence or absence of user contact with device 1700, the orientation or acceleration / deceleration of device 1700, and changes in the temperature of device 1700. Sensor assembly 1714 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1714 can further include an optical sensor, such as a CMOS or CCD image sensor used in imaging applications. In some examples, sensor assembly 1714 can further include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0182] Communication component 1716 is configured to promote wired or wireless communication between device 1700 and other equipment. Device 1700 can access a wireless network based on communication standards such as WiFi, 4G or a combination thereof. In one example, communication component 1716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one example, communication component 1716 can further include a near field communication (NFC) module for promoting short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0183] In one example, the apparatus 1700 may be implemented by one or more of the following to perform the above method: an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components.

[0184] The non-transitory computer-readable storage medium may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a flash memory, a hybrid drive or a solid-state hybrid drive (SSHD), a read-only memory (ROM), a compact disk read-only memory (CD-ROM), a magnetic tape, a floppy disk, etc.

[0185] Figure 18 is a flowchart illustrating an exemplary process for video encoding and decoding for motion compensated prediction using geometric partitioning according to some embodiments of the present disclosure.

[0186] In step 1802, the processor 1720 divides the video picture into multiple codec units (CUs), and at least one of the multiple CUs is further divided into two prediction units (PUs). The two PUs may include at least one geometrically shaped PU. For example, the geometrically shaped PUs may include a pair of triangular-shaped PUs, a pair of wedge-shaped PUs, or other geometrically shaped PUs.

[0187] In step 1804, processor 1720 constructs a first merge list including a plurality of candidates, each candidate being a motion vector including a list 0 motion vector or a list 1 motion vector. For example, processor 1720 may construct the first merge list based on a merge list construction process for conventional merge prediction. Processor 1720 may also obtain the first merge list from another electronic device or storage device.

[0188] In step 1806 , the processor 1720 locates a first candidate for the first PU according to the first index.

[0189] In step 1808 , the processor 1720 locates a second candidate for the second PU according to the second index.

[0190] In step 1810 , the processor 1720 obtains a first unidirectional prediction MV for the first PU by selecting the first candidate list X1MV according to a first binary reference list indication flag, where X1 takes a value of 0 or 1 and is indicated by the first binary reference list indication flag.

[0191] In step 1812, the processor 1720 obtains a second unidirectional prediction MV for the second PU by selecting the second candidate list X2MV according to the second binary reference list indication flag, where X2 takes a value of 0 or 1 and is indicated by the second binary reference list indication flag.

[0192] In some examples, a device for video encoding and decoding is provided. The device includes: a processor 1720; and a memory 1704, the memory 1704 being configured to store instructions executable by the processor; wherein, when the processor executes these instructions, it is configured to perform the following steps: Figure 18 The method shown in .

[0193] In some other examples, a non-transitory computer-readable storage medium 1704 is provided having instructions stored therein. When these instructions are executed by the processor 1720, the instructions cause the processor to perform the following steps: Figure 18 The method shown in .

[0194] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0195] The examples are chosen and described in order to explain the principles of the present disclosure and to enable others skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles as well as the various embodiments with various modifications as are suited to the particular use contemplated. Therefore, it will be understood that the scope of the present disclosure is not to be limited to the specific examples of the embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A video decoding method using geometric partitioning, comprising: Obtaining a video picture, wherein the video picture is partitioned into a plurality of coding units (CUs), at least one of the plurality of coding units being further partitioned along a non-diagonal line into two prediction units (PUs), a first PU and a second PU, the two prediction units including at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0 and / or a list 1 MV; Locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; Obtain a first unidirectional prediction MV for the first PU by determining the first candidate list X1 MV according to a first binary reference list indicator flag, where X1 takes a value of 0 or 1; and A second unidirectional prediction MV for the second PU is obtained by determining the second candidate list X2 MV according to a second binary reference list indication flag, where X2 takes a value of 0 or 1.

2. The video decoding method using geometric partitioning according to claim 1, wherein: When it is determined that the first index and the second index are the same, it is determined that the second binary reference list indication flag has an opposite value relative to the first binary reference list indication flag.

3. The video decoding method using geometric partitioning according to claim 1, wherein: When it is determined that the first candidate list X1 MV does not exist, selecting the unidirectional prediction zero MV as the first unidirectional prediction MV; or When it is determined that the second candidate list X2 MV does not exist, the unidirectional prediction zero MV is selected as the second unidirectional prediction MV.

4. The video decoding method using geometric partitioning according to claim 1, further comprising: A first unidirectional prediction MV for the first PU is obtained by selecting the first candidate list (1-X1) MV when it is determined that the first candidate list X1 MV does not exist.

5. The video decoding method using geometric partitioning according to claim 1 or 4, further comprising: A second unidirectional prediction MV for the second PU is obtained by selecting the second candidate list (1-X2)MV when it is determined that the second candidate list X2MV does not exist.

6. The video decoding method using geometric partitioning according to claim 1, wherein: When it is determined that backward prediction is used for the current picture, determining the second binary reference list indication flag to have an opposite value relative to the first binary reference list indication flag; as well as When it is determined that backward prediction is not used for the current picture, the second binary reference list indication flag is determined to have the same value as the first binary reference list indication flag.

7. The video decoding method using geometric partitioning according to claim 1, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on values ​​of the first index and the second index, respectively.

8. The video decoding method using geometric partitioning according to claim 1, wherein: The first binary reference list indication flag and the second binary reference list indication flag are encoded and decoded into CABAC context binary bits.

9. The video decoding method using geometric partitioning according to claim 1, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on parity checks of index values ​​of the first candidate and the second candidate in the first merged list, respectively.

10. A video decoding apparatus using geometric partitioning, comprising: one or more processors; as well as a memory configured to store instructions executable by the one or more processors; wherein the one or more processors, when executing the instructions, are configured to: Obtaining a video picture, wherein the video picture is partitioned into a plurality of coding units (CUs), at least one of the plurality of coding units being further partitioned along a non-diagonal line into two prediction units (PUs), a first PU and a second PU, the two prediction units including at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0 and / or a list 1 MV; Locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; Obtain a first unidirectional prediction MV for the first PU by determining the first candidate list X1 MV according to a first binary reference list indicator flag, where X1 takes a value of 0 or 1; and A second unidirectional prediction MV for the second PU is obtained by determining the second candidate list X2MV according to a second binary reference list indication flag, where X2 takes a value of 0 or 1.

11. The video decoding apparatus using geometric partitioning according to claim 10, wherein: When it is determined that the first index and the second index are the same, it is determined that the second binary reference list indication flag has an opposite value relative to the first binary reference list indication flag.

12. The video decoding apparatus using geometric partitioning according to claim 10, wherein: When it is determined that the first candidate list X1 MV does not exist, selecting the unidirectional prediction zero MV as the first unidirectional prediction MV; or When it is determined that the second candidate list X2 MV does not exist, the unidirectional prediction zero MV is selected as the second unidirectional prediction MV.

13. The video decoding apparatus using geometric partitioning according to claim 10, wherein: The one or more processors are further configured to: A first unidirectional prediction MV for the first PU is obtained by selecting the first candidate list (1-X1) MV when it is determined that the first candidate list X1 MV does not exist.

14. The video decoding apparatus using geometric partitioning according to claim 10 or 13, wherein: The one or more processors are further configured to: A second unidirectional prediction MV for the second PU is obtained by selecting the second candidate list (1-X2)MV when it is determined that the second candidate list X2MV does not exist.

15. The video decoding apparatus using geometric partitioning according to claim 10, wherein: When it is determined that backward prediction is used for the current picture, determining the second binary reference list indication flag to have an opposite value relative to the first binary reference list indication flag; as well as When it is determined that backward prediction is not used for the current picture, the second binary reference list indication flag is determined to have the same value as the first binary reference list indication flag.

16. The video decoding apparatus using geometric partitioning according to claim 10, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on values ​​of the first index and the second index, respectively.

17. The video decoding apparatus using geometric partitioning according to claim 10, wherein: The first binary reference list indication flag and the second binary reference list indication flag are encoded and decoded into CABAC context binary bits.

18. The video decoding apparatus using geometric partitioning according to claim 10, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on parity checks of index values ​​of the first candidate and the second candidate in the first merged list, respectively.

19. A non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning, the non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to: Get video pictures, where The video picture is partitioned into a plurality of coding units (CUs), at least one of the plurality of coding units is further partitioned along a non-diagonal line into two prediction units (PUs), a first PU and a second PU, the two prediction units including at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0 and / or a list 1 MV; Locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; Obtain a first unidirectional prediction MV for the first PU by determining the first candidate list X1 MV according to a first binary reference list indicator flag, where X1 takes a value of 0 or 1; as well as A second unidirectional prediction MV for the second PU is obtained by determining the second candidate list X2MV according to a second binary reference list indication flag, where X2 takes a value of 0 or 1.

20. The non-transitory computer-readable storage medium for video decoding using geometric partitioning according to claim 19, wherein When it is determined that the first index and the second index are the same, it is determined that the second binary reference list indication flag has an opposite value relative to the first binary reference list indication flag.

21. The non-transitory computer-readable storage medium for video decoding using geometric partitioning according to claim 19, wherein When it is determined that the first candidate list X1 MV does not exist, selecting the unidirectional prediction zero MV as the first unidirectional prediction MV; or When it is determined that the second candidate list X2 MV does not exist, the unidirectional prediction zero MV is selected as the second unidirectional prediction MV.

22. The non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning according to claim 19, wherein The actions also include: A first unidirectional prediction MV for the first PU is obtained by selecting the first candidate list (1-X1) MV when it is determined that the first candidate list X1 MV does not exist.

23. The non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning according to claim 19, wherein: The actions further include obtaining a second unidirectional prediction MV for the second PU by selecting the second candidate list (1-X2)MV when it is determined that the second candidate list X2MV does not exist.

24. The non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning according to claim 19, wherein When it is determined that backward prediction is used for the current picture, determining the second binary reference list indication flag to have an opposite value relative to the first binary reference list indication flag; as well as When it is determined that backward prediction is not used for the current picture, the second binary reference list indication flag is determined to have the same value as the first binary reference list indication flag.

25. The non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning according to claim 19, wherein The first binary reference list indication flag and the second binary reference list indication flag are determined based on values ​​of the first index and the second index, respectively.

26. The non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning according to claim 19, wherein The first binary reference list indication flag and the second binary reference list indication flag are encoded and decoded into CABAC context binary bits.

27. The non-transitory computer-readable storage medium for video decoding utilizing geometric partitioning according to claim 19, wherein The first binary reference list indication flag and the second binary reference list indication flag are determined based on parity checks of index values ​​of the first candidate and the second candidate in the first merged list, respectively.

28. A video encoding method using geometric partitioning, comprising: Splitting a video picture into a plurality of coding units (CUs), at least one of the plurality of coding units being further split into two prediction units (PUs) along a non-diagonal line, a first PU and a second PU, the two prediction units including at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0 and / or a list 1 MV; Locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; Obtain a first unidirectional prediction MV for the first PU by determining the first candidate list X1 MV according to a first binary reference list indicator flag, where X1 takes a value of 0 or 1; as well as A second unidirectional prediction MV for the second PU is obtained by determining the second candidate list X2 MV according to a second binary reference list indication flag, where X2 takes a value of 0 or 1.

29. The video encoding method using geometric partitioning according to claim 28, wherein: When it is determined that the first index and the second index are the same, it is determined that the second binary reference list indication flag has an opposite value relative to the first binary reference list indication flag.

30. The video encoding method using geometric partitioning according to claim 28, wherein: When it is determined that the first candidate list X1 MV does not exist, selecting the unidirectional prediction zero MV as the first unidirectional prediction MV; or When it is determined that the second candidate list X2 MV does not exist, the unidirectional prediction zero MV is selected as the second unidirectional prediction MV.

31. The video encoding method using geometric partitioning according to claim 28, further comprising: A first unidirectional prediction MV for the first PU is obtained by selecting the first candidate list (1-X1) MV when it is determined that the first candidate list X1 MV does not exist.

32. The video encoding method using geometric partitioning according to claim 28, further comprising: A second unidirectional prediction MV for the second PU is obtained by selecting the second candidate list (1-X2)MV when it is determined that the second candidate list X2MV does not exist.

33. The video encoding method using geometric partitioning according to claim 28, wherein: When it is determined that backward prediction is used for the current picture, determining the second binary reference list indication flag to have an opposite value relative to the first binary reference list indication flag; as well as When it is determined that backward prediction is not used for the current picture, the second binary reference list indication flag is determined to have the same value as the first binary reference list indication flag.

34. The video encoding method using geometric partitioning according to claim 28, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on values ​​of the first index and the second index, respectively.

35. The video encoding method using geometric partitioning according to claim 28, wherein: The first binary reference list indication flag and the second binary reference list indication flag are encoded and decoded into CABAC context binary bits.

36. The video encoding method using geometric partitioning according to claim 28, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on parity checks of index values ​​of the first candidate and the second candidate in the first merged list, respectively.

37. A video encoding apparatus using geometric partitioning, comprising: one or more processors; as well as a memory configured to store instructions executable by the one or more processors; wherein the one or more processors, when executing the instructions, are configured to: Splitting a video picture into a plurality of coding units (CUs), at least one of the plurality of coding units being further split into two prediction units (PUs) along a non-diagonal line, a first PU and a second PU, the two prediction units including at least one geometrically shaped PU; constructing a first merge list including a plurality of candidates based on a merge list construction process for conventional merge prediction, wherein each of the plurality of candidates is a motion vector (MV) including a list 0 and / or a list 1 MV; Locating a first candidate for the first PU according to a first index; locating a second candidate for the second PU according to a second index; Obtain a first unidirectional prediction MV for the first PU by determining the first candidate list X1 MV according to a first binary reference list indicator flag, where X1 takes a value of 0 or 1; as well as A second unidirectional prediction MV for the second PU is obtained by determining the second candidate list X2MV according to a second binary reference list indication flag, where X2 takes a value of 0 or 1.

38. The video encoding apparatus using geometric partitioning according to claim 37, wherein: When it is determined that the first index and the second index are the same, it is determined that the second binary reference list indication flag has an opposite value relative to the first binary reference list indication flag.

39. The video encoding apparatus using geometric partitioning according to claim 37, wherein: When it is determined that the first candidate list X1 MV does not exist, selecting the unidirectional prediction zero MV as the first unidirectional prediction MV; or When it is determined that the second candidate list X2 MV does not exist, the unidirectional prediction zero MV is selected as the second unidirectional prediction MV.

40. The video encoding apparatus using geometric partitioning according to claim 37, wherein: The one or more processors are further configured to: A first unidirectional prediction MV for the first PU is obtained by selecting the first candidate list (1-X1) MV when it is determined that the first candidate list X1 MV does not exist.

41. The video encoding apparatus using geometric partitioning according to claim 37, wherein: The one or more processors are further configured to: A second unidirectional prediction MV for the second PU is obtained by selecting the second candidate list (1-X2)MV when it is determined that the second candidate list X2MV does not exist.

42. The video encoding apparatus using geometric partitioning according to claim 37, wherein: When it is determined that backward prediction is used for the current picture, determining the second binary reference list indication flag to have an opposite value relative to the first binary reference list indication flag; as well as When it is determined that backward prediction is not used for the current picture, the second binary reference list indication flag is determined to have the same value as the first binary reference list indication flag.

43. The video encoding apparatus using geometric partitioning according to claim 37, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on values ​​of the first index and the second index, respectively.

44. The video encoding apparatus using geometric partitioning according to claim 37, wherein: The first binary reference list indication flag and the second binary reference list indication flag are encoded and decoded into CABAC context binary bits.

45. The video encoding apparatus using geometric partitioning according to claim 37, wherein: The first binary reference list indication flag and the second binary reference list indication flag are determined based on parity checks of index values ​​of the first candidate and the second candidate in the first merged list, respectively.

46. ​​A decoder-readable storage medium storing a bit stream decodable using the method of any one of claims 1 to 9.

47. An encoder-readable storage medium storing a bit stream generated using the method of any one of claims 28 to 36.

48. A computer program product comprising one or more programs for execution by a computing device having one or more processors, wherein: The program, when executed by the one or more processors, causes the computing device to perform the method of any one of claims 1 to 9 or 28 to 36.

49. A method for storing a bitstream, comprising: Generating a bit stream by performing the method of any one of claims 28 to 36; as well as The bitstream is stored.

Images