Method and apparatus for video encoding and decoding using triangular partitioning

Abstract

A method and apparatus are provided for video encoding and decoding using triangular partitioning. The method includes: segmenting a video image into a plurality of CUs, at least one of the plurality of CUs being further segmented into two PUs comprising at least one geometric PU; constructing a first merge list comprising a plurality of candidates, each candidate being an MV comprising a list 0MV and / or a list 1MV; locating a first candidate of the first PU and a second candidate of the second PU according to a first index and a second index, respectively; obtaining a first unidirectional predicted MV for the first PU by selecting a list X1MV of the first candidate according to a first binary reference list indicator; and obtaining a second unidirectional predicted MV for the second PU by selecting a list X2MV of the second candidate according to a second binary reference list indicator.

Description

[0001] This application is a divisional application of the patent application filed on December 31, 2020, with application number 202080089982.X and invention title "Method and apparatus for video encoding and decoding using triangular partitions". Technical Field

[0002] This application relates generally to video encoding and decoding and compression, and specifically, but not limited to, methods and apparatus for motion compensation prediction using triangle prediction units (i.e., a special case of geometric partition prediction units) in video encoding and decoding. Background Technology

[0003] Digital video is supported by various electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression. Digital video devices implement video coding and decoding technologies, such as those described in standards defined by Universal Video Coding (VVC), Joint Explore Test Model (JEM), MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-10, Advanced Video Coding (AVC), ITU-T H.265 / HEVC, and extensions to such standards.

[0004] Video encoding and decoding typically employ prediction methods that leverage redundancy present in video images or sequences (e.g., inter-frame prediction, intra-frame prediction). A key goal of video encoding and decoding technologies is to compress video data into a form that uses lower bitrates while avoiding or minimizing video quality degradation. As more and more video services become available, there is a need for encoding and decoding technologies with better coding efficiency. Block partitioning schemes in each standard are also evolving.

[0005] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video coding and decoding, video frames are divided into one or more stripes, each strip containing multiple video blocks, also known as coding tree units (CTUs). Each CTU may contain one coding unit (CU) or be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be encoded in intra-frame mode, inter-frame mode, or IBC mode. Video blocks in an intra-frame coded (I) strip of a video frame are encoded using spatial predictions about reference samples in adjacent blocks within the same video frame. Video blocks in an inter-frame coded (P or B) strip of a video frame can be encoded using spatial predictions about reference samples in adjacent blocks within the same video frame or temporal predictions about reference samples in other previous and / or future reference video frames.

[0006] A prediction block for the current video block to be encoded is derived based on spatial or temporal predictions of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on the residual block and the motion vector pointing to the reference block forming the prediction block in the reference frame. The process of determining the motion vector is often referred to as motion estimation. Intra-frame coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain (e.g., the frequency domain) to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which is then entropy-encoded into a video bitstream for even greater compression.

[0007] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or for direct wired or wireless transmission to the electronic device. The electronic device then performs video decompression (the reverse of the video compression process described above), for example, by parsing the encoded video bitstream to obtain syntax elements, and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and the electronic device displays the reconstructed digital video data on its display.

[0008] As digital video quality evolves from high definition to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while efficiently encoding / decoding video data remains a long-standing challenge.

[0009] At the Joint Video Experts Group (JVET) meeting, JVET defined the first proposal for Universal Video Coding (VVC) and the VVC Test Model 1 (VTM1) coding method. It was decided that a quadtree with nested multi-type trees using binary and ternary partitioning of coding blocks would be included as the initial new coding feature for VVC. Since then, reference software VTM for implementing the coding method and the proposed VVC decoding process have been developed during the JVET meeting. Summary of the Invention

[0010] Generally, this disclosure describes examples of techniques related to motion compensation prediction using geometry prediction units in video encoding and decoding.

[0011] According to a first aspect of this disclosure, a method for video encoding / decoding using geometric partitioning is provided, comprising: segmenting a video image into a plurality of coding units (CUs), wherein at least one of the plurality of CUs is further segmented into two prediction units (PUs), the two PUs including at least one geometric PU; constructing a first merging list comprising a plurality of candidates based on a merging list construction process for conventional merging prediction, wherein each of the plurality of candidates is a motion vector (MV), the MV including list 0MV or list 1MV or both list 0MV and list 1MV; obtaining a unidirectional prediction MV for the PU by selecting list 0MV or list 1MV of the plurality of candidates; constructing a unidirectional prediction merging list based on the unidirectional prediction MV; and performing a pruning operation on the unidirectional prediction MV while constructing the unidirectional prediction merging list.

[0012] According to a second aspect of this disclosure, an apparatus for video encoding / decoding using geometric partitioning is provided, comprising: one or more processors; and a memory configured to store instructions executable by the one or more processors; wherein, when executing the instructions, the one or more processors are configured to: segment a video image into a plurality of coding units (CUs), wherein at least one of the plurality of CUs is further segmented into two prediction units (PUs), the two PUs including at least one geometric PU; construct a first merging list comprising a plurality of candidates based on a merging list construction process for conventional merging prediction, wherein each of the plurality of candidates is a motion vector (MV), the MV including list 0MV or list 1MV or both list 0MV and list 1MV; obtain a unidirectional prediction MV for the PU by selecting list 0MV or list 1MV of the plurality of candidates; construct a unidirectional prediction merging list based on the unidirectional prediction MV; and perform a trimming operation on the unidirectional prediction MV while constructing the unidirectional prediction merging list.

[0013] According to a third aspect of this disclosure, a non-volatile computer-readable storage medium is provided for video encoding and decoding using geometric partitioning. The non-volatile computer-readable storage medium stores computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform actions, including: segmenting a video image into a plurality of coding units (CUs), wherein at least one of the plurality of CUs is further segmented into two prediction units (PUs), the two PUs including at least one geometric PU; constructing a first merging list comprising a plurality of candidates based on a merging list construction process for conventional merging prediction, wherein each of the plurality of candidates is a motion vector (MV), the MV including list 0MV or list 1MV or both list 0MV and list 1MV; obtaining a unidirectional prediction MV for the PU by selecting list 0MV or list 1MV of the plurality of candidates; constructing a unidirectional prediction merging list based on the unidirectional prediction MV; and performing a pruning operation on the unidirectional prediction MV while constructing the unidirectional prediction merging list. Attached Figure Description

[0014] A more specific description of the examples of this disclosure will be presented with reference to the specific examples shown in the accompanying drawings. Given that these drawings depict only a few examples and are therefore not intended to limit the scope, the examples will be described and explained using additional features and details through the use of the drawings.

[0015] Figure 1 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0016] Figure 2This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0017] Figure 3 This is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to some embodiments of the present disclosure.

[0018] Figure 4 This is a schematic diagram illustrating an example of a CTU (Central Unit) partitioned according to some embodiments of the present disclosure.

[0019] Figure 5 This is a schematic diagram illustrating a multi-type tree partitioning pattern according to some embodiments of the present disclosure.

[0020] Figure 6 This is a schematic diagram illustrating the division of a CU into triangular prediction units according to some embodiments of the present disclosure.

[0021] Figure 7 This is a schematic diagram showing the positions of adjacent blocks according to some embodiments of the present disclosure.

[0022] Figure 8 This is a schematic diagram illustrating the locations of spatial merging candidates according to some embodiments of the present disclosure.

[0023] Figure 9 This is a schematic diagram illustrating motion vector scaling of time merging candidates according to some embodiments of the present disclosure.

[0024] Figure 10 This is a schematic diagram illustrating the candidate positions of time merging candidates according to some embodiments of the present disclosure.

[0025] Figures 11A to 11B This is a schematic diagram illustrating an example of unidirectional predictive motion vector (MV) selection for a triangular prediction pattern according to some embodiments of the present disclosure.

[0026] Figures 12A to 12D This is a schematic diagram illustrating an example of unidirectional prediction MV selection for a triangular prediction pattern according to some embodiments of the present disclosure.

[0027] Figure 13 This is a schematic diagram illustrating an example of flexible one-way prediction MV selection for a triangular prediction pattern according to some embodiments of the present disclosure.

[0028] Figure 14 This is a schematic diagram illustrating an example of dividing a CU into geometric partitions according to some embodiments of the present disclosure.

[0029] Figure 15 This is a schematic diagram illustrating an example of a geometric partition representation according to some embodiments of the present disclosure.

[0030] Figures 16A to 16B This is a schematic diagram illustrating an example of unidirectional prediction MV selection for a triangular prediction / geometric prediction pattern according to some embodiments of the present disclosure.

[0031] Figure 17 This is a schematic diagram illustrating an example of unidirectional prediction MV selection for triangle prediction / geometric prediction that has performed MV trimming operations according to some embodiments of the present disclosure.

[0032] Figure 18 This is a schematic diagram illustrating an example of unidirectional prediction MV selection for triangle prediction / geometric prediction, in which an MV pruning operation is performed once when each MV candidate is selected, according to some embodiments of the present disclosure.

[0033] Figure 19 This is a block diagram illustrating an exemplary apparatus for video encoding and decoding according to some embodiments of the present disclosure.

[0034] Figure 20 This is a flowchart illustrating an exemplary process for video encoding and decoding using motion compensation prediction with a geometric prediction unit according to some embodiments of the present disclosure. Detailed Implementation

[0035] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives can be used. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0036] Throughout this specification, references to "an embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language indicate that a particular feature, structure, or characteristic described is included in at least one embodiment or example. Unless otherwise expressly stated, the features, structures, elements, or characteristics described in connection with one or more embodiments may also be applied to other embodiments.

[0037] Throughout this disclosure, unless otherwise expressly stated, the terms “first,” “second,” “third,” etc., are used only to refer to related elements (e.g., equipment, components, compositions, steps, etc.) and do not indicate any spatial or temporal order. For example, “first equipment” and “second equipment” can refer to two separately formed devices, or two parts, components, or operating states of the same device, and can be named arbitrarily.

[0038] As used herein, depending on the context, the terms "if" or "when" may be understood to mean "at the time of" or "in response to". If these terms appear in the claims, they do not necessarily indicate that the relevant limitation or feature is conditional or optional.

[0039] The terms "module," "submodule," "circuit," "subcircuit," "circuit system," "subcircuit system," "unit," or "subunit" may include memory (shared, dedicated, or combined) storing code or instructions executable by one or more processors. A module may include one or more circuits, with or without stored code or instructions. A module or circuit may include one or more components, directly or indirectly connected. These components may or may not be physically attached to each other or positioned adjacent to each other.

[0040] Units or modules can be implemented purely in software, purely in hardware, or a combination of hardware and software. In a purely software implementation, for example, a unit or module may include functionally related code blocks or software components that are directly or indirectly linked together to perform a specific function.

[0041] Figure 1 A block diagram is shown illustrating an exemplary block-based hybrid video encoder 100 that can be used in conjunction with many video codec standards that employ block-based processing. In encoder 100, video frames are segmented into multiple video blocks for processing. For each given video block, a prediction is formed based on either an inter-frame prediction method or an intra-frame prediction method. In inter-frame prediction, one or more prediction values ​​are formed based on pixels from previously reconstructed frames, through motion estimation and motion compensation. In intra-frame prediction, prediction values ​​are formed based on reconstructed pixels in the current frame. Through mode decision, the best prediction value can be selected to predict the current block.

[0042] The prediction residual, representing the difference between the current video block and its predicted value, is sent to transform circuit 102. The transform coefficients are then sent from transform circuit 102 to quantization circuit 104 for entropy reduction. The quantized coefficients are then fed to entropy coding circuit 106 to generate a compressed video bitstream. Figure 1 As shown, prediction-related information 110 (such as video block segmentation information, motion vectors, reference picture indexes, and intra-prediction modes) from the inter-frame prediction circuit and / or intra-frame prediction circuit 112 is also fed through the entropy coding circuit 106 and stored in the compressed video bitstream 114.

[0043] In encoder 100, for prediction purposes, decoder-related circuitry is also required to reconstruct pixels. First, the prediction residual is reconstructed via inverse quantization 116 and inverse transform circuit 118. This reconstructed prediction residual is combined with block prediction values ​​120 to generate unfiltered reconstructed pixels for the current video block.

[0044] Spatial prediction (or "intra-frame prediction") uses pixels from samples (called reference samples) of already encoded neighboring blocks in the same video frame as the current video block to predict the current video block.

[0045] Timing prediction (also known as "inter-frame prediction") uses reconstructed pixels from already encoded video frames to predict the current video block. Timing prediction reduces the inherent temporal redundancy in the video signal. A timing prediction signal for a given coding unit (CU) or coding block is typically sent by signaling one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its timing reference. Additionally, if multiple reference frames are supported, a reference frame index is sent to identify which reference frame in the reference frame memory the timing prediction signal originates from.

[0046] After performing spatial and / or temporal prediction, the intra / inter-frame mode decision circuit 121 in encoder 100 selects the optimal prediction mode, for example, based on a rate-distortion optimization method. The block prediction value 120 is then subtracted from the current video block; and the resulting prediction residual is decorrelated using transform circuit 102 and quantization circuit 104. The resulting quantized residual coefficients are dequantized by inverse quantization circuit 116 and inverse transformed by inverse transform circuit 118 to form the reconstruction residual, which is then added back to the prediction block to form the reconstructed CU signal. Before placing the reconstructed CU into the reference image memory of image buffer 117 for encoding and decoding subsequent video blocks, loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO), and / or adaptive loop filter (ALF), can be further applied to the reconstructed CU. To form the output video bitstream 114, the encoding mode (inter-frame or intra-frame), prediction mode information, motion information, and quantized residual coefficients are all sent to entropy coding unit 106 for further compression and packing to form the bitstream.

[0047] For example, deblocking filters are available in current versions of AVC, HEVC, and VVC. In HEVC, an additional loop filter called Sample Adaptive Offset (SAO) is defined to further improve coding efficiency. In the current version of the VVC standard, another loop filter called Adaptive Loop Filter (ALF) is under active investigation and is very likely to be included in the final standard.

[0048] These loop filter operations are optional. Performing these operations helps improve coding efficiency and visual quality. They can also be turned off based on decisions made by encoder 100 to save computational complexity.

[0049] It should be noted that intra-frame prediction is typically based on unfiltered reconstructed pixels, while inter-frame prediction is based on filtered reconstructed pixels (if these filter options are enabled in encoder 100).

[0050] Figure 2 This is a block diagram illustrating an exemplary block-based video decoder 200 that can be used in conjunction with many video codec standards. The decoder 200 is similar to [the one residing in...]. Figure 1 The reconstruction-related part is located in the encoder 100. In the decoder 200, the input video bitstream 201 is first decoded by entropy decoding 202 to obtain quantized coefficient levels and prediction-related information. Then, the quantized coefficient levels are processed by inverse quantization 204 and inverse transform 206 to obtain the reconstructed prediction residuals. The block prediction mechanism implemented in the intra / inter-frame mode selector 212 is configured to perform intra-frame prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residuals from inverse transform 206 and the prediction output generated by the block prediction mechanism using summer 214.

[0051] Before storing the reconstructed blocks in image buffer 213, which serves as a reference image memory, the reconstructed blocks can be further passed through loop filter 209. The reconstructed video in image buffer 213 can be sent to drive a display device and to predict subsequent video blocks. With loop filter 209 open, filtering operations are performed on these reconstructed pixels to obtain the final reconstructed video output 222.

[0052] The video encoding / decoding standards mentioned above (such as VVC, JEM, HEVC, MPEG-4, and Part 10) are conceptually similar. For example, they all use block-based processing. The following describes some block partitioning schemes in these standards.

[0053] High-Efficiency Video Coding (HEVC)

[0054] HEVC is based on a hybrid block-based motion-compensated transform coding architecture. The basic unit used for compression is called a coding unit (CTU). For the 4:2:0 chroma format, the maximum CTU size is defined as a block of up to 64 x 64 luma pixels and two 32 x 32 chroma pixels. Each CTU may contain one coding unit (CU) or be recursively divided into four smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more prediction units (PUs) and transform unit (TU) trees.

[0055] Generally, except for monochrome content, a CTU may include one Luminance Coded Tree Block (CTB) and two corresponding Chroma CTBs; a CU may include one Luminance Coded Block (CB) and two corresponding Chroma CBs; a PU may include one Luminance Prediction Block (PB) and two corresponding Chroma PBs; and a TU may include one Luminance Transform Block (TB) and two corresponding Chroma TBs. However, exceptions may occur because for both luminance and chroma, the minimum TB size is 4×4 (i.e., 2×2 Chroma TBs are not supported for the 4:2:0 color format), and regardless of the number of intra-luminance PBs in the corresponding intra-luminance CB, each intra-chroma CB always has only one intra-chroma PB.

[0056] For intra-frame CUs, the luma CB can be predicted by one or four luma PBs, and each of the two chroma CBs is always predicted by one chroma PB, where each luma PB has an intra-frame luma prediction mode, and the two chroma PBs share an intra-frame chroma prediction mode. Furthermore, for intra-frame CUs, the TB size cannot be larger than the PB size. Within each PB, intra-frame prediction is applied to predict samples within each TB from neighboring reconstructed samples of the TB. For each PB, in addition to 33 directional intra-frame prediction modes, a DC mode and a planar mode are supported to predict flat regions and gradually changing regions, respectively.

[0057] For each inter-frame PU, one of three prediction modes can be selected, including inter-frame, skip, and merge. Generally, a motion vector contention (MVC) scheme is introduced to select motion candidates from a given set of candidates, including spatial and temporal motion candidates. Multiple references for motion estimation allow finding the optimal reference among two possible lists of reconstructed reference images (i.e., list 0 and list 1). For the inter-frame mode (called AMVP mode, where AMVP stands for Advanced Motion Vector Prediction), an inter-frame prediction indicator (list 0, list 1, or bidirectional prediction), a reference index, a motion candidate index, a motion vector difference (MVD), and a prediction residual are transmitted. For the skip and merge modes, only the merge index is transmitted, and the current PU inherits the inter-frame prediction indicator, reference index, and motion vector from neighboring PUs referenced by the coded merge index. In the case of skip-coded CUs, the residual signal is also omitted.

[0058] Joint Exploratory Test Model (JEM)

[0059] The Joint Exploratory Test Model (JEM) is built upon the HEVC test model. The basic encoding and decoding workflow of HEVC remains unchanged in JEM; however, the design elements of the most important modules (including modules for block structure, intra- and inter-frame prediction, residual transform, loop filters, and entropy encoding / decoding) have been slightly modified, and additional encoding tools have been added. The following new encoding features are included in JEM.

[0060] In HEVC, the CTU is divided into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics. The decision to encode a picture region using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs depending on the PU partitioning type. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining residual blocks by applying the prediction process based on the PU partitioning type, the CU can be partitioned into transform units (TUs) according to another quadtree structure similar to a coding tree. One of the key features of the HEVC architecture is that it has multiple partitioning concepts including CUs, PUs, and TUs.

[0061] Figure 3 This is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to some embodiments of the present disclosure.

[0062] The QTBT structure eliminates the concept of multiple segmentation types; that is, it removes the distinction between CU, PU, ​​and TU concepts and supports greater flexibility in the shape of CU segments. In a QTBT block structure, CUs can have square or rectangular shapes. For example... Figure 3As shown, the coding tree unit (CTU) is first partitioned using a quadtree (i.e., a four-way tree) structure. The leaf nodes of the quadtree can be further partitioned using a binary tree structure. There are two types of partitioning in the binary tree: symmetrical horizontal partitioning and symmetrical vertical partitioning. The leaf nodes of the binary tree are called coding units (CUs), and such partitions are used for prediction and transform processing without further partitioning. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. In JEM, CUs sometimes consist of coding blocks (CBs) of different color components; for example, in the case of P and B stripes in a 4:2:0 chroma format, a CU contains one luma CB and two chroma CBs. Sometimes, CUs consist of CBs of a single component; for example, in the case of I stripes, a CU contains only one luma CB or only two chroma CBs.

[0063] The following parameters are defined for the QTBT segmentation scheme.

[0064] -CTU size: The size of the root node of the quadtree, the same concept as in HEVC;

[0065] -MinQTSize: The minimum allowed size of a quadleaf leaf node;

[0066] -MaxBTSize: The maximum allowed size of the root node of a binary tree;

[0067] -MaxBTDepth: The maximum allowed binary tree depth;

[0068] -MinBTSize: The minimum allowed size of a binary leaf node.

[0069] In one example of a QTBT segmentation structure, the CTU size is set to 128×128 luminance samples with two corresponding 64×64 chroma sample blocks (in a 4:2:0 chroma format), MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for both width and height) is set to 4×4, and MaxBTDepth is set to 4. Quadtree segmentation is first applied to the CTU to produce quadtree leaf nodes. Quadtree leaf nodes can have sizes ranging from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If a quadtree leaf node is 128×128, it will not be further segmented by a binary tree because its size exceeds MaxBTSize (i.e., 64×64). Otherwise, the quadtree leaf node can be further segmented by a binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, and its binary tree depth is 0. When the binary tree depth reaches MaxBTDepth (i.e., 4), no further partitioning is considered. When a binary tree node has a width equal to MinBTSize (i.e., 4), no further horizontal partitioning is considered. Similarly, when a binary tree node has a height equal to MinBTSize, no further vertical partitioning is considered. The leaf nodes of the binary tree are further processed through prediction and transformation, without any further partitioning. In JEM, the maximum CTU size is 256×256 luminance samples.

[0070] exist Figure 3 The diagram illustrates an example of block partitioning using the QTBT scheme and its corresponding tree representation. Solid lines indicate quadtree partitioning, and dashed lines indicate binary tree partitioning. Figure 3 As shown, the coding tree unit (CTU) 300 is first segmented using a quadtree structure, and then three of the four quadtree leaf nodes 302, 304, 306, and 308 are further segmented using either a quadtree or a binary tree structure. For example, quadtree leaf node 306 is further segmented using a quadtree partition; quadtree leaf node 304 is further segmented into two leaf nodes 304a and 304b using a binary tree partition; and quadtree leaf node 302 is also further segmented using a binary tree partition. In each partition (i.e., non-leaf) node of the binary tree, a flag is used to indicate which partition type (i.e., horizontal or vertical) is used, where 0 indicates a horizontal partition and 1 indicates a vertical partition. For example, quadtree leaf node 304 is flagged as 0 to indicate a horizontal partition, and quadtree leaf node 302 is flagged as 1 to indicate a vertical partition. For quadtree partitioning, there is no need to indicate the partitioning type, because quadtree partitioning always divides the block horizontally and vertically to produce 4 sub-blocks of equal size.

[0071] Furthermore, the QTBT scheme supports the ability for luma and chroma to have separate QTBT structures. Currently, for P and B stripes, the luma and chroma CTBs within a CTU share the same QTBT structure. However, for I stripes, the luma CTB is segmented into CUs using a QTBT structure, and the chroma CTB is segmented into chroma CUs using a separate QTBT structure. This means that a CU in an I stripe consists of either a block of code for the luma component or two blocks of code for the chroma components, while a CU in a P or B stripe consists of blocks of code for all three color components.

[0072] Universal Video Coding (VVC)

[0073] At the Joint Video Experts Group (JVET) meeting, JVET defined the first draft of Universal Video Coding (VVC) and the VVC Test Model 1 (VTM1) coding method. It was decided that quadtrees with nested multi-type trees using binary and tri-partitioned coding block structures would be included as the initial new coding feature for VVC.

[0074] In VVC, the image segmentation structure divides the input video into blocks called coding tree units (CTUs). CTUs are further divided into coding units (CUs) using a quadtree with a nested multi-type tree structure, where leaf coding units (CUs) define regions that share the same prediction pattern (e.g., intra-frame or inter-frame). Here, the term "unit" defines a region of the image that covers all components; the term "block" is used to define a region that covers a specific component (e.g., luma) and can differ spatially when considering chroma sampling formats (such as 4:2:0).

[0075] Segment the image into CTUs

[0076] Figure 4 This is a schematic diagram illustrating an example of a CTU (Central Unit) partitioned according to some embodiments of the present disclosure.

[0077] In VVC, images are divided into CTU sequences, and the CTU concept is the same as that in HEVC. For an image with three sample arrays, a CTU consists of an N×N luminance sample block and two corresponding chrominance sample blocks. Figure 4 An example of image 400 is shown, which is divided into CTU 402.

[0078] The maximum permissible size of the luminance block in the CTU is specified as 128×128 (although the maximum size of the luminance transform block is 64×64).

[0079] CTU is segmented using a tree structure.

[0080] Figure 5 This is a schematic diagram illustrating a multi-type tree partitioning pattern according to some embodiments of the present disclosure.

[0081] In HEVC, the CTU is divided into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics. The decision to encode a picture region using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the leaf CU level. Each leaf CU can be further divided into one, two, or four PUs depending on the PU partitioning type. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining the residual block by applying the prediction process based on the PU partitioning type, the leaf CU can be divided into transform units (TUs) according to another quadtree structure similar to a coding tree. One of the key features of the HEVC structure is that it has multiple partitioning concepts including CUs, PUs, and TUs.

[0082] In VVC, a quadtree with nested multi-type trees is used instead of the concept of multiple partitioning unit types for binary and ternary partitioning structures. That is, it removes the distinction between CU, PU, ​​and TU concepts (except for CUs with sizes too large for the maximum transform length), and supports greater flexibility in CU partitioning shapes. In the coding tree structure, CUs can have square or rectangular shapes. First, coding tree units (CTUs) are partitioned using a quadtree (i.e., a quadtree) structure. Then, the leaf nodes of the quadtrees can be further partitioned using multi-type tree structures. For example... Figure 5 As shown, there are four partition types in the multi-type tree structure: vertical binary partition 502 (SPLIT_BT_VER), horizontal binary partition 504 (SPLIT_BT_HOR), vertical tri-partition 506 (SPLIT_TT_VER), and horizontal tri-partition 508 (SPLIT_TT_HOR). The leaf nodes of the multi-type tree are called coding units (CUs), and unless the CU is too large for the maximum transform length, this partition is used for prediction and transform processing without any further partitioning. This means that in most cases, CUs, PUs, and TUs have the same block size in a quadtree with a nested multi-type tree coding block structure. Exceptions occur when the maximum supported transform length is less than the width or height of the color component of the CU. In VTM1, a CU consists of coding blocks (CBs) for different color components; for example, a CU contains one luma CB and two chroma CBs (unless the video is monochrome, i.e., has only one color component).

[0083] The CU is divided into multiple prediction units.

[0084] In VVC, for each CU segmented based on the structure described above, block content prediction can be performed on the entire CU block or on sub-blocks as explained in the following paragraphs. This unit of operation for prediction is called a prediction unit (or PU).

[0085] In intra-frame prediction (or intra-frame prediction), the size of the PU is typically equal to the size of the CU. In other words, prediction is performed on the entire CU block. For inter-frame prediction (or inter-frame prediction), the size of the PU can be equal to or smaller than the size of the CU. In other words, there are cases where the CU can be divided into multiple PUs for prediction.

[0086] Examples of PU sizes smaller than CU sizes include affine prediction mode, advanced time-level motion vector prediction (ATMVP) mode, and triangle prediction mode.

[0087] In affine prediction mode, the CU can be divided into multiple 4×4 prediction units (PUs). Motion vectors can be obtained for each 4×4 PU, and motion compensation can be performed accordingly. In ATMVP mode, the CU can be divided into one or more 8×8 prediction units (PUs). Motion vectors are obtained for each 8×8 PU, and motion compensation can be performed accordingly. In triangle prediction mode, the CU can be divided into two triangular prediction units. Motion vectors are obtained for each PU, and motion compensation is performed accordingly. Triangle prediction mode is supported for inter-frame prediction. More details about triangle prediction mode are described below.

[0088] Triangle Prediction Pattern

[0089] Figure 6 This is a schematic diagram illustrating the division of a CU into triangular prediction units according to some embodiments of the present disclosure.

[0090] The concept of the triangular prediction pattern is to introduce triangular partitioning for motion compensation prediction. The triangular prediction pattern can also be named the triangular prediction unit pattern. Figure 6 As shown, CU 602 or 604 is divided into two triangular prediction units PU1 and PU2 in the diagonal or anti-diagonal direction (i.e., a division from the top left to the bottom right as shown in CU 602, or a division from the top right to the bottom left as shown in CU 604). Each triangular prediction unit in the CU performs inter-frame prediction using its own unidirectional prediction motion vector obtained from the unidirectional prediction candidate list and the reference frame index. After predicting the triangular prediction units, an adaptive weighting process is performed on the diagonal edges. Then, a transform and quantization process is applied to the entire CU. Note that this mode is only applied to the skip and merge modes in the current VVC. Although in Figure 6 In the diagram, CU is shown as a square block, but the triangle prediction mode can also be applied to non-square (i.e., rectangular) CU shapes.

[0091] A one-way prediction candidate list may include one or more candidates, and each candidate may be a motion vector. Therefore, throughout this disclosure, the terms "one-way prediction candidate list," "one-way prediction motion vector candidate list," and "one-way prediction merged list" are used interchangeably; and the terms "one-way prediction merged candidate" and "one-way prediction motion vector" are also used interchangeably.

[0092] One-way prediction motion vector candidate list

[0093] Figure 7 This is a schematic diagram showing the positions of adjacent blocks according to some embodiments of the present disclosure.

[0094] In some examples, the list of unidirectional predicted motion vector candidates may include two to five unidirectional predicted motion vector candidates. In other examples, the number may be different. It is obtained from neighboring blocks. Figure 7 As shown, the candidate list of unidirectional predicted motion vectors is obtained from seven neighboring blocks, including five spatially adjacent blocks (1 to 5) and two temporally co-located blocks (6 to 7). The motion vectors of the seven neighboring blocks are collected into a first merged list. Then, a unidirectional predicted candidate list is formed based on the motion vectors in the first merged list according to a specific order. Based on this order, unidirectional predicted motion vectors from the first merged list are first added to the unidirectional predicted motion vector candidate list, followed by reference image list 0 or L0 motion vectors for bidirectional predicted motion vectors, then reference image list 1 or L1 motion vectors for bidirectional predicted motion vectors, and finally the average motion vector of the L0 and L1 motion vectors for bidirectional predicted motion vectors. At this point, if the number of candidates is still less than the target number (which can be 5), a zero motion vector is added to the list to meet the target number.

[0095] For each triangular PU, a predictor is obtained based on its motion vector. Notably, the obtained predictor covers a larger area than the actual triangular PU, resulting in an overlap between the two predictors along the shared diagonal edge of the two triangular PUs. A weighting process is applied to the diagonal edge region between the two predictors to obtain the final prediction for the CU. The current weighting factors used for the luminance and chrominance samples are {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8}, respectively.

[0096] Triangle prediction pattern syntax and signal transmission

[0097] The triangle prediction pattern is signaled using a triangle prediction flag. The triangle prediction flag is signaled when encoding a CU in skip or merge modes. For a given CU, if the triangle prediction flag has a value of 1, it indicates that the corresponding CU is encoded using the triangle prediction pattern. Otherwise, a prediction pattern other than the triangle prediction pattern is used to encode the CU.

[0098] For example, in skip mode or merge mode, the triangle prediction flag is conditionally signaled. First, a triangle prediction tool enable / disable flag is signaled in the sequence parameter set (or SPS). The triangle prediction flag is signaled at the CU level only when the triangle prediction tool enable / disable flag is true. Second, the triangle prediction tool is enabled only in B-strips. Therefore, the triangle prediction flag is signaled at the CU level only in B-strips. Third, the triangle prediction mode is signaled only for CUs with a size equal to or greater than a certain threshold. If the CU has a size smaller than the threshold, the triangle prediction flag is not signaled. Fourth, the triangle prediction mode can be signaled only for CUs as long as they are not encoded in a sub-block merge mode that includes both affine mode and ATMVP mode. In the four cases listed above, when the triangle prediction flag is not signaled, it is inferred as 0 on the decoder side.

[0099] When sending triangle prediction flags with signals, a context-adaptive binary arithmetic code (CABAC) entropy encoder with specific context is used to send the triangle prediction flags with signals. The context is formed based on the triangle prediction flag values ​​of the upper and left blocks of the current CU.

[0100] To encode or decode the triangle prediction flags of the current block (or current CU), we obtain the triangle prediction flags from both the block above (or CU) and the block to the left (or CU), and sum their values. This results in three possible contexts corresponding to the following cases:

[0101] 1) Both the left and top blocks have a triangle prediction flag of 0;

[0102] 2) Both the left and top blocks have a triangle prediction marker 1;

[0103] 3) Others.

[0104] Each of the three contexts maintains a separate probability. Once the context value of the current block is determined, the triangular prediction flag of the current block is encoded and decoded using the CABAC probability model corresponding to the context value.

[0105] If the triangle prediction flag is true, a triangle splitting direction flag is sent to indicate whether the splitting is from the top left corner to the bottom right corner or from the top right corner to the bottom left corner.

[0106] When the triangle prediction mode is used in the CU, two index values ​​are signaled to indicate the index values ​​of the first and second unidirectional prediction merge candidates used for triangle prediction, respectively. These two index values ​​are used to locate two merge candidates from the aforementioned list of unidirectional prediction motion vector candidates for the first and second partitions, respectively. For triangle prediction, the two index values ​​need to be different so that the two prediction values ​​for the two triangle partitions can be different from each other. Therefore, the first index value is signaled directly. To signal the second index value, if it is less than the first index value, its value is signaled directly. Otherwise, the value of the second index value is subtracted by 1 before signaling it to the decoder. On the decoder side, the first index is decoded and used directly. To decode the second index value, the value represented as "idx" is first decoded by the CABAC engine. If idx is less than the first index value, the second index value will be equal to the value of idx. Otherwise, the second index value will be equal to (idx+1).

[0107] Geometric merging mode

[0108] In geometric merging mode (i.e., GEO), the CU can be divided into two prediction units (i.e., geometric PU) based on geometric partitioning. Figure 14 Examples of geometric partitioning according to some embodiments of this disclosure are shown. For example... Figure 14 As shown, the CU can be divided into two PUs along a non-perfect diagonal line, and the two PUs can have geometric shapes, such as triangles, wedges, or trapezoids. The geometric merging pattern can be considered an extension of the triangular prediction pattern, or a superset of the triangular prediction pattern.

[0109] Figure 15 This is a schematic diagram illustrating an example of a geometric partition representation according to some embodiments of the present disclosure. For example... Figure 15 As shown, given a GEO partition, the signal transmission for the partition requires signal transmission for the angle α and the separation line displacement, represented as ρ, relative to the center of the block. Here, α represents the quantized angle between 0 degrees and 360 degrees.

[0110] Similar to the triangle prediction mode, under GEO segmentation, only unidirectional prediction is allowed for each of the two GEO partitions, ensuring that the memory bandwidth requirement on the decoder side in this mode does not exceed the memory bandwidth requirement of the bidirectional prediction block. Motion vector prediction for GEO segmentation is consistent with the triangle prediction mode. Also similar to the triangle prediction mode, a weighted average-based merging operation can be applied to the inner boundary between the two partitions in the geometry merging mode. Therefore, techniques related to motion compensation prediction using triangle prediction units can also be applied to motion compensation prediction using geometry prediction units.

[0111] Candidate list of motion vectors for regular merging mode

[0112] Based on the current VVC, in the regular merging mode where the entire CU is predicted without being divided into more than one PU, a different process is used to construct the motion vector candidate list or merging candidate list than the process used for the triangle prediction mode.

[0113] Figure 8 This is a schematic diagram illustrating the positions of spatial merging candidates according to some embodiments of the present disclosure, such as... Figure 8 As shown, firstly, spatial motion vector candidates are selected based on motion vectors from neighboring blocks. Among the spatial merging candidates for the current block 802, those located as shown in the diagram... Figure 8 You can select up to four merge candidates from the candidates at the positions depicted. Candidates are selected according to a specific order. One example yields the order A1→B1→B0→A0→(B2). Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable or intra-frame encoded / decoded. Note that other different orders can also be used.

[0114] Next, temporal merging candidates are obtained. In obtaining the temporal merging candidates, scaled motion vectors are derived based on the corresponding PUs of the images belonging to the given list of reference images that have the smallest picture order count (POC) difference with the current image. The list of reference images to be used to obtain the corresponding PUs is explicitly indicated in the strip header. Figure 9 Motion vector scaling of time merging candidates according to some embodiments of this disclosure is illustrated, such as Figure 9As shown by the dashed lines, the scaled motion vectors of the temporal merge candidates are obtained. The scaled motion vectors of the temporal merge candidates are scaled using the motion vectors of the co-located PU col_PU using the POC distances tb and td, where tb is defined as the POC difference between the reference image curr_ref of the current image curr_pic and the current image curr_pic, and td is defined as the POC difference between the reference image col_ref of the co-located image col_pic and the co-located image col_pic. The reference image index of the temporal merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC draft specification. For B-strips, two motion vectors (one for reference image list 0 and the other for reference image list 1) are obtained and combined to form a bidirectional prediction merge candidate.

[0115] Figure 10 This is a schematic diagram illustrating the candidate positions of time merging candidates according to some embodiments of the present disclosure.

[0116] like Figure 10 As described, the position of the corresponding PU is selected between two candidate positions C3 and H. If the PU at position H is unavailable, or is intra-frame encoded or decoded, or is outside the current CTU, then position C3 is used to obtain a time-merging candidate. Otherwise, position H is used to obtain a time-merging candidate.

[0117] After inserting both spatial and temporal motion vectors into the merge candidate list as described above, history-based merge candidates are added. These history-based merge candidates include motion vectors from previously encoded / decoded CUs, which are maintained in a separate motion vector list and managed according to specific rules.

[0118] After inserting history-based candidates, if the merged candidate list is not full, pairwise average motion vector candidates are further added to the list. As its name suggests, this type of candidate is constructed by averaging candidates already in the current list. More specifically, based on a specific order or rule, two candidates are taken from the merged candidate list each time, and the average motion vector of those two candidates is appended to the current list.

[0119] After inserting the pairwise average motion vectors, if the merged candidate list is still not full, zero motion vectors will be added to fill the list.

[0120] The first merge list for triangle prediction is constructed using the standard merge list construction process.

[0121] The triangular prediction model in the current VVC shares some similarities with the conventional merging prediction model throughout the process of forming predictions. For example, both prediction models require at least the neighboring spatial motion vectors and co-located motion vectors of the current CU to construct the merging list. However, the triangular prediction model also has some aspects that differ from the conventional merging prediction model.

[0122] For example, although a merge list needs to be constructed in both the triangular forecasting mode and the regular merge forecasting mode, the detailed process of constructing such a list differs.

[0123] These differences lead to additional costs in codec implementation due to the need for extra logic. The process and logic for constructing the merge list can be unified and shared between the triangular prediction mode and the regular merge prediction mode.

[0124] In some examples, when forming the unidirectional forecast (also known as single forecast) merge list for the triangular forecast pattern, the new motion vector is fully pruned against those already in the list before being added to the merge list. In other words, the new motion vector is compared with each motion vector already in the unidirectional forecast merge list, and the new motion vector is added to the list only if it differs from every motion vector in the merge list. Otherwise, the new motion vector is not added to the list.

[0125] Based on some examples of this disclosure, in the triangle prediction mode, a unidirectional prediction merge list is constructed from a regular merge mode motion vector candidate list (which may be referred to as the regular merge list).

[0126] More specifically, to construct a merged candidate list for the triangle prediction pattern, a first merged list is first built based on the merged list construction process used for regular merged prediction. The first merged list includes multiple candidates, each being a motion vector. Then, a unidirectional prediction merged list for the triangle prediction pattern is further constructed using the motion vectors from the first merged list.

[0127] It should be noted that the first merge list constructed in this case can be selected to have a list size different from the list size of the general merge mode or the regular merge mode. In one example of this disclosure, the first merge list has the same size as the list in the general merge mode. In another example of this disclosure, the constructed first merge list has a list size different from the list size in the general merge mode.

[0128] Construct a one-way predictive merge list from the first merge list.

[0129] Based on some examples of this disclosure, a unidirectional prediction merge list for a triangular prediction pattern can be constructed from a first merge list using one of the following methods.

[0130] In the example disclosed herein, to construct a one-way prediction merge list, firstly, the candidate prediction list 0 motion vector in the first merge list is examined and selected into the one-way prediction merge list. If the one-way prediction merge list is not full after this process (e.g., the number of candidates in this list is still less than the target number), then the candidate prediction list 1 motion vector in the first merge list is examined and selected into the one-way prediction merge list. If the one-way prediction merge list is still not full, then the prediction list 0 zero vector is added to the one-way prediction merge list. If the one-way prediction merge list is still not full, then the prediction list 1 zero vector is added to the one-way prediction merge list.

[0131] In another example of this disclosure, for each candidate in the first merge list, its prediction list 0 motion vector and prediction list 1 motion vector are added to the unidirectional prediction merge list in an interleaved manner. More specifically, for each candidate in the first merge list, if the candidate is a unidirectional prediction motion vector, it is added directly to the unidirectional prediction merge list. Otherwise, if the candidate is a bidirectional prediction motion vector in the first merge list, its prediction list 0 motion vector is first added to the unidirectional prediction merge list, followed by its prediction list 1 motion vector. Once all motion vector candidates in the first merge list have been checked and added, and the unidirectional prediction merge list is still not full, a unidirectional prediction zero motion vector can be added. For example, for each reference frame index, the prediction list 0 zero motion vector and the prediction list 1 zero motion vector can be added separately to the unidirectional prediction merge list until the list is full.

[0132] In yet another example of this disclosure, unidirectional predicted motion vectors from a first merge list are first selected into the unidirectional predicted merge list. If the unidirectional predicted merge list is not full after this process, for each bidirectional predicted motion vector in the first merge list, its prediction list 0 motion vector is first added to the unidirectional predicted merge list, followed by its prediction list 1 motion vector. After this process, if the unidirectional predicted merge list is still not full, a unidirectional predicted zero motion vector may be added. For example, for each reference frame index, the prediction list 0 zero motion vector and the prediction list 1 zero motion vector may be added separately to the unidirectional predicted merge list until the list is full.

[0133] In the above description, when a unidirectional predicted motion vector is added to the unidirectional predicted merging list, a motion vector pruning process can be performed to ensure that the new motion vector to be added is different from those already in the unidirectional predicted merging list. Such a motion vector pruning process can also be performed partially to achieve lower complexity; for example, checking only some, rather than all, motion vectors already in the unidirectional predicted merging list for the new motion vector to be added. In extreme cases, no motion vector pruning (i.e., a motion vector comparison operation) is performed during the process.

[0134] Based on image prediction configuration, a one-way prediction merge list is constructed from the first merge list.

[0135] In some examples disclosed herein, a one-way prediction merge list can be constructed adaptively based on whether backward prediction is used for the current image. For example, different methods can be used to construct the one-way prediction merge list depending on whether backward prediction is used for the current image. If the image order count (POC) value of all reference images is not greater than the POC value of the current image, it indicates that backward prediction is not used for the current image.

[0136] In the examples disclosed herein, when backward prediction is not used for the current image, or after it is determined that backward prediction is not used for the current image, the candidate prediction list 0 motion vectors in the first merge list are first checked and selected into the one-way prediction merge list, followed by those candidate prediction list 1 motion vectors; if the one-way prediction merge list is still not full, a one-way prediction zero motion vector can be added. Otherwise, if backward prediction is used for the current image, each candidate prediction list 0 and prediction list 1 motion vector in the first merge list is checked and selected into the one-way prediction merge list in an alternating manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, followed by the first candidate prediction list 1 motion vector, then the second candidate prediction list 0 motion vector, then the second candidate prediction list 1 motion vector, and so on. At the end of the process, if the one-way prediction merge list is still not full, a one-way prediction zero vector can be added.

[0137] In another example of this disclosure, if the current image does not use backward prediction, the candidate prediction list 1 motion vectors in the first merge list are first examined and selected into the one-way prediction merge list, followed by those candidate prediction list 0 motion vectors; if the one-way prediction merge list is still not full, a one-way prediction zero motion vector can be added. Otherwise, if the current image uses backward prediction, each candidate prediction list 0 and prediction list 1 motion vector in the first merge list is examined and selected into the one-way prediction merge list in an alternating manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, followed by the first candidate prediction list 1 motion vector, then the second candidate prediction list 0 motion vector, then the second candidate prediction list 1 motion vector, and so on. At the end of the process, if the one-way prediction merge list is still not full, a one-way prediction zero vector can be added.

[0138] In another example of this disclosure, if the current image does not use backward prediction, then only the candidate prediction list 0 motion vectors in the first merge list are checked first and selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is still not full, then the unidirectional prediction zero motion vector can be added. Otherwise, if the current image uses backward prediction, then each candidate prediction list 0 motion vector and prediction list 1 motion vector in the first merge list is checked and selected into the unidirectional prediction merge list in an alternating manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, then the first candidate prediction list 1 motion vector is added, then the second candidate prediction list 0 motion vector is added, then the second candidate prediction list 1 motion vector is added, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, then the unidirectional prediction zero vector can be added.

[0139] In another example of this disclosure, if the current image does not use backward prediction, then only the candidate prediction list 1 motion vectors in the first merge list are checked first and selected into the unidirectional prediction merge list. If the unidirectional prediction merge list is still not full, then the unidirectional prediction zero motion vector can be added. Otherwise, if the current image uses backward prediction, then each candidate prediction list 0 motion vector and prediction list 1 motion vector in the first merge list is checked and selected into the unidirectional prediction merge list in an alternating manner as described above, i.e., the first candidate prediction list 0 motion vector in the first merge list is added, then the first candidate prediction list 1 motion vector is added, then the second candidate prediction list 0 motion vector is added, then the second candidate prediction list 1 motion vector is added, and so on. At the end of the process, if the unidirectional prediction merge list is still not full, then the unidirectional prediction zero vector can be added.

[0140] In the example above, the unidirectional prediction merge list for triangle prediction is constructed by selecting motion vectors from the first merge list into the unidirectional prediction merge list. However, in practice, the method can be implemented in different ways (physically forming the unidirectional prediction (or single prediction) merge list or not physically forming the unidirectional prediction (or single prediction) merge list). In some examples, the first merge list can be used directly without physically creating the unidirectional prediction merge list. For example, the motion vectors of list 0 and / or list 1 for each candidate in the first merge list can be indexed based on a specific order and accessed directly from the first merge list. That is, after constructing a first merge list including multiple candidates (each candidate is one or more motion vectors) based on the merge list construction process used for regular merge prediction, instead of constructing the unidirectional prediction merge list, an index list including multiple reference indices (each reference index is a reference to the motion vectors of the candidates in the first merge list) is constructed. It should be noted that the order of the indices can follow any selection order described in the example of constructing the unidirectional prediction merge list.

[0141] In one example of this disclosure, when backward prediction is not used for the current image, the motion vectors of the candidate prediction list 0 in the first merge list are used as unidirectional prediction merge candidates and are indexed according to the same index order as they are in the first merge list. That is, after determining that the POC of the current image is greater than each POC in the reference image, the reference index is arranged according to the same order of the motion vectors of the candidate list 0 in the first merge list. Otherwise, if backward prediction is used for the current image, the motion vectors of each candidate list 0 and list 1 in the first merge list are used as unidirectional prediction merge candidates and are indexed based on an interleaved manner, i.e., first the motion vectors of the first candidate list 0 in the first merge list, then the motion vectors of the first candidate list 1, then the motion vectors of the second candidate list 0, then the motion vectors of the second candidate list 1, and so on. That is, after determining that the POC of the current image is less than at least one of the POCs in the reference image, when each candidate in the first merge list is a bidirectional prediction motion vector, the reference index is arranged according to the interleaved manner of the motion vectors of each candidate list 0 and list 1 in the first merge list. If a candidate in the first merge list is a unidirectional motion vector, the zero motion vector is indexed as the unidirectional prediction merge candidate following that candidate's motion vector. This ensures that, for the current image using backward prediction, each candidate in the first merge list (regardless of whether it is a bidirectional or unidirectional prediction motion vector) provides two unidirectional motion vectors as unidirectional prediction merge candidates.

[0142] In another example of this disclosure, when backward prediction is not used for the current image, the prediction list 0 motion vectors of the candidates in the first merge list are used as unidirectional prediction merge candidates and are indexed according to the same index order as they are in the first merge list. Otherwise, if backward prediction is used for the current image, the list 0 and list 1 motion vectors of each candidate in the first merge list are used as unidirectional prediction merge candidates and are indexed according to the staggered manner described above, i.e., first the list 0 motion vector of the first candidate in the first merge list, then the list 1 motion vector of the first candidate, then the list 0 motion vector of the second candidate, then the list 1 motion vector of the second candidate, and so on. When a candidate in the first merge list is a unidirectional motion vector, the motion vector is indexed by adding a specific motion offset to become the unidirectional prediction merge candidate following that candidate's motion vector.

[0143] During the above process, when examining new motion vectors to be added to the one-way prediction merge list, pruning can be performed fully or partially. When performed partially, this means comparing the new motion vector with some, but not all, motion vectors already in the one-way prediction merge list. In the extreme case, no motion vector pruning (i.e., no motion vector comparison operation) is performed during the process.

[0144] Motion vector trimming can also be adaptively performed when forming a unidirectional prediction merging list, depending on whether backward prediction is used for the current image. For example, in the example related to the construction of an index list based on image prediction configuration, this disclosure performs motion vector trimming fully or partially when backward prediction is not used for the current image. When backward prediction is used for the current image, no motion vector trimming is performed.

[0145] Select unidirectional prediction merging candidates for the triangular prediction pattern.

[0146] In addition to the examples above, other methods for constructing one-way predictive merge lists or selecting one-way predictive merge candidates are also disclosed.

[0147] In one example of this disclosure, once a first merge list for the regular merge pattern is constructed, unidirectional predictive merge candidates can be selected for triangular prediction according to the following rules:

[0148] For motion vector candidates in the first merge list, one and only one of the motion vectors in list 0 or list 1 is used for triangle prediction;

[0149] For a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is even, then its list 0 motion vector is used for triangle prediction if it is available, and if this motion vector candidate does not have a list 0 motion vector, then its list 1 motion vector is used for triangle prediction; and

[0150] For a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is odd, then its list 1 motion vector is used for triangle prediction if it is available, and if the motion vector candidate does not have a list 1 motion vector, then its list 0 motion vector is used for triangle prediction.

[0151] Figure 11A This example illustrates unidirectional predictive motion vector (MV) selection (or unidirectional predictive merged candidate selection) for a triangular prediction pattern. In the example, the first N (e.g., N equals 5) merged MV candidates obtained from the first merge list are indexed from 0 to (N-1); and each row has two columns, representing the list 0 motion vector and list 1 motion vector for the candidates in the first merge list. Each candidate in the list can be predicted unidirectionally or bidirectionally. For a unidirectional prediction candidate, it has only either a list 0 motion vector or a list 1 motion vector, not both. For a bidirectional prediction candidate, it has both a list 0 motion vector and a list 1 motion vector. Figure 11A In this context, for each merge index, the motion vectors marked "x" are those that are first used for triangle prediction if they are available. If the motion vectors marked "x" are unavailable, the unmarked motion vectors corresponding to the same merge index will be used for triangle prediction.

[0152] The above concept can be extended to other examples. Figure 11B Another example of unidirectional predictive motion vector (MV) selection for a triangular prediction pattern is shown. According to Figure 11B The rules for selecting unidirectional prediction merge candidates for triangle prediction are as follows:

[0153] For motion vector candidates in the first merge list, one and only one of the motion vectors in list 0 or list 1 is used for triangle prediction;

[0154] For a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is even, then its list 1 motion vectors are used for triangle prediction if available, and if this motion vector candidate does not have list 1 motion vectors, then its list 0 motion vectors are used for triangle prediction; and

[0155] For a given motion vector candidate in the first merge list, if the merge index value of the motion vector candidate in the list is odd, then its list 0 motion vector is used for triangle prediction if it is available, and if the motion vector candidate does not have a list 0 motion vector, then its list 1 motion vector is used for triangle prediction.

[0156] In some examples, other different orders can be defined and used to select unidirectional prediction merge candidates for triangle prediction from those motion vector candidates in the first merge list. More specifically, for a given motion vector candidate in the first merge list, the decision of whether to use its list 0 motion vector or list 1 motion vector first when that motion vector candidate is available for triangle prediction does not necessarily depend on the parity of the index values ​​of the candidates in the first merge list as described above. For example, the following rule can also be used:

[0157] For motion vector candidates in the first merge list, one and only one of the motion vectors in list 0 or list 1 is used for triangle prediction;

[0158] Based on a predefined pattern, for several motion vector candidates in the first merge list, motion vectors in list 0 are used for triangle prediction if available, and if motion vectors in list 0 are not available, the corresponding motion vectors in list 1 are used for triangle prediction; and

[0159] Based on the same predefined pattern, for the remaining motion vector candidates in the first merge list, the List 1 motion vectors are used for triangle prediction if they are available, and the corresponding List 0 motion vectors are used for triangle prediction if the List 1 motion vectors are not available.

[0160] Figures 12A to 12D Examples of predefined patterns in the selection of unidirectional prediction motion vectors (MVs) for triangle prediction are shown. For each merge index, motion vectors marked "x" are those that are used first for triangle prediction if they are available. If motion vectors marked "x" are unavailable, the unmarked motion vectors corresponding to the same merge index will be used for triangle prediction.

[0161] exist Figure 12A In the process, for the first three motion vector candidates in the first merged list, their list 0 motion vectors are checked first. Only if the list 0 motion vector is unavailable, the corresponding list 1 motion vector is used for triangle prediction. For the fourth and fifth motion vector candidates in the first merged list, their list 1 motion vectors are checked first. Only if the list 1 motion vector is unavailable, the corresponding list 0 motion vector is used for triangle prediction. Figures 12B to 12DThree other modes for selecting unidirectional predictive merge candidates from the first merge list are shown. The examples shown in the figures are not limiting, and other examples exist. For example, it is also possible to use... Figures 12A to 12D The horizontal and / or vertical mirror versions of those patterns shown.

[0162] The concepts illustrated by some examples can be used in conjunction with the concepts illustrated by some other examples in this disclosure. According to one example of this disclosure, for a given motion vector candidate in a first merge list, the decision of whether to use its list 0 motion vector or list 1 motion vector first when its list 0 motion vector and list 1 motion vector are available for triangle and / or geometric prediction may additionally depend on whether backward prediction is used for the current image and / or stripe. Figures 16A to 16B This is a schematic diagram illustrating an example of unidirectional prediction MV selection for a triangular prediction pattern according to some embodiments of the present disclosure. Figure 16A As shown, if the current image and / or strip uses backward prediction, the selection order between candidate list 0 motion vectors and list 1 motion vectors is based on the parity of the index values ​​of the candidates in the first merged list. If the current image and / or strip does not use backward prediction, the candidate list x (where x is 0 or 1) motion vectors (if available) are preferred and selected first, and examples where x equals 0 are shown in... Figure 16B As shown in the diagram. Similar to other examples previously explained, when a unidirectional predicted motion vector is not available for a specific candidate in the first merge list, the corresponding unidirectional predicted motion vector from the same candidate in another reference list is selected. Therefore, for each candidate in the first merge list, a unidirectional predicted motion vector is obtained and used for the triangle and / or geometric prediction pattern. The obtained unidirectional predicted motion vector shares the same index value as the merged candidates in the first merge list.

[0163] Flexible merge list construction and candidate index signaling for triangular prediction patterns

[0164] In one example of this disclosure, once a first merge list for a regular merge pattern is constructed, a unidirectional prediction motion vector for triangle prediction is directly selected from said list. To indicate which list 0 or list 1 motion vector is used for triangle prediction, firstly, an index value is signaled to indicate which candidate from the first merge list is selected. Next, a binary reference list indication flag (referred to as L0L1_flag) is signaled to indicate whether the selected candidate's list 0 or list 1 motion vector from the first merge list is selected for the first partition of triangle prediction. The same signaling method is used to indicate a second list 0 or list 1 motion vector to be used for the second partition of triangle prediction. For example, the signaling syntax for the triangle pattern encoding CU may include index1, L0L1_flag1, index2, and L0L1_flag2. Here, index1 and index2 are the merge index values ​​of the two candidates selected from the first merge list for the first and second partitions, respectively. index1 and index2 are used to locate the candidates for the first and second partitions (i.e., the first candidate and the second candidate). L0L1_flag1 is a binary flag for the first partition, indicating whether to select either list 0 or list 1 motion vectors from the first merge list based on index 1. L0L1_flag2 is a binary flag for the second partition, indicating whether to select either list 0 or list 1 motion vectors from the first merge list based on index 2. The unidirectional prediction MV for the first partition is obtained by selecting the first candidate list X1 MV according to the first binary reference list indicator flag L0L1_flag1, where X1 takes a value of 0 or 1. The unidirectional prediction MV for the second partition is obtained by selecting the second candidate list X2 MV according to the second binary reference list indicator flag L0L1_flag2, where X2 takes a value of 0 or 1.

[0165] Figure 13 This is a schematic diagram illustrating an example of flexible unidirectional prediction MV selection for a triangular prediction pattern according to some embodiments of the present disclosure. Figure 13As shown, in triangular prediction mode, each list 0 motion vector and / or list 1 motion vector indicated by the symbol "x" in the rectangle can be instructed / signed to the decoder for prediction of the first partition, and each list 0 motion vector and / or list 1 motion vector indicated by the symbol "x" in the rectangle can be instructed / signed to the decoder for prediction of the second partition. Therefore, selecting unidirectional prediction motion vectors from the first merge list becomes very flexible. Given a first merge list of size N candidates, up to 2N unidirectional prediction motion vectors can be used for each of the two triangular partitions. The two merge index values ​​for the two partitions in triangular prediction mode do not have to be different from each other. In other words, they can take the same value. The index values ​​are signaled directly before being sent without adjustment. More specifically, unlike what is defined in the current VVC, the second index value is signaled directly to the decoder without any adjustment to that value before signaling.

[0166] In another example of this disclosure, when the two index values ​​are the same, it is not necessary to signal the binary flag L0L1_flag2 for the second partition. Instead, it is inferred to have the opposite value relative to the binary flag L0L1_flag1 for the first partition. In other words, in this case, L0L1_flag2 can take the value (1-L0L1_flag1).

[0167] In yet another example of this disclosure, binary reference list indicator flags (i.e., L0L1_flag1 and L0L1_flag2) can be encoded as CABAC context bits. The context for L0L1_flag1 can be separate from the context for L0L1_flag2. The CABAC probability under each context can be initialized at the beginning of the video sequence and / or at the beginning of the image and / or at the beginning of the tile group. In this example, different context modeling methods can be used to encode the two flags. In one method, the context model selection for the second flag can depend on the value of the first flag. In other words, at least two context models can be used to encode the second flag, where the context model is selected based on the value of the first flag transmitted by the signal. In another method, the CABAC probability under each context model can be initialized differently depending on whether the current image uses backward prediction. If the image uses backward prediction, the probability for the flag can be initialized higher to indicate a specific list, such as list 0. In yet another method, the first flag can be encoded as bypass bits, and only the second flag can be encoded as context bits. The methods mentioned here can be used individually or in combination.

[0168] In yet another example of this disclosure, when the motion vector indicated by the merge index value and the associated binary reference list indicator flag (i.e., L0L1_flag) is absent, a one-way predicted zero motion vector may be used instead. That is, when it is determined that the first candidate list X1 MV does not exist, the one-way predicted zero MV is selected as the one-way predicted MV for the first partition; and / or when it is determined that the second candidate list X2 MV does not exist, the one-way predicted zero MV is selected as the one-way predicted MV for the second partition.

[0169] In yet another example of this disclosure, when the motion vector indicated by the merge index value and the associated L0L1_flag is absent, a corresponding motion vector indicated by the same merge index value but from another list (i.e., list (1-L0L1_flag)) may be used instead. That is, a one-way predicted MV for the first partition is obtained by selecting the first candidate list (1-X1)MV when the first candidate list X1 MV is determined to be absent; and / or a one-way predicted MV for the second partition is obtained by selecting the second candidate list (1-X2)MV when the second candidate list X2 MV is determined to be absent.

[0170] In another example of this disclosure, for a triangular pattern encoded CU, a second L0L1_flag (i.e., L0L1_flag2) associated with a second index (i.e., index2) is always inferred without signaling. In this case, the syntax of index1, L0L1_flag1, and index2 still needs to be signaled. In one approach, L0L1_flag2 is inferred based on the value of L0L1_flag1 and whether backward prediction is used for the current image. More specifically, for a triangular pattern encoded CU, if backward prediction is used for the current image, the value of L0L1_flag2 is inferred to be the inverse binary value of L0L1_flag1 (i.e., 1-L0L1_flag1); if backward prediction is not used for the current image, the value of L0L1_flag2 is inferred to be the same as L0L1_flag1. Furthermore, if backward prediction is not used for the current image, the value of index2 can be further implemented as a different value from the value of index1, since both motion vectors (one for each triangular partition) come from the same prediction list. If index2 has a value equal to index1, this means the same motion vector will be used for both triangular partitions, which is useless from a coding efficiency perspective. In this case, when signaling the value of index2, a corresponding adjustment to the value of index2 can be performed before index binarization, which is the same as the adjustment for signaling index2 in the current VVC design. For example, if the actual value of index1 is less than the actual value of index2, the CABAC binarized codeword corresponding to (index2-1) is used to signal the value of index2; otherwise, the CABAC binarized codeword corresponding to index2 is used to signal the value of index2. Based on this example of the present disclosure, optionally, along with the same Index2 value adjustment for CABAC binarization, an implementation of Index2 with a value different from index1 can also be applied when the current image uses backward prediction.

[0171] In yet another example of this disclosure, for the triangle pattern encoding CU, neither of the L0L1_flags is signaled. Instead, they are all inferred. In this case, the index1 and index2 syntax still needs to be signaled, representing the merge index values ​​of two candidates selected from the first merge lists for the first and second partitions, respectively. Given the merge candidate index values, a specific method can be defined or used to determine whether the motion vector of list 0 or list 1 corresponding to the merge candidate from the first list is selected for triangle pattern prediction. In one method, for index1, Figure 11A The pattern shown is used to determine from which prediction list to select merged candidate motion vectors for triangle pattern prediction; and for index2, Figure 11B The pattern shown is used to determine from which prediction list the motion vector of the merged candidate is selected for triangle pattern prediction. In other words, if index1 is even, the motion vector from candidate list 0 indicated by index1 is selected, and if index1 is odd, the motion vector from candidate list 1 indicated by index1 is selected. For index2, if it is even, the motion vector from list 1 is selected, and if it is odd, the motion vector from list 0 is selected. In cases where no motion vector corresponds to a particular prediction list, a specific default motion vector, such as the zero motion vector or a corresponding motion vector from another prediction list, can be used instead. When determining from which prediction list the motion vector of the merged candidate is selected for triangle pattern prediction, Figure 11B The pattern shown is used for index1 and Figure 11A The pattern shown can also be used for index2. That is, L0L1_flag can be determined based on the values ​​of index1 and index2.

[0172] Although the triangular prediction pattern is used as an example to illustrate the methods in this disclosure, it is assumed that the geometric merging pattern can be considered an extension or superset of the triangular prediction pattern, and therefore the methods shown in this disclosure can naturally be applied to the geometric merging pattern as well. All the merge list construction methods shown in this disclosure can be naturally used in the geometric merging pattern without any modification. Similarly, all the signaling methods for merging indices shown in this disclosure can be naturally used in the geometric merging pattern without any modification. In one example, according to Figure 11A The same merge list construction process was used for the geometric merge pattern. In another example, according to Figure 11B The same merge list construction process is used for the geometric merge pattern. Furthermore, it's worth noting that if both the triangle prediction pattern and the geometric merge pattern exist in VVC, the same merge list construction process can be shared and used for both patterns.

[0173] It should be mentioned that although some examples in this disclosure use a first merge list containing 5 merge candidates, the size of the first merge list can actually be defined differently, for example, 6 or 4, or some other value. The method shown in the examples is applicable to cases where the first merge list has a size other than 5.

[0174] In the examples above, motion vector pruning can also be performed. This pruning can be done completely or partially. When performed partially, this means comparing the new motion vector with some, rather than all, motion vectors already in the unidirectional prediction merging list. It can also mean that only some, rather than all, of the new motion vectors need to be pruned before being used as merging candidates for triangle prediction. A specific example is pruning the second motion vector only against the first motion vector before using it as a merging candidate for triangle prediction, instead of pruning all other motion vectors. In an extreme case, no motion vector pruning (i.e., no motion vector comparison operation) is performed in this process.

[0175] Another example of performing partial motion vector trimming is controlling the total number of motion vector trimming operations performed in generating the unidirectional prediction merge list to no more than N, where N is a positive integer. As an exemplary implementation, a counter can be used during this process; each time a motion vector trimming operation is performed, the counter increments by 1 regardless of the result; once the counter has a value equal to N, no further motion vector trimming operations are performed when selecting the remaining unidirectional prediction motion vectors into the unidirectional prediction merge list. Figure 17 The following example illustrates this situation. For instance, the arrows indicate the order in which unidirectional predicted motion vectors from the first merge list are checked. For each candidate in the first merge list, its predicted list 0 motion vector and predicted list 1 motion vector are checked and added to the unidirectional predicted merge list in an alternating order. More specifically, for each candidate in the first merge list, if it is a unidirectional predicted motion vector, it is added directly to the unidirectional predicted merge list. Otherwise, for a bidirectional predicted motion vector in the first merge list, its list 0 motion vector is checked first and added to the unidirectional predicted merge list, followed by its list 1 motion vector. Once all motion vector candidates in the first merge list have been checked and the unidirectional predicted merge list is not yet full, a unidirectional predicted zero motion vector can be added. For example, for each reference frame index, list-0 zero vector and list-1 zero vector can be added to the unidirectional predicted merge list respectively until the list is full. During this process, each time a motion vector is checked to be added to the unidirectional predicted merge list, a motion vector pruning operation is performed to check if the motion vector is the same as any of the motion vectors already selected in the unidirectional predicted merge list. If the pruning operation determines that a motion vector is different from those motion vectors it was compared with, then the motion vector is added to the unidirectional prediction merging list. Otherwise, the motion vector is not selected or added to the list. If the total number of motion vector pruning operations performed reaches N, no further pruning operations are performed while selecting the remaining motion vectors into the list.

[0176] Another example of partial motion vector pruning is to perform the motion vector pruning operation only during the generation of the first M candidates in the unidirectional prediction merge list, where M is a positive integer. For example, M can take values ​​such as 2 or 3. Once M motion vectors have been selected into the unidirectional prediction merge list, the motion vector pruning operation will not be performed again when selecting the remaining unidirectional prediction motion vectors into the list.

[0177] In another example of partial motion vector trimming, the motion vector trimming operation is performed only on the top K candidates from the first merge list, where K is a positive integer. Figure 17 In the example shown, a motion vector pruning operation is performed when the first K motion vectors are examined in the order indicated by those arrows. Starting from the (K+1)th candidate, a motion vector pruning operation is no longer performed when selecting the remaining unidirectional predicted motion vectors into the unidirectional predicted merge list.

[0178] Another example of partial motion vector trimming is allowing a fixed number of trimming operations, L, to select each motion vector, where L is a positive integer. If L is 1, it means that only one motion vector trimming operation is allowed when selecting each motion vector. Based on this method, in Figure 18 The diagram provides a concrete example where arrows indicate the order in which the position of each unidirectional predicted motion vector from the first merged list is examined. In this diagram, each row has two blocks (or columns) representing the list 0 and list 1 motion vectors from the first merged list, respectively. Each candidate in the list can be a unidirectional or bidirectional predicted motion vector. A unidirectional predicted candidate has only one list 0 motion vector or a list 1 motion vector, but not both. A bidirectional predicted candidate has both list 0 and list 1 motion vectors. Figure 18In this model, for each merge index, motion vectors marked with "x" (if available) are the first to be examined for triangle prediction / geometric prediction. For the first motion vector marked "x" with index 0, if it is available, it is used for triangle prediction / geometric prediction. Otherwise, the unmarked motion vector corresponding to the same merge index is used for triangle prediction / geometric prediction. Starting with the second motion vector (with an index value of 1 or greater), if the motion vector marked "x" is available and it differs from its immediate preceding motion vector marked "x", or if the unmarked motion vector corresponding to the same merge index is unavailable, the motion vector marked "x" is used for triangle prediction / geometric prediction. Otherwise, the unmarked motion vector corresponding to the same merge index is used for triangle prediction / geometric prediction. In this case, only one motion vector pruning operation is performed when selecting each motion vector candidate, except for the first candidate. More importantly, in triangle prediction / geometric prediction mode, this design has very low decoder complexity when locating the corresponding unidirectional predicted motion vector from the first merge list given a merge index. On the decoder side, if the decoded merge index value is 0, the first motion vector marked "x" in the first merge list is selected (if it is available). Otherwise, the unmarked motion vector corresponding to the same merge index is selected. If the decoded merge index has a value greater than 0, motion vectors marked "x" with the same merge index are checked. If it is available and different from the immediately preceding motion vector marked "x" in the first merge list, or if the unmarked motion vector corresponding to the same merge index in the first merge list is unavailable (i.e., the original candidate with the same merge index in the first merge list is not a bidirectional predictive motion vector), the motion vector marked "x" is selected. Otherwise, the unmarked unidirectional motion vector corresponding to the same merge index in the first merge list is selected.

[0179] Figure 19 This is a block diagram illustrating an apparatus for video encoding and decoding according to some embodiments of the present disclosure. Apparatus 1900 may be a terminal, such as a mobile phone, tablet computer, digital broadcasting terminal, tablet device, or personal digital assistant.

[0180] like Figure 19 As shown, device 1900 may include one or more of the following components: processing component 1902, memory 1904, power supply component 1906, multimedia component 1908, audio component 1910, input / output (I / O) interface 1912, sensor component 1914, and communication component 1916.

[0181] Processing component 1902 typically controls the overall operation of device 1900, such as operations related to display, telephone calls, data communication, camera operation, and recording. Processing component 1902 may include one or more processors 1920 for executing instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 1902 may include one or more modules for facilitating interaction between processing component 1902 and other components. For example, processing component 1902 may include a multimedia module for facilitating interaction between multimedia component 1908 and processing component 1902.

[0182] Memory 1904 is configured to store different types of data to support the operation of device 1900. Examples of such data include instructions for any application or method operating on device 1900, contact data, phonebook data, messages, pictures, videos, etc. Memory 1904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, and memory 1904 may be static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0183] Power supply assembly 1906 supplies power to various components of device 1900. Power supply assembly 1906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1900.

[0184] Multimedia component 1908 includes a screen that provides an output interface between device 1900 and the user. In some examples, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen that receives input signals from the user. The touch panel may include one or more touch sensors for sensing touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some examples, multimedia component 1908 may include a front-facing camera and / or a rear-facing camera. When device 1900 is in an operating mode such as shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data.

[0185] Audio component 1910 is configured to output and / or input audio signals. For example, audio component 1910 includes a microphone (MIC). When device 1900 is in an operating mode (such as call mode, recording mode, and voice recognition mode), the microphone is configured to receive external audio signals. The received audio signals may be further stored in memory 1904 or transmitted via communication component 1916. In some examples, audio component 1910 also includes a speaker for outputting audio signals.

[0186] I / O interface 1912 provides an interface between processing component 1902 and peripheral interface modules. These peripheral interface modules can be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0187] Sensor assembly 1914 includes one or more sensors for providing state assessment in various aspects of device 1900. For example, sensor assembly 1914 may detect the on / off state of device 1900 and the relative position of components. Components, for example, are the display and keyboard of device 1900. Sensor assembly 1914 may also detect changes in position of device 1900 or its components, the presence or absence of user contact on device 1900, the orientation or acceleration / deceleration of device 1900, and temperature changes of device 1900. Sensor assembly 1914 may include a proximity sensor configured to detect the presence of nearby objects without any physical touch. Sensor assembly 1914 may also include optical sensors, such as CMOS or CCD image sensors used in imaging applications. In some examples, sensor assembly 1914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0188] Communication component 1916 is configured to facilitate wired or wireless communication between device 1900 and other devices. Device 1900 may access a wireless network based on communication standards such as WiFi, 4G, or combinations thereof. In the example, communication component 1916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In the example, communication component 1916 may also include a near-field communication (NFC) module for facilitating short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0189] In the example, device 1900 may be implemented by one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components to perform the methods described above.

[0190] Non-volatile computer-readable storage media can be, for example, hard disk drives (HDDs), solid-state drives (SSDs), flash memory, hybrid drives or solid-state hybrid drives (SSHDs), read-only memory (ROMs), optical disc read-only memory (CD-ROMs), magnetic tapes, floppy disks, etc.

[0191] Figure 20 This is a flowchart illustrating an exemplary process for video encoding and decoding using motion compensation prediction with a geometric prediction unit according to some embodiments of the present disclosure.

[0192] In step 2002, the processor 1920 segments the video image into a plurality of coding units (CUs), wherein at least one of the plurality of CUs is further segmented into two prediction units (PUs). The two PUs may include at least one geometric PU. For example, the geometric PUs may include a pair of triangular shaped PUs, a pair of wedge-shaped PUs, or other geometric PUs.

[0193] In step 2004, processor 1920 constructs a first merge list comprising multiple candidates, wherein each candidate is a motion vector comprising motion vectors of list 0 and / or list 1. For example, processor 1920 may construct the first merge list based on a merge list construction process used for conventional merge prediction. Processor 1920 may also obtain the first merge list from other electronic devices or memory.

[0194] In step 2006, the processor 1920 obtains a one-way prediction MV for the PU by selecting multiple candidate lists 0MV or lists 1MV.

[0195] In step 2008, processor 1920 constructs a one-way prediction merge list based on one-way prediction MV.

[0196] In step 2010, processor 1920 performs a pruning operation on the one-way prediction MV while constructing the one-way prediction merge list.

[0197] In some examples, an apparatus for video encoding and decoding is provided. The apparatus includes a processor 1920; and a memory 1904 configured to store instructions executable by the processor; wherein the processor, when executing the instructions, is configured to perform actions such as... Figure 20 The method shown.

[0198] In some other examples, a non-volatile computer-readable storage medium 1904 is provided, having instructions stored therein. When the instructions are executed by processor 1920, the instructions cause the processor to perform actions such as Figure 20 The method shown.

[0199] The description in this disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited thereto. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings.

[0200] The examples were chosen and described to explain the principles of this disclosure and to enable others skilled in the art to understand the various embodiments of this disclosure and, best of all, to utilize the basic principles and the various embodiments with modifications suitable for the intended particular purpose. Therefore, it will be understood that the scope of this disclosure is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of this disclosure.

Claims

1. A method for video decoding, comprising: Multiple coding units (CUs) segmented from video images are obtained from the video bitstream, wherein at least one of the multiple CUs is further segmented into two prediction units (PUs), the two PUs including at least one geometric PU; Multiple candidates are obtained, wherein each of the multiple candidates includes a motion vector MV, the MV including list 0MV or list 1MV or both list 0MV and list 1MV; One-way prediction MV candidates for the PU are obtained by selecting one MV from each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates. The step of obtaining a unidirectional prediction MV candidate for the PU by selecting one MV from each of the plurality of candidates includes performing the following operation on the first candidate among the plurality of candidates: If the first candidate corresponds to the bidirectional prediction mode, then the list of the first candidates, 0MV, is selected as the unidirectional prediction MV candidate for the PU.

2. The video decoding method according to claim 1, wherein, The steps to construct a one-way prediction merge list include: The total number of pruning operations performed is less than or equal to N, where N is a positive integer; A counter records the total number of times the trimming operation is performed, wherein the counter increments by 1 each time a trimming operation is performed on MV, regardless of the result of the trimming operation; and When the counter reaches a value equal to N, the pruning operation on the remaining one-way predicted MV candidates is stopped.

3. The video decoding method according to claim 1, wherein, The steps to construct a one-way prediction merge list include: For each of the plurality of candidates, the pruning operation is performed on either list 0MV or list 1MV, which is added to the one-way prediction merge list by checking for MVs that exist in the one-way prediction merge list.

4. The video decoding method according to claim 1, wherein, The steps to construct a one-way prediction merge list include: The pruning operation is performed when the first M candidates in the unidirectional prediction merging list are obtained, where M is a positive integer; and When M unidirectional prediction MV candidates are obtained in the unidirectional prediction merging list, the pruning operation on the remaining unidirectional prediction MV candidates is stopped.

5. The video decoding method according to claim 1, wherein, The steps to construct a one-way prediction merge list include: The pruning operation is performed on the first K candidates out of the plurality of candidates, where K is a positive integer; and When K candidates out of the plurality of candidates have been examined, the pruning operation for obtaining the remaining one-way predicted MV candidates is stopped.

6. The video decoding method according to claim 1, wherein, The steps to construct a one-way prediction merge list include: Perform at most one pruning operation on each of the plurality of candidates to obtain only one unidirectional prediction MV candidate from each of the plurality of candidates.

7. The video decoding method according to claim 1, wherein, The step of obtaining a unidirectional prediction MV candidate for the PU by selecting one MV from each of the plurality of candidates includes performing the following operation on the first candidate among the plurality of candidates: If the first candidate corresponds to the one-way prediction pattern, then the first candidate is selected as the one-way prediction MV candidate for the PU.

8. The video decoding method according to claim 1, wherein, The step of obtaining a unidirectional prediction MV candidate for the PU by selecting one MV from each of the plurality of candidates includes performing the following operations on the candidates corresponding to the bidirectional prediction modes among the plurality of candidates: If the candidate has an odd merge index number, then the list 1MV of the candidate is selected as the one-way prediction MV candidate for the PU; as well as If the candidate has an even-numbered merge index number, then the list of candidates 0MV is selected as the one-way prediction MV candidate for the PU.

9. A video decoding apparatus, comprising: One or more processors; as well as A memory is configured to store instructions executable by the one or more processors; wherein the one or more processors are configured to: Multiple coding units (CUs) segmented from video images are obtained from the video bitstream, wherein at least one of the multiple CUs is further segmented into two prediction units (PUs), the two PUs including at least one geometric PU; Multiple candidates are obtained, wherein each of the multiple candidates includes a motion vector MV, the MV including list 0MV or list 1MV or both list 0MV and list 1MV; One-way prediction MV candidates for the PU are obtained by selecting one MV from each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates. The one or more processors are further configured to perform the following operation on the first candidate of the plurality of candidates: If the first candidate corresponds to the bidirectional prediction mode, then the list of the first candidates, 0MV, is selected as the unidirectional prediction MV candidate for the PU.

10. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to: The total number of pruning operations performed is less than or equal to N, where N is a positive integer; The total number of times the pruning operation is performed is recorded by a counter, wherein, Each time a trimming operation is performed on MV, regardless of the result of the trimming operation, the counter increments by 1; as well as When the counter reaches a value equal to N, the pruning operation on the remaining one-way predicted MV candidates is stopped.

11. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to: For each of the plurality of candidates, the pruning operation is performed on either list 0MV or list 1MV, which is added to the one-way prediction merge list by checking for MVs that exist in the one-way prediction merge list.

12. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to: The pruning operation is performed when the first M candidates in the unidirectional prediction merging list are obtained, where M is a positive integer; and When M unidirectional prediction MV candidates are obtained in the unidirectional prediction merging list, the pruning operation on the remaining unidirectional prediction MV candidates is stopped.

13. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to: The pruning operation is performed on the top K candidates out of the plurality of candidates, wherein, K is a positive integer; as well as When K candidates out of the plurality of candidates have been examined, the pruning operation for obtaining the remaining one-way predicted MV candidates is stopped.

14. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to: Perform at most one pruning operation on each of the plurality of candidates to obtain only one unidirectional prediction MV candidate from each of the plurality of candidates.

15. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to perform the following operation on a first candidate of the plurality of candidates: If the first candidate corresponds to the one-way prediction pattern, then the first candidate is selected as the one-way prediction MV candidate for the PU.

16. The video decoding apparatus of claim 9, wherein the one or more processors are further configured to perform the following operations on the candidate corresponding to the bidirectional prediction mode among the plurality of candidates: If the candidate has an odd-numbered merge index number, then the list 1MV of the candidate is selected as the one-way prediction MV candidate for the PU; and If the candidate has an even-numbered merge index number, then the list of candidates 0MV is selected as the one-way prediction MV candidate for the PU.

17. A non-volatile computer-readable storage medium for video decoding, the non-volatile computer-readable storage medium storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform actions, the actions including: Multiple coding units (CUs) segmented from video images are obtained from the video bitstream, wherein at least one of the multiple CUs is further segmented into two prediction units (PUs), the two PUs including at least one geometric PU; Multiple candidates are obtained, wherein each of the multiple candidates includes a motion vector MV, the MV including list 0MV or list 1MV or both list 0MV and list 1MV; One-way prediction MV candidates for the PU are obtained by selecting one MV from each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates. The method of obtaining a unidirectional prediction MV candidate for the PU by selecting one MV from each of the plurality of candidates further includes performing the following operation on the first candidate among the plurality of candidates: If the first candidate corresponds to the bidirectional prediction mode, then the list of the first candidates, 0MV, is selected as the unidirectional prediction MV candidate for the PU.

18. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes: The total number of pruning operations performed is less than or equal to N, where N is a positive integer; A counter records the total number of times the trimming operation is performed, wherein the counter increments by 1 each time a trimming operation is performed on MV, regardless of the result of the trimming operation; and When the counter reaches a value equal to N, the pruning operation on the remaining one-way predicted MV candidates is stopped.

19. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes: For each of the plurality of candidates, the pruning operation is performed on either list 0MV or list 1MV, which is added to the one-way prediction merge list by checking for MVs that exist in the one-way prediction merge list.

20. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes: The pruning operation is performed when the first M candidates in the unidirectional prediction merging list are obtained, where M is a positive integer; and When M unidirectional prediction MV candidates are obtained in the unidirectional prediction merging list, the pruning operation on the remaining unidirectional prediction MV candidates is stopped.

21. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes: The pruning operation is performed on the first K candidates out of the plurality of candidates, where K is a positive integer; and When K candidates out of the plurality of candidates have been examined, the pruning operation for obtaining the remaining one-way predicted MV candidates is stopped.

22. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes: Perform at most one pruning operation on each of the plurality of candidates to obtain only one unidirectional prediction MV candidate from each of the plurality of candidates.

23. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes performing the following operation on the first candidate among the plurality of candidates: If the first candidate corresponds to the one-way prediction pattern, then the first candidate is selected as the one-way prediction MV candidate for the PU.

24. The non-volatile computer-readable storage medium for video decoding according to claim 17, wherein, The action also includes performing the following operations on the candidates among the plurality of candidates that correspond to the bidirectional prediction mode: If the candidate has an odd merge index number, then the list 1MV of the candidate is selected as the one-way prediction MV candidate for the PU; as well as If the candidate has an even-numbered merge index number, then the list of candidates 0MV is selected as the one-way prediction MV candidate for the PU.

25. A method for transmitting a bit stream, comprising: Perform the encoding method to generate a bitstream; as well as Send the bit stream to the decoding device. The encoding method includes: The video image is segmented into multiple coding units (CUs), wherein at least one of the multiple CUs is further segmented into two prediction units (PUs), and the two PUs include at least one geometric PU. Multiple candidates are obtained, wherein each of the multiple candidates includes a motion vector MV, the MV including list 0MV or list 1MV or both list 0MV and list 1MV; One-way prediction MV candidates for the PU are obtained by selecting one MV from each of the plurality of candidates; and By performing a pruning operation, a unidirectional prediction merge list is constructed based on the unidirectional prediction MV candidates. The step of obtaining a unidirectional prediction MV candidate for the PU by selecting one MV from each of the plurality of candidates includes performing the following operation on the first candidate among the plurality of candidates: If the first candidate corresponds to the bidirectional prediction mode, then the list of the first candidates, 0MV, is selected as the unidirectional prediction MV candidate for the PU.

Images