Adaptive coding of motion information for multiple hypothesis prediction for video coding - Patents.com

Abstract

An exemplary device for decoding video data includes one or more processors configured to: generate a first predictive block for a current block of the video data using a base inter-prediction mode; code a merge mode syntax element for the second predictive block representing an additional prediction hypothesis, the merge mode syntax element indicating whether the motion information for the second predictive block is coded using a merge mode; code motion information for the second predictive block according to the merge mode syntax element, where, for coding the motion information, the one or more processors are configured to form a merge candidate list including merge candidates representing respective sets of uni-predictive motion information; generate the second predictive block for the current block of the video data using the motion information; form a multi-hypothesis predictive block from the first predictive block and the second predictive block; and decode the current block using the multi-hypothesis predictive block.

Description

[Technical field]

[0001]

[0001] This application claims priority to U.S. Patent Application No. 17 / 655,919, filed March 22, 2022, and U.S. Provisional Application No. 63 / 167,480, filed March 29, 2021, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 17 / 655,919, filed March 22, 2022, claims the benefit of U.S. Provisional Application No. 63 / 167,480, filed March 29, 2021.

[0002] FIELD OF THE DISCLOSURE

[0002] This disclosure relates to video coding, including video encoding and video decoding. [Background technology]

[0003] Digital video capabilities may be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio telephones, so-called "smartphones," video teleconferencing devices, video streaming devices, etc. Digital video devices implement video coding techniques such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), ITU-T H.266 / Versatile Video Coding (VVC), and extensions to such standards, as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. By implementing such video coding techniques, video devices may more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0004]

[0004] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. In block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs) and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame. Summary of the Invention

[0005]

[0005] Generally, this disclosure describes techniques for multiple hypothesis inter-prediction during video coding. Video coding generally partitions a series of pictures into respective blocks and then codes (encodes or decodes) each of the blocks. Coding a block generally includes predicting the block and coding a residual of the block, i.e., a set of differences between the predicted block and the actual block. The prediction may be intra-picture (i.e., predicting the current block using data of a picture that includes the block) or inter-picture (i.e., predicting the current block using data of one or more previously coded pictures). Multiple hypothesis inter-prediction refers to using multiple reference blocks in one or more previously coded pictures to predict a current block. For example, a video coder may generate two or more predictive blocks and then combine values ​​of the predictive blocks using averaging or weighted prediction, etc.

[0006]

[0006] Inter prediction may be signaled using various coding modes, such as merge mode and advanced motion vector prediction (AMVP) mode. Generally, in merge mode, the motion information of the signaled neighboring block is used as the motion information of the current block, whereas in AMVP mode, the motion information of the signaled neighboring block is used to predict the motion vector of the current block, and other motion information (e.g., motion vector differential value and reference picture identification information) is coded to form the motion information of the current block. This disclosure describes various examples of techniques related to signaling of multiple hypothesis inter prediction information using merge mode, among others.

[0007]

[0007] In one example, a method for decoding video data includes generating a first prediction block for a current block of the video data using a base inter-prediction mode; coding a merge mode syntax element for a second prediction block, the second prediction block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second prediction block is coded using a merge mode; coding motion information for the second prediction block in accordance with the merge mode syntax element; and coding the motion information for the second prediction block includes forming a merge candidate list including one or more merge candidates when the merge mode syntax element indicates that the motion information for the second prediction block is coded using the merge mode, the merge candidate list including one or more merge candidates when the merge mode syntax element indicates that the motion information for the second prediction block is coded using a merge mode. generating a second predictive block for a current block of the video data using the motion information representing a respective set of motion information, forming a multi-hypothesis prediction block for the current block as a combination of the first predictive block and the second predictive block, and decoding the current block using the multi-hypothesis prediction block.

[0008]

[0008] In another example, a device for decoding video data includes a memory configured to store the video data and one or more processors implemented in a circuit, the one or more processors generating a first predictive block for a current block of the video data using a base inter prediction mode, and coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, wherein the merge mode syntax element indicates whether motion information for the second predictive block is coded using the merge mode. and wherein, for coding the motion information, the one or more processors are configured, when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, to form a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information, to generate a second predictive block for a current block of the video data using the motion information, form a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block, and decode the current block using the multi-hypothesis predictive block.

[0009]

[0009] In another example, a computer-readable storage medium and coding the motion information for the second predictive block according to the merge mode syntax element. When the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, coding the motion information for the second predictive block according to the merge mode syntax element includes forming a merge candidate list including one or more merge candidates when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, each of the merge candidates representing a respective set of uni-predictive motion information.

[0010]

[0010] In another example, a device for decoding video data includes means for generating a first predictive block for a current block of the video data using a base inter prediction mode; means for coding a merge mode syntax element for the second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode; means for coding the motion information for the second predictive block in accordance with the merge mode syntax element; and when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, the means for coding the motion information for the second predictive block includes means for forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information, means for generating a second predictive block for the current block of the video data using the motion information; means for forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and means for decoding the current block using the multi-hypothesis predictive block.

[0011]

[0011] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0012] [Figure 1]

[0012] A block diagram illustrating an example video encoding and decoding system in which the techniques of this disclosure may be implemented. [Diagram 2]

[0013] 1 is a conceptual diagram illustrating a table depicting an example scheme for uni-prediction motion vector selection in geometric partitioning mode (GPM). [Diagram 3]

[0014] 1 is a block diagram illustrating an example video encoder that may implement the techniques of this disclosure. [Figure 4]

[0015] 1 is a block diagram illustrating an example video decoder that may implement the techniques of this disclosure. [Diagram 5]

[0016] 6 is a flowchart illustrating an example method for encoding a current block in accordance with techniques of this disclosure. [Figure 6]

[0017] 6 is a flowchart illustrating an example method for decoding a current block in accordance with techniques of this disclosure. [Figure 7]

[0018] 1 is a flowchart illustrating an example method for decoding video data in accordance with techniques of this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013]

[0019] Video coding generally partitions a series of pictures into respective blocks and then codes (encodes or decodes) each of the blocks. Coding a block generally involves predicting the block and coding the block's residual, i.e., a set of differences between the predicted block and the actual block. Prediction can be intra-picture (i.e., predicting the current block using data of the picture that contains the block) or inter-picture (i.e., predicting the current block using data of one or more previously coded pictures). Multiple hypothesis inter prediction refers to using multiple reference blocks in one or more previously coded pictures to predict the current block. For example, a video coder may generate two or more predictive blocks and then combine the values ​​of the predictive blocks using averaging or weighted prediction, etc.

[0014]

[0020] Inter prediction may be signaled using various coding modes, such as merge mode and advanced motion vector prediction (AMVP) mode. Generally, in merge mode, the signaled motion information of neighboring blocks is used as the motion information of the current block, whereas in AMVP mode, the signaled motion information of neighboring blocks is used to predict the motion vector of the current block, and other motion information (e.g., motion vector differential values, and reference picture identification information) is coded to form the motion information of the current block.

[0015]

[0021] This disclosure describes various example techniques that may be used to apply merge mode signaling to multiple hypothesis inter prediction. Rather than always signaling one or more of a reference picture index, a motion vector predictor index, and / or a motion vector difference for multiple hypothesis inter prediction, the techniques of this disclosure may include coding a merge flag that indicates whether motion information for an additional hypothesis is coded using a merge mode or an AMVP mode. In particular, signaling motion information for each motion vector may result in signaling excessive data, which may excessively increase overhead signaling of the video bitstream and require excessive encoding, decoding, and other processing operations. By using merge mode to code motion information for multiple hypothesis inter prediction, signaling overhead, as well as encoding, decoding, and other processing operations, may be reduced without degrading video quality.

[0016]

[0022] In some examples, a video coder (encoder or decoder) may construct a merge candidate list for an additional hypothesis such that all merge candidates in the merge candidate list are uni-predictive motion information candidates when the motion information for the additional hypothesis is coded using a merge mode. For example, if a neighboring block used as a merge candidate is coded using bi-prediction including first motion information and second motion information, the video coder may construct a merge candidate from the neighboring block using only one of the first motion information or the second motion information. In some examples, the video coder may construct two merge candidates from the neighboring block coded using bi-prediction, i.e., a first merge candidate using the first motion information and a second merge candidate using the second motion information.

[0017]

[0023] In some examples, the video coder may apply local illumination compensation (LIC) to the additional hypothesis when LIC is applied to the base hypothesis. The video coder may perform LIC in this manner in addition to or instead of the techniques described above.

[0018]

[0024] In some examples, the video coder may determine an interpolation filter to apply when calculating values ​​for half-sample (half pixel or half subpixel) locations of the luminance data. For example, if the motion information for the additional hypothesis is coded using a merge mode, the interpolation filter may be determined according to the interpolation filter of the merge candidate instead of the base mode. As another example, if the motion information for the additional hypothesis is coded using AMVP, the interpolation filter may be determined according to the base mode. The video coder may implement interpolation filter inheritance in any of these ways in addition to or instead of the techniques described above.

[0019]

[0025] 1 is a block diagram illustrating an example video encoding and decoding system 100 that may implement the techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and / or decoding) video data. In general, video data includes any data for processing video. Thus, video data may include raw uncoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0020]

[0026] 1, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116, in this example. In particular, source device 102 provides the video data to destination device 116 via a computer-readable medium 110. Source device 102 and destination device 116 may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, mobile devices, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, broadcast receiver devices, and the like. In some cases, source device 102 and destination device 116 may be equipped for wireless communication and thus may be referred to as wireless communication devices.

[0021]

[0027] In the example of FIG. 1, source device 102 includes a video source 104, a memory 106, a video encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a video decoder 300, a memory 120, and a display device 118. According to this disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 may be configured to apply techniques for coding motion information for multiple hypothesis inter prediction. Thus, source device 102 represents an example of a video encoding device, and destination device 116 represents an example of a video decoding device. In other examples, the source device and destination device may include other components or configurations. For example, source device 102 may receive video data from an external video source, such as an external camera. Similarly, destination device 116 may interface with an external display device rather than including an integrated display device.

[0022]

[0028] The system 100 shown in FIG. 1 is only an example. In general, any digital video encoding and / or decoding device may implement techniques for coding motion information for multiple hypothesis inter prediction. The source device 102 and the destination device 116 are only examples of coding devices, such that the source device 102 generates coded video data for transmission to the destination device 116. This disclosure refers to a "coding" device as a device that performs coding (encoding and / or decoding) of data. Thus, the video encoder 200 and the video decoder 300 represent examples of coding devices, in particular video encoders and video decoders, respectively. In some examples, the source device 102 and the destination device 116 may operate substantially symmetrically, such that each of the source device 102 and the destination device 116 includes video encoding and video decoding components. Thus, the system 100 may support one-way or two-way video transmission between the source device 102 and the destination device 116, for example, video streaming, video playback, video broadcasting, or video telephony.

[0023]

[0029] Generally, the video source 104 represents a source of video data (i.e., raw, uncoded video data) and provides a continuous series of pictures (also called "frames") of the video data to the video encoder 200, which encodes the data for the pictures. The video source 104 of the source device 102 may include a video capture device such as a video camera, a video archive containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source 104 may generate computer graphics-based data as the source video, or a combination of live, archived, and computer-generated video. In each case, the video encoder 200 encodes the captured, pre-captured, or computer-generated video data. The video encoder 200 may reorder the pictures from a received order (sometimes called a "display order") into a coding order for coding. The video encoder 200 may generate a bitstream including the encoded video data. The source device 102 may then output the encoded video data onto a computer-readable medium 110 via an output interface 108, for receipt and / or retrieval by, for example, an input interface 122 of a destination device 116.

[0024]

[0030] The memory 106 of the source device 102 and the memory 120 of the destination device 116 represent general-purpose memories. In some examples, the memories 106, 120 may store raw video data, e.g., raw video from the video source 104 and raw decoded video data from the video decoder 300. Additionally or alternatively, the memories 106, 120 may store software instructions executable by, e.g., the video encoder 200 and the video decoder 300, respectively. Although the memories 106 and 120 are shown in this example separately from the video encoder 200 and the video decoder 300, it should be understood that the video encoder 200 and the video decoder 300 may also include internal memories for functionally similar or equivalent purposes. Additionally, the memories 106, 120 may store encoded video data, e.g., the output from the video encoder 200 and the input to the video decoder 300. In some examples, portions of the memory 106, 120 may be allocated as one or more video buffers, for example, to store raw decoded and / or encoded video data.

[0025]

[0031] The computer-readable medium 110 may represent any type of medium or device capable of transporting encoded video data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium for enabling the source device 102 to transmit the encoded video data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. In accordance with a communication standard, such as a wireless communication protocol, the output interface 108 may modulate a transmission signal including the encoded video data, and the input interface 122 may demodulate a received transmission signal. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the source device 102 to the destination device 116.

[0026]

[0032] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0027]

[0033] In some examples, source device 102 may output the encoded video data to a file server 114 or another intermediate storage device, which may store the encoded video data generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or download.

[0028]

[0034] The file server 114 may be any type of server device capable of storing encoded video data and transmitting the encoded video data to a destination device 116. The file server 114 may represent a web server (e.g., for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a HyperText Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or enhanced MBMS (eMBMS) server, and / or a network attached storage (NAS) device. The file server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, etc.

[0029]

[0035] The destination device 116 may access the encoded video data from the file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both, that is suitable for accessing the encoded video data stored on the file server 114. The input interface 122 may be configured to operate according to any one or more of the various protocols described above for retrieving or receiving media data from the file server 114, or other such protocols for retrieving media data.

[0030]

[0036] The output interface 108 and the input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples in which the output interface 108 and the input interface 122 comprise wireless components, the output interface 108 and the input interface 122 may be configured to transfer data, such as encoded video data, according to a cellular communication standard, such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc. In some examples in which the output interface 108 comprises a wireless transmitter, the output interface 108 and the input interface 122 may be configured to transfer data, such as encoded video data, according to other wireless standards, such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee), the Bluetooth standard, etc. In some examples, the source device 102 and / or the destination device 116 may include respective system-on-chip (SoC) devices. For example, the source device 102 may include a SoC device for performing functions attributable to the video encoder 200 and / or the output interface 108, and the destination device 116 may include a SoC device for performing functions attributable to the video decoder 300 and / or the input interface 122.

[0031]

[0037] The techniques of this disclosure may be applied to video coding supporting any of a variety of multimedia applications, such as over-the-air television broadcast, cable television transmission, satellite television transmission, Internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0032]

[0038] The input interface 122 of the destination device 116 receives the encoded video bitstream from the computer-readable medium 110 (e.g., a communication medium, the storage device 112, the file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200 that is also used by the video decoder 300, such as syntax elements having values ​​that describe characteristics and / or processing of video blocks or other coded units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded pictures of the decoded video data to a user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0033]

[0039] Although not shown in FIG. 1, in some examples, the video encoder 200 and the video decoder 300 may each be integrated with an audio encoder and / or an audio decoder and may include appropriate MUX-DEMUX units or other hardware and / or software to handle multiplexed streams that include both audio and video in a common data stream.

[0034]

[0040] The video encoder 200 and the video decoder 300 may each be implemented as any of a variety of suitable encoder and / or decoder circuits, or any combination thereof, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, etc. When the techniques are implemented partially in software, the device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of this disclosure. Each of the video encoder 200 and the video decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. The device including the video encoder 200 and / or the video decoder 300 may comprise an integrated circuit, a microprocessor, and / or a wireless communication device such as a cellular phone.

[0035]

[0041] The video encoder 200 and the video decoder 300 may operate according to a video coding standard, such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC), or extensions thereof, such as multiview and / or scalable video coding extensions. Alternatively, the video encoder 200 and the video decoder 300 may operate according to other proprietary or industry standards, such as ITU-T H.266, also referred to as Generic Video Coding (VVC). In other examples, the video encoder 200 and the video decoder 300 may operate according to a proprietary video codec / format, such as AOMedia Video 1 (AV1), an extension of AV1, and / or a successor version of AV1 (e.g., AV2). In other examples, the video encoder 200 and the video decoder 300 may operate according to other proprietary formats or industry standards. However, the techniques of this disclosure are not limited to any particular coding standard or format. In general, video encoder 200 and video decoder 300 may be configured to implement the techniques of this disclosure with any video coding technique that codes motion information for multiple hypothesis inter prediction.

[0036]

[0042] Generally, the video encoder 200 and the video decoder 300 may perform block-based coding of pictures. The term “block” generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used in an encoding and / or decoding process). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Generally, the video encoder 200 and the video decoder 300 may code video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, rather than coding red, green, and blue (RGB) data for a sample of a picture, the video encoder 200 and the video decoder 300 may code a luminance component and a chrominance component, where the chrominance component may include both red and blue hues of chrominance components. In some examples, the video encoder 200 converts received RGB format data to a YUV representation prior to encoding, and the video decoder 300 converts the YUV representation to the RGB format. Alternatively, pre-processing and post-processing units (not shown) may perform these conversions.

[0037]

[0043] This disclosure may generally refer to coding (e.g., encoding and decoding) a picture to include processes of encoding or decoding data for a picture. Similarly, this disclosure may refer to coding a block of a picture to include processes of encoding or decoding data for the block, e.g., predictive and / or residual coding. A coded video bitstream generally includes a series of values ​​for syntax elements that represent coding decisions (e.g., coding modes) and partitioning of a picture into blocks. Thus, references to coding a picture or a block should be understood generally as coding values ​​for the syntax elements that form the picture or block.

[0038]

[0044] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video coder (such as video encoder 200) partitions coding tree units (CTUs) into CUs according to a quadtree structure. That is, the video coder partitions CTUs and CUs into four equal non-overlapping squares, and each node of the quadtree has either zero or four child nodes. A node without children may be called a "leaf node", and a CU of such a leaf node may include one or more PUs and / or one or more TUs. The video coder may further partition PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partition of TUs. In HEVC, a PU represents inter-predicted data, and a TU represents residual data. An intra-predicted CU includes intra-prediction information, such as an intra-mode indication.

[0039]

[0045] As another example, the video encoder 200 and the video decoder 300 may be configured to operate according to VVC. According to VVC, a video coder (such as the video encoder 200) partitions a picture into multiple coding tree units (CTUs). The video encoder 200 may partition the CTUs according to a tree structure, such as a quad-tree binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple partition types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels, a first level partitioned according to a quad-tree partition and a second level partitioned according to a binary tree partition. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

[0040]

[0046] In the MTT partitioning structure, blocks may be partitioned using quad tree (QT) partitioning, binary tree (BT) partitioning, and one or more types of triple tree (TT) (also called ternary tree (TT)) partitioning. A triple or ternary tree partitioning is a partitioning in which a block is split into three sub-blocks. In some examples, a triple or ternary tree partitioning splits a block into three sub-blocks without splitting the original block through the center. The partition types in MTT (e.g., QT, BT, and TT) may be symmetric or asymmetric.

[0041]

[0047] When operating according to the AV1 codec, the video encoder 200 and the video decoder 300 may be configured to code the video data in blocks. In AV1, the largest coding block that may be processed is called a superblock. In AV1, a superblock may be either 128×128 luma samples or 64×64 luma samples. However, in successor video encoding formats (e.g., AV2), a superblock may be defined by a different (e.g., larger) luma sample size. In some examples, a superblock is the top level of a quadtree of blocks. The video encoder 200 may further partition the superblock into smaller coding blocks. The video encoder 200 may partition the superblock and other coding blocks into smaller blocks using square or non-square partitions. The non-square blocks may include N / 2×N, N×N / 2, N / 4×N, and N×N / 4 blocks. The video encoder 200 and the video decoder 300 may perform separate prediction and transformation processes on each of the coding blocks.

[0042]

[0048] AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that may be coded independently of other tiles. That is, video encoder 200 and video decoder 300 may encode and decode coding blocks within a tile, respectively, without using video data from other tiles. However, video encoder 200 and video decoder 300 may perform filtering across tile boundaries. Tiles may be uniform or non-uniform in size. Tile-based coding may enable parallel processing and / or multithreading for encoder and decoder implementation.

[0043]

[0049] In some examples, video encoder 200 and video decoder 300 may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, and in other examples, video encoder 200 and video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luminance component and another QTBT / MTT structure for both chrominance components (or two QTBT / MTT structures for each chrominance component).

[0044]

[0050] Video encoder 200 and video decoder 300 may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, superblock partitioning, or other partitioning structures.

[0045]

[0051] In some examples, the CTU includes a coding tree block (CTB) of luma samples, two corresponding CTBs of chroma samples for a picture with three sample arrays, or a CTB of samples for a monochrome picture, or a picture coded using three separate color planes and syntax structures used to code the samples. The CTB may be an N×N block of samples for some value of N such that the division of the components into the CTB is partitioned. The components are arrays or single samples from one of three arrays (luma and two chromas) for pictures in 4:2:0, 4:2:2, or 4:4:4 color formats, or arrays or single samples of arrays for pictures in monochrome formats. In some examples, the coding block is an M×N block of samples for some value of M and N such that the division of the CTB into coding blocks is partitioned.

[0046]

[0052] Blocks (e.g., CTUs or CUs) may be grouped in various ways in a picture. As an example, a brick may refer to a rectangular region of a CTU row in a particular tile in a picture. A tile may be a rectangular region of a CTU in a particular tile column and a particular tile row in a picture. A tile column refers to a rectangular region of a CTU having a height equal to the height of the picture and a width specified by a syntax element (e.g., in a picture parameter set). A tile row refers to a rectangular region of a CTU having a height specified by a syntax element (e.g., in a picture parameter set) and a width equal to the width of the picture.

[0047]

[0053] In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile may not be referred to as a tile. Bricks in a picture may also be arranged in slices. A slice may be an integer number of bricks of a picture that may be contained entirely in a single Network Abstraction Layer (NAL) Unit. In some examples, a slice includes either several complete tiles or only a continuous sequence of complete bricks of one tile.

[0048]

[0054] This disclosure may use "NxN" and "N by N", e.g., 16x16 samples or 16 by 16 samples, interchangeably to refer to sample dimensions of a block (such as a CU or other video block) in terms of vertical and horizontal dimensions. Generally, a 16x16 CU has 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an NxN CU generally has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU may be arranged in rows and columns. Moreover, a CU does not necessarily have to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may comprise NxM samples, where M is not necessarily equal to N.

[0049]

[0055] The video encoder 200 encodes video data for a CU that represents prediction and / or residual information, as well as other information. The prediction information indicates how the CU should be predicted to form a predictive block for the CU. The residual information generally represents sample-by-sample differences between samples of the CU prior to encoding and the predictive block.

[0050]

[0056] To predict a CU, the video encoder 200 may generally form a predictive block for the CU through inter-prediction or intra-prediction. Inter-prediction generally refers to predicting a CU from data of a previously coded picture, while intra-prediction generally refers to predicting a CU from previously coded data of the same picture. To perform inter-prediction, the video encoder 200 may generate a predictive block using one or more motion vectors. The video encoder 200 may generally perform a motion search to identify a reference block that closely matches the CU, for example, with respect to the difference between the CU and the reference block. The video encoder 200 may calculate a difference metric using a sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculation to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 may predict the current CU using uni-directional prediction or bi-directional prediction.

[0051]

[0057] In HEVC, as an example, a bi-directionally predicted (also called bi-predictive) signal is generated by averaging two prediction signals obtained from two different reference pictures and / or by using two different motion vectors. In VVC, as another example, the bi-predictive mode is extended beyond simple averaging to allow for weighted averaging of two prediction signals. The following equation represents an example of generation of a prediction block using bi-prediction:

[0052]

number

[0053]

[0058] In the above formula (1), P bi-predwhere P0 represents a first prediction block formed from a first motion information set and a first reference picture, P1 represents a second prediction block formed from a second motion information set and a second reference picture, w represents a weight value, and ">>" represents a bitwise right shift operator. In VVC, five weight values ​​w are allowed in weighted averaging bi-prediction, w∈{-2,3,4,5,10}. For each bi-predictive CU, the weight w is determined in one of two ways: 1) for non-merged CUs, the weight index is signaled after the motion vector differential; 2) for merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. In VVC, bi-prediction with CU level weighting (BCW) is only applied to CUs with 256 or more luma samples (i.e., CU width x CU height is greater than or equal to 256). For low-latency pictures, all five weights are used. For non-low-latency pictures, only three weights (w∈{3,4,5}) are used.

[0054]

[0059] In HEVC, a video coder may code a motion vector difference (MVD) representing a difference between a motion vector of a CU and a predicted motion vector in units of 1 / 4 luma samples when use_integer_mv_flag (i.e., a syntax element indicating whether integer level precision or sub-integer level precision is used) is equal to 0 (indicating sub-integer precision) in the slice header. VVC introduced a CU-level adaptive motion vector resolution (AMVR) scheme. AMVR allows the MVD of a CU to be coded with different precisions. Depending on the mode for the current CU in VVC (e.g., AMVP mode or affine AMVP mode), the MVD of the current CU may be adaptively selected as follows:

[0055] Normal AMVP mode: 1 / 4-luma sample, 1 / 2 luma sample, integer luma sample, or 4 luma samples.

[0056] Affine AMVP mode: 1 / 4 luma sample, integer luma sample, or 1 / 16 luma sample.

[0057]

[0060] The video coder may conditionally code the CU-level MVD resolution if the current CU has at least one non-zero MVD component per VVC. If all MVD components (i.e., both horizontal and vertical MVDs in reference list L0 and reference list L1) are zero, the video coder may infer that a 1 / 4 luma sample MVD resolution is used without coding additional data to represent the precision (resolution).

[0058]

[0061] For a CU with at least one non-zero MVD component in VVC, a first flag is signaled to indicate whether 1 / 4 luma sample MVD precision is used for the CU. If the first flag is 0, no further signaling is required and 1 / 4 luma sample MVD precision is used for the current CU. Otherwise, the video coder codes a second flag to indicate whether 1 / 2 luma sample MVD precision or other MVD precision (integer or 4 luma sample) is used for the normal AMVP CU. In the case of 1 / 2 luma sample, a six-tap interpolation filter is used for 1 / 2 luma sample positions in VVC instead of the default 8-tap interpolation filter. Otherwise, a third flag is signaled to indicate whether integer luma sample MVD precision or 4 luma sample MVD precision is used for the normal AMVP CU in VVC. For affine AMVP CUs, a second flag is used to indicate whether integer luma sample MVD precision or 1 / 16 luma sample MVD precision is used. To ensure that the reconstructed MV has the intended precision (1 / 4 luma sample, 1 / 2 luma sample, integer luma sample, or 4 luma sample), the video coder may round the motion vector predictor for the CU to the same precision as that of the MVD before adding the predictor with the MVD. The video coder may round the motion vector predictor towards 0 (i.e., negative motion vector predictors are rounded towards positive infinity and positive motion vector predictors are rounded towards negative infinity).

[0059]

[0062] FIG. 2 is a conceptual diagram illustrating a table showing an example scheme for uni-predictive motion vector selection for geometric partitioning mode (GPM). VVC includes GPM for inter prediction. In VVC, the geometric partitioning mode is signaled using a CU level flag as a kind of merge mode, and other merge modes include normal merge mode, merge with MVD (MMVD) mode, cross-component intra prediction (CIIP) mode, and sub-block merge mode. A total of 64 partitions are divided by the geometric partitioning mode into w×h=2, m, n∈{3…6} except for 8×64 and 64×8. m ×2 n Supported for each possible CU size.

[0060]

[0063] When this mode is used, in VVC, a CU is divided into two parts by a geometrically arranged straight line. Each part of the geometric partition within a CU is inter-predicted using its own motion, and only uni-prediction is allowed for each partition, i.e., each part has one motion vector and one reference index. A uni-prediction motion constraint is applied to ensure that only two motion-compensated predictions are required for each CU, just like in traditional bi-prediction. The uni-prediction motion for each partition is derived using the process described next.

[0061] The uni-prediction candidate list is directly derived from the merge candidate list constructed according to the extended merge prediction process in VVC. We denote n as the index of the uni-prediction motion in the geometric uni-prediction candidate list. The LX motion vector (where X is equal to the parity of n) of the n-th extended merge candidate is used as the n-th uni-prediction motion vector of the geometric partition mode. These motion vectors are denoted by "x" in FIG. 2. If the corresponding LX motion vector of the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the uni-prediction motion vector of the geometric partition mode.

[0062]

[0064] If a geometric partition mode is used for the current CU, in VVC, the video coder may further code a geometric partition index (angle and offset) that indicates the partition mode of the geometric partition, as well as two merge indexes (one for each partition). According to VVC, the number of maximum GPM candidate sizes is explicitly coded in the sequence parameter set (SPS) to specify the syntax binarization of the GPM merge index. After predicting each of the portions of the geometric partition, the video coder may adjust the sample values ​​along the edges of the geometric partition using a blending process with adaptive weights.

[0063]

[0065] In nearly all past and present video coding standards, a video coder may utilize either unidirectional or bidirectional techniques to temporally predict blocks of video data that are single prediction hypotheses. Multiple hypothesis prediction (MHP) is documented in Winken et al., "Multi-hypothesis Inter-prediction," 10th Meeting of the JVET of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 10-20 April 2018, Document No. JVET-J0041; Winken et al., "CE10: Multi-Hypothesis Inter Prediction (Tests 1.5 - 1.8)," 11th Meeting of the JVET of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 10-18 July 2018, Document JVET-K0269; Winken et al., "CE10: Multi-Hypothesis Inter Prediction (Tests 1.2.a - 1.2.c)," ITU-T SG16 WP3 and ISO / IEC and Winken et al., "CE10: Multi-hypothesis inter prediction (Test 10.1.2)", ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 13th Meeting of JVET, 9-18 January 2019, Document No. JVET-M0425. In MHP in VVC, the inter prediction method allows weighted overlap of three or more motion compensated prediction signals. The resulting global prediction signal is obtained by sample-wise weighted overlap. The uni / bi-prediction signal p uni / bi and the first additional inter prediction signal / hypothesis h3, in VVC the resulting prediction signal p3 is obtained as follows:

[0064]

number

[0065]

[0066] In equation (2), p3 represents the obtained predicted signal, and p uni / bi where h1 represents the first predicted block from the first prediction hypothesis, h2 represents the second predicted block from the second prediction hypothesis, and α represents a weighting value. In VVC, the weighting factor α is specified by the syntax element add_hyp_weight_idx according to the following mapping:

[0066] [Table 1]

[0067]

[0067] As above, two or more additional prediction signals may be used. In VVC, the resulting overall prediction signal may be iteratively accumulated with each additional prediction signal as follows:

[0068]

number

[0069] The obtained overall prediction signal is the last p n (i.e., p with maximum index n n ) is obtained.

[0070] For inter-predicted blocks using MERGE mode (but not SKIP mode), an additional inter-prediction signal may be specified. For the additional prediction signal, in VVC, one of two AMVP candidate lists is used.

[0071] · If the Picture Order Count (POC) of the reference picture of the additional prediction signal is equal to the POC of the used list1 reference picture, then the list1 AMVP candidate list is used.

[0072] Otherwise, the list0 AMVP candidate list is used.

[0073]

[0070] Chang et al., "Compression efficiency methods beyond VVC," 21st Meeting of JVET of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29, 6-15 January 2021, Document No. JVET-U0100, describes local illumination compensation (LIC). LIC is an inter-prediction technique that models the local illumination variation between a current block and its predicted block as a function between a current block template and a reference block template. The parameters of the function can be represented by a scale α and an offset β, which forms a linear equation, i.e., α*p[x]+β, to compensate for the illumination change, where p[x] is the reference sample pointed to by the MV at position x on the reference picture. Since α and β can be derived based on the current block template and the reference block template, no signaling overhead is required for them, except that a LIC flag is signaled in AMVP mode to indicate the use of LIC.

[0074]

[0071] Local illumination compensation for uni-predictive inter CUs is also described in Seregin et al., "CE4-3.1a and CE4-3.1b: Unidirectional local illumination compensation with affine prediction," July 3-12, 2019, Document No. JVET-O0066, with the following modifications:

[0075] · Intra adjacent samples can be used for LIC parameters.

[0076] ·LIC is disabled for blocks with less than 32 luma samples.

[0077] · In both non-subblock and affine modes, the LIC parameter derivation is performed based on the template block samples corresponding to the current CU instead of the partial template block samples corresponding to the first top-left 16x16 unit.

[0078] · The reference block template samples are generated by using MC together with the block MV without rounding it to integer pixel precision.

[0079]

[0072] Referring again to FIG. 1, according to various techniques of this disclosure, the video encoder 200 and the video decoder 300 may be configured to implement a merge mode for multiple hypothesis inter prediction. For example, in a merge mode for an additional hypothesis, instead of always coding a reference picture index, a motion vector predictor index, and a motion vector differential for the additional hypothesis (as in the AMVP mode), the video encoder 200 and the video decoder 300 may code a merge flag to indicate whether the motion information for the additional hypothesis is coded using the MERGE mode or the AMVP mode. If the merge flag is true, the video encoder 200 and the video decoder 300 may code a merge index to indicate a merge candidate of the additional hypothesis. If the merge flag is false, the video encoder 200 and the video decoder 300 may code a reference picture index, a motion vector predictor index, and a motion vector differential for the additional hypothesis.

[0080]

[0073] In some examples, for one or more additional hypotheses, the video encoder 200 and the video decoder 300 may construct merge candidates for the merge mode for the additional hypotheses such that each merge candidate represents uni-predictive motion. In some examples, the video encoder 200 and the video decoder 300 may use a uni-predictive candidate list similar to the geometric partitioning mode, for example, as in VVC. Thus, the merge mode for the additional hypotheses may share the same merge candidate list as in the geometric partitioning mode.

[0081]

[0074] In other words, to build a merge candidate list, when the neighboring blocks are predicted using bi-prediction, the video encoder 200 and the video decoder 300 may build one or two merge candidates from the neighboring blocks. When the neighboring blocks are predicted using bi-prediction, the neighboring blocks include both the first motion information for the first motion vector and the second motion information for the second motion vector. The video encoder 200 and the video decoder 300 may build a merge candidate from either or both of the first motion information and / or the second motion information. Thus, the video encoder 200 and the video decoder 300 may form a uni-prediction merge candidate from the neighboring blocks, including using one of the first motion information or the second motion information. In addition, the video encoder 200 and the video decoder 300 may form a second uni-prediction merge candidate from the neighboring blocks using the second motion information.

[0082] In some examples, the video encoder 200 and the video decoder 300 may code a merge flag of the additional hypothesis as a merge flag of a base mode (where "base mode" refers to a mode on which the additional hypothesis is applied, e.g., unidirectional prediction or bidirectional prediction). In some examples, the video encoder 200 and the video decoder 300 may use the same CABAC context to code this merge flag. In other examples, the video encoder 200 and the video decoder may code the merge flag for the additional hypothesis using a new CABAC context that is different from the CABAC context used for the merge flag signaled in the base mode.

[0083] In some examples, the video encoder 200 and the video decoder 300 may code the merge index for the additional hypothesis using the same CABAC context used to signal the merge index for the MERGE mode. In other examples, the video encoder 200 and the video decoder 300 may code the merge index for the additional hypothesis using a different CABAC context than the CABAC context used to code the merge index for the MERGE mode.

[0084]

[0077] In accordance with the techniques of this disclosure, by allowing a merge mode to be used to code motion information for an additional hypothesis, the motion information coded in the bitstream may be reduced. That is, rather than coding each of the candidate index, the motion vector differential, the reference picture list indicator, and the reference index as the motion information for the additional hypothesis, the video encoder 200 and the video decoder 300 may code a merge index indicating whether a merge mode is used to code the motion information for the additional hypothesis, and if so, may simply code the candidate index for the merge mode. In this manner, a more accurate prediction block may be formed for the current block, and the bit rate associated with the coding data used to form the prediction block may be reduced.

[0085]

[0078] In some examples, when the merge mode is used for multiple additional hypotheses, the video encoder 200 and the video decoder 300 may remove merge candidates used in previous additional hypotheses from the merge candidate list of the current additional hypothesis. Thus, the number of merge candidates in the current additional hypothesis may be reduced. Thus, the signaling of the merge index may be correspondingly modified to save bits. For example, if merge candidate i is used in the first additional hypothesis in the order and the total number of candidates is N, the video encoder 200 and the video decoder 300 may remove merge candidate i from the merge candidate list of the second additional hypothesis in the order. The total number of merge candidates in the second additional hypothesis becomes N-1. Thus, the merge index in the second additional hypothesis may be coded using a truncated binarization code, for example, a truncated unary code with a maximum codeword length of N-1 instead of N.

[0086] In some examples, the video encoder 200 and the video decoder 300 may apply local illumination compensation (LIC) to the additional hypothesis in the merge mode and / or the AMVP mode if LIC is applied to the base mode. In other examples, the video encoder 200 and the video decoder 300 may infer that LIC is applied to the additional hypothesis if the additional hypothesis uses the AMVP mode and LIC is applied to the base mode. If the additional hypothesis uses the merge mode and LIC is applied to the base mode, the video encoder 200 and the video decoder 300 may code a LIC flag of the merge candidate that indicates whether LIC should be applied to the additional hypothesis.

[0087]

[0080] In VVC, for 1 / 2 luma sample MVD resolution, a 6-tap interpolation filter is used for 1 / 2 luma sample positions instead of the default 8-tap interpolation filter. The merge candidate may inherit such filter switching. According to the techniques of this disclosure, when motion information for an additional hypothesis is coded using a merge mode, whether to apply an alternative switching filter for interpolating 1 / 2 luma sample positions or other sample positions may depend on the merge candidate of the additional hypothesis instead of that of the base mode. Alternatively, when the additional hypothesis is an AMVP mode, whether to apply an alternative switching filter for interpolating 1 / 2 luma sample positions or other positions may depend on whether an interpolation filter switching is implemented in the base mode.

[0088] Some examples of VVC also provide an affine motion compensation mode, which may be considered an inter-prediction mode. In an affine motion compensation mode, video encoder 200 may determine two or more motion vectors that represent non-translational motion, such as zooming in or out, rotation, perspective motion, or other irregular motion types.

[0089]

[0082] To perform intra prediction, the video encoder 200 may select an intra prediction mode to generate a prediction block. Some examples of VVC provide 67 intra prediction modes, including various directional modes, as well as planar and DC modes. In general, the video encoder 200 selects an intra prediction mode that describes the neighboring samples for a current block (e.g., a block of a CU) from which the samples of the current block should be predicted. Such samples may generally be above, to the upper left, or to the left of the current block in the same picture as the current block, assuming that the video encoder 200 codes the CTUs and CUs in raster scan order (from left to right, from top to bottom).

[0090]

[0083] The video encoder 200 encodes data representing a prediction mode for the current block. For example, in an inter prediction mode, the video encoder 200 may encode data representing which of various available inter prediction modes is used, as well as motion information for the corresponding mode. For example, in unidirectional or bidirectional inter prediction (), the video encoder 200 may encode a motion vector using advanced motion vector prediction (AMVP) or merge mode. The video encoder 200 may use a similar mode to encode a motion vector for an affine motion compensation mode.

[0091]

[0084] AV1 includes two general techniques for encoding and decoding coding blocks of video data. The two general techniques are intra prediction (e.g., intraframe prediction or spatial prediction) and inter prediction (e.g., interframe prediction or temporal prediction). In the context of AV1, when predicting a block of a current frame of video data using an intra prediction mode, the video encoder 200 and the video decoder 300 do not use video data from other frames of the video data. In most intra prediction modes, the video encoder 200 encodes a block of a current frame based on a difference between a sample value in the current block and a predicted value generated from a reference sample in the same frame. The video encoder 200 determines a predicted value generated from a reference sample based on an intra prediction mode.

[0092]

[0085] Following prediction, such as intra- or inter-prediction, of a block, the video encoder 200 may calculate residual data for the block. The residual data, such as a residual block, represents sample-by-sample differences between the block and a predictive block for the block formed using a corresponding prediction mode. The video encoder 200 may apply one or more transforms to the residual block to produce transformed data in a transform domain rather than a sample domain. For example, the video encoder 200 may apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 may apply a secondary transform, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc., following the first transform. The video encoder 200 produces transform coefficients following application of the one or more transforms.

[0093]

[0086] As mentioned above, following any transformation to produce transform coefficients, the video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, providing further compression. By performing a quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 may truncate an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift of the value to be quantized.

[0094]

[0087] Following quantization, the video encoder 200 may scan the transform coefficients to produce a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan may be designed to place higher energy (and therefore lower frequency) transform coefficients at the front of the vector and lower energy (and therefore higher frequency) transform coefficients at the rear of the vector. In some examples, the video encoder 200 may utilize a predefined scan order to scan the quantized transform coefficients to produce a serialized vector, and then entropy code the quantized transform coefficients of the vector. In other examples, the video encoder 200 may perform an adaptive scan. After scanning the quantized transform coefficients to form the one-dimensional vector, the video encoder 200 may entropy code the one-dimensional vector, for example, according to context adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropy code values ​​for syntax elements describing metadata associated with the encoded video data for use by the video decoder 300 in decoding the video data.

[0095] To implement CABAC, the video encoder 200 may assign a context in a context model to a symbol to be transmitted. The context may relate, for example, to whether neighboring values ​​of the symbol are zero-valued or not. The probability decision may be based on the context assigned to the symbol.

[0096]

[0089] Video encoder 200 may further generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, for video decoder 300, e.g., in a picture header, block header, slice header, or other syntax data, such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS). Video decoder 300 may similarly decode such syntax data to determine how to decode corresponding video data.

[0097] In this manner, the video encoder 200 may generate encoded video data, e.g., a bitstream including syntax elements that describe partitions of a picture into blocks (e.g., CUs) and predictive and / or residual information for the blocks. Finally, the video decoder 300 may receive the bitstream and decode the encoded video data.

[0098]

[0091] In general, the video decoder 300 performs a reverse process of that performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may decode values ​​for syntax elements of the bitstream using CABAC in a manner that is reversed but substantially similar to the CABAC encoding process of the video encoder 200. The syntax elements may define partition information for partitioning a picture into CTUs and partitions of each CTU according to a corresponding partition structure, such as a QTBT structure, to define CUs of the CTU. The syntax elements may further define prediction and residual information for blocks (e.g., CUs) of video data.

[0099]

[0092] The residual information may be represented, for example, by quantized transform coefficients. The video decoder 300 may dequantize and inverse transform the quantized transform coefficients of the block to reconstruct a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra- or inter-prediction) and associated prediction information (e.g., motion information for inter-prediction) to form a predictive block for the block. The video decoder 300 may then combine (sample-by-sample) the predictive block and the residual block to reconstruct the original block. The video decoder 300 may perform additional processing, such as performing a deblocking process to reduce visual artifacts along block boundaries.

[0100]

[0093] This disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to communication of values ​​for syntax elements and / or other data used to decode encoded video data. That is, video encoder 200 may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As described above, source device 102 may transport the bitstream to destination device 116 in substantially real-time or may transport the bitstream to destination device 116 in non-real-time, such as may be done when storing syntax elements in storage device 112 for later retrieval by destination device 116.

[0101]

[0094] Thus, video encoder 200 and video decoder 300 represent example devices for decoding (and / or encoding) video data, the example devices including a memory configured to store the video data and one or more processors implemented in a circuit, the one or more processors configured to generate a first predictive block for a current block of the video data using a base inter prediction mode and code a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether motion information for the second predictive block is coded using the merge mode. and coding motion information for the second predictive block according to a merge mode syntax element, where, for coding the motion information, the one or more processors are configured to form a merge candidate list including one or more merge candidates when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, each of the merge candidates representing a respective set of uni-predictive motion information, generate a second predictive block for a current block of the video data using the motion information, form a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block, and decode the current block using the multi-hypothesis predictive block.

[0102]

[0095] Figure 3 is a block diagram illustrating an example video encoder 200 that may implement the techniques of this disclosure. Figure 3 is provided for illustrative purposes and should not be considered as limiting the techniques broadly illustrated and described in this disclosure. For illustrative purposes, this disclosure describes a video encoder 200 according to VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265) techniques. However, the techniques of this disclosure may be implemented by video encoding devices configured for other video coding standards and video coding formats, such as AV1 and successors of the AV1 video coding format.

[0103] 3, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 may be implemented in one or more processors or processing circuits. For example, the units of video encoder 200 may be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Moreover, video encoder 200 may include additional or alternative processors or processing circuits for performing these and other functions.

[0104]

[0097] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. The video encoder 200 may receive video data stored in the video data memory 230, for example, from the video source 104 (FIG. 1). The DPB 218 may act as a reference picture memory that stores reference video data for use in predicting subsequent video data by the video encoder 200. The video data memory 230 and the DPB 218 may be formed by any of a variety of memory devices, such as synchronous dynamic random access memory (DRAM), including DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 230 and the DPB 218 may be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 may be on-chip with the other components of the video encoder 200, as shown, or off-chip relative to those components.

[0105] In this disclosure, references to video data memory 230 should not be construed as limited to memory internal to video encoder 200 unless specifically so described, or to memory external to video encoder 200 unless specifically so described. Instead, references to video data memory 230 should be understood as a reference memory that stores video data that video encoder 200 receives for encoding (e.g., video data for a current block to be encoded). Memory 106 of FIG. 1 may also provide temporary storage of outputs from various units of video encoder 200.

[0106]

[0099] The various units in FIG. 3 are shown to aid in understanding the operations performed by the video encoder 200. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function and is preconfigured with respect to the operations that may be performed. A programmable circuit refers to a circuit that may be programmed to perform various tasks and provide flexible functionality in the operations that may be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. A fixed-function circuit may execute software instructions (e.g., to receive parameters or output parameters), but the type of operations that the fixed-function circuit performs is generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be an integrated circuit.

[0107]

[0100] The video encoder 200 may include an arithmetic logic unit (ALU), a basic functional unit (EFU), a programmable core formed of digital circuits, analog circuits, and / or programmable circuits. In examples in which the operations of the video encoder 200 are implemented using software executed by a programmable circuit, the memory 106 (FIG. 1) may store instructions (e.g., object code) of the software that the video encoder 200 receives and executes, or another memory (not shown) within the video encoder 200 may store such instructions.

[0108]

[0101] The video data memory 230 is configured to store the received video data. The video encoder 200 may retrieve pictures of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be raw video data to be encoded.

[0109] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes. By way of example, the mode selection unit 202 may include a palette unit, an intra block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0110]

[0103] The mode selection unit 202 generally coordinates multiple encoding passes to test combinations of encoding parameters and the resulting rate-distortion values ​​for such combinations. The encoding parameters may include partitioning of CTUs into CUs, prediction modes for CUs, transform types for residual data of CUs, quantization parameters for residual data of CUs, etc. The mode selection unit 202 may finally select a combination of encoding parameters that has a rate-distortion value that is better than other tested combinations.

[0111]

[0104] The video encoder 200 may partition a picture retrieved from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs in a slice. The mode selection unit 202 may partition the CTUs of the picture according to a tree structure, such as the MTT structure, the QTBT structure, the superblock structure, or the quadtree structure described above. As described above, the video encoder 200 may form one or more CUs from partitioning the CTUs according to the tree structure. Such a CU may also be generally referred to as a "video block" or a "block."

[0112]

[0105] In general, the mode selection unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for a current block (e.g., the current CU, or in HEVC, the overlapping portion of the PU and TU). For inter prediction of the current block, the motion estimation unit 222 may perform motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in the DPB 218). In particular, the motion estimation unit 222 may calculate a value representing how similar a potential reference block is to the current block according to, for example, the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), and the like. The motion estimation unit 222 may generally perform these calculations using the sample-by-sample difference between the current block and the reference block under consideration. Motion estimation unit 222 may identify the reference block having the lowest value resulting from these calculations, which indicates the reference block that most closely matches the current block.

[0113]

[0106] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of a current block in a current picture. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, in uni-directional inter-prediction, the motion estimation unit 222 may provide a single motion vector, while in bi-directional inter-prediction, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then use the motion vectors to generate a predictive block. For example, the motion compensation unit 224 may use the motion vectors to retrieve data of the reference block. As another example, if the motion vector has fractional sample precision, the motion compensation unit 224 may interpolate values ​​for the predictive block according to one or more interpolation filters. Moreover, in bidirectional inter prediction, the motion compensation unit 224 may retrieve data for two reference blocks identified by the respective motion vectors and combine the retrieved data, for example, through sample-wise averaging or weighted averaging.

[0114] When performing multiple-hypothesis inter prediction, the motion estimation unit 222 and the motion compensation unit 224 may generate one or more additional prediction blocks as additional hypotheses for the current block using data of the DPB 218. Furthermore, the video encoder 200 may be configured to apply any of the various techniques of this disclosure to encode motion information for the one or more additional prediction hypotheses.

[0115] For example, the motion compensation unit 224 may generate both the base mode hypothesis and the additional hypothesis. The motion compensation unit 224 may generate the base mode hypothesis using unidirectional inter prediction or bidirectional inter prediction. The motion compensation unit 224 may also generate the additional hypothesis using unidirectional inter prediction or bidirectional inter prediction. In some examples, the additional hypothesis may be limited to unidirectional prediction only. According to the techniques of this disclosure, the mode selection unit 202 may determine whether to encode the motion information for the additional hypothesis using a merge mode or an AMVP mode. In response to determining that the motion information for the additional hypothesis should be encoded using the merge mode, the mode selection unit 202 may assign a value to a merge mode syntax element indicating that the motion information for the additional hypothesis is encoded using the merge mode, and encode the motion information using the merge mode.

[0116]

[0109] In particular, to encode motion information for additional hypotheses using a merge mode, the mode selection unit 202 may build a merge candidate list. That is, the mode selection unit 202 may determine motion information from blocks neighboring the current block being encoded, and add motion information for the particular neighboring block to the merge candidate list. The mode selection unit 202 may be configured to limit the merge candidates to only uni-directional prediction candidates (also referred to as "uni-prediction candidates"). If the neighboring blocks are predicted using bi-prediction, the mode selection unit 202 may generate one or two merge candidates from the neighboring blocks, as described above with respect to FIG. 2. For example, the mode selection unit 202 may build a first merge candidate from a first set of motion information for the neighboring blocks (e.g., L0 motion information), and in some examples, build a second merge candidate from a second set of motion information for the neighboring blocks (e.g., L1 motion information).

[0117]

[0110] The mode selection unit 202 may also form a merge index having a value indicating which merge candidate should be used to encode the motion information for the additional hypothesis. In particular, the mode selection unit 202 may determine whether any of the merge candidates in the merge candidate list have motion information that matches the motion information formed by the motion estimation unit 222 for the additional hypothesis. The mode selection unit 202 may select one of the merge candidates in the merge candidate list that has the matching motion information as the merge candidate to be used to encode the motion information for the additional hypothesis.

[0118]

[0111] The motion compensation unit 224 may further combine the additional hypothesis with the base hypothesis, for example as an average or a weighted combination of each sample on a sample-by-sample basis. After combining the additional hypothesis with the base hypothesis, the motion compensation unit 224 may provide the resulting prediction block to the residual generation unit 204 and the reconstruction unit 214.

[0119]

[0112] Additionally or alternatively, the mode selection unit 202 may determine whether to apply local illumination compensation (LIC) to the additional hypothesis depending on whether LIC was applied to the base hypothesis. For example, the mode selection unit 202 may determine that LIC should be applied to the additional hypothesis if LIC was applied to the base hypothesis, and LIC should not be applied otherwise. As another example, if LIC is applied to the base hypothesis and the motion information for the additional hypothesis is encoded using AMVP, the mode selection unit 202 may determine that LIC should be performed on the additional hypothesis. Alternatively, if LIC is applied to the base hypothesis and the motion information for the additional hypothesis is encoded using merge mode, the mode selection unit 202 may determine whether to perform LIC on the additional hypothesis and encode a LIC syntax element (e.g., a LIC flag) indicating whether LIC should be performed on the additional hypothesis.

[0120] Additionally or alternatively, the mode selection unit 202 may determine which of various interpolation filters to use to interpolate values ​​for half-sample positions. In particular, if the motion vector for the base hypothesis of the previous block has half-sample accuracy and is coded using AMVP, and the previous block is selected as a candidate for coding the motion information of the additional hypothesis, the mode selection unit 202 may determine that a 6-tap interpolation filter should be used to interpolate values ​​for the half-sample positions.

[0121]

[0114] When operating according to the AV1 video coding format, the motion estimation unit 222 and the motion compensation unit 224 may be configured to encode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using translational motion compensation, affine motion compensation, overlapped block motion compensation (OBMC), and / or hybrid inter-intra prediction.

[0122]

[0115] As another example, for intra prediction, or intra predictive coding, the intra prediction unit 226 may generate a predictive block from samples adjacent to a current block. For example, in a directional mode, the intra prediction unit 226 may generally mathematically combine values ​​of adjacent samples and populate these calculated values ​​in a defined direction across the current block to produce a predictive block. As another example, in a DC mode, the intra prediction unit 226 may calculate an average of the adjacent samples for the current block and generate a predictive block to include this obtained average for each sample of the predictive block.

[0123] When operating according to the AV1 video coding format, the intra prediction unit 226 may be configured to encode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, chroma-from-luma (CFL) prediction, intra block copy (IBC), and / or color palette modes. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes.

[0124]

[0117] The mode selection unit 202 provides the prediction block to the residual generation unit 204. The residual generation unit 204 receives a raw uncoded version of the current block from the video data memory 230 and receives the prediction block from the mode selection unit 202. The residual generation unit 204 calculates sample-by-sample differences between the current block and the prediction block. The resulting sample-by-sample differences define a residual block for the current block. In some examples, the residual generation unit 204 may also determine differences between sample values ​​in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0125]

[0118] In an example where the mode selection unit 202 partitions a CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs having various sizes. As indicated above, the size of a CU may refer to the size of the luma coding block of the CU, and the size of a PU may refer to the size of the luma prediction unit of the PU. Assuming that the size of a particular CU is 2N×2N, the video encoder 200 may support a PU size of 2N×2N or N×N for intra prediction, and a symmetric PU size of 2N×2N, 2N×N, N×2N, N×N, or the like for inter prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning for PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0126] In an example in which the mode selection unit 202 does not further partition the CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As described above, the size of a CU may refer to the size of the luma coding block of the CU. The video encoder 200 and the video decoder 300 may support CU sizes of 2N×2N, 2N×N, or N×2N.

[0127]

[0120] In other video coding techniques, such as intra block copy mode coding, affine mode coding, and linear model (LM) mode coding, as some examples, the mode selection unit 202 generates a prediction block for the current block being coded via a respective unit associated with the coding technique. In some examples, such as palette mode coding, the mode selection unit 202 may not generate a prediction block, but instead generates a syntax element that indicates how the block should be reconstructed based on a selected palette. In such modes, the mode selection unit 202 may provide these syntax elements to be coded to the entropy coding unit 220.

[0128]

[0121] As described above, the residual generation unit 204 receives video data for a current block and a corresponding predictive block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates sample-by-sample differences between the predictive block and the current block.

[0129]

[0122] Transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, transform processing unit 206 may perform multiple transforms on the residual block, e.g., a linear transform and a secondary transform such as a rotation transform. In some examples, transform processing unit 206 does not apply a transform to the residual block.

[0130]

[0123] When operating according to AV1, transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a horizontal / vertical transform combination, which may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), an inverse ADST (e.g., a reverse ADST), and an identity transform (IDTX). When using the identity transform, the transform is skipped in either the vertical or horizontal direction. In some examples, the transform process may be skipped.

[0131]

[0124] The quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. The quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. The video encoder 200 (e.g., via the mode selection unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may result in loss of information, and thus the quantized transform coefficients may have lower precision than the original transform coefficients produced by the transform processing unit 206.

[0132]

[0125] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform to the quantized transform coefficient block, respectively, to reconstruct a residual block from the transform coefficient block. The reconstruction unit 214 may produce a reconstructed block corresponding to the current block (potentially with some distortion) based on the reconstructed residual block and the predictive block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the predictive block generated by the mode selection unit 202 to produce a reconstructed block.

[0133]

[0126] Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of a CU. The operations of filter unit 216 may be skipped in some examples.

[0134]

[0127] When operating according to AV1, filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of the CU. In another example, filter unit 216 may apply a constrained direction emphasis filter (CDEF) that may be applied after deblocking and may include application of a non-separable nonlinear low-pass directional filter based on the estimated edge direction. Filter unit 216 may also include a loop restoration filter applied after the CDEF and may include a separable symmetric normalized Wiener filter or a dual self-guiding filter.

[0135]

[0128] The video encoder 200 stores the reconstructed block in the DPB 218. For example, in an example where the operation of the filter unit 216 is not performed, the reconstruction unit 214 may store the reconstructed block in the DPB 218. In an example where the operation of the filter unit 216 is performed, the filter unit 216 may store the filtered reconstructed block in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 may retrieve a reference picture formed from the reconstructed (and potentially filtered) block from the DPB 218 to inter predict blocks of a picture to be subsequently encoded. Furthermore, the intra prediction unit 226 may use the reconstructed block in the DPB 218 of the current picture to intra predict other blocks in the current picture.

[0136] In general, the entropy encoding unit 220 may entropy encode syntax elements received from other functional components of the video encoder 200. For example, the entropy encoding unit 220 may entropy encode quantized transform coefficient blocks from the quantization unit 208. As another example, the entropy encoding unit 220 may entropy encode predictive syntax elements (e.g., motion information for inter prediction or intra mode information for intra prediction) from the mode selection unit 202. The entropy encoding unit 220 may perform one or more entropy encoding operations on syntax elements, which are another example of video data, to generate entropy encoded data. For example, the entropy encoding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioned entropy (PIPE) coding operation, an exponential-Golomb coding operation, or another type of entropy coding operation on the data. In some examples, the entropy encoding unit 220 may operate in a bypass mode in which syntax elements are not entropy coded.

[0137]

[0130] Video encoder 200 may output a bitstream that includes entropy-encoded syntax elements needed to reconstruct blocks of a slice or picture. In particular, entropy encoding unit 220 may output the bitstream.

[0138] According to AV1, the entropy coding unit 220 may be configured as an inter-symbol adaptive multi-symbol coder. A syntax element in AV1 includes an alphabet of N elements, and a context (e.g., a probability model) includes a set of N probabilities. The entropy coding unit 220 may store the probabilities as an n-bit (e.g., 15-bit) cumulative distribution function (CDF). The entropy coding unit 220 may perform recursive scaling with an update factor based on the alphabet size to update the context.

[0139]

[0132] The operations described above are described with respect to blocks. Such descriptions should be understood as being operations for luma coding blocks and / or chroma coding blocks. As described above, in some examples, the luma coding blocks and chroma coding blocks are luma and chroma components of a CU. In some examples, the luma coding blocks and chroma coding blocks are luma and chroma components of a PU.

[0140]

[0133] In some examples, the operations performed with respect to the luma coding block do not need to be repeated for the chroma coding block. As an example, the operations for identifying a motion vector (MV) and a reference picture for the luma coding block do not need to be repeated to identify the MV and the reference picture for the chroma block. Rather, the MV for the luma coding block may be scaled to determine the MV for the chroma block, and the reference picture may be the same. As another example, the intra prediction process may be the same for the luma coding block and the chroma coding block.

[0141]

[0134] Figure 4 is a block diagram illustrating an example video decoder 300 that may implement the techniques of this disclosure. Figure 4 is provided for illustrative purposes and is not intended to limit the techniques broadly illustrated and described in this disclosure. For illustrative purposes, this disclosure describes the video decoder 300 in accordance with VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265) techniques. However, the techniques of this disclosure may be implemented by video coding devices configured for other video coding standards.

[0142] In the example of FIG. 4, the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 314 may be implemented in one or more processors or processing circuits. For example, the units of the video decoder 300 may be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, an ASIC, or an FPGA. Moreover, the video decoder 300 may include additional or alternative processors or processing circuits for performing these and other functions.

[0143]

[0136] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units for performing prediction according to other prediction modes. By way of example, the prediction processing unit 304 may include a palette unit, an intra block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0144] When operating according to AV1, the compensation unit 316 may be configured to decode coding blocks of the video data (e.g., both luma coding blocks and chroma coding blocks) using translational motion compensation, affine motion compensation, OBMC, and / or hybrid inter-intra prediction, as described above. The intra prediction unit 318 may be configured to decode coding blocks of the video data (e.g., both luma coding blocks and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, CFL, intra block copy (IBC), and / or color palette mode, as described above.

[0145]

[0138] The CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of the video decoder 300. The video data stored in the CPB memory 320 may be obtained, for example, from the computer-readable medium 110 (FIG. 1). The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. The CPB memory 320 may also store video data other than syntax elements of a coded picture, such as temporary data representing output from various units of the video decoder 300. The DPB 314 generally stores decoded pictures that the video decoder 300 may output and / or use as reference video data when decoding subsequent data or pictures of the encoded video bitstream. The CPB memory 320 and the DPB 314 may be formed by any of a variety of memory devices, such as synchronous dynamic random access memory (DRAM), including DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The CPB memory 320 and the DPB 314 may be provided by the same memory device or separate memory devices. In various examples, the CPB memory 320 may be on-chip with other components of the video decoder 300 or off-chip relative to those components.

[0146] Additionally or alternatively, in some examples, the video decoder 300 may retrieve coded video data from memory 120 (FIG. 1). That is, memory 120 may store data as described above along with CPB memory 320. Similarly, memory 120 may store instructions to be executed by the video decoder 300 when some or all of the functionality of the video decoder 300 is implemented in software to be executed by processing circuitry of the video decoder 300.

[0147]

[0140] The various units shown in FIG. 4 are shown to aid in understanding the operations performed by the video decoder 300. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. As in FIG. 3, a fixed-function circuit refers to a circuit that provides a specific function and is preconfigured with respect to the operations that may be performed. A programmable circuit refers to a circuit that may be programmed to perform various tasks and provide flexible functionality in the operations that may be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. A fixed-function circuit may execute software instructions (e.g., to receive parameters or output parameters), but the type of operations that the fixed-function circuit performs is generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be an integrated circuit.

[0148]

[0141] The video decoder 300 may include a programmable core formed from an ALU, an EFU, a digital circuit, an analog circuit, and / or a programmable circuit. In examples in which the operation of the video decoder 300 is performed by software executing on a programmable circuit, an on-chip or off-chip memory may store instructions (e.g., object code) of the software that the video decoder 300 receives and executes.

[0149]

[0142] The entropy decoding unit 302 may receive the encoded video data from the CPB and entropy decode the video data to recover the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0150]

[0143] Generally, the video decoder 300 reconstructs a picture on a block-by-block basis. The video decoder 300 may perform a reconstruction operation on each block individually (wherein the block currently being reconstructed, i.e., decoded, may be referred to as the "current block").

[0151]

[0144] The entropy decoding unit 302 may entropy decode syntax elements defining the quantized transform coefficients of the quantized transform coefficient block, as well as transform information such as a quantization parameter (QP) and / or a transform mode indication(s). The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine the degree of quantization and, similarly, the degree of inverse quantization that the inverse quantization unit 306 should apply. The inverse quantization unit 306 may perform, for example, a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block including the transform coefficients.

[0152] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse transform, or another inverse transform to the transform coefficient block.

[0153]

[0146] Furthermore, the prediction processing unit 304 generates a prediction block according to the prediction information syntax element entropy decoded by the entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, the motion compensation unit 316 may generate a prediction block. In this case, the prediction information syntax element may indicate a reference picture in the DPB 314 from which the reference block should be retrieved, as well as a motion vector that identifies the location of the reference block in the reference picture relative to the location of the current block in the current picture. The motion compensation unit 316 may generally perform an inter-prediction process in a manner substantially similar to that described with respect to the motion compensation unit 224 (FIG. 3).

[0154] When performing multiple hypothesis inter prediction, the motion compensation unit 316 may generate one or more additional prediction blocks as additional hypotheses for the current block using data of the DPB 314. Furthermore, the video decoder 300 may be configured to apply any of the various techniques of this disclosure to decode motion information for the one or more additional prediction hypotheses.

[0155] For example, the motion compensation unit 316 may generate both the base mode hypothesis and the additional hypothesis. The motion compensation unit 316 may generate the base mode hypothesis using unidirectional inter prediction or bidirectional inter prediction. The motion compensation unit 316 may also generate the additional hypothesis using unidirectional inter prediction or bidirectional inter prediction. In some examples, the additional hypothesis may be limited to only unidirectional prediction. According to the techniques of this disclosure, the prediction processing unit 304 may determine whether to decode the motion information for the additional hypothesis using merge mode or AMVP mode. In particular, the prediction processing unit 304 may receive a value of a merge mode syntax element indicating whether the motion information for the additional hypothesis should be decoded using merge mode or AMVP mode, and decode the motion information using the indicated mode.

[0156]

[0149] To decode motion information for additional hypotheses using a merge mode, prediction processing unit 304 may build a merge candidate list. That is, prediction processing unit 304 may determine motion information from blocks neighboring the current block being coded, and add motion information for the particular neighboring block to the merge candidate list. Prediction processing unit 304 may be configured to limit merge candidates to only unidirectional prediction candidates (also referred to as "uni-prediction candidates"). If the neighboring block is predicted using bi-prediction, prediction processing unit 304 may generate one or two merge candidates from the neighboring blocks, as described above with respect to FIG. 2. For example, prediction processing unit 304 may build a first merge candidate from a first set of motion information for the neighboring blocks (e.g., L0 motion information), and in some examples, build a second merge candidate from a second set of motion information for the neighboring blocks (e.g., L1 motion information).

[0157]

[0150] The prediction processing unit 304 may also receive a merge index having a value indicating which merge candidate should be used to decode the motion information for the additional hypothesis. The prediction processing unit 304 may determine one of the merge candidates in the merge candidate list corresponding to the merge index as the merge candidate to be used to decode the motion information for the additional hypothesis.

[0158]

[0151] The motion compensation unit 316 may further combine the additional hypothesis with the base hypothesis, for example as an average or a weighted combination of each sample on a sample-by-sample basis. After combining the additional hypothesis with the base hypothesis, the motion compensation unit 316 may provide the resulting prediction block to the reconstruction unit 310.

[0159]

[0152] Additionally or alternatively, the prediction processing unit 304 may determine whether to apply local illumination compensation (LIC) to the additional hypothesis depending on whether LIC was applied to the base hypothesis. For example, the prediction processing unit 304 may determine that LIC should be applied to the additional hypothesis if LIC was applied to the base hypothesis, and LIC should not be applied otherwise. As another example, if LIC is applied to the base hypothesis and the motion information for the additional hypothesis is decoded using AMVP, the prediction processing unit 304 may determine that LIC should be performed on the additional hypothesis. Alternatively, if LIC is applied to the base hypothesis and the motion information for the additional hypothesis is decoded using merge mode, the prediction processing unit 304 may decode a LIC syntax element (e.g., a LIC flag) indicating whether to perform LIC on the additional hypothesis.

[0160]

[0153] Additionally or alternatively, the prediction processing unit 304 may determine which of various interpolation filters to use to interpolate values ​​for half-sample positions. In particular, if the motion vector for the base hypothesis of the previous block has half-sample accuracy and is decoded using AMVP, and the previous block is selected as a candidate for decoding motion information of an additional hypothesis, the prediction processing unit 304 may determine that a 6-tap interpolation filter should be used to interpolate values ​​for half-sample positions.

[0161]

[0154] As another example, if the prediction information syntax element indicates that the current block is intra predicted, the intra prediction unit 318 may generate a prediction block according to the intra prediction mode indicated by the prediction information syntax element. Again, the intra prediction unit 318 may generally perform an intra prediction process in a manner substantially similar to that described with respect to the intra prediction unit 226 (FIG. 3). The intra prediction unit 318 may retrieve data of neighboring samples for the current block from the DPB 314.

[0162]

[0155] The reconstruction unit 310 may reconstruct the current block using the predictive block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the predictive block to reconstruct the current block.

[0163]

[0156] The filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blockiness artifacts along edges of the reconstructed block. The operations of the filter unit 312 are not necessarily performed in all examples.

[0164]

[0157] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in an example where the operation of the filter unit 312 is not performed, the reconstruction unit 310 may store the reconstructed block in the DPB 314. In an example where the operation of the filter unit 312 is performed, the filter unit 312 may store the filtered reconstructed block in the DPB 314. As described above, the DPB 314 may provide reference information to the prediction processing unit 304, such as samples of a current picture for intra prediction and previously decoded pictures for subsequent motion compensation. Moreover, the video decoder 300 may output the decoded picture (e.g., decoded video) from the DPB 314 for subsequent presentation on a display device, such as the display device 118 of FIG. 1.

[0165]

[0158] Figure 5 is a flowchart illustrating an example method for encoding a current block according to the techniques of this disclosure. The current block may comprise a current CU. Although described with respect to video encoder 200 (Figures 1 and 3), it should be understood that other devices may be configured to implement a method similar to that of Figure 5.

[0166]

[0159] In this example, the video encoder 200 first predicts the current block (350). For example, the video encoder 200 may form a predictive block for the current block. In particular, the video encoder 200 may form the predictive block using multiple hypothesis inter prediction. The video encoder 200 may then calculate a residual block for the current block (352). To calculate the residual block, the video encoder 200 may calculate a difference between an original uncoded block and a predictive block for the current block. The video encoder 200 may then transform the residual block and quantize transform coefficients of the residual block (354). Next, the video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or following the scan, the video encoder 200 may entropy code the coefficients and the prediction information (358). For example, the video encoder 200 may encode the data using CAVLC or CABAC. Moreover, video encoder 200 may encode the motion information for the one or more additional hypotheses using a merge mode in accordance with any of the various techniques of this disclosure. Video encoder 200 may then output entropy coded data for the block (360).

[0167]

[0160] The video encoder 200 may also decode the current block after encoding the current block to use the decoded version of the current block as reference data for later coded data (e.g., in an inter-prediction mode or an intra-prediction mode). Thus, the video encoder 200 may dequantize and inverse transform the coefficients to reconstruct the residual block (362). The video encoder 200 may combine the residual block with the predictive block to form a decoded block (364). In some examples, the video encoder 200 may perform local illumination compensation (LIC) according to any of the various techniques of this disclosure. The video encoder 200 may then store the decoded block in the DPB 218 (366).

[0168]

[0161] Figure 6 is a flowchart illustrating an example method for decoding a current block of video data in accordance with the techniques of this disclosure. The current block may comprise a current CU. Although described with respect to video decoder 300 (Figures 1 and 4), it should be understood that other devices may be configured to perform a method similar to that of Figure 6.

[0169]

[0162] The video decoder 300 may receive entropy coded data for the current block, such as the entropy coded prediction information and the entropy coded data for the transform coefficients of the residual block corresponding to the current block (370). The video decoder 300 may entropy decode the entropy coded data to determine the prediction information for the current block and to reconstruct the transform coefficients of the residual block (372). For example, the video decoder 300 may entropy decode motion information for one or more additional prediction hypotheses in a merge mode using any of the various techniques of this disclosure. The video decoder 300 may predict the current block, for example, using a multiple hypothesis inter prediction mode indicated by the prediction information for the current block, to compute a predictive block for the current block (374). The video decoder 300 may then inverse scan the reconstructed transform coefficients to create a block of quantized transform coefficients (376). The video decoder 300 may then inverse quantize the transform coefficients and apply an inverse transform to the transform coefficients to produce the residual block (378). The video decoder 300 may finally decode the current block by combining the predictive block and the residual block (380). In some examples, the video decoder 300 may further perform local illumination compensation (LIC) on the decoded block in accordance with any of the various techniques of this disclosure.

[0170]

[0163] FIG. 7 is a flow chart illustrating an example method of decoding video data according to the techniques of this disclosure. The method of FIG. 7 may be implemented by a video encoder, such as the video encoder 200 of FIG. 1 and FIG. 3. When implemented as part of a video encoding process, the method of FIG. 7 may correspond to steps 350 and 362-364 of FIG. 5. Alternatively, the method of FIG. 7 may be implemented by a video decoder, such as the video decoder 300 of FIG. 1 and FIG. 4. When implemented as part of a video decoding process, the method of FIG. 7 may correspond to steps 374 and 380 of FIG. 6. Although the method of FIG. 7 is described as a method of decoding video data, the video encoder both encodes and decodes the video data for use as subsequent prediction data, as described above. Thus, the method of FIG. 7 may be implemented by both a video encoder and a video decoder. For illustrative purposes, the method of FIG. 7 is described with reference to the video decoder 300, although certain aspects are also described with reference to the video encoder 200.

[0171]

[0164] First, the video decoder 300 may form a base hypothesis prediction block for a current block of video data (400). For example, the video decoder 300 may perform unidirectional prediction (using a single set of motion information) or bidirectional prediction (using two sets of motion information) to form the base hypothesis prediction block.

[0172]

[0165] The video decoder 300 may also code (i.e., decode) a merge mode syntax element for the additional hypothesis block relative to the current block (402). The value of the merge mode syntax element may indicate whether the motion information for the additional hypothesis block is coded using merge mode or AMVP mode. If implemented by video encoder 200, video encoder 200 may encode the value of the merge mode syntax element.

[0173]

[0166] When the merge mode syntax element indicates that the motion information for the additional hypothesis block is predicted using the merge mode, the video decoder 300 may form a merge candidate list (404). The merge candidate list may correspond to merge candidates that represent the motion information of blocks neighboring the current block. The video decoder 300 may ensure that each of the merge candidates in the merge candidate list includes only unidirectional motion information. If the motion information for one of the neighboring blocks is bi-predictively coded, the video decoder 300 may form one or two merge candidates from one of the neighboring blocks, for example, a single merge candidate including one of the two sets of motion information, or a first merge candidate including one of the two sets of motion information (e.g., L0 motion information) and a second merge candidate including the other of the two sets of motion information (e.g., L1 motion information). In this manner, each merge candidate in the merge candidate list may be restricted to unidirectional prediction.

[0174]

[0167] Then, the video decoder 300 may code the motion information for the additional hypothesis block (406). For example, assuming that the motion information for the additional hypothesis block is coded in merge mode, the video decoder 300 may decode a merge index that identifies one of the merge candidates in the merge candidate list. The video decoder 300 may then use the motion information of the identified one of the merge candidates as the motion information to form the additional hypothesis block. On the other hand, if the motion information for the additional hypothesis block is coded in AMVP mode, the video decoder 300 may decode an AMVP candidate index, a reference picture list identifier, a reference picture index, and motion vector difference information. Alternatively, if the method is implemented by the video encoder 200, the video encoder 200 may encode the merge index or the AMVP candidate index, the reference picture list identifier, the reference picture index, and the motion vector difference information.

[0175]

[0168] The video decoder 300 may then generate additional hypothesis blocks using the motion information (408). The video decoder 300 may further form a multi-hypothesis prediction block from the base hypothesis prediction block and the additional hypothesis block (410). For example, the video decoder 300 may average co-located samples of the base hypothesis prediction block and the additional hypothesis block, or perform a weighted combination of co-located samples of the base hypothesis prediction block and the additional hypothesis block. In some examples, the video decoder 300 may perform local illumination compensation (LIC) on either or both of the base hypothesis block and / or the additional hypothesis block before forming the multi-hypothesis prediction block.

[0176]

[0169] Finally, the video decoder 300 may decode (i.e., reconstruct) the current block using the multi-hypothesis prediction block (412). For example, the video decoder 300 may decode quantized transform coefficients for the current block, dequantize and transform the quantized transform coefficients to form residual samples, and inverse scan the residual samples to reconstruct the residual block. The video decoder 300 may then combine the multi-hypothesis prediction block with the residual block on a sample-by-sample basis to reconstruct (decode) the current block.

[0177]

[0170]

[0046] Thus, the method of FIG. 7 represents an example of a method for decoding (and / or encoding) video data, the method including: generating a first predictive block for a current block of video data using a base inter-prediction mode; coding a merge mode syntax element for the second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode; coding the motion information for the second predictive block in accordance with the merge mode syntax element; and when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information.

[0178]

[0171] Various examples of the techniques of the present disclosure are summarized in the following paragraphs.

[0179]

[0172] Clause 1: A method for decoding decoded video data, comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; generating a second predictive block for the current block of video data as an additional prediction hypothesis; determining whether to apply local illumination compensation (LIC) to the second predictive block depending on whether LIC has been applied to the first predictive block; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0180]

[0173] Clause 2: The method of clause 1, wherein the base inter prediction mode comprises one of unidirectional inter prediction or bidirectional inter prediction.

[0181]

[0174] Clause 3: A method as described in any of clauses 1 and 2, further comprising applying LIC to the first predictive block, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block.

[0182]

[0175] Clause 4: A method as described in any of clauses 1 and 2, further comprising applying LIC to a first predictive block and coding motion information for the second predictive block using an advanced motion vector prediction (AMVP) mode, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block and coding the motion information for the second predictive block using the AMVP mode.

[0183]

[0176] Clause 5: A method according to any of clauses 1 and 2, further comprising applying LIC to a first predictive block and coding motion information for the second predictive block using a merge mode, wherein determining whether to apply LIC to the second predictive block comprises coding a LIC flag for a merge candidate of the second predictive block and determining whether to apply LIC to the second predictive block according to a value of the LIC flag for the merge candidate of the second predictive block.

[0184]

[0177] Clause 6: A method for decoding video data, the method comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode; coding motion information for the second predictive block in accordance with the merge mode syntax element; generating a second predictive block for the current block of video data using the motion information; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0185]

[0178] Clause 7: A method for coding video data, comprising a combination of the method according to any of clauses 1 to 5 and the method according to clause 6.

[0186]

[0179] Clause 8: A method according to any of clauses 6 and 7, wherein when the merge mode syntax element indicates that a merge mode is used, coding motion information for the second predictive block comprises coding a merge index indicating a merge candidate from which to inherit the motion information for the second predictive block.

[0187]

[0180] Clause 9: A method according to any of clauses 6 and 7, wherein when the merge mode syntax element indicates that merge mode is not used, coding motion information for the second predictive block comprises coding a reference picture index, a motion vector predictor index, and a motion vector differential for the second predictive block.

[0188]

[0181] Clause 10: A method described in any of clauses 6 to 9, wherein coding the motion information for the second predictive block comprises forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of unidirectional predictive motion information.

[0189]

[0182] Clause 11: The method described in clause 10, wherein the second prediction block comprises one additional motion hypothesis among a plurality of additional motion hypotheses, and further comprises coding motion information for each of the additional motion hypotheses using the same merge candidate list.

[0190]

[0183] Clause 12: A method according to any of clauses 10 and 11, wherein forming the merge candidate list comprises forming the merge candidate list according to a geometric partitioning mode (GPM) merge candidate list construction process.

[0191]

[0184] Clause 13: The method of any of clauses 6 to 9, wherein the second prediction block comprises one additional motion hypothesis among a plurality of additional motion hypotheses, wherein coding the motion information for the second prediction block comprises forming a merge candidate list including two or more merge candidates, selecting one of the merge candidates from the merge candidate list for the motion information for the second prediction block, and coding the motion information for the second prediction block using one of the merge candidates, the method further comprising removing one of the merge candidates from the merge candidate list to form a reduced merge candidate list, and coding the motion information for a third prediction candidate using one of the remaining merge candidates in the reduced merge candidate list.

[0192]

[0185] Clause 14: The method of clause 13, further comprising: coding a first merge candidate index for a second prediction block, the merge candidate list having N merge candidates, N being a positive integer greater than 1, the first merge candidate index having a maximum possible truncated binary value of N; coding a second merge candidate index for a third prediction block, the reduced merge candidate list having N-1 merge candidates, the second merge candidate index having a maximum possible truncated binary value of N-1.

[0193]

[0186] Clause 15: The method of any of clauses 6 to 14, wherein the merge mode syntax element comprises a merge mode syntax element for a base inter prediction mode of the first prediction block.

[0194]

[0187] Clause 16: A method as described in any of clauses 6 to 14, wherein the merge mode syntax element comprises a second merge mode syntax element, and the method further comprises coding a first merge mode syntax element for a base inter prediction mode of the first predictive block, and the second merge mode syntax element is different from the first merge mode syntax element.

[0195]

[0188] Clause 17: The method of clause 16, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and coding the second merge mode syntax element comprises coding the second merge mode syntax element using the first CABAC context in response to coding the first merge mode syntax element using the first CABAC context.

[0196]

[0189] Clause 18: The method according to clause 17, wherein the first CABAC context is a CABAC context used for the merge mode.

[0197]

[0190] Clause 19: The method according to clause 17, wherein the first CABAC context is different from the second CABAC context used for the merge mode.

[0198]

[0191] Clause 20: The method of clause 16, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using a second CABAC context.

[0199]

[0192] Clause 21: The method of clause 20, wherein the second CABAC context is different from the first CABAC context.

[0200]

[0193] Clause 22: The method according to any of clauses 6 to 21, further comprising coding motion information of a third prediction block using an advanced motion vector prediction (AMVP) mode, wherein coding the motion information of the third prediction block includes coding motion vector difference (MVD) values ​​having half-luma sample resolution, the third prediction block being a base inter-prediction block of an adjacent block to the current block, and coding a candidate index identifying the third prediction block as a motion information candidate for the second prediction block, wherein generating the second prediction block comprises interpolating half-pixel samples of a reference block of the second prediction block using a 6-tap interpolation filter.

[0201]

[0194] Clause 23: A method for decoding video data, the method comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; generating a second predictive block for the current block of video data as an additional prediction hypothesis; determining whether to apply local illumination compensation (LIC) to the second predictive block depending on whether LIC has been applied to the first predictive block; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0202]

[0195] Clause 24: The method of clause 23, wherein the base inter prediction mode comprises one of unidirectional inter prediction or bidirectional inter prediction.

[0203]

[0196] Clause 25: The method of clause 23, further comprising applying LIC to the first predictive block, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block.

[0204]

[0197] Clause 26: The method of clause 23, further comprising applying LIC to the first predictive block and coding motion information for the second predictive block using an advanced motion vector prediction (AMVP) mode, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block and coding the motion information for the second predictive block using AMVP.

[0205]

[0198] Clause 27: The method of clause 23, further comprising applying LIC to a first predictive block and coding motion information for the second predictive block using a merge mode, wherein determining whether to apply LIC to the second predictive block comprises coding a LIC flag for a merge candidate of the second predictive block and determining whether to apply LIC to the second predictive block according to a value of the LIC flag for the merge candidate of the second predictive block.

[0206]

[0199] Clause 28: A method of decoding video data, the method comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode, coding motion information for the second predictive block in accordance with the merge mode syntax element, generating a second predictive block for the current block of video data using the motion information, forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block, and decoding the current block using the multi-hypothesis predictive block.

[0207]

[0200] Clause 29: The method described in clause 28, when the merge mode syntax element indicates that a merge mode is used, coding motion information for the second predictive block comprises coding a merge index indicating a merge candidate from which to inherit the motion information for the second predictive block.

[0208]

[0201] Clause 30: The method described in clause 28, wherein when the merge mode syntax element indicates that merge mode is not used, coding motion information for the second predictive block comprises coding a reference picture index, a motion vector predictor index, and a motion vector differential for the second predictive block.

[0209]

[0202] Clause 31: The method described in clause 28, wherein coding the motion information for the second predictive block comprises forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of unidirectional predictive motion information.

[0210]

[0203] Clause 32: The method described in Clause 31, wherein the second prediction block comprises one additional prediction hypothesis among a plurality of additional motion hypotheses, and the method further comprises coding motion information for each of the additional motion hypotheses using the same merge candidate list.

[0211]

[0204] Clause 33: The method of clause 31, wherein forming the merge candidate list comprises forming the merge candidate list according to a geometric partitioning mode (GPM) merge candidate list construction process.

[0212]

[0205] Clause 34: The method according to Clause 28, wherein the second predictive block comprises one additional prediction hypothesis among a plurality of additional motion hypotheses, wherein coding the motion information for the second predictive block comprises forming a merge candidate list including two or more merge candidates, selecting one of the merge candidates from the merge candidate list for the motion information for the second predictive block, and coding the motion information for the second predictive block using one of the merge candidates, and the method comprises removing one of the merge candidates from the merge candidate list to form a reduced merge candidate list, and coding the motion information for the third predictive block using one of the remaining merge candidates in the reduced merge candidate list.

[0213]

[0206] Clause 35: The method of clause 34, coding a first merge candidate index for a second predictive block, where the merge candidate list has N merge candidates, N being a positive integer greater than 1, and the first merge candidate index has the largest possible truncated binary value of N; coding a second merge candidate index for a third predictive block, where the reduced merge candidate list has N-1 merge candidates, and the second merge candidate index has the largest possible truncated binary value of N-1.

[0214]

[0207] Clause 36: The method of clause 28, wherein the merge mode syntax element comprises a merge mode syntax element for a base inter prediction mode of the first predictive block.

[0215]

[0208] Clause 37: The method described in Clause 28, wherein the merge mode syntax element comprises a second merge mode syntax element, and the method further comprises coding a first merge mode syntax element for a base inter prediction mode of the first predictive block, and the second merge mode syntax element is different from the first merge mode syntax element.

[0216]

[0209] Clause 38: The method of clause 37, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using the first CABAC context in response to coding the first merge mode syntax element using the first CABAC context.

[0217]

[0210] Clause 39: The method according to clause 38, wherein the first CABAC context is a CABAC context used for the merge mode.

[0218]

[0211] Clause 40: The method according to clause 38, wherein the first CABAC context is different from the second CABAC context used for the merge mode.

[0219]

[0212] Clause 41: The method of clause 37, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using a second CABAC context.

[0220]

[0213] Clause 42: The method of clause 41, wherein the second CABAC context is different from the first CABAC context.

[0221]

[0214] Clause 43: The method further comprises coding motion information of a third prediction block using an advanced motion vector prediction (AMVP) mode, wherein coding the motion information of the third prediction block includes coding motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third prediction block being a base inter prediction block of an adjacent block to the current block, and coding a candidate index identifying the third prediction block as a motion information candidate for the second prediction block, wherein generating the second prediction block comprises interpolating half-pixel samples of a reference block of the second prediction block using a 6-tap interpolation filter. The method described in Clause 28.

[0222]

[0215] Clause 44: The method of any of clauses 1 to 43, further comprising encoding the current block prior to decoding the current block.

[0223]

[0216] Clause 45: A device for decoding video data, the device comprising one or more means for performing the method according to any of clauses 1 to 44.

[0224]

[0217] Clause 46: A device as described in clause 45, wherein the one or more means comprise one or more processors implemented in circuitry.

[0225]

[0218] Clause 47: The device of any of clauses 45 and 46, further comprising a display configured to display the decoded video data.

[0226]

[0219] Clause 48: A device according to any of clauses 45 to 47, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0227]

[0220] Clause 49: The device of any one of clauses 45 to 48, further comprising a memory configured to store video data.

[0228]

[0221] Clause 50: A computer-readable storage medium storing instructions which, when executed, cause a processor of a device for decoding video data to perform a method according to any of clauses 1 to 44.

[0229]

[0222] Clause 51: A device for decoding video data, the device comprising: means for generating a first predictive block for a current block of video data using a base inter prediction mode; means for generating a second predictive block for the current block of video data as an additional prediction hypothesis; means for determining whether to apply local illumination compensation (LIC) to the second predictive block depending on whether LIC has been applied to the first predictive block; means for forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and means for decoding the current block using the multi-hypothesis predictive block.

[0230]

[0223] Clause 52: A device for decoding video data, the device comprising: means for generating a first predictive block for a current block of video data using a base inter prediction mode; means for coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode, means for coding motion information for the second predictive block in accordance with the merge mode syntax element, means for generating a second predictive block for the current block of video data using the motion information, means for forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block, and means for decoding the current block using the multi-hypothesis predictive block.

[0231]

[0224] Clause 53: A method of decoding video data, the method comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; coding a merge mode syntax element for the second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode; coding the motion information for the second predictive block in accordance with the merge mode syntax element; and when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information. The method comprises: generating a second predictive block for the current block of video data using the motion information; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0232]

[0225] Clause 54: The method described in clause 53, when the merge mode syntax element indicates that a merge mode is used, coding motion information for the second predictive block comprises coding a merge index indicating a merge candidate from which to inherit the motion information for the second predictive block.

[0233]

[0226] Clause 55: The method described in clause 53, when the merge mode syntax element indicates that merge mode is not used, coding motion information for the second predictive block comprises coding a reference picture index, a motion vector predictor index, and a motion vector differential for the second predictive block.

[0234]

[0227] Clause 56: The method of clause 53, comprising: forming a merge candidate list comprising determining that an adjacent block to the second predictive block is bi-predicted using the first motion information and the second motion information; forming at least one uni-predictive merge candidate from the adjacent block; and forming the at least one uni-predictive merge candidate comprising selecting only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate.

[0235]

[0228] Clause 57: The method described in clause 56, wherein forming a merge candidate list comprises forming a first uni-predictive merge candidate using first motion information, and forming a second uni-predictive merge candidate using second motion information.

[0236]

[0229] Clause 58: The method described in Clause 53, wherein the second prediction block comprises one additional prediction hypothesis among a plurality of additional motion hypotheses, and the method further comprises coding motion information for each of the additional motion hypotheses using the same merge candidate list.

[0237]

[0230] Clause 59: The method of clause 53, wherein forming the merge candidate list comprises forming the merge candidate list according to a geometric partitioning mode (GPM) merge candidate list construction process.

[0238]

[0231] Clause 60: The method described in Clause 53, wherein the second prediction block comprises one additional prediction hypothesis among a plurality of additional motion hypotheses, and wherein coding the motion information for the second prediction block comprises forming a merge candidate list including two or more merge candidates, selecting one of the merge candidates from the merge candidate list for the motion information for the second prediction block, and coding the motion information for the second prediction block using one of the merge candidates, and the method further comprises removing one of the merge candidates from the merge candidate list to form a reduced merge candidate list, and coding the motion information for a third prediction candidate using one of the remaining merge candidates in the reduced merge candidate list.

[0239]

[0232] Clause 61: The method of clause 60, further comprising coding a first merge candidate index for the second predictive block, the merge candidate list having N merge candidates, N being a positive integer greater than 1, and the first merge candidate index having the largest possible truncated binary value of N; coding a second merge candidate index for the third predictive block, the reduced merge candidate list having N-1 merge candidates, and the second merge candidate index having the largest possible truncated binary value of N-1.

[0240]

[0233] Clause 62: The method of clause 53, wherein the merge mode syntax element comprises a merge mode syntax element for a base inter prediction mode of the first predictive block.

[0241]

[0234] Clause 63: The method described in Clause 53, wherein the merge mode syntax element comprises a second merge mode syntax element, and the method further comprises coding a first merge mode syntax element for a base inter prediction mode of the first predictive block, and the second merge mode syntax element is different from the first merge mode syntax element.

[0242]

[0235] Clause 64: The method of clause 63, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using the first CABAC context in response to coding the first merge mode syntax element using the first CABAC context.

[0243]

[0236] Clause 65: The method according to clause 64, wherein the first CABAC context is a CABAC context used for the merge mode.

[0244]

[0237] Clause 66: The method of clause 64, wherein the first CABAC context is different from the second CABAC context used for the merge mode.

[0245]

[0238] Clause 67: The method of clause 63, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using a second CABAC context.

[0246]

[0239] Clause 68: The method of clause 67, wherein the second CABAC context is different from the first CABAC context.

[0247]

[0240] Clause 69: The method of clause 53, further comprising coding motion information of a third predictive block using an advanced motion vector prediction (AMVP) mode, and wherein coding the motion information of the third predictive block includes coding motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third predictive block being a base inter prediction block of an adjacent block to the current block, and coding a candidate index identifying the third predictive block as a motion information candidate for the second predictive block, wherein generating the second predictive block comprises interpolating half-pixel samples of a reference block of the second predictive block using a 6-tap interpolation filter.

[0248]

[0241] Clause 70: The method of clause 53, further comprising determining whether to apply local illumination compensation (LIC) to the second predicted block depending on whether LIC has been applied to the first predicted block.

[0249]

[0242] Clause 71: The method of clause 70, wherein the base inter prediction mode comprises one of unidirectional inter prediction or bidirectional inter prediction.

[0250]

[0243] Clause 72: The method of clause 70, further comprising applying LIC to the first predictive block, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block.

[0251]

[0244] Clause 73: The method of clause 70, further comprising applying LIC to the first predictive block and coding motion information for the second predictive block using an advanced motion vector prediction (AMVP) mode, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block and coding the motion information for the second predictive block using AMVP.

[0252]

[0245] Clause 74: The method of clause 70, further comprising applying LIC to a first predictive block and coding motion information for the second predictive block using a merge mode, wherein determining whether to apply LIC to the second predictive block comprises coding a LIC flag for a merge candidate of the second predictive block and determining whether to apply LIC to the second predictive block according to a value of the LIC flag for the merge candidate of the second predictive block.

[0253]

[0246] Clause 75: The method of clause 53, further comprising encoding the current block prior to decoding the current block.

[0254]

[0247] Clause 76: A device for decoding video data, the device comprising: a memory configured to store the video data; and one or more processors implemented in a circuit, the one or more processors generating a first predictive block for a current block of the video data using a base inter prediction mode; and coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, wherein the merge mode syntax element indicates whether motion information for the second predictive block is coded using the merge mode. and coding the motion information using the first and second predictive blocks, wherein, for coding the motion information, the one or more processors are configured to form a merge candidate list including one or more merge candidates when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, each of the merge candidates representing a respective set of uni-predictive motion information.

[0255]

[0248] Clause 77: The device described in Clause 76, configured to: determine, to form a merge candidate list, that an adjacent block to the second predictive block is bi-predicted using the first motion information and the second motion information; and form at least one uni-predictive merge candidate from the adjacent block, wherein the one or more processors are configured to select only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate.

[0256]

[0249] Clause 78: A device as described in Clause 77, wherein to form a merge candidate list, one or more processors are configured to form a first uni-predictive merge candidate using the first motion information and to form a second uni-predictive merge candidate using the second motion information.

[0257]

[0250] Clause 79: The device described in Clause 76, wherein the one or more processors are further configured to code motion information of a third predictive block using an advanced motion vector prediction (AMVP) mode, wherein, for coding the motion information of the third predictive block, the one or more processors are configured to code motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third predictive block being a base inter prediction block of an adjacent block to the current block, and to code a candidate index identifying the third predictive block as a motion information candidate for a second predictive block, the third predictive block being a base inter prediction block of an adjacent block to the current block, wherein, to generate the second predictive block, the one or more processors are configured to interpolate half-pixel samples of a reference block of the second predictive block using a 6-tap interpolation filter.

[0258]

[0251] Clause 80: The device described in Clause 76, wherein the one or more processors are further configured to determine whether to apply local illumination compensation (LIC) to the second predicted block depending on whether LIC has been applied to the first predicted block.

[0259]

[0252] Clause 81: The device of clause 76, wherein the one or more processors are further configured to encode the current block prior to decoding the current block.

[0260]

[0253] Clause 82: The device of clause 76, further comprising a display configured to display the decoded video data.

[0261]

[0254] Clause 83: The device of clause 76, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0262]

[0255] Clause 84: A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to generate a first predictive block for a current block of video data using a base inter prediction mode; code a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using a merge mode; and code motion information for the second predictive block in accordance with the merge mode syntax element. 22. The method of claim 21, further comprising: coding motion information for a current block of video data using a merge mode syntax element that includes generating a second predictive block for the current block of video data using the motion information; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0263]

[0256] Clause 85: A computer-readable storage medium as described in Clause 84, comprising instructions for causing a processor to form a merge candidate list, the instructions causing the processor to determine that an adjacent block to the second predictive block is bi-predicted using the first motion information and the second motion information, and to form at least one uni-predictive merge candidate from the adjacent block, and forming the at least one uni-predictive merge candidate includes instructions for causing the processor to select only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate.

[0264]

[0257] Clause 86: A computer-readable storage medium as described in Clause 85, wherein the instructions for causing a processor to form a merge candidate list include instructions for causing the processor to form a first uni-predictive merge candidate using the first motion information and to form a second uni-predictive merge candidate using the second motion information.

[0265]

[0258] Clause 87: A computer-readable storage medium as described in Clause 84, wherein the instructions that cause the processor to code motion information of a third predictive block using an advanced motion vector prediction (AMVP) mode, and wherein the instructions that cause the processor to code the motion information of the third predictive block comprise instructions that cause the processor to code motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third predictive block being a base inter prediction block of an adjacent block to the current block, and further comprise instructions that cause the processor to code a candidate index that identifies the third predictive block as a motion information candidate for a second predictive block, the third predictive block being a base inter prediction block of an adjacent block to the current block, and wherein the instructions that cause the processor to generate the second predictive block comprise instructions that cause the processor to interpolate half-pixel samples of a reference block of the second predictive block using a 6-tap interpolation filter.

[0266]

[0259] Clause 88: A computer-readable storage medium as described in clause 84, further comprising instructions for causing a processor to determine whether local lighting compensation (LIC) should be applied to the second predicted block depending on whether LIC has been applied to the first predicted block.

[0267]

[0260] Clause 89: The computer-readable storage medium of clause 84, further comprising instructions that cause the processor to encode the current block prior to decoding the current block.

[0268]

[0261] Clause 90: A device for decoding video data, the device comprising: means for generating a first predictive block for a current block of the video data using a base inter prediction mode; means for coding a merge mode syntax element for the second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode; means for coding the motion information for the second predictive block in accordance with the merge mode syntax element; and when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, the means for coding the motion information for the second predictive block includes means for forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information, means for generating a second predictive block for the current block of the video data using the motion information; means for forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and means for decoding the current block using the multi-hypothesis predictive block.

[0269]

[0262] Clause 91: A method for decoding video data, the method comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; coding a merge mode syntax element for the second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using the merge mode; coding the motion information for the second predictive block in accordance with the merge mode syntax element; and when the merge mode syntax element indicates that the motion information for the second predictive block is coded using the merge mode, forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information. The method comprises: generating a second predictive block for the current block of video data using the motion information; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0270]

[0263] Clause 92: The method described in Clause 91, when the merge mode syntax element indicates that a merge mode is used, coding motion information for the second predictive block comprises coding a merge index indicating a merge candidate from which to inherit the motion information for the second predictive block.

[0271]

[0264] Clause 93: A method according to any of clauses 91 and 92, wherein when the merge mode syntax element indicates that merge mode is not used, coding motion information for the second predictive block comprises coding a reference picture index, a motion vector predictor index, and a motion vector differential for the second predictive block.

[0272]

[0265] Clause 94: A method according to any of clauses 91 to 93, comprising: forming a merge candidate list comprising determining that an adjacent block to the second predictive block is bi-predicted using the first motion information and the second motion information; forming at least one uni-predictive merge candidate from the adjacent block; and forming the at least one uni-predictive merge candidate comprising selecting only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate.

[0273]

[0266] Clause 95: The method described in clause 94, wherein forming a merge candidate list comprises forming a first uni-predictive merge candidate using first motion information, and forming a second uni-predictive merge candidate using second motion information.

[0274]

[0267] Clause 96: A method as described in any of clauses 91 to 95, wherein the second prediction block comprises one additional prediction hypothesis among a plurality of additional motion hypotheses, and the method further comprises coding motion information for each of the additional motion hypotheses using the same merge candidate list.

[0275]

[0268] Clause 97: The method of any of clauses 91 to 96, wherein forming the merge candidate list comprises forming the merge candidate list according to a geometric partitioning mode (GPM) merge candidate list construction process.

[0276]

[0269] Clause 98: A method according to any of clauses 91 to 97, wherein the second prediction block comprises one additional prediction hypothesis among a plurality of additional motion hypotheses, and wherein coding the motion information for the second prediction block comprises forming a merge candidate list including two or more merge candidates, selecting one of the merge candidates from the merge candidate list for the motion information for the second prediction block, and coding the motion information for the second prediction block using one of the merge candidates, and the method further comprises removing one of the merge candidates from the merge candidate list to form a reduced merge candidate list, and coding the motion information for the third prediction candidate using one of the remaining merge candidates in the reduced merge candidate list.

[0277]

[0270] Clause 99: The method of clause 98, further comprising: coding a first merge candidate index for a second predictive block, the merge candidate list having N merge candidates, N being a positive integer greater than 1, and the first merge candidate index having the largest possible truncated binary value of N; coding a second merge candidate index for a third predictive block, the reduced merge candidate list having N-1 merge candidates, and the second merge candidate index having the largest possible truncated binary value of N-1.

[0278]

[0271] Clause 100: The method of any of clauses 91 to 99, wherein the merge mode syntax element comprises a merge mode syntax element for a base inter prediction mode of the first predictive block.

[0279]

[0272] Clause 101: A method according to any of clauses 91 to 100, wherein the merge mode syntax element comprises a second merge mode syntax element, and the method further comprises coding a first merge mode syntax element for a base inter prediction mode of the first predictive block, and the second merge mode syntax element is different from the first merge mode syntax element.

[0280]

[0273] Clause 102: The method of clause 101, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using the first CABAC context in response to coding the first merge mode syntax element using the first CABAC context.

[0281]

[0274] Clause 103: The method according to clause 102, wherein the first CABAC context is a CABAC context used for the merge mode.

[0282]

[0275] Clause 104: The method of clause 102, wherein the first CABAC context is different from the second CABAC context used for the merge mode.

[0283]

[0276] Clause 105: The method described in Clause 101, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and coding the second merge mode syntax element comprises coding the second merge mode syntax element using a second CABAC context.

[0284]

[0277] Clause 106: The method of clause 105, wherein the second CABAC context is different from the first CABAC context.

[0285]

[0278] Clause 107: A method according to any of clauses 91 to 106, further comprising coding motion information of a third predictive block using an advanced motion vector prediction (AMVP) mode, wherein the coding of the motion information of the third predictive block includes coding motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third predictive block being a base inter prediction block of an adjacent block to the current block, and coding a candidate index identifying the third predictive block as a motion information candidate for the second predictive block, wherein generating the second predictive block comprises interpolating half-pixel samples of a reference block of the second predictive block using a 6-tap interpolation filter.

[0286]

[0279] Clause 108: A method as described in any of clauses 91 to 107, further comprising determining whether to apply local illumination compensation (LIC) to the second predicted block depending on whether LIC has been applied to the first predicted block.

[0287]

[0280] Clause 109: The method of clause 108, wherein the base inter prediction mode comprises one of unidirectional inter prediction or bidirectional inter prediction.

[0288]

[0281] Clause 110: A method as described in any of clauses 108 and 109, further comprising applying LIC to the first predictive block, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block.

[0289]

[0282] Clause 111: A method according to any of clauses 108 to 110, further comprising applying LIC to the first predictive block and coding motion information for the second predictive block using an advanced motion vector prediction (AMVP) mode, wherein determining whether to apply LIC to the second predictive block comprises determining that LIC should be applied to the second predictive block in response to applying LIC to the first predictive block and coding the motion information for the second predictive block using AMVP.

[0290]

[0283] Clause 112: A method according to any of clauses 108 to 111, further comprising applying LIC to a first predictive block and coding motion information for the second predictive block using a merge mode, wherein determining whether to apply LIC to the second predictive block comprises coding a LIC flag for a merge candidate of the second predictive block and determining whether to apply LIC to the second predictive block according to a value of the LIC flag for the merge candidate of the second predictive block.

[0291]

[0284] Clause 113: The method of any of clauses 91 to 112, further comprising encoding the current block prior to decoding the current block.

[0292]

[0285] Clause 114: A device for decoding video data, the device comprising: a memory configured to store the video data; and one or more processors implemented in a circuit, the one or more processors generating a first predictive block for a current block of the video data using a base inter prediction mode; and coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, wherein the merge mode syntax element indicates whether motion information for the second predictive block is coded using the merge mode. a multi-hypothesis prediction block for the current block of video data as a combination of the first and second predictive blocks; and a multi-hypothesis prediction block for the current block of video data.

[0293]

[0286] Clause 115: The device described in Clause 114, wherein to form a merge candidate list, one or more processors are configured to determine that an adjacent block to the second predictive block is bi-predicted using the first motion information and the second motion information, and to form at least one uni-predictive merge candidate from the adjacent block, wherein the one or more processors are configured to select only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate.

[0294]

[0287] Clause 116: A device as described in Clause 115, wherein to form a merge candidate list, one or more processors are configured to form a first uni-predictive merge candidate using the first motion information and to form a second uni-predictive merge candidate using the second motion information.

[0295]

[0288] Clause 117: A device described in any of clauses 114 to 116, wherein one or more processors are configured to code motion information of a third predictive block using an advanced motion vector prediction (AMVP) mode, where, for coding the motion information of the third predictive block, the one or more processors are configured to code motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third predictive block being a base inter predictive block of an adjacent block to the current block, coding a candidate index identifying the third predictive block as a motion information candidate for the second predictive block, and where, for generating the second predictive block, the one or more processors are configured to interpolate half-pixel samples of a reference block of the second predictive block using a 6-tap interpolation filter.

[0296]

[0289] Clause 118: A device described in any of clauses 114 to 117, wherein the one or more processors are further configured to determine whether to apply local illumination compensation (LIC) to the second predicted block depending on whether LIC has been applied to the first predicted block.

[0297]

[0290] Clause 119: A device described in any of clauses 114 to 118, further configured to encode the current block prior to the one or more processors decoding the current block.

[0298]

[0291] Clause 120: A device described in any of clauses 114 to 119, further comprising a display configured to display the decoded video data.

[0299]

[0292] Clause 121: A device according to any of clauses 114 to 120, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0300]

[0293] Clause 122: A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to generate a first predictive block for a current block of video data using a base inter prediction mode; code a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether the motion information for the second predictive block is coded using a merge mode; and code motion information for the second predictive block in accordance with the merge mode syntax element. 22. The method of claim 21, further comprising: coding the motion information for the current block of the video data using the motion information of the first predictive block and the second predictive block; forming a merge candidate list including one or more merge candidates when a merge mode syntax element indicates that the motion information for the second predictive block is coded using a merge mode, each of the merge candidates representing a respective set of uni-predictive motion information; forming a multi-hypothesis predictive block for the current block as a combination of the first predictive block and the second predictive block; and decoding the current block using the multi-hypothesis predictive block.

[0301]

[0294] Clause 123: A computer-readable storage medium as described in Clause 122, comprising instructions that cause the processor to determine that an adjacent block to the second predictive block is bi-predicted using the first motion information and the second motion information, and form at least one uni-predictive merge candidate from the adjacent block, and forming the at least one uni-predictive merge candidate includes instructions that cause the processor to select only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate.

[0302]

[0295] Clause 124: A computer-readable storage medium as described in Clause 123, wherein the instructions for causing a processor to form a merge candidate list include instructions for causing the processor to form a first uni-predictive merge candidate using the first motion information and to form a second uni-predictive merge candidate using the second motion information.

[0303]

[0296] Clause 125: A computer-readable storage medium as described in clauses 122 to 124, wherein the instructions for causing the processor to code motion information of a third predictive block using an advanced motion vector prediction (AMVP) mode, and wherein the instructions for causing the processor to code the motion information of the third predictive block comprise instructions for causing the processor to code motion vector difference (MVD) values ​​having 1 / 2 luma sample resolution, the third predictive block being a base inter prediction block of an adjacent block to the current block, and further comprise instructions for causing the processor to code a candidate index identifying the third predictive block as a motion information candidate for a second predictive block, the third predictive block being a base inter prediction block of an adjacent block to the current block, and wherein the instructions for causing the processor to generate the second predictive block comprise instructions for causing the processor to interpolate half-pixel samples of a reference block of the second predictive block using a 6-tap interpolation filter.

[0304]

[0297] Clause 126: A computer-readable storage medium as described in clauses 122 to 125, further comprising instructions to cause a processor to determine whether local illumination compensation (LIC) should be applied to the second predicted block depending on whether LIC has been applied to the first predicted block.

[0305]

[0298] Clause 127: The computer-readable storage medium of any one of clauses 122 to 126, further comprising instructions that cause a processor to encode the current block prior to decoding the current block.

[0306]

[0299] It should be appreciated that, depending on the example, some acts or events of any of the techniques described herein may be performed in a different sequence, added, merged, or omitted entirely (e.g., not all described acts or events may be required to practice the techniques). Moreover, in some examples, acts or events may be performed simultaneously rather than sequentially, for example, through multi-threaded processing, interrupt processing, or multiple processors.

[0307]

[0300] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or may include a communication medium, which includes any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this manner, the computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0308]

[0301] By way of example and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer readable media.

[0309]

[0302] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the terms "processor" and "processing circuitry" as used herein may refer to any of the above structures, or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functions described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a composite codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0310]

[0303] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or sets of ICs (e.g., chipsets). In this disclosure, various components, modules, or units have been described to highlight functional aspects of devices configured to implement the disclosed techniques, but those components, modules, or units do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors described above, along with suitable software and / or firmware.

[0311]

[0304] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. 1. A method for decoding video data, the method comprising: generating a first predictive block for a current block of video data using a base inter prediction mode; coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether motion information for the second predictive block is coded using a merge mode. coding the motion information for the second predictive block in accordance with the merge mode syntax element and coding the motion information for the second predictive block includes, when the merge mode syntax element indicates that the motion information for the second predictive block is coded using a merge mode, forming a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information. generating the second predictive block for the current block of video data using the motion information; and forming a multi-hypothesis prediction block for the current block as a combination of the first prediction block and the second prediction block; decoding the current block using the multi-hypothesis prediction block; A method comprising:

2. 2. The method of claim 1, wherein, when the merge mode syntax element indicates that a merge mode is used, coding the motion information for the second predictive block comprises coding a merge index indicating a merge candidate from which to inherit the motion information for the second predictive block, and / or, when the merge mode syntax element indicates that a merge mode is not used, coding the motion information for the second predictive block comprises coding a reference picture list identifier, a reference picture index, a motion vector predictor index, and a motion vector differential for the second predictive block.

3. Forming a merge candidate list involves: determining that a neighboring block to the second predictive block is bi-predicted using the first motion information and the second motion information; forming at least one uni-predictive merge candidate from the neighboring blocks, and forming the at least one uni-predictive merge candidate includes selecting only one of the first motion information or the second motion information for the at least one uni-predictive merge candidate. Equipped with Optionally, forming the merge candidate list comprises: forming a first uni-predictive merge candidate using the first motion information; forming a second uni-predictive merge candidate using the second motion information; and The method of claim 1 , comprising:

4. 2. The method of claim 1 , wherein the second predictive block comprises one additional prediction hypothesis of a plurality of additional prediction hypotheses, the method further comprising coding motion information for each of the additional prediction hypotheses using a same merge candidate list, and / or forming the merge candidate list comprises forming the merge candidate list according to a geometric partitioning mode (GPM) merge candidate list construction process.

5. the second predictive block comprises an additional prediction hypothesis of a plurality of additional prediction hypotheses, wherein coding the motion information for the second predictive block comprises: forming a merge candidate list including two or more merge candidates; selecting one of the merging candidates from the merging candidate list for the motion information for the second predictive block; coding the motion information for the second predictive block using the one of the merging candidates; and Equipped with The method comprises: removing the one of the merge candidates from the merge candidate list to form a reduced merge candidate list; coding motion information for a third prediction candidate using one of the remaining merge candidates in the reduced merge candidate list; and and optionally further comprising: coding a first merging candidate index for the second predictive block, the merging candidate list having N merging candidates, N being a positive integer greater than 1, and the first merging candidate index having a maximum possible truncated binary value of N. coding a second merging candidate index for the third prediction candidate, the reduced merging candidate list having N-1 merging candidates, the second merging candidate index having N-1 maximum possible truncated binary values. The method of claim 1 further comprising:

6. 2. The method of claim 1 , wherein the merge mode syntax element comprises a merge mode syntax element for the base inter prediction mode of the first predictive block, and / or further comprises encoding the current block prior to decoding the current block.

7. 2. The method of claim 1 , wherein the merge mode syntax element comprises a second merge mode syntax element, the method further comprising coding a first merge mode syntax element for the base inter prediction mode of the first predictive block, the second merge mode syntax element being different from the first merge mode syntax element.

8. 8. The method of claim 7, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using the first CABAC context in response to coding the first merge mode syntax element using the first CABAC context, optionally wherein the first CABAC context is a CABAC context used for a merge mode, or wherein the first CABAC context is a CABAC context used for a merge mode.

9. 8. The method of claim 7, wherein coding the first merge mode syntax element comprises coding the first merge mode syntax element using a first context-adaptive binary arithmetic coding (CABAC) context, and wherein coding the second merge mode syntax element comprises coding the second merge mode syntax element using a second CABAC context, optionally the second CABAC context different from the first CABAC context.

10. coding motion information of a third prediction block using an advanced motion vector prediction (AMVP) mode; and coding the motion information of the third prediction block includes coding a motion vector difference (MVD) value having ½ luma sample resolution, the third prediction block being a base inter prediction block of a neighboring block relative to the current block. coding a candidate index that identifies the third predictive block as a motion information candidate for the second predictive block; Further equipped with 2. The method of claim 1, wherein generating the second predictive block comprises interpolating half-pixel samples of a reference block for the second predictive block using a 6-tap interpolation filter.

11. 2. The method of claim 1, further comprising: determining whether to apply local illumination compensation (LIC) to the second predicted block depending on whether LIC has been applied to the first predicted block; and optionally, the base inter prediction mode comprises one of unidirectional inter prediction or bidirectional inter prediction.

12. applying LIC to the first predicted block, and wherein determining whether to apply LIC to the second predicted block comprises determining, in response to applying LIC to the first predicted block, to apply LIC to the second predicted block. applying LIC to the first predictive block and coding the motion information for the second predictive block using an advanced motion vector prediction (AMVP) mode, wherein determining whether to apply LIC to the second predictive block comprises determining to apply LIC to the second predictive block in response to applying LIC to the first predictive block and coding the motion information for the second predictive block using AMVP; or applying LIC to the first predictive block and coding motion information for the second predictive block using a merge mode, wherein determining whether to apply LIC to the second predictive block includes: coding a LIC flag for a merging candidate of the second predicted block; and determining whether to apply LIC to the second predicted block according to a value of the LIC flag for the merging candidate of the second predicted block; Equipped with The method of claim 11 further comprising:

13. 1. A device for decoding video data, said device comprising: a memory configured to store video data; one or more processors implemented in the circuit; wherein the one or more processors: generating a first predictive block for a current block of video data using a base inter prediction mode; coding a merge mode syntax element for a second predictive block, the second predictive block representing an additional prediction hypothesis for the current block, where the merge mode syntax element indicates whether motion information for the second predictive block is coded using a merge mode. coding motion information for the second predictive block according to the merge mode syntax element, where, for coding the motion information, the one or more processors are configured, when the merge mode syntax element indicates that the motion information for the second predictive block is coded using a merge mode, to form a merge candidate list including one or more merge candidates, each of the merge candidates representing a respective set of uni-predictive motion information. generating the second predictive block for the current block of video data using the motion information; and forming a multi-hypothesis prediction block for the current block as a combination of the first prediction block and the second prediction block; decoding the current block using the multi-hypothesis prediction block; A device configured to:

14. The device of claim 13, wherein the one or more processors are further configured to execute a method according to any one of claims 1 to 12.

15. A computer readable storage medium having instructions stored thereon which, when executed, cause a processor to perform the method of any one of claims 1 to 12.

Images