Methods, apparatus and computer-readable storage media for decoding video data

Abstract

An example device for decoding video data includes one or more processors configured to: determine that a current block predicted by a dual prediction mode for the video data has a first weight and a second weight; determine whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the MHP mode with the dual prediction mode as the base mode should be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block based on the dual prediction mode; generate a second prediction block based on the additional inter-frame prediction mode; generate a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Patent Application No. 17 / 644,519, filed December 15, 2021, and U.S. Provisional Application No. 63 / 130,232, filed December 23, 2020, the entire contents of which are incorporated herein by reference. U.S. Patent Application No. 17 / 644,519, filed December 15, 2021, claims the benefit of U.S. Provisional Application No. 63 / 130,232, filed December 23, 2020. Technical Field

[0003] This disclosure relates to video decoding, including video encoding and video decoding. Background Technology

[0004] Digital video capabilities can be integrated into a wide variety of devices, including digital televisions, digital direct broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablets, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones, so-called "smartphones," video conferencing equipment, video streaming devices, and the like. Digital video devices implement video decoding technologies, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Decoding (AVC), ITU-T H.264 High Efficiency Video Decoding (HEVC), and extensions to these standards. Video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information by implementing such video decoding technologies.

[0005] Video decoding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove inherent redundancy in video sequences. For block-based video decoding, video slices (e.g., video pictures or portions of video pictures) can be divided into video blocks, which may also be referred to as decoding tree units (CTUs), decoding units (CUs), and / or decoding nodes. For reference samples in adjacent blocks within the same picture, spatial prediction is used to encode video blocks in the intra-frame decoding (I) slice of the picture. Video blocks in the inter-frame decoding (P or B) slice of the picture can use spatial prediction for reference samples in adjacent blocks within the same picture, or temporal prediction for reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. Summary of the Invention

[0006] Generally, this disclosure describes techniques for inter-frame prediction in video decoding (encoding and decoding). In particular, these techniques involve multi-hypothesis prediction. In video decoding, video is represented by a series of pictures. Each picture can be segmented into blocks, and each block can be decoded individually. Typically, predicted blocks and residual blocks are used to decode blocks. Video decoders can typically form predicted blocks based on inter-frame prediction (relative to previously decoded pictures) or intra-frame prediction (relative to previously decoded blocks of the same picture). In inter-frame prediction, blocks can be predicted using unidirectional (one motion vector) or bidirectional (two motion vectors) prediction. Furthermore, in inter-frame prediction, blocks can be predicted using multi-hypothesis prediction, i.e., using more than two motion vectors.

[0007] In one example, a method for decoding video data includes determining that a first weight and a second weight are assigned to a current block of video data using a dual prediction mode for inter-frame prediction decoding, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight are assigned, determining whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the MHP mode with the dual prediction mode as the base mode is to be used to predict the current block, determining an additional inter-frame prediction mode of the MHP mode; generating a first prediction block based on the dual prediction mode; generating a second prediction block based on the additional inter-frame prediction mode; using the first prediction block and the second prediction block, generating a final prediction block for the current block based on the MHP mode; and decoding the current block using the final prediction block.

[0008] In another example, an apparatus for decoding video data includes: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determine whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the MHP mode with the dual prediction mode as the base mode should be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block based on the dual prediction mode; generate a second prediction block based on the additional inter-frame prediction mode; generate a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

[0009] In another example, a computer-readable storage medium having instructions thereon, which, when executed, cause a processor to: determine that a first weight and a second weight are assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight are assigned, determine whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the MHP mode with the dual prediction mode as the base mode is to be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block based on the dual prediction mode; generate a second prediction block based on the additional inter-frame prediction mode; generate a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

[0010] In another example, an apparatus for decoding video data includes: means for determining a first weight and a second weight for a current block of video data using a dual prediction mode for inter-frame prediction decoding, wherein the first weight is different from the second weight; means for determining, in response to determining that the first weight and the second weight are specified, whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; means for determining an additional inter-frame prediction mode of the MHP mode in response to determining that the MHP mode with the dual prediction mode as the base mode is to be used to predict the current block; means for generating a first prediction block based on the dual prediction mode; means for generating a second prediction block based on the additional inter-frame prediction mode; means for generating a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and means for decoding the current block using the final prediction block.

[0011] Details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating an example video encoding and decoding system that can perform the techniques of this disclosure.

[0013] Figure 2A and Figure 2B This is a conceptual diagram illustrating an example quadtree binary tree (QTBT) structure and the corresponding decoding tree unit (CTU).

[0014] Figure 3 This is a block diagram illustrating an example video encoder that can perform the techniques of this disclosure.

[0015] Figure 4This is a block diagram illustrating an example video decoder that can perform the techniques disclosed herein.

[0016] Figure 5 This is a flowchart illustrating an example method for encoding the current block according to the technology of this disclosure.

[0017] Figure 6 This is a flowchart illustrating an example method for decoding the current block according to the technology of this disclosure.

[0018] Figure 7 This is a conceptual diagram illustrating an example of applying dual prediction (BCW) and multiple hypothesis prediction (MHP) with CU level weighting as an extension of BCW.

[0019] Figure 8 This is a flowchart illustrating an example method for decoding (e.g., reproducing) a current block of video data according to the techniques of this disclosure. Detailed Implementation

[0020] In video decoding, video is represented by a series of images. Each image can be divided into blocks, and each block can be decoded individually. Typically, prediction blocks and residual blocks are used to decode blocks. Video decoders can usually form prediction blocks based on inter-frame prediction (relative to previously decoded images) or intra-frame prediction (relative to previously decoded blocks of the same image). In inter-frame prediction, blocks can be predicted using unidirectional (one motion vector) or bidirectional (two motion vectors) prediction.

[0021] The video decoder can further decode motion vectors, for example, using a merge mode or an Advanced Motion Vector Prediction (AMVP) mode. In merge mode, the video decoder decodes merge candidates, representing neighboring blocks of the current block, from which motion information of the motion vector is inferred (e.g., reference image index, reference image list, and the motion vector itself, such as the x and y components of the motion vector). The x component represents the horizontal displacement of the motion vector, while the y component represents the vertical displacement. In AMVP mode, the video decoder decodes motion vector prediction candidates, representing neighboring blocks of the current block from which the motion vector is predicted, and motion vector difference (MVD) information representing the offsets to be applied to the x and y components of the motion vector prediction candidates. In AMVP, the video decoder also explicitly decodes reference image identification information, such as reference image indexes and reference image lists.

[0022] In ITU-T H.265 High-Efficiency Video Decoding (HEVC), the video decoder decodes the MVD value (i.e., the difference between the motion vector predictor and the motion vector of the current CU) in quarter-luminance samples (when sub-pixel precision is enabled, for example, when "use_integer_mv_flag" has a value of 0 in the slice header). ITU-T H.26 / Various Video Decoding (VVC) introduces a CU-level Adaptive Motion Vector Resolution (AMVR) scheme, which allows decoding of the CU's MVD with different precisions. Depending on the current CU's mode (normal AMVP mode or affine AMVP mode), the video decoder can adaptively select the MVD resolution according to the VVC, as follows: In normal AMVP mode, the video decoder can select from quarter-luminance samples, half-luminance samples, integer luminance samples, or four luminance samples; in affine AMVP mode, the video decoder can select from 1 / 16 luminance samples, quarter-luminance samples, or integer luminance samples.

[0023] In VVC, for a CU with at least one non-zero MVD component, the video decoder can decode a first flag to indicate whether a quarter-luminance sample MVD precision is used for that CU. If the first flag is 0, no further signaling is required, and the quarter-luminance sample MVD precision is used for the current CU. Otherwise, the video decoder can decode a second flag to indicate whether a half-luminance sample or another MVD precision (integer or four luminance samples) is used for a normal AMVP CU. In the case of half-luminance samples, the video decoder can apply a 6-tap interpolation filter to the half-luminance sample location instead of the default 8-tap interpolation filter. Otherwise, the video decoder can decode a third flag to indicate whether an integer luminance sample or four luminance sample MVD precision is used for a normal AMVP CU.

[0024] In the case of an affine AMVP CU in VVC, the video decoder can use a second flag to indicate whether to use integer luminance samples or 1 / 16th of a luminance sample MVD precision. To ensure the reconstructed MV has the desired precision (quarter luminance sample, half luminance sample, integer luminance sample, or four luminance samples), the video decoder can round the CU's motion vector predictor to the same precision as the MVD before adding the motion vector predictor along with the MVD. The video decoder can round the motion vector predictor to zero (i.e., negative motion vector predictors round to positive infinity, and positive motion vector predictors round to negative infinity).

[0025] In HEVC, the video decoder generates a dual prediction signal by averaging two prediction signals obtained from two different reference images and / or using two different motion vectors. In VVC, the dual prediction mode is extended beyond simple averaging to allow a weighted average of the two prediction signals, as follows:

[0026] P bi-pred(双预测) =((8-w)*P0+w*P1+4)>>3

[0027] In VVC, weighted average double prediction allows five weights w, w∈{-2, 3, 4, 5, 10}. For each double prediction decoding unit (CU, i.e., block), the weights w are determined in one of two ways: 1) for non-merged CUs, the video decoder decodes the data representing the weight index following the motion vector difference data in the bitstream; 2) for merged CUs, the video decoder infers the weight index from neighboring blocks based on the merge candidate index. In VVC, double prediction with CU horizontal weighting (BCW) is only applied to CUs with 256 or more luma samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency images, all 5 weights are used. For non-low-latency images, only 3 weights (w∈{3, 4, 5}) are used.

[0028] Furthermore, in inter-frame prediction, multiple hypothesis prediction (MHP) can be used to predict blocks, that is, using more than two motion vectors. MHP is described below:

[0029] • Winken et al., “Multi-hypothesis Inter-prediction”, ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 Joint Video Experts Group (JVET), 10th Meeting: San Diego, USA, April 10–20, 2018, Document JVET-J0041-v2;

[0030] • Winken et al., “CE10: Multi-Hypothesis Inter Prediction (Tests 1.5–1.8)”, ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 Joint Video Expert Group (JVET), 11th Meeting: Ljubljana, Slovenia, July 10–18, 2018, Document JVET-K0269;

[0031] • Winken et al., “CE10: Multi-Hypothesis Inter Prediction (Tests 1.2.a–1.2.c)”, ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 Joint Video Expert Group (JVET), 12th Meeting: Macau, China, 3-12 October 2018, Document JVET-L0148-v3; and

[0032] • Winken et al., “CE10: Multi-hypothesis inter prediction (Test 10.1.2)”, ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 Joint Video Expert Group (JVET), 13th Meeting: Marrakech, Morocco, 9–18 January 2019, Document JVET-M0425-v2.

[0033] In MHP, inter-frame prediction techniques allow for the weighted superposition of two or more motion-compensated prediction signals (e.g., using more than two motion vectors). The video decoder can obtain the overall prediction signal by weighted superposition of samples. This can be achieved using single / dual prediction signals p. uni / bi (Single / Dual), the first additional inter-frame prediction signal / hypothesis h3 and weighting factor α, the video decoder can obtain the predicted signal p3 as follows:

[0034] p3=(1-α)p uni / bi +αh3

[0035] According to MHP, the video decoder can decode the data of the weighting factor α in the syntax element add_hyp_weight_idx (additional hypothesis weight index), for example, according to the following mapping in Table 1:

[0036] Table 1

[0037] 0 1 / 4 1 -1 / 8

[0038] Similar to the techniques discussed above, video decoders can use more than one additional prediction signal. The video decoder can iteratively accumulate the resulting overall prediction signal with each additional prediction signal, as shown below:

[0039] p n+1 =(1-α) n+1 )p n +α n+1 h n+1

[0040] The video decoder can obtain the overall predicted signal as the final p n (That is, p with the largest index n) n ).

[0041] For inter-frame prediction blocks using the merge mode (instead of the SKIP mode) according to these traditional MHP techniques, the video decoder can also specify an additional inter-frame prediction signal. For the additional prediction signal, the video decoder can use one of two AMVP candidate lists:

[0042] • If the Picture Order Count (POC) of the reference picture for the additional prediction signal is equal to the POC of the reference picture used in List 1, then the video decoder can use the List 1 AMVP candidate list.

[0043] Otherwise, the video decoder can use list 0AMVP candidate list.

[0044] This disclosure recognizes that the aforementioned conventional MHP signaling can be improved. The techniques disclosed herein can achieve improvements in video decoding related to the use of MHP because these techniques can reduce signaling overhead, reduce processing requirements, and / or improve data fidelity by increasing the prediction accuracy of motion vector predictors and / or prediction blocks.

[0045] Figure 1 This is a block diagram illustrating an example video encoding and decoding system 100 that can perform the techniques of this disclosure. The techniques of this disclosure are generally directed to decoding (encoding and / or decoding) video data. Typically, video data includes any data used for processing video. Therefore, video data can include raw, undecoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata such as signaling data.

[0046] like Figure 1 As shown, system 100 includes source device 102, which in this example provides encoded video data to be decoded and displayed by destination device 116. Specifically, source device 102 provides video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a variety of devices, including desktop computers, laptops, mobile devices, tablets, set-top boxes, handheld phones such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some cases, source device 102 and destination device 116 may be equipped for wireless communication and therefore may be referred to as wireless communication devices.

[0047] exist Figure 1In the example, source device 102 includes a video source 104, memory 106, video encoder 200, and output interface 108. Destination device 116 includes an input interface 122, video decoder 300, memory 120, and display device 118. According to this disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply techniques for decoding data for multiple hypothesis prediction. Thus, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source device and destination device may include other components or arrangements. For example, source device 102 may receive video data from an external video source such as an external camera. Similarly, destination device 116 may be connected to an external display device, rather than including an integrated display device.

[0048] like Figure 1 The system 100 shown is merely an example. Typically, any digital video encoding and / or decoding device can perform techniques for decoding data for multiple hypothesis predictions. The source device 102 and destination device 116 are merely examples of such decoding devices, where the source device 102 generates decoded video data for transmission to the destination device 116. This disclosure refers to a “decoding” device as a device that decodes (encodes and / or decodes) data. Thus, video encoder 200 and video decoder 300 represent examples of decoding devices, specifically a video encoder and a video decoder, respectively. In some examples, the source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each of the source device 102 and destination device 116 includes video encoding and decoding components. Therefore, system 100 can support unidirectional or bidirectional video transmission between the source device 102 and destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0049] Typically, video source 104 represents the source of video data (i.e., raw, undecoded video data) and provides a series of consecutive pictures (also called “frames”) of video data to video encoder 200, which encodes the data of the pictures. Video source 104 of source device 102 may include video capture devices such as video cameras, video archives containing previously captured raw video, and / or video feed interfaces for receiving video from video content providers. As a further alternative, video source 104 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes the captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the pictures from the order of reception (sometimes referred to as “display order”) to a decoding order for decoding. Video encoder 200 may generate a bitstream comprising the encoded video data. Source device 102 may then output the encoded video data via output interface 108 to a computer-readable medium 110 for reception and / or retrieval by, for example, an input interface 122 of destination device 116.

[0050] The memory 106 of source device 102 and the memory 120 of destination device 116 represent general-purpose memory. In some examples, memories 106 and 120 may store raw video data, such as raw video from video source 104 and raw, decoded video data from video decoder 300. Additionally or alternatively, memories 106 and 120 may store software instructions executable by, for example, video encoder 200 and video decoder 300, respectively. Although memories 106 and 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include internal memory for similar functions or equivalent purposes. Furthermore, memories 106 and 120 may store encoded video data, such as data output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106 and 120 may be allocated as one or more video buffers, for example, for storing raw, decoded, and / or encoded video data.

[0051] Computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to transmit encoded video data directly to destination device 116 in real time, such as via a radio frequency network or a computer-based network. Output interface 108 can modulate the transmitted signal including the encoded video data according to communication standards such as wireless communication protocols, and input interface 122 can demodulate the received transmitted signal. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium can include a router, a switch, a base station, or any other device that may help facilitate communication from source device 102 to destination device 116.

[0052] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0053] In some examples, source device 102 can output encoded video data to file server 114 or another intermediate storage device, which can store the encoded video information generated by source device 102. Destination device 116 can access the stored video data from file server 114 via streaming or download.

[0054] File server 114 can be any type of server device capable of storing encoded video data and sending it to destination device 116. File server 114 can represent a web server (e.g., for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Transfer over One-Way Transfer (FLUTE) protocol), a Content Delivery Network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) server, and / or a Network Attached Storage (NAS) device. Additionally or alternatively, file server 114 can implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Real-Time Streaming (HLS), Real-Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, or similar protocols.

[0055] Destination device 116 can access encoded video data from file server 114 via any standard data connection, including an internet connection. This can include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber line (DSL), cable modems, etc.), or combinations thereof, suitable for accessing encoded video data stored on file server 114. Input interface 122 can be configured to operate according to any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114, or according to other such protocols for retrieving media data.

[0056] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired network component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data, such as encoded video data according to cellular communication standards (e.g., 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, or similar). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data, such as data according to other wireless standards (e.g., the IEEE 802.11 specification, the IEEE 802.15 specification, e.g., ZigBee). TM ),Bluetooth TM The encoded video data is of standard or similar quality. In some examples, source device 102 and / or destination device 116 may include their respective system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing functions attributed to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing functions attributed to video decoder 300 and / or input interface 122.

[0057] The technology disclosed herein can be applied to video decoding that supports any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission, dynamic adaptive streaming over HTTP (DASH), digital video encoded onto a data storage medium, decoding digital video stored on a data storage medium, or other applications.

[0058] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, or the like). The encoded video bitstream may include signaling information defined by the video encoder 200 (which is also used by the video decoder 300), such as syntax elements having values ​​describing the characteristics and / or processing of video blocks or other decoding units (e.g., slices, pictures, picture groups, sequences, or the like). The display device 118 displays a decoded picture of the decoded video data to the user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0059] although Figure 1 Not shown, but in some examples, the video encoder 200 and video decoder 300 may each be integrated with the audio encoder and / or audio decoder, and may include appropriate MUX-DEMUX units or other hardware and / or software to process multiplexed streams of both audio and video in a common data stream. Where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP).

[0060] Each of the video encoder 200 and the video decoder 300 can be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and may execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the video encoder 200 and the video decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the video encoder 200 and / or the video decoder 300 may include integrated circuits, microprocessors, and / or wireless communication devices such as cellular phones.

[0061] The video encoder 200 and video decoder 300 can operate according to video decoding standards such as ITU-TH.265 (also known as High Efficiency Video Coding (HEVC)) or its extensions, such as MultiView and / or Scalable Video Coding Extensions. Alternatively, the video encoder 200 and video decoder 300 can operate according to other proprietary or industry standards, such as Versatile Video Coding (VVC). A draft of the VVC standard is described in “Versatile Video Coding (Draft 9)” by Bross et al., ITU-T SG 16WP 3 and the Joint Video Experts Group (JVET) of ISO / IEC JTC 1 / SC 29 / WG 11, 18th meeting: April 15-24, JVET-R2001-v8 (hereinafter referred to as “VVC Draft 9”). However, the technology disclosed herein is not limited to any particular decoding standard.

[0062] Typically, video encoder 200 and video decoder 300 can perform block-based decoding of images. The term "block" generally refers to a structure that includes the data to be processed (e.g., encoded, decoded, or otherwise used during encoding and / or decoding). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Typically, video encoder 200 and video decoder 300 can decode video data represented in YUV (e.g., Y, Cb, Cr) format. That is, video encoder 200 and video decoder 300 can decode luminance and chrominance components, rather than decoding the red, green, and blue (RGB) data of image samples, where chrominance components may include red hue and blue hue chrominance components. In some examples, video encoder 200 converts received RGB format data to YUV representation before encoding, and video decoder 300 converts YUV representation to RGB format. Alternatively, preprocessing and post-processing units (not shown) can perform these conversions.

[0063] This disclosure can generally refer to the decoding (e.g., encoding and decoding) of images, including the process of encoding or decoding data of an image. Similarly, this disclosure can refer to the decoding of blocks of images, including the process of encoding or decoding data of blocks, such as prediction and / or residual decoding. Encoded video bitstreams typically include a series of values ​​representing decoding decisions (e.g., decoding modes) and syntax elements that divide images into blocks. Therefore, reference to decoding an image or block should generally be understood as the decoded values ​​of the syntax elements that form the image or block.

[0064] HEVC defines various blocks, including decoding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as a video encoder 200) divides the decoding tree unit (CTU) into CUs according to a quadtree structure. That is, the video decoder divides the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. Nodes without child nodes can be called "leaf nodes," and the CU of such leaf nodes can include one or more PUs and / or one or more TUs. The video decoder can further divide the PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of TUs. In HEVC, PUs represent inter-frame prediction data, while TUs represent residual data. CUs that are intra-frame predicted include intra-frame prediction information, such as intra-frame mode indication.

[0065] As another example, video encoder 200 and video decoder 300 can be configured to operate according to VVC. According to VVC, the video decoder (such as video encoder 200) partitions the image into multiple decoding tree units (CTUs). Video encoder 200 can partition the CTUs according to a tree structure such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple partitioning types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure consists of two levels: the first level is partitioned according to a quadtree, and the second level is partitioned according to a binary tree. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to decoding units (CUs).

[0066] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT), binary tree (BT), and one or more types of ternary tree (TT) partitions. A ternary or triple tree partition splits a block into three sub-blocks. In some examples, a ternary or triple tree partition divides a block into three sub-blocks without splitting the original block by a center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0067] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MMTT structure for the luminance component and another QTBT / MMTT structure for the two chrominance components (or two QTBT / MTT structures for the corresponding chrominance components).

[0068] The video encoder 200 and video decoder 300 can be configured to use per-HEVC quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures. For illustrative purposes, the description of the techniques of this disclosure is based on QTBT partitioning. However, it should be understood that the techniques of this disclosure can also be applied to video decoders configured to use quadtree partitioning or other types of partitioning.

[0069] In some examples, a CTU includes a decoded tree block (CTB) of luminance samples, two corresponding CTBs of chrominance samples for an image with three sample arrays, or a CTB of samples for a monochrome image or an image decoded using three separate color planes and a syntax structure for the decoded samples. A CTB can be an N×N block of samples for some value of N, such that dividing the component into a CTB is a partition. The component can be one of three arrays (luminance and two chrominance) for an image in 4:2:0, 4:2:2, or 4:4:4 color format, or a single sample from one, or an array or a single sample from an array for a monochrome image. In some examples, a decoded block is an M×N block of samples for some values ​​of M and N, such that dividing the CTB into a decoded block is a partition.

[0070] Blocks (e.g., CTUs or CUs) can be grouped in various ways within an image. As an example, a brick can refer to a rectangular area of ​​a row of CTUs within a specific tile in an image. A tile can be a rectangular area of ​​CTUs within a specific tile column and a specific tile row in an image. A tile column is a rectangular area of ​​CTUs with a height equal to the image height and a width specified by a syntax element (e.g., such as an image parameter set). A tile row is a rectangular area of ​​CTUs with a height specified by a syntax element (e.g., such as an image parameter set) and a width equal to the image width.

[0071] In some examples, a tile may be divided into multiple rectangular blocks, each of which may include one or more CTU rows within the tile. A tile that is not divided into multiple rectangular blocks may also be referred to as a rectangular block. However, a rectangular block that is a true subset of a tile may not be referred to as a tile.

[0072] Rectangular blocks in an image can also be arranged into a slice. A slice can be an integer number of rectangular blocks in the image, which can be contained within a single Network Abstraction Layer (NAL) unit. In some examples, a slice includes multiple complete tiles, or a continuous sequence of complete rectangular blocks that contain only one tile.

[0073] This disclosure uses "N×N" and "N by N" interchangeably to refer to the sample dimensions of a block (such as a CU or other video block) in the vertical and horizontal dimensions, for example, 16×16 samples or 16 by 16 samples. Typically, a 16×16 CU has 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an N×N CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in rows and columns. Furthermore, a CU does not necessarily have the same number of samples in the horizontal direction as it does in the vertical direction. For example, a CU can include N×M samples, where M is not necessarily equal to N.

[0074] The video encoder 200 encodes video data of the CU (Complex Unit), which represents prediction and / or residual information, as well as other information. The prediction information indicates how to predict the CU in order to form a prediction block for the CU. The residual information typically represents the sample-by-sample difference between the CU samples and the prediction block before encoding.

[0075] To predict the Cubic Frame (CU), the video encoder 200 typically forms a predicted block of the CU using either inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting the CU based on data from previously decoded images, while intra-frame prediction typically refers to predicting the CU from data previously decoded from the same image. To perform inter-frame prediction, the video encoder 200 can use one or more motion vectors to generate the predicted block. The video encoder 200 can typically perform a motion search to identify a reference block that closely matches the CU, for example, based on the difference between the CU and a reference block. The video encoder 200 can calculate the difference using sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 can use unidirectional or bidirectional prediction to predict the current CU.

[0076] Some examples of VVC also provide affine motion compensation modes, which can be considered as inter-frame prediction modes. In affine motion compensation modes, the video encoder 200 can determine two or more motion vectors representing non-translational motion, such as zooming in or out, rotation, perspective motion, or other irregular motion types.

[0077] To perform intra-frame prediction, the video encoder 200 can select an intra-frame prediction mode to generate prediction blocks. Some examples of VVC provide sixty-seven intra-frame prediction modes, including various directional modes as well as planar and DC modes. Typically, the video encoder 200 selects an intra-frame prediction mode that describes the neighboring samples to the current block (e.g., the block of the CU), from which it predicts samples for the current block. Assuming the video encoder 200 decodes the CTU and CU in raster scan order (from left to right, from top to bottom), such samples are typically above, above, to the left, or to the right of the current block in the same frame as the current block.

[0078] The video encoder 200 encodes data representing the prediction mode of the current block. For example, for inter-frame prediction modes, the video encoder 200 may encode data indicating which of the various available inter-frame prediction models is used, along with motion information for the corresponding mode. For example, for unidirectional or bidirectional inter-frame prediction, the video encoder 200 may encode motion vectors using Advanced Motion Vector Prediction (AMVP) or merging modes. The video encoder 200 may use similar modes to encode motion vectors for affine motion compensation modes.

[0079] After block prediction, such as intra-frame or inter-frame prediction, the video encoder 200 can compute residual data for that block. The residual data (such as a residual block) represents the sample-by-sample difference between the block formed using the corresponding prediction mode and the predicted block used for that block. The video encoder 200 can apply one or more transforms to the residual block to produce transformed data in the transform domain rather than the sample domain. For example, the video encoder 200 can apply a Discrete Cosine Transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. Furthermore, the video encoder 200 can apply a secondary transform after the first transform, such as a mode-dependent inseparable secondary transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), or the like. The video encoder 200 produces transform coefficients after applying one or more transforms.

[0080] As described above, after performing any transformation to produce transform coefficients, the video encoder 200 can perform quantization of the transform coefficients. Quantization generally refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent them, thereby providing further compression. By performing quantization, the video encoder 200 can reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 can round an n-bit value down to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 can perform a bit-by-bit right shift on the value to be quantized.

[0081] After quantization, the video encoder 200 can scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place higher-energy (and therefore lower-frequency) coefficients before the vector and lower-energy (and therefore higher-frequency) transform coefficients after the vector. In some examples, the video encoder 200 can utilize a predefined scan order to scan the quantized transform coefficients to produce a serialized vector, and then entropy-encode the set of quantized transform coefficients of that vector. In other examples, the video encoder 200 can perform an adaptive scan. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can entropy-encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic decoding (CABAC). The video encoder 200 can also entropy-encode the values ​​of syntax elements describing metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0082] To perform CABAC, the video encoder 200 can assign context within a context model to the symbols to be transmitted. The context may involve, for example, whether the neighboring values ​​of a symbol are zero. Probability determination can be based on the context assigned to the symbols.

[0083] The video encoder 200 can further generate syntax data for the video decoder 300, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, for example in picture headers, block headers, slice headers, or other syntax data, such as sequence parameter sets (SPS), picture parameter sets (PPS), or video parameter sets (VPS). The video decoder 300 can also decode such syntax data to determine how to decode the corresponding video data.

[0084] In this way, the video encoder 200 can generate a bitstream that includes encoded video data, such as syntax elements describing the division of images into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0085] Typically, the video decoder 300 performs the reverse process of the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 can use CABAC to decode the values ​​of the syntax elements of the bitstream in a manner substantially similar to but opposite to the CABAC encoding process of the video encoder 200. The syntax elements can define partitioning information used to divide the image into CTUs, and each CTU is partitioned according to a corresponding partitioning structure (such as a QTBT structure) to define the CUs of the CTU. The syntax elements can further define prediction and residual information for blocks (e.g., CUs) of the video data.

[0086] The residual information can be represented, for example, by quantized transform coefficients. The video decoder 300 can inversely quantize and inversely transform the quantized transform coefficients of a block to reconstruct the residual block of that block. The video decoder 300 uses the prediction mode (intra-frame or inter-frame prediction) transmitted with the signal and associated prediction information (e.g., motion information for inter-frame prediction) to form a prediction block of the block. The video decoder 300 can then combine the prediction block and the residual block (on a sample-by-sample basis) to reconstruct the original block. The video decoder 300 can perform additional processing, such as performing deblocking to reduce visual artifacts along block boundaries.

[0087] According to the technology disclosed herein, the video encoder 200 and the video decoder 300 can be configured to perform multiple hypothesis prediction (MHP). In particular, the video encoder 200 and the video decoder 300 can be configured to perform any or all of the following technologies related to MHP in any combination.

[0088] In one example, the video encoder 200 and video decoder 300 may apply MHP to non-merging mode only if the dual predictions with CU-level weighted (BCW) use unequal weights (i.e., the predictors from different references have different weights). Therefore, the video encoder 200 and video decoder 300 may apply MHP as an extension of BCW. For example, when applied to VVC, when BCW uses weights other than "4", the video encoder 200 and video decoder 300 will only decode the "additional prediction signal" in non-merging mode. Note that the weights of the "additional prediction signal" also imply unequal weights.

[0089] In another example, in addition to or as an alternative to the techniques described above, for the non-merging mode, the MVD resolution of the "additional prediction signal" is the same as the selected MVD resolution in the basic mode. Alternatively, the video encoder 200 and the video decoder 300 can decode the MVD resolution of the "additional prediction signal," as in AMVR in the basic mode. Therefore, the MVD resolution of the "additional prediction signal" can be adaptive and different from that in the basic mode. The additional mode refers to the mode in which the "additional prediction signal" is sent (merging mode or non-merging mode).

[0090] In another example, in addition to or as an alternative to the techniques described above, the video encoder 200 and video decoder 300 can be configured to apply MHP using only different motion vectors or different reference images. This reduces the signaling cost of the additional motion vectors and ensures that the final prediction is not a repetition of the BCW.

[0091] In another example, in addition to or as an alternative to the techniques described above, the video encoder 200 and video decoder 300 can be configured not to apply MHP to blocks with motion vectors decoded using non-merging mode when the AMVR mode is half-luminance sample, where different interpolation filters are applied for motion compensation. The predictor can be generated from a smoothing interpolation filter in the half-luminance sample AMVR mode, and therefore applying MHP (using a weighted average) to generate a smooth predictor is not very helpful in this mode. Avoiding the signaling of the additional predictor signal can reduce some of the overhead in this mode.

[0092] In another example, in addition to or as an alternative to the techniques described above, the video encoder 200 and video decoder 300 can be configured to determine whether to apply MHP based on a selected interpolation filter (e.g., a selected interpolation filter index) used to interpolate sub-integer pixel values ​​of sub-integer precision motion vectors. In some designs, a set of interpolation filters with different characteristics can be used. The selection of interpolation filters can be block-wise, signal-transmitted, or inherited as filter indexes. The interpolation filters can be different for different fractional (sub-integer) positions (phase). For certain filter combinations, MHP may be disabled. In one example, the interpolation filter set may include sharpening filters and smoothing filters. Then, if a smoothing filter is selected, the video encoder 200 and video decoder 300 can disable block-wise MHP. Typically, a "sharpening" filter means assigning more weight to some samples than others, while a "smoothing" filter means assigning relatively similar weights across different samples.

[0093] In another example, in addition to or as an alternative to the techniques described above, the video encoder 200 and video decoder 300 do not select the AMVP candidate list for the motion vectors used to append the predicted signal based on whether the POC of the reference image list is equal to the POC of the reference image in list 1. Instead, the video encoder 200 and video decoder 300 can be configured to determine the list to use based on the reference index. This simplifies the process and provides decoding improvements.

[0094] In another example, in addition to or as an alternative to the techniques described above, the video encoder 200 and video decoder 300 can be configured to have additional weighting factors available for MHP, besides those in Table 1. For example, the video encoder 200 and video decoder 300 can be configured to use the weighting factors in Table 2:

[0095] Table 2

[0096] 0 1 / 4 1 -1 / 8 2 1 / 2

[0097] In the example above, an additional weight of 1 / 2 can be used. This allows the video encoder 200 to place greater emphasis on the additional assumptions than traditional methods.

[0098] This disclosure can generally refer to "signaling" certain information, such as syntax elements. The term "signaling" can generally refer to communication used to decode the values ​​of syntax elements and / or other data in encoded video data. That is, video encoder 200 can signal the values ​​of syntax elements in the bitstream. Generally, signaling refers to generating a value in the bitstream. As described above, source device 102 can transmit the bitstream to destination device 116 substantially in real time or non-real time, such as when syntax elements are stored in storage device 112 for later retrieval by destination device 116.

[0099] Figure 2A and Figure 2B This is a conceptual diagram illustrating an example Quadtree Binary Tree (QTBT) structure 130 and a corresponding Decoding Tree Unit (CTU) 132. Solid lines represent quadtree splits, and dashed lines represent binary tree splits. In each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (i.e., horizontal or vertical) is used, where in this example 0 represents a horizontal split and 1 represents a vertical split. For quadtree splits, it is not necessary to indicate the split type because quadtree nodes split blocks horizontally and vertically into four equal-sized sub-blocks. Therefore, the video encoder 200 can encode syntax elements (such as split information) (i.e., solid lines) for the region tree level of the QTBT structure 130 and syntax elements (such as split information) (i.e., dashed lines) for the prediction tree level of the QTBT structure 130, and the video decoder 300 can decode them. The video encoder 200 can encode video data such as prediction and transform data of the CU represented by the terminal leaf nodes of the QTBT structure 130, and the video decoder 300 can decode it.

[0100] generally, Figure 2B The CTU 132 can be associated with parameters that define the size of the block corresponding to the node in the QTBT structure 130 at the first and second levels. These parameters can include the CTU size (representing the size of the CTU 132 in the sample), the minimum quadtree size (minimum QT size, representing the minimum allowed size of the leaf node in the quadtree), the maximum binary tree size (maximum BT size, representing the maximum allowed size of the root node in the binary tree), the maximum binary tree depth (maximum BT depth, representing the maximum allowed depth in the binary tree), and the minimum binary tree size (minimum BT size, representing the minimum allowed size of the leaf node in the binary tree).

[0101] The root node of a QTBT structure corresponding to a CTU can have four child nodes at the first level of the QTBT structure, each child node being partitioned according to a quadtree partition. That is, a first-level node is either a leaf node (with no child nodes) or has four child nodes. An example of QTBT structure 130 represents such a node as having a parent node and child nodes with solid lines for branching. If a first-level node is not larger than the maximum allowed binary tree root node size (maximum BT size), the node can be further partitioned by the corresponding binary tree. The binary tree partitioning of a node can be iterated until the partitioned node reaches the minimum allowed binary tree leaf node size (minimum BT size) or the maximum allowed binary tree depth (maximum BT depth). An example of QTBT structure 130 represents such a node as having dashed lines for branching. The binary tree leaf nodes are called decoding units (CUs), which are used for prediction (e.g., intra-frame or inter-frame prediction) and transformation without requiring any further partitioning. As discussed above, CUs can also be referred to as “video blocks” or “blocks”.

[0102] In an example of a QTBT partitioning structure, the CTU size is set to 128x128 (luminance samples and two corresponding 64x64 chrominance samples), the minimum QT size is set to 16x16, the maximum BT size is set to 64x64, the minimum BT size (both width and height) is set to 4, and the maximum BT depth is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes ranging from 16x16 (i.e., the minimum QT size) to 128x128 (i.e., the CTU size). If a quadtree leaf node is 128x128, it will not be further partitioned into a binary tree because its size exceeds the maximum BT size (i.e., 64x64 in this example). Otherwise, the quadtree leaf node can be further partitioned into a binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, and the binary tree depth is 0. When the binary tree depth reaches the maximum BT depth (4 in this example), further partitioning is not allowed. A binary tree node with a width equal to the minimum BT size (4 in this example) means that further vertical splits (i.e., width partitions) are not allowed for that binary tree node. Similarly, a binary tree node with a height equal to the minimum BT size means that further horizontal splits (i.e., height partitions) are not allowed for that binary tree node. As mentioned above, the leaf nodes of the binary tree are called CUs and are further processed based on prediction and transformation without further partitioning.

[0103] Figure 3 This is a block diagram illustrating an example video encoder 200 that can perform the techniques of this disclosure. Figure 3This disclosure is provided for illustrative purposes and should not be construed as limiting the techniques extensively illustrated and described herein. For illustrative purposes, this disclosure describes video encoders 200 within the context of video decoding standards such as the ITU-T H.265 / HEVC video decoding standard and the VVC video decoding specification under development. However, the techniques of this disclosure are not limited to these video decoding standards and are generally applicable to other video encoding and decoding standards.

[0104] exist Figure 3 In the example, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, mode selection unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, DPB 218, and entropy coding unit 220 can be implemented in one or more processors or in processing circuitry. For example, units of the video encoder 200 can be implemented as one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Furthermore, the video encoder 200 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0105] The video data storage device 230 can store video data to be encoded by the components of the video encoder 200. The video encoder 200 can obtain data from, for example, a video source 104 (…). Figure 1 The video data memory 230 receives video data stored in the video data memory 230. The DPB 218 can act as a reference picture memory, storing reference video data for prediction of subsequent video data by the video encoder 200. The video data memory 230 and DPB 218 can be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 230 and DPB 218 can be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 can be on-chip with other components of the video encoder 200, as shown, or off-chip relative to these components.

[0106] In this disclosure, references to video data memory 230 should not be construed as limited to memory within video encoder 200 unless specifically described as such, or memory external to video encoder 200 unless specifically described as such. Rather, references to video data memory 230 should be understood as storing reference memory that video encoder 200 receives for encoding video data (e.g., video data of the current block to be encoded). Figure 1 The memory 106 can also provide temporary storage for the outputs from the various units of the video encoder 200.

[0107] Figure 3 The various units are shown to aid in understanding the operations performed by the video encoder 200. These units can be implemented as fixed-function circuits, programmable circuits, or combinations thereof. A fixed-function circuit is a circuit that provides a specific function and is pre-defined in the executable operation. A programmable circuit is a circuit that can be programmed to perform various tasks and provide flexible functionality in the executable operation. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions from software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is typically immutable. In some examples, one or more of these units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more units can be integrated circuits.

[0108] The video encoder 200 may include an arithmetic logic unit (ALU), an essential function unit (EFU), digital circuits, analog circuits, and / or a programmable core formed by programmable circuits. In an example where software executed by programmable circuits is used to perform the operation of the video encoder 200, memory 106 ( Figure 1 The video encoder 200 may store instructions (e.g., object code) of the software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0109] The video data storage unit 230 is configured to store the received video data. The video encoder 200 can retrieve images of the video data from the video data storage unit 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data storage unit 230 can be the raw video data to be encoded.

[0110] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra-frame prediction unit 226. The mode selection unit 202 may include additional functional units for performing video prediction based on other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, or the like.

[0111] Mode selection unit 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and the rate-distortion values ​​produced by these combinations. Encoding parameters may include dividing the CTU into CUs, the prediction mode of the CUs, the transformation type of the residual data of the CUs, the quantization parameters of the residual data of the CUs, and so on. Mode selection unit 202 can ultimately select a combination of encoding parameters that has a better rate-distortion value than other tested combinations. According to the techniques of this disclosure, mode selection unit 202 can choose whether to use multiple hypothesis prediction (MHP) according to the techniques of this disclosure, as well as any other various factors discussed above (e.g., weights applied to the prediction block, MVD resolution, and similar) to predict the current block.

[0112] The video encoder 200 can divide images retrieved from the video data storage 230 into a series of CTUs and encapsulate one or more CTUs within a slice. The mode selection unit 202 can divide the image into CTUs according to a tree structure, such as the QTBT structure described above or the quadtree structure of HEVC. As mentioned above, the video encoder 200 can form one or more CUs by dividing CTUs according to a tree structure. Such CUs can also generally be referred to as "video blocks" or "blocks".

[0113] Typically, mode selection unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra-frame prediction unit 226) to generate predicted blocks for the current block (e.g., the current CU, or, in HEVC, the overlapping portion of PU and TU). For inter-frame prediction of the current block, motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously decoded pictures stored in DPB 218). In particular, motion estimation unit 222 may calculate values ​​representing the similarity between a potential reference block and the current block, for example, based on sum of absolute differences (SAD), sum of squared differences, mean absolute difference (MAD), mean squared difference (MSD), or similar. Motion estimation unit 222 may typically perform these calculations using the sample-by-sample difference between the current block and the reference blocks under consideration. Motion estimation unit 222 may identify reference blocks with the lowest values ​​obtained from these calculations, indicating the reference block that is closest to the current block.

[0114] The motion estimation unit 222 can generate one or more motion vectors (MVs) that define the position of a reference block in a reference image relative to the position of a current block in the current image. The motion estimation unit 222 can then provide these motion vectors to the motion compensation unit 224. For example, for unidirectional inter-frame prediction, the motion estimation unit 222 can provide a single motion vector, while for bidirectional inter-frame prediction, the motion estimation unit 222 can provide two motion vectors. For MHP, the motion estimation unit 222 can provide additional motion vectors.

[0115] Motion compensation unit 224 can then use motion vectors and various weights to generate prediction blocks, and mode selection unit 202 can ultimately determine the appropriate weight. For example, motion compensation unit 224 can use motion vectors to retrieve data from reference blocks. As another example, if the motion vectors have fractional-sample accuracy, motion compensation unit 224 can interpolate the values ​​of the prediction blocks according to one or more interpolation filters. Furthermore, for bidirectional inter-frame prediction and / or MHP, motion compensation unit 224 can retrieve data from two reference blocks identified by corresponding motion vectors and combine the retrieved data, for example, by per-sample averaging or weighted averaging.

[0116] In some examples, mode selection unit 202 can determine to apply MHP as an extension of dual prediction with CU level weighting (BCW). For example, mode selection unit 202 can determine to use MHP as an extension of BCW to decode blocks to produce optimal rate-distortion optimization (RDO) values. Specifically, motion estimation unit 222 can determine motion information for generating intermediate prediction blocks in the basic dual prediction mode, and motion information for generating additional prediction blocks using additional prediction modes (e.g., dual prediction or single prediction). Mode selection unit 202 can determine that prediction blocks generated using BCW will be combined using unequal weights (e.g., weight values ​​other than 4).

[0117] Motion compensation unit 224 can generate two inter-frame prediction blocks (intermediate prediction blocks for BCW prediction) and combine the two intra-frame prediction blocks with unequal weights to form an intermediate prediction block. Motion compensation unit 224 can then use an additional prediction mode to generate an additional prediction block. Motion compensation unit 224 can further combine the intermediate prediction block and the additional prediction block according to the MHP. Furthermore, mode selection unit 202 can determine whether to use a non-merging mode such as AMVP to encode motion information.

[0118] The mode selection unit 202 can also determine the weighting factors used to combine intermediate prediction blocks with additional prediction blocks according to the MHP. The weighting factors of the MHP differ from the unequal weights of the BCW. For example, the mode selection unit 202 can provide the values ​​of the weighting factors to the entropy encoding unit 220 for encoding according to a table of additional hypothesis weight index values ​​(e.g., add_hyp_weight_idx) specifying the weighting factors. The entropy encoding unit 220 can determine the values ​​of the additional hypothesis weight index values ​​from this table. Therefore, the entropy encoding unit 220 can encode the data representing the unequal weights of the BCW and the weighting factors of the MHP.

[0119] The mode selection unit 202 can provide the weight values ​​and motion information of the basic dual prediction mode and the additional prediction mode to the entropy coding unit 220. In some examples, the motion estimation unit 222 and the motion compensation unit 224 can be configured to use the MVD accuracy of the motion information of the additional prediction signal, which is the same as the MVD accuracy of the basic dual prediction mode. Therefore, the mode selection unit 202 does not need to provide the entropy coding unit 220 with data representing the MVD accuracy of the additional prediction mode.

[0120] In other examples, if mode selection unit 202 determines to use a dual prediction mode with equal weights to predict the current block, mode selection unit 202 can prevent the use of additional prediction signals for MHP. Therefore, when the weights are equal for BCW, entropy coding unit 220 does not need to decode any additional motion information.

[0121] As another example, for intra-prediction or intra-prediction decoding, intra-prediction unit 226 can generate a prediction block from samples adjacent to the current block. For example, in directional mode, intra-prediction unit 226 can typically mathematically combine the values ​​of adjacent samples and fill these calculated values ​​in a defined direction across the current block to generate a prediction block. As another example, in DC mode, intra-prediction unit 226 can calculate the average of the adjacent samples to the current block and generate a prediction block to include this generated average for each sample in the prediction block.

[0122] Mode selection unit 202 provides the prediction block to residual generation unit 204. Residual generation unit 204 receives the raw, undecoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block of the current block. In some examples, residual generation unit 204 can also determine the difference between sample values ​​in the residual block to generate the residual block using residual differential pulse decode-modulation (RDPCM). In some examples, one or more subtractor circuits performing binary subtraction can be used to form residual generation unit 204.

[0123] In the example where mode selection unit 202 divides a CU into PUs, each PU can be associated with a luma prediction unit and a corresponding chroma prediction unit. Video encoder 200 and video decoder 300 can support PUs of various sizes. As mentioned above, the size of a CU can refer to the size of its luma decoding block, and the size of a PU can refer to the size of its luma prediction unit. Assuming a specific CU size is 2N×2N, video encoder 200 can support PU sizes of 2N×2N or N×N for intra-frame prediction, and symmetrical PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar sizes for inter-frame prediction. Video encoder 200 and video decoder 300 can also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0124] In the example where mode selection unit 202 does not further divide the CU into PUs, each CU can be associated with a luminance decoding block and a corresponding chrominance decoding block. As mentioned above, the size of the CU can refer to the size of the luminance decoding block of the CU. The video encoder 200 and the video decoder 300 can support CU sizes of 2N×2N, 2N×N, or N×2N.

[0125] For other video decoding techniques, such as intra-block copy mode decoding, affine mode decoding, and linear model (LM) mode decoding, as some examples, mode selection unit 202 generates a prediction block for the current block being encoded via a corresponding unit associated with the decoding technique. In some examples, such as palette mode decoding, mode selection unit 202 may not generate a prediction block, but may instead generate syntax elements indicating how the block can be reconstructed based on the selected palette. In such modes, mode selection unit 202 may provide these syntax elements to entropy coding unit 220 for encoding.

[0126] As described above, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0127] Transform processing unit 206 applies one or more transformations to the residual block to produce a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transformations to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a Discrete Cosine Transform (DCT), direction transformation, Karhunen-Loeve Transform (KLT), or a conceptually similar transformation to the residual block. In some examples, transform processing unit 206 may perform multiple transformations on the residual block, such as primary and secondary transformations, such as rotation transformations. In some examples, transform processing unit 206 does not apply any transformations to the residual block.

[0128] Quantization unit 208 quantizes the transform coefficients in a transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 quantizes the transform coefficients of the transform coefficient block based on the quantization parameter (QP) value associated with the current block. Video encoder 200 (e.g., via mode selection unit 202) can adjust the quantization degree applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss, and therefore the quantized transform coefficients may have lower precision than the original transform coefficients generated by transform processing unit 206.

[0129] The inverse quantization unit 210 and the inverse transform processing unit 212 can apply inverse quantization and inverse transform to the quantized transform coefficient block, respectively, to reconstruct the residual block from the transform coefficient block. The reconstruction unit 214 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the prediction block generated by the mode selection unit 202 (although it may have some degree of distortion). For example, the reconstruction unit 214 can add samples of the reconstructed residual block to the corresponding samples of the prediction block generated by the mode selection unit 202 to generate the reconstructed block.

[0130] Filter unit 216 can perform one or more filter operations on the reconstructed block. For example, filter unit 216 can perform a deblocking operation to reduce blockiness artifacts along the edges of the CU. In some examples, the operations of filter unit 216 can be skipped.

[0131] The video encoder 200 stores reconstructed blocks in the DPB 218. For example, in an example where the operation of the filter unit 216 is not required, the reconstruction unit 214 can store the reconstructed blocks in the DPB 218. In an example where the operation of the filter unit 216 is required, the filter unit 216 can store the filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can retrieve a reference image from the DPB 218, which is formed by the reconstructed (and possibly filtered) blocks, to perform inter-frame prediction of blocks in subsequently encoded images. Furthermore, the intra-frame prediction unit 226 can use the reconstructed blocks in the DPB 218 of the current image to perform intra-frame prediction of other blocks in the current image.

[0132] Typically, entropy coding unit 220 can entropy code syntax elements received from other functional components of video encoder 200. For example, entropy coding unit 220 can entropy code a block of quantized transform coefficients from quantization unit 208. As another example, entropy coding unit 220 can entropy code predictive syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from mode selection unit 202. Entropy coding unit 220 can perform one or more entropy coding operations on syntax elements that are another instance of video data to produce entropy-coded data. For example, entropy coding unit 220 can perform context-adaptive variable-length decoding (CAVLC), CABAC, variable-to-variable (V2V) length decoding, syntax-based context-adaptive binary arithmetic decoding (SBAC), probability interval partitioning entropy (PIPE) decoding, exponential Golomb coding, or another type of entropy coding operation on the data. In some examples, entropy coding unit 220 can operate in a bypass mode where no entropy coding is performed on syntax elements.

[0133] The video encoder 200 can output a bitstream that includes entropy coding syntax elements required to reconstruct slices or blocks of images. In particular, the entropy coding unit 220 can output a bitstream.

[0134] The operations described above are described in terms of a block. Such a description should be understood as operations on the luma decoding block and / or chroma decoding block. As mentioned above, in some examples, the luma decoding block and chroma decoding block are the luma and chroma components of the CU. In some examples, the luma decoding block and chroma decoding block are the luma and chroma components of the PU.

[0135] In some examples, the operations performed on the luma decoder block do not need to be repeated for the chroma decoder block. As an example, the operations for identifying the motion vector (MV) and reference image of the luma decoder block do not need to be repeated for identifying the MV and reference image of the chroma block. Instead, the MV of the luma decoder block can be scaled to determine the MV of the chroma decoder block, and the reference image can be the same. As another example, the intra-frame prediction process can be the same for both the luma and chroma decoder blocks.

[0136] Figure 4 This is a block diagram illustrating an example video decoder 300 that can perform the techniques disclosed herein. Figure 4 This disclosure is provided for illustrative purposes and does not limit the techniques extensively illustrated and described herein. For illustrative purposes, this disclosure describes a video decoder 300 based on VVC and HEVC (ITU-T H.265) technologies. However, the techniques of this disclosure can be implemented by video decoding devices configured to other video decoding standards.

[0137] exist Figure 4 In the example, the video decoder 300 includes a decoded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 314 can be implemented in one or more processors or in processing circuitry. For example, units of the video decoder 300 can be implemented as one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Furthermore, the video decoder 300 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0138] The prediction processing unit 304 includes a motion compensation unit 316 and an intra-frame prediction unit 318. The prediction processing unit 304 may include additional units for performing predictions based on other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, or the like. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0139] CPB memory 320 can store video data to be decoded by components of video decoder 300, such as encoded video bitstreams. The video data stored in CPB memory 320 can be, for example, from computer-readable medium 110 (…). Figure 1 The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, the CPB memory 320 may store video data other than syntax elements of the decoded picture, such as temporary data representing the output from various units of the video decoder 300. The DPB 314 typically stores decoded pictures that the video decoder 300 may output and / or use as reference video data while decoding subsequent data or pictures of the encoded video bitstream. The CPB memory 320 and DPB 314 may be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, the CPB memory 320 may be on-chip with other components of the video decoder 300, or off-chip relative to those components.

[0140] Additionally or alternatively, in some examples, the video decoder 300 can be drawn from the memory 120 ( Figure 1 The decoded video data is retrieved. That is, memory 120 can store data as discussed above with CPB memory 320. Similarly, when some or all of the functions of video decoder 300 are implemented in software to be executed by the processing circuitry of video decoder 300, memory 120 can store instructions to be executed by video decoder 300.

[0141] Figure 4 The various units shown are illustrated to aid in understanding the operations performed by the video decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Similar to... Figure 3 Fixed-function circuits refer to circuits that provide a specific function and have predefined operations that can be performed. Programmable circuits, on the other hand, are circuits that can be programmed to perform various tasks and provide flexible functionality in the operations they can perform. For example, a programmable circuit can execute software or firmware that causes it to operate in a manner defined by instructions in the software or firmware. Fixed-function circuits can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is typically immutable. In some examples, the one or more units may be different circuit blocks (fixed-function or programmable), and in some examples, the one or more units may be integrated circuits.

[0142] The video decoder 300 may include an ALU, an EFU, digital circuitry, analog circuitry, and / or a programmable core formed by programmable circuitry. In an example where the operation of the video decoder 300 is performed by software executed on the programmable circuitry, on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0143] The entropy decoding unit 302 can receive encoded video data from the CPB and perform entropy decoding on the video data to reproduce the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 can generate decoded video data based on the syntax elements extracted from the bitstream.

[0144] Typically, the video decoder 300 reconstructs the image on a block-by-block basis. The video decoder 300 can perform the reconstruction operation on each block individually (where the block currently being reconstructed (i.e. decoded) can be referred to as the "current block").

[0145] Entropy decoding unit 302 can entropy decode the syntax elements of the quantized transform coefficients defining the quantized transform coefficient block, as well as transform information (such as quantization parameters (QP) and / or (multiple) transform mode indications). Inverse quantization unit 306 can use the QP associated with the quantized transform coefficient block to determine the quantization degree, and similarly, determine the inverse quantization degree to be applied by inverse quantization unit 306. For example, inverse quantization unit 306 can perform a bit-by-bit left shift operation to inverse quantize the quantized transform coefficients. Inverse quantization unit 306 can thereby form a transform coefficient block including the transform coefficients.

[0146] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 can apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 can apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse direction transform, or another inverse transform to the transform coefficient block.

[0147] Furthermore, prediction processing unit 304 generates prediction blocks based on prediction information syntax elements entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax elements indicate that the current block is inter-frame predicted, motion compensation unit 316 can generate prediction blocks. In this case, the prediction information syntax elements may indicate a reference image in DPB 314 from which the reference block is retrieved, and a motion vector identifying the position of the reference block in the reference image relative to the current block in the current image. Motion compensation unit 316 can typically be configured with respect to motion compensation unit 224 ( Figure 3The method described is essentially the same as the one used to perform the inter-frame prediction process.

[0148] For example, motion compensation unit 316 can be configured to perform unidirectional or bidirectional prediction, or MHP according to the techniques of this disclosure. Prediction processing unit 304 can be configured according to the techniques of this disclosure to determine whether MHP is available for the current block, and accordingly control motion compensation unit 316 to perform or not perform MHP. In some examples, entropy decoding unit 302 can decode data representing MVD weights of motion vectors and / or weights to be applied to the prediction block to form a weighted combination of prediction blocks.

[0149] In some examples, the entropy decoding unit 302 can decode data representing the weights of the BCW modes of the current block. When the weights of the BCW modes are unequal, the entropy decoding unit 302 can further determine that additional motion information of the additional prediction signal will be entropy decoded. Therefore, the entropy decoding unit 302 can entropy decode the additional motion information and the motion information of the basic dual prediction modes of BCW, and provide the weights and all motion information to the prediction processing unit 304. The entropy decoding unit 302 can further entropy decode the weighting factor index value of the MHP, which represents the weighting factor to be used when combining prediction blocks according to the MHP. The entropy decoding unit 302 can use a weighting factor table that maps index values ​​to weighting factors to determine the weighting factor from the weighting factor index.

[0150] Motion compensation unit 316 can generate two inter-frame prediction blocks (intermediate prediction blocks for BCW prediction) and combine the two intra-frame prediction blocks with unequal weights to form an intermediate prediction block. Motion compensation unit 316 can then use an additional prediction mode to generate an additional prediction block. Motion compensation unit 316 can further combine the intermediate prediction block and the additional prediction block using a weighting factor based on the MHP. Again, the weighting factor of the MHP differs from the unequal weights of the BCW.

[0151] In some examples, the motion compensation unit 316 can be configured to use the MVD accuracy of the motion information of the additional prediction signal, which is the same as the MVD accuracy of the basic dual prediction mode. Therefore, the entropy decoding unit 302 may not decode the data representing the MVD accuracy of the additional prediction mode.

[0152] In other examples, if the entropy decoding unit 302 decodes an indication that a dual prediction mode with equal weights will be used to predict the data for the current block, then the entropy decoding unit 302 can determine not to decode additional motion information for the current block. Therefore, subsequent data in the bitstream can correspond to syntax elements different from those with additional motion information.

[0153] As another example, if the prediction information syntax element indicates that the current block is intra-predictable, then the intra-prediction unit 318 can generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Again, the intra-prediction unit 318 can generally be configured in a manner substantially similar to that of the intra-prediction module 226 ( Figure 3 The intra-frame prediction process is performed in the manner described above. The intra-frame prediction unit 318 can retrieve data of neighboring samples of the current block from the DPB 314.

[0154] Reconstruction unit 310 can use the prediction block and the residual block to reconstruct the current block. For example, reconstruction unit 310 can add samples from the residual block to the corresponding samples from the prediction block to reconstruct the current block.

[0155] Filter unit 312 can perform one or more filter operations on the reconstructed block. For example, filter unit 312 can perform a deblocking operation to reduce blockiness artifacts along the edges of the reconstructed block. The operation of filter unit 312 is not required in all examples.

[0156] The video decoder 300 can store reconstructed blocks in the DPB 314. For example, in an example where the filter unit 312 is not operated, the reconstruction unit 310 can store the reconstructed blocks in the DPB 314. In an example where the filter unit 312 is operated, the filter unit 312 can store the filtered reconstructed blocks in the DPB 314. As discussed above, the DPB 314 can provide reference information to the prediction processing unit 304, such as samples of the current image for intra-frame prediction and previously decoded images for subsequent motion compensation. Furthermore, the video decoder 300 can output the decoded images from the DPB 314 for subsequent display on a display device (such as...). Figure 1 It is displayed on the display device 118.

[0157] Figure 5 This is a flowchart illustrating an example method for encoding the current block according to the technology of this disclosure. The current block may include the current CU. Although regarding video encoder 200 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform similar actions. Figure 5 The method.

[0158] In this example, the video encoder 200 initially predicts the current block (350). For example, the video encoder 200 can form a prediction block for the current block. According to the techniques of this disclosure, the video encoder 200 can form a prediction block based on the MHP discussed above. The video encoder 200 can then compute the residual block of the current block (352). To compute the residual block, the video encoder 200 can compute the difference between the original, undecoded block and the prediction block of the current block. The video encoder 200 can then transform and quantize the coefficients of the residual block (354). Next, the video encoder 200 can scan the quantized transform coefficients of the residual block (356). During or after the scan, the video encoder 200 can entropy encode the coefficients (358). For example, the video encoder 200 can encode the coefficients using CAVLC or CABAC. The video encoder 200 can then output the entropy-encoded data of the block (360).

[0159] The video encoder 200 can also decode the current block after encoding it, using the decoded version of the current block as reference data for subsequent decoded data (e.g., in inter-frame or intra-frame prediction modes). Therefore, the video encoder 200 can perform inverse quantization and inverse transform on the coefficients to reproduce the residual block (362). The video encoder 200 can combine the residual block with the prediction block to form a decoded block (364). The video encoder 200 can then store the decoded block in the DPB 218 (366).

[0160] In this way, Figure 5 The method represents an example of a method for decoding (and / or encoding) the current block, including determining that a first weight and a second weight are assigned to the current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight are assigned, determining whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the current block is to be predicted using an MHP mode with the dual prediction mode as the base mode, determining an additional inter-frame prediction mode of the MHP mode; generating a first prediction block based on the dual prediction mode; generating a second prediction block based on the additional inter-frame prediction mode; using the first prediction block and the second prediction block, generating a final prediction block for the current block according to the MHP mode; and decoding the current block using the final prediction block.

[0161] Figure 6 This is a flowchart illustrating an example method for decoding the current block according to the technology disclosed herein. The current block may include the current CU. Although regarding the video decoder 300 ( Figure 1 and Figure 4 The description is provided, but it should be understood that other devices can be configured to perform similar actions. Figure 6 The method.

[0162] The video decoder 300 can receive entropy-coded data of the current block, such as entropy-coded prediction information and entropy-coded data of the coefficients of the residual block corresponding to the current block (370). The video decoder 300 can entropy decode the entropy-coded data to determine the prediction information of the current block and reproduce the coefficients of the residual block (372). The video decoder 300 can predict the current block (374), for example, by calculating the prediction block of the current block using an intra-frame or inter-frame prediction mode indicated by the prediction information of the current block. According to the techniques of this disclosure, the video decoder 300 can form a prediction block based on the MHP discussed above. The video decoder 300 can then perform an inverse scan on the reproduced coefficients (376) to create a quantized transform coefficient block. The video decoder 300 can then perform inverse quantization and inverse transform on the quantized transform coefficients to produce a residual block (378). The video decoder 300 can finally decode the current block by combining the prediction block and the residual block (380).

[0163] In this way, Figure 6 The method represents an example of a method for decoding the current block, including determining that a first weight and a second weight are assigned to the current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight are assigned, determining whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the current block is to be predicted using an MHP mode with the dual prediction mode as the base mode, determining an additional inter-frame prediction mode of the MHP mode; generating a first prediction block based on the dual prediction mode; generating a second prediction block based on the additional inter-frame prediction mode; using the first prediction block and the second prediction block, generating a final prediction block for the current block according to the MHP mode; and decoding the current block using the final prediction block.

[0164] Figure 7 This is a conceptual diagram illustrating an example of applying dual prediction (BCW) and multiple hypothesis prediction (MHP) with CU level weighting as an extension of BCW. About Figure 1 and 4 The video decoder 300 is used to describe Figure 7 Examples are provided. However, the video encoder 200 can also be configured to perform these or similar techniques.

[0165] Initially, the video decoder 300 can form a first dual prediction (BP) block 402 and a second dual prediction block 404. The video decoder 300 can use a first motion vector to form the first dual prediction block 402 and a second motion vector to form the second dual prediction block 404. The video decoder 300 can also decode motion information representing the first and second motion vectors, for example, according to a mode other than a merging mode (such as AMVP). Therefore, the video decoder 300 can decode, for example, AMVP candidate indices identifying adjacent blocks for use as motion vector predictors, motion vector difference (MVD) values ​​representing the difference between the MVP and the actual first and second motion vectors, reference picture list identifiers, and reference picture index values ​​identifying reference pictures in the corresponding reference picture lists. MVD values ​​can have a specific resolution, such as full pixels, half pixels, quarter pixels, eighth pixels, or similar.

[0166] The video decoder 300 can further decode the data representing weights W1 406 and W2 408. Weights W1 406 and W2 408, when added together, form a total value of 8. The video decoder 300 can apply weight W1 406 to samples of the first dual prediction block 402 and weight W2 408 to samples of the second dual prediction block 404. That is, the video decoder 300 can multiply weight W1 406 by the value of each sample of the first dual prediction block 402 and weight W2 408 by the value of each sample of the second dual prediction block 404. The video decoder 300 can perform a bit-by-bit left shift operation based on the values ​​of the corresponding weights, instead of performing an explicit multiplication function. The video decoder 300 can then combine the weighted samples of the first dual prediction block 402 with the weighted samples of the second dual prediction block 404 and divide the sum of each sample by the total value of the weights (e.g., 8). The video decoder 300 can perform a 3-bit bit-by-bit right shift, instead of performing an explicit division operation. The obtained blocks are in Figure 7 The middle (int.) prediction block 410 is referred to as the intermediate prediction block.

[0167] When weights W1 406 and W2 408 are not equal (e.g., neither W1 406 nor W2 408 equals 4), the video decoder 300 can determine to perform multiple hypothesis prediction as an extension of BCW. Therefore, the video decoder 300 can further decode motion information from the video bitstream for the additional inter-frame prediction mode. The video decoder 300 can use, for example, an AMVP mode or another non-merging mode to decode the motion information. In some examples, the video decoder 300 can determine that the MVD value for the additional inter-frame prediction mode has the same MVD resolution as the motion information used to form the dual-prediction motion information of the first dual-prediction block 402 and the second dual-prediction block 404, such that additional data representing the MVD resolution for the additional inter-frame prediction mode does not need to be decoded. The video decoder 300 can also use the motion information to generate the additional prediction block 412.

[0168] The video decoder 300 can also decode data representing weighting factor values ​​WF1 414 and WF2 416. For example, the video decoder 300 can decode weighting factor index values, such as the value of the add_hyp_weight_idx syntax element. The video decoder 300 can determine the values ​​of WF1 414 and WF2 416 based on the weighting factor index values, for example, using a weighting factor index table. The video decoder 300 can then apply WF1 414 to samples of intermediate prediction block 410 and WF2 416 to samples of additional prediction block 412. Finally, the video decoder 300 can combine the weighted values ​​of the samples of intermediate prediction block 410 with the weighted values ​​of the samples of additional prediction block 412 to generate a final prediction block 418. The video decoder 300 can then use the final prediction block 418 to decode (reconstruct) the corresponding blocks, for example, by adding the samples of the final prediction block 418 to the corresponding samples of the residual blocks.

[0169] Figure 8 This is a flowchart illustrating an example method for decoding (e.g., reproducing) a current block of video data according to the techniques of this disclosure. Figure 8 The method can be performed by the video encoder 200 (during the decoding loop of the encoding process) or by the video decoder 300. For example, Figure 8 The method can usually correspond to Figure 5 Step 350 or Figure 6 Step 374. For illustrative and explanatory purposes, let's explain for video decoder 300. Figure 8 The method.

[0170] Initially, the video decoder 300 can generate a first prediction block (430) and a second prediction block (432). For example, the video decoder 300 can receive motion information encoded in AMVP mode, including the corresponding motion vector difference (MVD) value, AMVP candidate identifier, reference list identifier and reference list index, and weights to be applied to form a dual prediction block with CU-level weighted (BCW).

[0171] In this example, the video decoder 300 can determine that the weights are unequal values, for example, neither weight is equal to 4. Therefore, the video decoder 300 can determine to apply Multiple Hypothesis Prediction (MHP) as an extension of BCW. The video decoder 300 can then continue to apply the weights to the first prediction block and the second prediction block (434). The video decoder 300 can also combine the first weighted prediction block and the second weighted prediction block to form an intermediate prediction block for MHP (436).

[0172] In response to the determination of unequal weights, the video decoder 300 can decode additional motion information for the additional prediction mode, such as the AMVP candidate index, MVD, reference list identifier, and reference list index (or multiple of these values ​​if the additional prediction mode is a dual prediction). The video decoder 300 can then use the additional motion information to generate an additional prediction block (438). The video decoder 300 can further determine the weights of the intermediate prediction block and the additional prediction block. Such weights can be predetermined or signaled, for example, using an index from a weighting factor table. The video decoder 300 can then apply the weights to the intermediate prediction block and the additional prediction block (440), and combine the weighted intermediate prediction block with the weighted additional prediction block to form the final prediction block (442).

[0173] Finally, the video decoder 300 can use the final predicted block to decode the current block (444). For example, the video decoder 300 can combine samples from the final predicted block with corresponding samples from the reconstructed residual block, such as regarding Figure 6 Steps 376 to 380 are discussed. When the video encoder 200 executes... Figure 8 In this method, the video encoder 200 can subtract the samples of the final prediction block from the corresponding samples of the residual block, as shown in the following example. Figure 5 Steps 352 to 358, as discussed, are used to encode the current block. Furthermore, the video encoder 200 can add samples from the final predicted block to the corresponding samples of the current block, as per [reference to...]. Figure 5 Steps 362 to 366 are discussed to decode the current block.

[0174] In this way, Figure 8The method represents an example of a method for decoding (and / or encoding) the current block, including determining that a first weight and a second weight are assigned to the current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight are assigned, determining whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; in response to determining that the current block is to be predicted using an MHP mode with the dual prediction mode as the base mode, determining an additional inter-frame prediction mode of the MHP mode; generating a first prediction block based on the dual prediction mode; generating a second prediction block based on the additional inter-frame prediction mode; using the first prediction block and the second prediction block, generating a final prediction block for the current block according to the MHP mode; and decoding the current block using the final prediction block.

[0175] The various techniques of this disclosure are summarized in the following clauses:

[0176] Clause 1: A method for decoding video data, the method comprising: determining to perform inter-frame predictive decoding of a current block of data using at least two motion vectors with assigned weights, wherein the at least two motion vectors are decoded using a mode other than a merging mode; in response to determining that weights are assigned, determining whether to use multiple hypothesis prediction (MHP) to predict the current block; in response to determining that MHP should be used to predict the current block, forming a prediction block for the current block based on the MHP using the at least two motion vectors; and decoding the current block using the prediction block.

[0177] Clause 2: According to the method of Clause 1, wherein determining whether to use MHP to predict the current block includes a weight value of 4, it is determined that MHP should be used to predict the current block.

[0178] Clause 3: A method for decoding video data, the method comprising: determining inter-frame predictive decoding of a current block of data using two or more motion vectors, wherein the two or more motion vectors are decoded using a mode other than a merging mode, the two or more motion vectors including a base motion vector and an additional motion vector; determining that the precision of the motion vector difference (MVD) value of the additional motion vector is equal to the precision of the MVD value of the base motion vector; forming a prediction block for the current block using the two or more motion vectors according to multiple hypothesis prediction (MHP); and decoding the current block using the prediction block.

[0179] Clause 4: A method, including the method of any one of Clauses 1 and 2 and the method of Clause 3.

[0180] Clause 5: The method according to any one of Clauses 3 and 4 further includes: decoding the data representing the precision of the MVD value of the basic motion vector; and inferring the precision of the MVD value of the additional motion vector from the precision of the MVD value of the basic motion vector without decoding the additional data representing the precision of the MVD value of the additional motion vector.

[0181] Clause 6: A method for decoding video data, the method comprising: determining inter-frame predictive decoding of a current block of data using two or more motion vectors, the two or more motion vectors including a basic motion vector and an additional motion vector; decoding data of a first precision representing motion vector difference (MVD) values ​​of the additional motion vectors; decoding data of a second precision representing MVD values ​​of the basic motion vectors; forming a prediction block for the current block using the two or more motion vectors according to multiple hypothesis prediction (MHP); and decoding the current block using the prediction block.

[0182] Clause 7: A method, including the method of any one of Clauses 1 and 2 and the method of Clause 6.

[0183] Clause 8: A method for decoding video data, the method comprising: determining to perform inter-frame prediction decoding of a current block of data using two or more sets of motion information according to multiple hypothesis prediction (MHP), the first set of motion information including a first motion vector and first reference picture identification data representing a first reference picture to which the first motion vector points; in response to determining to perform inter-frame prediction decoding of the current block using two or more sets of motion information according to MHP, determining that a second set of motion information includes at least one of a second motion vector different from the first motion vector or second reference picture identification data representing a second reference picture different from the first reference picture; forming a prediction block for the current block using the two or more sets of motion information according to multiple hypothesis prediction (MHP); and decoding the current block using the prediction block.

[0184] Clause 9: A method, including the method of any one of Clauses 1 to 7 and the method of Clause 8.

[0185] Clause 10: A method for decoding video data, the method comprising: determining that inter-frame predictive decoding of a current block of data is performed using at least one motion vector; determining that the advanced motion vector resolution (AMVR) of the at least one motion vector is half-luminance sample resolution; in response to determining that the AMVR of the at least one motion vector is half-luminance sample resolution, determining that the current block is not predicted using multiple hypothesis prediction (MHP); using at least one motion vector to form a prediction block for the current block without using MHP; and decoding the current block using the prediction block.

[0186] Clause 11: A method, including the method of any one of Clauses 1 to 9 and the method of Clause 10.

[0187] Clause 12: A method for decoding video data, the method comprising: determining inter-frame predictive decoding of a current block of data using at least one motion vector having sub-pixel precision; decoding data representing an interpolation filter to be used to interpolate values ​​of subpixels of a reference picture; determining, based on the data representing the interpolation filter, whether to use multiple hypothesis prediction (MHP) to predict the current block; forming a prediction block for the current block using at least one motion vector; and decoding the current block using the prediction block.

[0188] Clause 13: A method, including the method of any one of Clauses 1 to 11 and the method of Clause 12.

[0189] Clause 14: The method of any one of Clauses 12 and 13, wherein determining whether to use MHP to predict the current block comprises: determining whether the interpolation filter assigns more weight to a first sample of the reference image than to a second sample of the reference image; and determining that MHP will be used to predict the current block when the interpolation filter assigns more weight to the first sample of the reference image than to the second sample of the reference image.

[0190] Clause 15: A method for decoding video data, the method comprising: determining inter-frame predictive decoding of a current block of data using two or more sets of motion information based on multiple hypothesis prediction (MHP), the first set of motion information including a first reference index and the second set of motion information including a second reference index; determining a first motion vector prediction candidate list based on the first reference index; decoding a first motion vector of the first set of motion information using the first motion vector prediction candidate list; determining a second motion vector prediction candidate list based on the second reference index; decoding a second motion vector of the second set of motion information using the second motion vector prediction candidate list; forming a prediction block for the current block using the first motion vector and the second motion vector based on multiple hypothesis prediction (MHP); and decoding the current block using the prediction block.

[0191] Clause 16: A method, including the method of any one of Clauses 1 to 14 and the method of Clause 15.

[0192] Clause 17: A method for decoding video data, the method comprising: determining inter-frame predictive decoding of a current block of data using two or more motion vectors according to multiple hypothesis prediction (MHP); forming a prediction block for the current block using the two or more motion vectors according to MHP, including: forming a first intermediate prediction block using a first motion vector of the two or more motion vectors; forming a second intermediate prediction block using a second motion vector of the two or more motion vectors; determining a weight to be applied to the second intermediate prediction block, the weight including one of 1 / 4, -1 / 8, or 1 / 2; and using the weight to combine the first intermediate prediction block and the second intermediate prediction block to form the prediction block; and decoding the current block using the prediction block.

[0193] Clause 18: A method, including the method of any one of Clauses 1 to 16 and the method of Clause 17.

[0194] Clause 19: The method of any one of Clauses 17 and 18, wherein determining the weights to be applied to the second intermediate prediction block comprises: decoding the value of the add_hyp_weight_idx syntax element; determining that the weights include 1 / 4 when the value of the add_hyp_weight_idx syntax element is 0; determining that the weights include -1 / 8 when the value of the add_hyp_weight_idx syntax element is 1; and determining that the weights include 1 / 2 when the value of the add_hyp_weight_idx syntax element is 2.

[0195] Clause 20: The method according to any one of Clauses 1 to 19 also includes encoding the current block before decoding the current block.

[0196] Clause 21: An apparatus for decoding video data, the apparatus comprising one or more components for performing the methods of any one of Clauses 1 to 20.

[0197] Clause 22: A device pursuant to Clause 21, wherein the one or more components include one or more processors implemented in a circuit.

[0198] Clause 23: The device pursuant to Clause 21 also includes a display configured to display decoded video data.

[0199] Clause 24: Devices pursuant to Clause 21, wherein the device includes one or more of a camera, computer, mobile device, broadcast receiver device or set-top box.

[0200] Clause 25: The device pursuant to Clause 21 also includes a memory configured to store video data.

[0201] Clause 26: A computer-readable storage medium having instructions stored thereon, which, when executed, cause a processor of a device for decoding video data to perform any one of the methods of Clauses 1 to 20.

[0202] Clause 27: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using at least two motion vectors with assigned weights, wherein the at least two motion vectors are decoded using a mode other than a merging mode; means for determining, in response to determining that weights are assigned, whether to use multiple hypothesis prediction (MHP) to predict the current block; means for forming a prediction block for the current block using the at least two motion vectors according to the MHP in response to determining that MHP should be used to predict the current block; and means for decoding the current block using the prediction block.

[0203] Clause 28: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using two or more motion vectors, wherein the decoding of the two or more motion vectors is performed using a mode other than a merging mode, the two or more motion vectors including a basic motion vector and an additional motion vector; means for determining that the precision of the motion vector difference (MVD) value of the additional motion vectors is equal to the precision of the MVD value of the basic motion vector; means for forming a prediction block for the current block using the two or more motion vectors according to multiple hypothesis prediction (MHP); and means for decoding the current block using the prediction block.

[0204] Clause 29: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using two or more motion vectors, the two or more motion vectors including a basic motion vector and an additional motion vector; means for decoding data with a first precision representing a motion vector difference (MVD) value of the additional motion vector; means for decoding data with a second precision representing an MVD value of the basic motion vector; means for forming a prediction block for the current block using the two or more motion vectors according to multiple hypothesis prediction (MHP); and means for decoding the current block using the prediction block.

[0205] Clause 30: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using two or more sets of motion information according to multiple hypothesis prediction (MHP), the first set of motion information including a first motion vector and first reference image identification data representing a first reference image to which the first motion vector points; means for determining, in response to determining inter-frame predictive decoding of the current block using two or more sets of motion information according to MHP, a second set of motion information including at least one of a second motion vector different from the first motion vector or second reference image identification data representing a second reference image different from the first reference image; means for forming a prediction block for the current block according to multiple hypothesis prediction (MHP) using the two or more sets of motion information; and means for decoding the current block using the prediction block.

[0206] Clause 31: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using at least one motion vector; means for determining that the advanced motion vector resolution (AMVR) of the at least one motion vector is half-luminance sample resolution; means for determining, in response to determining that the AMVR of the at least one motion vector is half-luminance sample resolution, not to use multiple hypothesis prediction (MHP) to predict the current block; means for forming a prediction block for the current block using at least one motion vector without using MHP; and means for decoding the current block using the prediction block.

[0207] Clause 32: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using at least one motion vector having sub-pixel precision; means for decoding data representing an interpolation filter to be used for interpolating values ​​of subpixels of a reference picture; means for determining, based on the data representing the interpolation filter, whether to use multiple hypothesis prediction (MHP) to predict the current block; means for forming a prediction block for the current block using at least one motion vector; and means for decoding the current block using the prediction block.

[0208] Clause 33: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using two or more sets of motion information according to multiple hypothesis prediction (MHP), the first set of motion information including a first reference index and the second set of motion information including a second reference index; means for determining a first motion vector prediction candidate list based on the first reference index; means for decoding a first motion vector of the first set of motion information using the first motion vector prediction candidate list; means for determining a second motion vector prediction candidate list based on the second reference index; means for decoding a second motion vector of the second set of motion information using the second motion vector prediction candidate list; means for forming a prediction block for the current block using the first motion vector and the second motion vector according to multiple hypothesis prediction (MHP); and means for decoding the current block using the prediction block.

[0209] Clause 34: An apparatus for decoding video data, the apparatus comprising: means for determining inter-frame predictive decoding of a current block of data using two or more motion vectors according to multiple hypothesis prediction (MHP); means for forming a prediction block for the current block using the two or more motion vectors according to MHP, including: means for forming a first intermediate prediction block using a first motion vector of the two or more motion vectors; means for forming a second intermediate prediction block using a second motion vector of the two or more motion vectors; means for determining a weight to be applied to the second intermediate prediction block, the weight including one of 1 / 4, -1 / 8, or 1 / 2; means for combining the first intermediate prediction block and the second intermediate prediction block using the weight to form the prediction block; and means for decoding the current block using the prediction block.

[0210] Clause 35: A method for decoding video data, the method comprising: determining that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determining whether to use a multi-hypothesis prediction (MHP) mode having the dual prediction mode as a base mode to predict the current block; in response to determining that the MHP mode having the dual prediction mode as a base mode should be used to predict the current block, determining an additional inter-frame prediction mode of the MHP mode; generating a first prediction block based on the dual prediction mode; generating a second prediction block based on the additional inter-frame prediction mode; generating a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decoding the current block using the final prediction block.

[0211] Clause 36: The method of Clause 35, wherein determining whether to use MHP to predict the current block includes determining whether to use MHP to predict the current block when the first weight is not equal to 4 and the second weight is not equal to 4.

[0212] Clause 37: The method of Clause 35, wherein determining whether to use MHP to predict the current block includes determining whether to use MHP to predict the current block when the first weight is not equal to the second weight.

[0213] Clause 38: The method according to Clause 35 further includes: decoding a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction modes; decoding a second motion vector using an AMVP mode for dual prediction modes; and decoding a third motion vector for an additional inter-frame prediction mode, wherein generating the first prediction block includes: generating a first intermediate prediction block using the first motion vector, generating a second intermediate prediction block using the second motion vector, applying a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block, applying a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block, and combining the first weighted intermediate prediction block and the second weighted intermediate prediction block to form the first prediction block, and wherein generating the second prediction block includes generating the second prediction block using the third motion vector.

[0214] Clause 39: The method according to Clause 38 further includes: decoding data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and determining that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0215] Clause 40: The method according to Clause 35, wherein generating the final prediction block comprises: determining a third weight and a fourth weight of the MHP pattern; applying the third weight to a first prediction block to form a first weighted prediction block; applying the fourth weight to a second prediction block to form a second weighted prediction block; and combining the first weighted prediction block with the second weighted prediction block to form the final prediction block.

[0216] Clause 41: The method according to Clause 40, wherein determining the third weight comprises: decoding the index value; and determining the third weight to which the index value is mapped in the mapping table, and wherein determining the fourth weight comprises calculating the fourth weight as one minus the third weight.

[0217] Clause 42: The method according to Clause 35 also includes encoding the current block using the final predicted block before decoding the current block.

[0218] Clause 43: An apparatus for decoding video data, the apparatus comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determine whether to use a multi-hypothesis prediction (MHP) mode having a dual prediction mode as the base mode to predict the current block; in response to determining that an MHP mode having a dual prediction mode as the base mode is to be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block based on the dual prediction mode; generate a second prediction block based on the additional inter-frame prediction mode; generate a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

[0219] Clause 44: A device pursuant to Clause 43, wherein, in order to determine whether to use MHP to predict the current block, the one or more processors are configured to determine whether to use MHP to predict the current block when a first weight is not equal to 4 and a second weight is not equal to 4.

[0220] Clause 45: A device pursuant to Clause 43, wherein, in order to determine whether to use MHP to predict the current block, the one or more processors are configured to determine whether to use MHP to predict the current block when a first weight is not equal to a second weight.

[0221] Clause 46: The apparatus according to Clause 43, wherein the one or more processors are further configured to: decode a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction mode; decode a second motion vector using an AMVP mode for dual prediction mode; and decode a third motion vector for an additional inter-frame prediction mode, wherein, in order to generate a first prediction block, the one or more processors are configured to: generate a first intermediate prediction block using the first motion vector, generate a second intermediate prediction block using the second motion vector, apply a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block, apply a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block, and combine the first weighted intermediate prediction block with the second weighted intermediate prediction block to form a first prediction block, and wherein, in order to generate a second prediction block, the one or more processors are configured to generate a second prediction block using the third motion vector.

[0222] Clause 47: A device pursuant to Clause 46, wherein the one or more processors are further configured to: decode data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and determine that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0223] Clause 48: A device pursuant to Clause 43, wherein, in order to generate a final prediction block, the one or more processors are configured to: determine a third weight and a fourth weight of the MHP mode; apply the third weight to a first prediction block to form a first weighted prediction block; apply the fourth weight to a second prediction block to form a second weighted prediction block; and combine the first weighted prediction block with the second weighted prediction block to form a final prediction block.

[0224] Clause 49: A device pursuant to Clause 48, wherein, in order to determine a third weight, the one or more processors are configured to: decode an index value; and determine in a mapping table the third weight to which the index value is mapped, and wherein, in order to determine a fourth weight, the one or more processors are configured to calculate the fourth weight as a minus the third weight.

[0225] Clause 50: A device pursuant to Clause 43, wherein the one or more processors are configured to encode the current block using the final predicted block before decoding the current block.

[0226] Clause 51: The device pursuant to Clause 43 also includes a display configured to display decoded video data.

[0227] Clause 52: Devices pursuant to Clause 43, wherein the device includes one or more of a camera, computer, mobile device, broadcast receiver device, or set-top box.

[0228] Clause 53: A computer-readable storage medium having instructions thereon, which, when executed, cause a processor to: determine that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determine whether to use a multi-hypothesis prediction (MHP) mode having a dual prediction mode as the basic mode to predict the current block; in response to determining that an MHP mode having a dual prediction mode as the basic mode should be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block according to the dual prediction mode; generate a second prediction block according to the additional inter-frame prediction mode; generate a final prediction block for the current block according to the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

[0229] Clause 54: A computer-readable storage medium pursuant to Clause 53, wherein causing a processor to determine whether to use MHP to predict instructions for the current block includes causing the processor to determine, when a first weight is not equal to 4 and a second weight is not equal to 4, to use MHP to predict instructions for the current block.

[0230] Clause 55: A computer-readable storage medium pursuant to Clause 53, wherein enabling a processor to determine whether to use MHP to predict instructions for the current block includes enabling the processor to determine, when a first weight is not equal to a second weight, to use MHP to predict instructions for the current block.

[0231] Clause 56: The computer-readable storage medium pursuant to Clause 53 further includes instructions that cause the processor to perform: decoding a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction mode; decoding a second motion vector using an AMVP mode for dual prediction mode; and decoding a third motion vector for an additional inter-frame prediction mode, wherein the instructions that cause the processor to generate the first prediction block include instructions that cause the processor to perform: generating a first intermediate prediction block using the first motion vector; generating a second intermediate prediction block using the second motion vector; applying a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block; applying a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block; and combining the first weighted intermediate prediction block with the second weighted intermediate prediction block to form the first prediction block, and wherein the instructions that cause the processor to generate the second prediction block include instructions that cause the processor to generate the second prediction block using the third motion vector.

[0232] Clause 57: The computer-readable storage medium pursuant to Clause 56 further includes instructions that cause the processor to perform: decoding data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and determining that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0233] Clause 58: A computer-readable storage medium pursuant to Clause 53, wherein instructions for causing a processor to generate a final prediction block include instructions for causing the processor to perform: determining a third weight and a fourth weight of an MHP mode; applying the third weight to a first prediction block to form a first weighted prediction block; applying the fourth weight to a second prediction block to form a second weighted prediction block; and combining the first weighted prediction block with the second weighted prediction block to form a final prediction block.

[0234] Clause 59: A computer-readable storage medium pursuant to Clause 58, wherein the instructions for causing a processor to determine a third weight include instructions for causing the processor to perform: decoding an index value; and determining a third weight to which the index value is mapped in a mapping table, and wherein the instructions for causing a processor to determine a fourth weight include instructions for causing the processor to calculate the fourth weight as a subtraction of the third weight.

[0235] Clause 60: A computer-readable storage medium pursuant to Clause 53 also includes instructions that cause the processor to encode the current block using the final predicted block before decoding the current block.

[0236] Clause 61: An apparatus for decoding video data, the apparatus comprising: means for determining a first weight and a second weight assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; means for determining, in response to determining that the first weight and the second weight are assigned, whether to use a multi-hypothesis prediction (MHP) mode having the dual prediction mode as a base mode to predict the current block; means for determining an additional inter-frame prediction mode of the MHP mode in response to determining that the MHP mode having the dual prediction mode as a base mode is to be used to predict the current block; means for generating a first prediction block based on the dual prediction mode; means for generating a second prediction block based on the additional inter-frame prediction mode; means for generating a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and means for decoding the current block using the final prediction block.

[0237] Clause 62: The device pursuant to Clause 61, wherein the component for determining whether to use MHP to predict the current block includes a component for determining whether to use MHP to predict the current block when the first weight is not equal to 4 and the second weight is not equal to 4.

[0238] Clause 63: The device pursuant to Clause 61, wherein the component for determining whether to use MHP to predict the current block includes a component for determining whether to use MHP to predict the current block when the first weight is not equal to the second weight.

[0239] Clause 64: The apparatus according to Clause 61 further includes: means for decoding a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction mode; means for decoding a second motion vector using an AMVP mode for dual prediction mode; and means for decoding a third motion vector for an additional inter-frame prediction mode, wherein the means for generating the first prediction block includes: means for generating a first intermediate prediction block using the first motion vector; means for generating a second intermediate prediction block using the second motion vector; means for applying a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block; means for applying a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block; and means for combining the first weighted intermediate prediction block and the second weighted intermediate prediction block to form the first prediction block, and wherein the means for generating the second prediction block includes means for generating the second prediction block using the third motion vector.

[0240] Clause 65: The apparatus pursuant to Clause 64 further includes: means for decoding data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and means for determining that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0241] Clause 66: The apparatus according to Clause 61, wherein the components for generating the final prediction block include: components for determining a third weight and a fourth weight of the MHP mode; components for applying the third weight to a first prediction block to form a first weighted prediction block; components for applying the fourth weight to a second prediction block to form a second weighted prediction block; and components for combining the first weighted prediction block and the second weighted prediction block to form the final prediction block.

[0242] Clause 67: The device pursuant to Clause 66, wherein the components for determining the third weight include: components for decoding the index value; and components for determining the third weight to which the index value is mapped in the mapping table, and wherein the components for determining the fourth weight include components for calculating the fourth weight as a subtraction of the third weight.

[0243] Clause 68: The apparatus pursuant to Clause 61 further includes a component for encoding the current block using the final predicted block before decoding the current block.

[0244] Clause 69: A method for decoding video data, the method comprising: determining that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determining whether to use a multi-hypothesis prediction (MHP) mode having the dual prediction mode as a base mode to predict the current block; in response to determining that the MHP mode having the dual prediction mode as a base mode should be used to predict the current block, determining an additional inter-frame prediction mode of the MHP mode; generating a first prediction block based on the dual prediction mode; generating a second prediction block based on the additional inter-frame prediction mode; generating a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decoding the current block using the final prediction block.

[0245] Clause 70: The method of Clause 69, wherein determining whether to use MHP to predict the current block includes determining whether to use MHP to predict the current block when the first weight is not equal to 4 and the second weight is not equal to 4.

[0246] Clause 71: The method of Clause 69, wherein determining whether to use MHP to predict the current block includes determining whether to use MHP to predict the current block when the first weight is not equal to the second weight.

[0247] Clause 72: The method according to any one of Clauses 69 to 71 further includes: decoding a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for a dual prediction mode; decoding a second motion vector using an AMVP mode for a dual prediction mode; and decoding a third motion vector for an additional inter-frame prediction mode, wherein generating the first prediction block includes: generating a first intermediate prediction block using the first motion vector, generating a second intermediate prediction block using the second motion vector, applying a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block, applying a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block, and combining the first weighted intermediate prediction block and the second weighted intermediate prediction block to form the first prediction block, and wherein generating the second prediction block includes generating the second prediction block using the third motion vector.

[0248] Clause 73: The method according to Clause 72 further includes: decoding data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and determining that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0249] Clause 74: The method according to any one of Clauses 69 to 73, wherein generating the final prediction block comprises: determining a third weight and a fourth weight of the MHP model; applying the third weight to a first prediction block to form a first weighted prediction block; applying the fourth weight to a second prediction block to form a second weighted prediction block; and combining the first weighted prediction block with the second weighted prediction block to form the final prediction block.

[0250] Clause 75: The method according to Clause 74, wherein determining the third weight comprises: decoding the index value; and determining the third weight to which the index value is mapped in the mapping table, and wherein determining the fourth weight comprises calculating the fourth weight as one minus the third weight.

[0251] Clause 76: The method according to any one of Clauses 69 to 75 further includes encoding the current block using the final predicted block before decoding the current block.

[0252] Clause 77: An apparatus for decoding video data, the apparatus comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determine whether to use a multi-hypothesis prediction (MHP) mode having a dual prediction mode as the base mode to predict the current block; in response to determining that an MHP mode having a dual prediction mode as the base mode should be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block based on the dual prediction mode; generate a second prediction block based on the additional inter-frame prediction mode; generate a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

[0253] Clause 78: A device pursuant to Clause 77, wherein, in order to determine whether to use MHP to predict the current block, the one or more processors are configured to determine whether to use MHP to predict the current block when a first weight is not equal to 4 and a second weight is not equal to 4.

[0254] Clause 79: A device pursuant to Clause 77, wherein, in order to determine whether to use MHP to predict the current block, the one or more processors are configured to determine whether to use MHP to predict the current block when a first weight is not equal to a second weight.

[0255] Clause 80: An apparatus pursuant to any one of Clauses 77 to 79, wherein the one or more processors are further configured to: decode a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction mode; decode a second motion vector using an AMVP mode for dual prediction mode; and decode a third motion vector for an additional inter-frame prediction mode, wherein, in order to generate a first prediction block, the one or more processors are configured to: generate a first intermediate prediction block using the first motion vector; generate a second intermediate prediction block using the second motion vector; apply a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block; apply a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block; and combine the first weighted intermediate prediction block with the second weighted intermediate prediction block to form a first prediction block, and wherein, in order to generate a second prediction block, the one or more processors are configured to generate a second prediction block using the third motion vector.

[0256] Clause 81: The apparatus according to Clause 80, wherein the one or more processors are further configured to: decode data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and determine that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0257] Clause 82: An apparatus pursuant to any one of Clauses 77 to 81, wherein, in order to generate a final prediction block, the one or more processors are configured to: determine a third weight and a fourth weight of the MHP mode; apply the third weight to a first prediction block to form a first weighted prediction block; apply the fourth weight to a second prediction block to form a second weighted prediction block; and combine the first weighted prediction block with the second weighted prediction block to form a final prediction block.

[0258] Clause 83: A device pursuant to Clause 82, wherein, in order to determine a third weight, the one or more processors are configured to: decode an index value; and determine in a mapping table the third weight to which the index value is mapped, and wherein, in order to determine a fourth weight, the one or more processors are configured to calculate the fourth weight as a minus the third weight.

[0259] Clause 84: A device pursuant to any one of Clauses 77 to 83, wherein the one or more processors are configured to encode the current block using the final predicted block before decoding the current block.

[0260] Clause 85: The device pursuant to any one of Clauses 77 to 84 also includes a display configured to display decoded video data.

[0261] Clause 86: Devices pursuant to any one of Clauses 77 to 85, wherein the device includes one or more of a camera, computer, mobile device, broadcast receiver device, or set-top box.

[0262] Clause 87: A computer-readable storage medium having instructions thereon, which, when executed, cause a processor to: determine that a first weight and a second weight have been assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; in response to determining that the first weight and the second weight have been assigned, determine whether to use a multi-hypothesis prediction (MHP) mode having a dual prediction mode as the basic mode to predict the current block; in response to determining that an MHP mode having a dual prediction mode as the basic mode should be used to predict the current block, determine an additional inter-frame prediction mode of the MHP mode; generate a first prediction block according to the dual prediction mode; generate a second prediction block according to the additional inter-frame prediction mode; generate a final prediction block for the current block according to the MHP mode using the first prediction block and the second prediction block; and decode the current block using the final prediction block.

[0263] Clause 88: A computer-readable storage medium pursuant to Clause 87, wherein causing a processor to determine whether to use MHP to predict instructions for the current block includes causing the processor to determine, when a first weight is not equal to 4 and a second weight is not equal to 4, to use MHP to predict instructions for the current block.

[0264] Clause 89: A computer-readable storage medium pursuant to Clause 87, wherein causing a processor to determine whether to use MHP to predict instructions for the current block includes causing the processor to determine, when a first weight is not equal to a second weight, to use MHP to predict instructions for the current block.

[0265] Clause 90: A computer-readable storage medium pursuant to any one of Clauses 87 to 89 further includes instructions that cause the processor to perform: decoding a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction mode; decoding a second motion vector using an AMVP mode for dual prediction mode; and decoding a third motion vector for an additional inter-frame prediction mode, wherein the instructions causing the processor to generate the first prediction block include instructions that cause the processor to: generate a first intermediate prediction block using the first motion vector; generate a second intermediate prediction block using the second motion vector; apply a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block; apply a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block; and combine the first weighted intermediate prediction block with the second weighted intermediate prediction block to form the first prediction block, and wherein the instructions causing the processor to generate the second prediction block include instructions that cause the processor to generate the second prediction block using the third motion vector.

[0266] Clause 91: The computer-readable storage medium pursuant to Clause 90 further includes instructions that cause the processor to perform: decoding data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and determining that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0267] Clause 92: A computer-readable storage medium pursuant to any one of Clauses 87 to 91, wherein the instructions for causing a processor to generate a final prediction block include instructions for causing the processor to perform: determining a third weight and a fourth weight of an MHP mode; applying the third weight to a first prediction block to form a first weighted prediction block; applying the fourth weight to a second prediction block to form a second weighted prediction block; and combining the first weighted prediction block with the second weighted prediction block to form a final prediction block.

[0268] Clause 93: A computer-readable storage medium pursuant to Clause 92, wherein the instructions for causing a processor to determine a third weight include instructions for causing the processor to perform: decoding an index value; and determining, in a mapping table, a third weight to which the index value is mapped, and wherein the instructions for causing a processor to determine a fourth weight include instructions for causing the processor to calculate the fourth weight as a subtraction of the third weight.

[0269] Clause 94: A computer-readable storage medium pursuant to any one of Clauses 87 to 93 may also include instructions that cause the processor to encode the current block using the final predicted block before decoding the current block.

[0270] Clause 95: An apparatus for decoding video data, the apparatus comprising: means for determining a first weight and a second weight assigned to a current block of video data for inter-frame prediction decoding using a dual prediction mode, wherein the first weight is different from the second weight; means for determining, in response to determining that the first weight and the second weight are assigned, whether to use a multi-hypothesis prediction (MHP) mode having the dual prediction mode as a base mode to predict the current block; means for determining an additional inter-frame prediction mode of the MHP mode in response to determining that the MHP mode having the dual prediction mode as a base mode is to be used to predict the current block; means for generating a first prediction block based on the dual prediction mode; means for generating a second prediction block based on the additional inter-frame prediction mode; means for generating a final prediction block for the current block based on the MHP mode using the first prediction block and the second prediction block; and means for decoding the current block using the final prediction block.

[0271] Clause 96: The device pursuant to Clause 95, wherein the component for determining whether to use MHP to predict the current block includes a component for determining whether to use MHP to predict the current block when the first weight is not equal to 4 and the second weight is not equal to 4.

[0272] Clause 97: The device pursuant to Clause 95, wherein the component for determining whether to use MHP to predict the current block includes a component for determining whether to use MHP to predict the current block when the first weight is not equal to the second weight.

[0273] Clause 98: The apparatus pursuant to any one of Clauses 95 to 97 further includes: means for decoding a first motion vector using an Advanced Motion Vector Prediction (AMVP) mode for dual prediction mode; means for decoding a second motion vector using an AMVP mode for dual prediction mode; and means for decoding a third motion vector for an additional inter-frame prediction mode, wherein the means for generating the first prediction block includes: means for generating a first intermediate prediction block using the first motion vector; means for generating a second intermediate prediction block using the second motion vector; means for applying a first weight to the first intermediate prediction block to form a first weighted intermediate prediction block; means for applying a second weight to the second intermediate prediction block to form a second weighted intermediate prediction block; and means for combining the first weighted intermediate prediction block and the second weighted intermediate prediction block to form the first prediction block, and wherein the means for generating the second prediction block includes means for generating the second prediction block using the third motion vector.

[0274] Clause 99: The apparatus pursuant to Clause 98 further includes: means for decoding data representing the motion vector difference (MVD) resolution of the first motion vector and the second motion vector; and means for determining that the third motion vector has MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

[0275] Clause 100: An apparatus pursuant to any one of Clauses 95 to 99, wherein the components for generating a final prediction block include: components for determining a third weight and a fourth weight of an MHP mode; components for applying the third weight to a first prediction block to form a first weighted prediction block; components for applying the fourth weight to a second prediction block to form a second weighted prediction block; and components for combining the first weighted prediction block and the second weighted prediction block to form a final prediction block.

[0276] Clause 101: The device according to Clause 100, wherein the components for determining the third weight include: components for decoding the index value; and components for determining the third weight to which the index value is mapped in the mapping table, and wherein the components for determining the fourth weight include components for calculating the fourth weight as a subtraction of the third weight.

[0277] Clause 102: The apparatus pursuant to any one of Clauses 95 to 101 further includes a component for encoding the current block using the final predicted block before decoding the current block.

[0278] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different order, and may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for the practice of the technique). Furthermore, in some examples, actions or events may be performed concurrently, for example, through multithreading, interrupt handling, or multiple processors, rather than sequentially.

[0279] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that includes any medium that facilitates, for example, the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the techniques described in this disclosure. Computer program products may include computer-readable media.

[0280] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Similarly, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are sent from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. Disks and optical discs as used herein include CDs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0281] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the terms "processor" and "processing circuitry" can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Similarly, the technique can be fully implemented in one or more circuit or logic elements.

[0282] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented through different hardware units. Rather, as described above, various units can be combined in a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above combined with suitable software and / or firmware.

[0283] Various examples have been described. These and other examples are within the scope of the appended claims.

Claims

1. A method for decoding video data, the method comprising: Receive bitstreams including encoded video data; Based on the encoded video data, a first weight and a second weight are assigned to the current block of video data that uses a dual prediction mode for inter-frame prediction decoding, wherein the first weight is different from the second weight. In response to determining that the first weight and the second weight have been specified, and based on the fact that the first weight is different from the second weight, it is determined that a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode should be used to predict the current block. In response to determining that the MHP mode with the dual prediction mode as the basic mode should be used to predict the current block, an additional inter-frame prediction mode of the MHP mode is determined. A first prediction block is generated based on the dual prediction mode; A second prediction block is generated based on the additional inter-frame prediction mode; Using the first and second prediction blocks, a final prediction block is generated for the current block according to the MHP pattern; and The current block is decoded using the final predicted block.

2. The method according to claim 1, wherein, Determining whether to use MHP to predict the current block includes determining that the first weight is not equal to 4 and the second weight is not equal to 4.

3. The method according to claim 1, further comprising: The first motion vector is decoded using the Advanced Motion Vector Prediction (AMVP) mode for the dual prediction mode; The second motion vector is decoded using the AMVP mode for the dual prediction mode; and Decode the third motion vector used for the additional inter-frame prediction mode. Generating the first prediction block includes: The first intermediate prediction block is generated using the first motion vector. The second motion vector is used to generate a second intermediate prediction block. The first weight is applied to the first intermediate prediction block to form a first weighted intermediate prediction block. The second weight is applied to the second intermediate prediction block to form a second weighted intermediate prediction block, and The first weighted intermediate prediction block is combined with the second weighted intermediate prediction block to form the first prediction block, and Generating the second prediction block includes generating the second prediction block using the third motion vector.

4. The method according to claim 3, further comprising: Decode the data representing the motion vector difference (MVD) resolution between the first motion vector and the second motion vector; and The third motion vector is determined to have the MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

5. The method according to claim 1, wherein, Generating the final prediction block includes: Determine the third and fourth weights of the MHP pattern; The third weight is applied to the first prediction block to form a first weighted prediction block; The fourth weight is applied to the second prediction block to form a second weighted prediction block; and The first weighted prediction block is combined with the second weighted prediction block to form the final prediction block.

6. The method according to claim 5, in, Determining the third weight includes: Decode the index value; and In the mapping table, determine the third weight to which the index value is mapped, and Determining the fourth weight includes calculating the fourth weight as a minus the third weight.

7. The method of claim 1, further comprising encoding the current block using the final predicted block before decoding the current block.

8. An apparatus for decoding video data, the apparatus comprising: A memory configured to store video data; as well as One or more processors, said one or more processors being implemented in a circuit and configured to: Receive bitstreams including encoded video data; Based on the encoded video data, a first weight and a second weight are assigned to the current block of video data that uses a dual prediction mode for inter-frame prediction decoding, wherein the first weight is different from the second weight. In response to determining that the first weight and the second weight have been specified, and based on the fact that the first weight is different from the second weight, it is determined that a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode should be used to predict the current block. In response to determining that the MHP mode with the dual prediction mode as the basic mode should be used to predict the current block, an additional inter-frame prediction mode of the MHP mode is determined. A first prediction block is generated based on the dual prediction mode; A second prediction block is generated based on the additional inter-frame prediction mode; Using the first and second prediction blocks, a final prediction block is generated for the current block according to the MHP pattern; and The current block is decoded using the final predicted block.

9. The device according to claim 8, wherein, In order to determine whether to use MHP to predict the current block, the one or more processors are configured to determine that the first weight is not equal to 4 and the second weight is not equal to 4.

10. The device according to claim 8, wherein, The one or more processors are further configured to: The first motion vector is decoded using the Advanced Motion Vector Prediction (AMVP) mode for the dual prediction mode; The second motion vector is decoded using the AMVP mode for the dual prediction mode; and Decode the third motion vector used for the additional inter-frame prediction mode. In order to generate the first prediction block, the one or more processors are configured to: The first intermediate prediction block is generated using the first motion vector. The second motion vector is used to generate a second intermediate prediction block. The first weight is applied to the first intermediate prediction block to form a first weighted intermediate prediction block. The second weight is applied to the second intermediate prediction block to form a second weighted intermediate prediction block, and The first weighted intermediate prediction block is combined with the second weighted intermediate prediction block to form the first prediction block, and In order to generate the second prediction block, the one or more processors are configured to generate the second prediction block using the third motion vector.

11. The device according to claim 10, wherein, The one or more processors are further configured to: Decode the data representing the motion vector difference (MVD) resolution between the first motion vector and the second motion vector; and The third motion vector is determined to have the MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

12. The device according to claim 8, wherein, In order to generate the final prediction block, the one or more processors are configured to: Determine the third and fourth weights of the MHP pattern; The third weight is applied to the first prediction block to form a first weighted prediction block; The fourth weight is applied to the second prediction block to form a second weighted prediction block; and The first weighted prediction block is combined with the second weighted prediction block to form the final prediction block.

13. The device according to claim 12, in, To determine the third weight, the one or more processors are configured to: Decode the index value; and In the mapping table, determine the third weight to which the index value is mapped, and In order to determine the fourth weight, the one or more processors are configured to calculate the fourth weight as a minus the third weight.

14. The device according to claim 8, wherein, The one or more processors are configured to encode the current block using the final predicted block before decoding the current block.

15. The device of claim 8, further comprising a display configured to display decoded video data.

16. The device according to claim 8, wherein, The device includes one or more of a camera, computer, mobile device, broadcast receiver device, or set-top box.

17. A computer-readable storage medium having instructions stored thereon, the instructions causing a processor, when executed, to: Receive bitstreams including encoded video data; Based on the encoded video data, a first weight and a second weight are assigned to the current block of video data that uses a dual prediction mode for inter-frame prediction decoding, wherein the first weight is different from the second weight. In response to determining that the first weight and the second weight have been specified, it is determined whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the current block; In response to determining that the MHP mode with the dual prediction mode as the basic mode should be used to predict the current block, an additional inter-frame prediction mode of the MHP mode is determined. A first prediction block is generated based on the dual prediction mode; A second prediction block is generated based on the additional inter-frame prediction mode; Using the first and second prediction blocks, a final prediction block is generated for the current block according to the MHP pattern; and The current block is decoded using the final predicted block.

18. The computer-readable storage medium according to claim 17, wherein, The instruction that causes the processor to determine that MHP should be used to predict the current block includes an instruction that causes the processor to determine that the first weight is not equal to 4 and the second weight is not equal to 4.

19. The computer-readable storage medium of claim 17, further comprising instructions that cause the processor to perform the following: The first motion vector is decoded using the Advanced Motion Vector Prediction (AMVP) mode for the dual prediction mode; The second motion vector is decoded using the AMVP mode for the dual prediction mode; and Decode the third motion vector used for the additional inter-frame prediction mode. in, The instructions that cause the processor to generate the first prediction block include instructions that cause the processor to execute the following: The first intermediate prediction block is generated using the first motion vector. The second motion vector is used to generate a second intermediate prediction block. The first weight is applied to the first intermediate prediction block to form a first weighted intermediate prediction block. The second weight is applied to the second intermediate prediction block to form a second weighted intermediate prediction block, and The first weighted intermediate prediction block is combined with the second weighted intermediate prediction block to form the first prediction block, and The instruction that causes the processor to generate the second prediction block includes an instruction that causes the processor to generate the second prediction block using the third motion vector.

20. The computer-readable storage medium of claim 19, further comprising instructions that cause the processor to perform the following: Decode the data representing the motion vector difference (MVD) resolution between the first motion vector and the second motion vector; and The third motion vector is determined to have the MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

21. The computer-readable storage medium according to claim 17, wherein, The instructions that cause the processor to generate the final prediction block include instructions that cause the processor to execute the following: Determine the third and fourth weights of the MHP pattern; The third weight is applied to the first prediction block to form a first weighted prediction block; The fourth weight is applied to the second prediction block to form a second weighted prediction block; and The first weighted prediction block is combined with the second weighted prediction block to form the final prediction block.

22. The computer-readable storage medium according to claim 21, in, The instructions that cause the processor to determine the third weight include instructions that cause the processor to perform the following: Decode the index value; and In the mapping table, determine the third weight to which the index value is mapped, and The instruction that causes the processor to determine the fourth weight includes an instruction that causes the processor to calculate the fourth weight as a subtraction of the third weight.

23. The computer-readable storage medium of claim 17, further comprising instructions that cause the processor to encode the current block using the final predicted block before decoding the current block.

24. An apparatus for decoding video data, the apparatus comprising: A component used to receive bitstreams including encoded video data; For video data based on the encoded video data, a component is assigned a first weight and a second weight to determine the current block of video data for inter-frame predictive decoding using a dual prediction mode, wherein the first weight is different from the second weight; In response to determining that the first weight and the second weight have been specified, it determines whether to use a multi-hypothesis prediction (MHP) mode with the dual prediction mode as the base mode to predict the components of the current block. A component for determining an additional inter-frame prediction mode of the MHP mode in response to determining that the current block should be predicted using the MHP mode having the dual prediction mode as the basic mode. Components used to generate a first prediction block according to the dual prediction mode; Components for generating a second prediction block based on the additional inter-frame prediction mode; A component for generating a final prediction block for the current block according to the MHP mode using the first prediction block and the second prediction block; as well as A component used to decode the current block using the final predicted block.

25. The device according to claim 24, wherein, The component for determining whether to use MHP to predict the current block includes a component for determining that the first weight is not equal to 4 and the second weight is not equal to 4.

26. The apparatus of claim 24, further comprising: Components for decoding the first motion vector using the Advanced Motion Vector Prediction (AMVP) mode for the dual prediction mode; Components for decoding the second motion vector using the AMVP mode for the dual prediction mode; as well as The component used to decode the third motion vector for the additional inter-frame prediction mode. The component used to generate the first prediction block includes: A component for generating a first intermediate prediction block using the first motion vector. A component for generating a first intermediate prediction block using the first motion vector. A component for applying the first weight to the first intermediate prediction block to form a first weighted intermediate prediction block. Components for applying the second weight to the second intermediate prediction block to form a second weighted intermediate prediction block, and A component for combining the first weighted intermediate prediction block and the second weighted intermediate prediction block to form the first prediction block, and The component for generating the second prediction block includes a component for generating the second prediction block using the third motion vector.

27. The apparatus of claim 26, further comprising: A component for decoding data representing the motion vector difference (MVD) resolution between the first motion vector and the second motion vector; as well as A component used to determine that the third motion vector has the MVD resolution without decoding additional data representing the MVD resolution of the third motion vector.

28. The device according to claim 24, wherein, The components used to generate the final prediction block include: Components used to determine the third and fourth weights of the MHP mode; Components for applying the third weight to the first prediction block to form a first weighted prediction block; Components for applying the fourth weight to the second prediction block to form a second weighted prediction block; and Components for combining the first weighted prediction block with the second weighted prediction block to form the final prediction block.

29. The device according to claim 28, in, The component used to determine the third weight includes: The component used for decoding index values; and The component used to determine the third weight to which the index value is mapped in the mapping table, and The component for determining the fourth weight includes a component for calculating the fourth weight as a subtraction of the third weight.

30. The apparatus of claim 24, further comprising means for encoding the current block using the final predicted block before decoding the current block.

Images