Unified processing and syntax for general prediction in video encoding / decoding

Abstract

At least a method and apparatus for efficiently encoding or decoding a video is provided. For example, a plurality of different motion prediction modes are obtained for a current block. The current block is encoded or decoded based on a combination of a plurality of different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes or an inter-frame prediction mode and an intra-frame prediction mode. Indication of triangle prediction and multi-hypothesis prediction is allowed in one or more lists of possible motion vector candidates, such as, for example, indicated in an advanced motion vector prediction (AMVP) mode.

Description

Technical Field

[0001] At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding, and more particularly to a method or apparatus for efficiently providing video compression and / or decompression utilizing a unified process for general prediction and a unified syntax and / or one or more related processes. Background Art

[0002] In order to achieve high compression efficiency, image and video coding schemes usually adopt prediction including motion vector prediction and transform to exploit spatial and temporal redundancy in video content. Generally, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image and the predicted image (usually expressed as prediction error or prediction residual) is transformed, quantized and entropy encoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy coding, quantization, transformation and prediction.

[0003] Recent additions to video compression technology include versions of various industry standards, reference software and / or documents, such as the Joint Exploration Model (JEM) and later VTM (Versatile Video Coding (VVC) Test Model) being developed by the JVET (Joint Video Exploration Team) group with the goal of further improving the existing HEVC (High Efficiency Video Coding) standard. Summary of the invention

[0004] The shortcomings and disadvantages of the prior art are addressed and resolved by one or more aspects described in this application.

[0005] According to one embodiment, a method for video encoding is provided, comprising: obtaining a plurality of different motion prediction modes for a current block; encoding the current block based on a combination of the plurality of different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights comprises at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and allowing indication of triangle prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0006] According to another embodiment, a method for video decoding is provided, comprising: obtaining multiple different motion prediction modes for a current block; decoding the current block based on a combination of the multiple different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and allowing indication of triangular prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0007] According to another embodiment, a device for video encoding is provided, comprising: a component for obtaining multiple different motion prediction modes for a current block; a component for encoding the current block based on a combination of the multiple different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and allowing indication of triangular prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0008] According to another embodiment, a device for video decoding is provided, comprising: a component for obtaining multiple different motion prediction modes for a current block; a component for decoding the current block based on a combination of the multiple different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and allowing indication of triangular prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0009] According to another embodiment, a device for video encoding is presented, comprising: one or more processors, wherein the one or more processors are configured to: obtain multiple different motion prediction modes for a current block; encode the current block based on a combination of the multiple different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and allow indication of triangular prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0010] According to another embodiment, a device for video decoding is presented, comprising: one or more processors, wherein the one or more processors are configured to: obtain multiple different motion prediction modes for a current block; decode the current block based on a combination of the multiple different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and allow indication of triangular prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0011] According to another embodiment, a signal including encoded video is formed by: obtaining multiple different motion prediction modes for a current block; encoding the current block based on a combination of the multiple different motion prediction modes and corresponding weights, wherein the combination with the corresponding weights includes at least two inter-frame prediction modes, or an inter-frame prediction mode and an intra-frame prediction mode; and forming a bitstream including the encoded current block; and allowing indication of triangular prediction and multi-hypothesis prediction in one or more lists of possible motion vector candidates.

[0012] According to another embodiment, obtaining a plurality of different motion prediction modes is based on processing a unified syntax including the plurality of different motion prediction modes and a first flag indicating whether the current block is bi-directionally predicted.

[0013] According to another embodiment, the current block is partitioned into two triangular sub-blocks, each with its own motion vector predictor.

[0014] According to another embodiment, one or more motion vector predictors for a diagonal boundary between two triangular sub-blocks are obtained based on a blend of the motion vector predictors of the two triangular sub-blocks.

[0015] According to another embodiment, when the combination with the corresponding weight includes at least an inter prediction mode and an intra prediction mode, the corresponding weight depends on the intra direction of the intra prediction mode. According to another embodiment, the corresponding weight also depends on the distance between the current sample of the current block and the intra reference sample of the intra prediction mode.

[0016] According to another embodiment, the corresponding weights decrease as the distance increases. According to another embodiment, the corresponding weights are applied via one or more masks. According to another embodiment, one or more masks are applied using a shifting process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A block diagram illustrating an embodiment of a video encoder.

[0018] Figure 2 A block diagram illustrating an embodiment of a video decoder.

[0019] Figure 3 Block diagram illustrating a system in which aspects of the present embodiments may be implemented.

[0020] Figure 4 An overview of the inter-frame prediction process is illustrated.

[0021] Figure 5 The example splits the CU into two triangular prediction units.

[0022] Figure 6 The weighting process on the diagonal edge between two triangular prediction units is illustrated.

[0023] Figure 7 Multi-hypothesis prediction in the case of combining inter mode and intra mode is exemplified.

[0024] Figure 8 Illustrate intra prediction regions and their corresponding weights in multiple hypothesis mode.

[0025] Fig. 9 The bidirectional prediction process in HEVC is illustrated.

[0026] Fig.10 Example optical flow trajectory.

[0027] Fig.11A Instantiate the compatibility tree.

[0028] Fig. 11B Instantiate the current syntax tree (ununified).

[0029] Fig.12 An exemplary process for a unified syntax for Advanced Motion Vector Prediction (AMVP), merge and skip modes is illustrated.

[0030] Fig.13 An exemplary process for unified syntax and tool extensions for the AMVP pattern is illustrated.

[0031] Fig.14 An exemplary process for uni-directional predictive multiple-hypothesis syntax is illustrated.

[0032] Fig.15 Another exemplary process for uni-directional predictive multiple-hypothesis syntax is illustrated.

[0033] Fig.16 An exemplary process for unifying merge grammars is illustrated.

[0034] Fig.17 Another exemplary process for unifying merge grammars is illustrated.

[0035] Fig.18 An exemplary process for unifying skip syntax is illustrated.

[0036] Fig.19 A general process of using a combination of masks is illustrated.

[0037] Fig. 20 A general process using shifting is illustrated. DETAILED DESCRIPTION

[0038] For clarity of description, the following description will describe various aspects with reference to embodiments involving video compression techniques such as, for example, HEVC, JEM and / or H.266. However, the described aspects are applicable to other video processing techniques and standards.

[0039] This application describes various aspects including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described and, at least to illustrate the various features, are often described in a manner that may sound restrictive. However, this is for the purpose of clarity of description and does not limit the application or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. In addition, these aspects can also be combined and interchanged with aspects described in previous applications.

[0040] The various examples of the embodiments described and contemplated in this document may be implemented in many different forms. Figure 1 , 2 and 3 provide some examples of embodiments described below, but other embodiments are contemplated, and Figure 1 , 2 The discussion of 3 does not limit the breadth of implementations. At least one of these aspects generally relates to video encoding and decoding, and at least another aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects may be implemented as methods, apparatus, a computer-readable storage medium having stored thereon instructions for encoding / decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods. .

[0041] In this application, the terms "reconstructed" and "decoded" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture", and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, and "decoded" is used on the decoder side.

[0042] The terms HDR (high dynamic range) and SDR (standard dynamic range) are used in this disclosure. These terms generally convey a specific value of dynamic range to one of ordinary skill in the art. However, additional embodiments are also intended in which references to HDR are understood to mean "higher dynamic range" and references to SDR are understood to mean "lower dynamic range". Such additional embodiments are not bound by any specific value of dynamic range, which may often be associated with the terms "high dynamic range" and "standard dynamic range".

[0043] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0044] Various methods and other aspects described in this application can be used to modify modules, e.g. Figure 1 and Figure 2 Intra-frame prediction, entropy encoding and / or decoding modules (160, 360, 145, 330) of the video encoder 100 and decoder 200 are shown. Moreover, the present aspects are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0045] Various numerical values ​​are used in this application, such as with respect to block size. The specific values ​​are for example purposes, and the described aspects are not limited to these specific values.

[0046] Figure 1 An exemplary encoder 100 is illustrated. Variations of the encoder 100 may be envisioned, but for clarity, the encoder 100 is described below without describing all contemplated variations. Before being encoded, the video sequence may undergo pre-encoding processing (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input image components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0047] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (102) and processed, for example in units of CUs. Each unit is encoded, for example, using intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use for the coding unit, and indicates the intra / inter decision, for example, by a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0048] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients are entropy encoded (145) along with motion vectors and other syntax elements to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process.

[0049] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture to, for example, perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0050] Figure 2 2 is a block diagram of an exemplary video decoder 200. In the decoder 200, a bitstream is decoded by decoder elements, as described below. The video decoder 200 generally performs the same Figure 1The encoder 100 also generally performs video decoding as part of the encoded video data. Specifically, the input to the decoder includes a video bitstream that may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other encoding information. Picture partition information indicates how the picture is partitioned. Thus, the decoder can divide (235) the picture according to the decoded picture partition information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored at a reference picture buffer (280).

[0051] The decoded picture may further undergo post-decoding processing (285), such as inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping to perform the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0052] Figure 3 A block diagram illustrating an example of a system in which various aspects and embodiments are implemented. System 1000 may be embodied as a device including various components described below, and is configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances and servers. The elements of system 1000 are embodied in a single integrated circuit, multiple ICs and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or by dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described in this document.

[0053] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 1010 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 1000 may also include at least one memory 1020 (e.g., a volatile memory device, a non-volatile memory device). The system 1000 may include a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, magnetic disk drive, and / or optical disk drive. As non-limiting examples, the storage device 1040 may include an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network accessible storage device.

[0054] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a (multiple) module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, as known to those skilled in the art, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000, or may be incorporated into the processor 1010 as a combination of hardware and software.

[0055] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during the execution of the processes described herein. Such stored items include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operational logic.

[0056] In some embodiments, memory internal to the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or storage device 1040, such as dynamic volatile memory and / or non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store, for example, an operating system for the television. In at least one embodiment, fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by JVET (Joint Video Experts Group)).

[0057] Input to the elements of system 1000 may be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion for receiving an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal (or a collection of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Fig.10 Other examples not shown include composite video.

[0058] In various embodiments, the input device of block 1130 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) again band-limiting the signal to a narrower band of frequencies to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet. The RF portion of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (e.g., cable) media, and by filtering, down-convert and filter again to the desired frequency band to perform frequency selection. Various embodiments rearrange the order of the above-mentioned (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF part includes an antenna. In addition, USB and / or HDMI terminals can include corresponding interface processors for connecting system 1000 to other electronic devices across USB and / or HDMI. It is to be understood that the various aspects of input processing, such as Reed-Solomon error correction, can be, for example, implemented as needed in a separate input processing IC or in processor 1010. Similarly, the various aspects of USB or HDMI interface processing can be implemented as needed in a separate interface IC or in processor 1010. The demodulated, error corrected and demultiplexed stream is provided to various processing elements including, for example, a processor 1010 and an encoder / decoder 1030 operating in conjunction with memory and storage elements to process the data stream for presentation on an output device.

[0059] The various elements of system 1000 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data may be transferred between them using a suitable connection arrangement, such as an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and printed circuit boards.

[0060] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0061] In various embodiments, a wireless network Wi-Fi network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers) is used to transmit data streams to or otherwise provide to the system 1000. The Wi-Fi signals of these embodiments are received on a communication channel 1060 and a communication interface 1050 adapted to Wi-Fi communication. The communication channel 1060 of these embodiments is usually connected to an access point or router, which provides access to an external network including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to the system 1000, which is connected to the HDMI of the input block 1130 to pass data. Other embodiments also use the RF connection of the input block 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming mode. In addition, various embodiments use a wireless network other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0062] The system 1000 can provide output signals to various output devices including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 1100 can also be integrated with other components (e.g., as in a smart phone), or independent (e.g., an external monitor for a laptop computer). Other peripheral devices 1120 include one or more independent digital video discs (or digital versatile discs) (DVR, for both), disc players, stereo systems, and lighting systems in various examples of embodiments. Various embodiments use one or more peripheral devices 1120 that provide functions based on the output of the system 1000. For example, a disc player performs the function of playing the output of the system 1000.

[0063] In various embodiments, control signals are transmitted between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling communications such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be communicatively coupled to the system 1000 via dedicated connections through various interfaces 1070, 1080, and 1090. Alternatively, the output device can be connected to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and speaker 1110 can be integrated with other components of the system 1000 into a single unit in an electronic device (such as, for example, a television). In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0064] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection (including, for example, an HDMI port, a USB port, or a COMP output).

[0065] The embodiments may be implemented by computer software implemented by the processor 1010 or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. As a non-limiting example, the memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed memory, and removable memory. As a non-limiting example, the processor 1010 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0066] Figure 1 The encoder 100, Figure 2 The decoder 200 and Figure 3 The system 1000 is suitable for implementing at least one of the embodiments described below.

[0067] Video encoding and decoding can use different tools for inter-frame prediction. Figure 4 An overview of the inter prediction process is shown with tools associated with each stage of the pipeline.

[0068] The following paragraphs describe the inter prediction tools and some of their other aspects.

[0069] Extensions of HEVC's bidirectional prediction were proposed in the combination stage of the pipeline and were later called combination tools: unidirectional prediction, similar to HEVC bidirectional prediction; bidirectional prediction: similar to HEVC bidirectional prediction, triangular prediction (TRIANGLE), multiple hypothesis (MH) and generalized bidirectional prediction (GBi).

[0070] TRIANGLE is a prediction composed of 2 predictions, but it is not a simple mix, each prediction will cover part of the PU. The boundary between the two predictions is mixed. Figure 5 As shown, a coding unit (CU) is partitioned into two triangular prediction units (PUs) in a diagonal or anti-diagonal direction. Each triangular prediction unit in a CU is inter-predicted using its own motion vector and a reference frame index derived from a merge candidate list.

[0071] An adaptive weighting process is applied to the prediction samples of the diagonal or anti-diagonal edges between two triangle prediction units to derive the final prediction sample value for the entire CU, such as Figure 5 shown.

[0072] Triangle prediction unit mode is only applied to CUs in skip or merge mode. When triangle prediction unit mode is applied to a CU, an index (triangle_merge_idx) indicating the direction to split the CU into two triangle prediction units is signaled, as well as the motion vectors of the two triangle prediction units. Partitions can be generalized to other partition shapes.

[0073] Multi-hypothesis mode combines regular inter prediction and intra prediction to form a block prediction. The blend between the 2 predictions depends on the intra direction. More precisely, multi-hypothesis combines the inter prediction performed in merge mode (signaling the merge index to derive motion information for motion compensated prediction) with intra prediction mode or another inter mode (e.g. uni-prediction advanced motion vector prediction (AMVP), skip or merge). The final prediction is a weighted average of the merge index prediction and the prediction generated by the intra prediction mode, where different weights are applied depending on the intra direction and the distance between the current sample and the intra reference sample. The intra prediction mode is signaled (it can be a subset of the complete set of prediction modes (e.g. 4)).

[0074] like Figure 8For example, the current block is divided into 4 equal-area regions. As the region moves away from the intra reference sample, the weight gradually decreases. Each weight set, denoted as (w_intrai, w_interi), where i ranges from 1 to 4 and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5) and (w_intra4, w_inter4) = (2, 6) will be applied to the corresponding region, as shown in Figure 2. Figure 7 The example for intra vertical prediction is shown in . When DC or planar mode is selected, or the CU width is higher or less than 4, the weights are applied to all samples. For intra prediction in a multi-hypothesis CU, the chroma components use direct mode (same intra direction as luma).

[0075] The Generic Bidirectional Prediction (GBi) mode predicts a block by combining two motion compensated prediction blocks using block-level adaptive weights from a predefined set of candidate weights. This way, the prediction process for GBi can reuse the existing weighted prediction logic and does not introduce additional decoding burden. In HEVC, the averaging of the two unidirectional prediction signals for bidirectional prediction is done with higher precision than the input or internal bit depth, such as Fig. 9 shown.

[0076] The bidirectional prediction formula is shown in Equation 1, where the offset and shift are used to normalize the final predictor to the input bit depth.

[0077] P bidir =(P L0 +P L1 +offset)>>shift

[0078] Equation 1

[0079] The HEVC interpolation filter allows certain implementation optimizations since no rounding is performed at the intermediate stages.

[0080] Multiple weights can be used to average 2 unidirectional predictions to obtain bidirectional prediction. Typically, the weights used are {-1 / 4, 5 / 4}, {3 / 8, 5 / 8} or {1 / 2, 1 / 2} (such as HEVC), and the bidirectional prediction formula is modified as shown in Equation 2. Only one weight is used for the entire block.

[0081] P bidir =((1-w1)*P L0 +w1*P L1 +offset)>>shiftEquation 2

[0082] Another tool is Bidirectional Optical Flow (BIO), a sample-based motion refinement performed on top of block-based motion compensation for bidirectional prediction. Sample-level motion refinement uses no signaling. In the case of bidirectional prediction, the goal of BIO is to refine the motion for each sample based on the assumption of a linear displacement between the two reference pictures, as in the case of Hermite based optical flow interpolation. Fig.10 shown.

[0083] However, these inter prediction tools are currently exclusive and not always available in all possible coding modes (SKIP / Merge / AMVP), as shown in Figure 11. This makes the coding less efficient and the syntax more complex.

[0084] Fig. 11B The current parse tree (not unified) of the currently used syntax is shown. The combined modes are in the bitstream and are encoded sequentially in an exclusive manner. Furthermore, not all tools are available in all modes (skip, merge, AMVP). For example, "multiple hypothesis" or "triangle" are not available in AMVP. It has to be noted that some tools like MMVD (Merging of Motion Vector Differences) are independent of the inter prediction mode, i.e. it can be applied to merge, affine merge or triangle partitioning. That is why the mmvd syntax is not shown in the following proposed syntax.

[0085] describe

[0086] At least one embodiment is directed to improving bi-directional prediction syntax and processing in inter-coded blocks of video compression systems.

[0087] In at least one embodiment, a new syntax is proposed to support available modes related to general prediction to provide:

[0088] - a unified process for reconstructing common predictions,

[0089] - proposes a unified syntax, and extensions for all modes (skip, merge and AMVP),

[0090] - Improved Context-Adaptive Binary Arithmetic Coding (CABAC) contexts due to unified syntax.

[0091] Such an embodiment would provide advantages such as:

[0092] - Unified syntax to allow more efficient use of CABAC contexts,

[0093] -Generalization of the AMVP model,

[0094] - compatibility with the use of unified / consistent candidate derivation,

[0095] - More clear standardization of grammar.

[0096] Unified Grammar

[0097] According to a first aspect, the present disclosure is directed to a unified grammar for general prediction processing. The grammar consists of Figure 3 The system 1000 uses, more specifically, a Figure 1 The elements 145, 170, 175 of the encoder 100 and the Figure 2 Elements 230 and 275 of the video decoder 200 are used.

[0098] In at least one embodiment, a process or apparatus is proposed for providing or utilizing a unified syntax for AMVP, merge, and skip modes.

[0099] In at least one embodiment, a process or apparatus for providing or utilizing a unified grammar for an AMVP mode is provided, Fig.12 The left part corresponds to, allowing triangles and multiple assumptions in AMVP mode:

[0100] -After reading the direction flag of the AMVP CU, if the direction is 3 (corresponding to a bidirectional prediction block), then:

[0101] - Read the first flag, weighted_bipred, to signal whether the block will use weighted bi-directional prediction or regular (HEVC) bi-directional prediction. If this flag is false, a uniform mask with a weight of 0.5 is generated.

[0102] - If the flag weighted_bipred is 1, the flag partitioned_bipred is read. If this flag is false, it corresponds to GBi mode and weight_index is decoded (corresponding to the index in the weight pair table).

[0103] - If the flag partitioned_bipred is 1, then read the flag has_intra_prediction. If this flag is false, it corresponds to triangle mode, and read partition_type. For example, for a mode with only triangle partitioning, 0 and 1 correspond to Figure 5 For patterns with more partitions, the partition index is decoded.

[0104] - If the flag has_intra_prediction is 1, the intra_dir value is decoded, corresponding to the intra direction of the intra part of the block. In current VTM3, only 4 directions are possible: DC, planar, horizontal and vertical, and are encoded using the 3MPM prediction system.

[0105] Fig.13 This syntax is also illustrated.

[0106] In at least one variant embodiment, the multiple hypothesis mode consists of unidirectional prediction and intra prediction. Fig.14 Given.

[0107] When the block is bi-directionally predicted (ie dir==3), the previously described syntax applies. In case is_intra_prediction==1 for bi-directional prediction, only one motion data (ref_idx, mvd and mvp idx) is decoded.

[0108] In at least one variant embodiment, the multiple hypothesis mode may consist of bi-prediction and intra prediction. The bi-prediction mode of the inter part of the prediction is signaled using the flag bi_inter. The decoding process is as follows Fig.15 Shown.

[0109] In at least one embodiment, a process or apparatus for providing or utilizing a unified grammar for merging modes is provided, such as Fig.16 shown.

[0110] In such an embodiment:

[0111] - First, decode the merge flag, and if true, decode the flag partitioned_bipred.

[0112] - If the flag partitioned_bipred is false, normal processing continues: the decoding of the affine flag and merge_idx or affine_merge_idx is the same as before.

[0113] - If flag partitioned_bipred is true, decode flag has_intra_prediction.

[0114] - If the flag has_intra_prediction is false, decode partition_type and then decode triangle_predictors_idx (one index per predictor) following the normal process for triangle mode decoding.

[0115] - If the flag has_intra_prediction is true, decode the intra_dir value and decode the merge_idx of the inter prediction quantity.

[0116] Note that the GBi mode is derived from the used predictor. The prediction mode for the inter part of the multiple hypothesis mode is derived from the decoded merge candidates.

[0117] In a variant embodiment, the multiple hypotheses assumed are unidirectionally predicted. In this case, the merge list derivation is different from the regular merge list derivation because there is no bidirectional prediction mode. Therefore, the merge index for this mode is determined by Fig.17 This is signaled by a different value called inter_merge_idx in .

[0118] The derivation process of the merge list in the case where the multiple hypotheses only contain unidirectional prediction candidates can be shared with the triangle merge list derivation of the first list.

[0119] In at least one embodiment, a unified syntax for skip mode is proposed. In multiple hypothesis mode, since intra mode is used, the residual is (almost) always coded, so this mode is removed from skip.

[0120] Fig.18 Instantiate a parse tree for merge mode. In this mode:

[0121] - First, decode the skip flag, if true, then decode the flag partitioned_bipred.

[0122] - If the flag partitioned_bipred is false, normal processing continues: the decoding of the affine flag and merge_idx or affine_merge_idx is the same as before.

[0123] - If the flag partitioned_bipred is true, the mode is assumed to be triangle and triangle_predictors_idx (one index per predictor) is decoded following the normal process for triangle mode decoding.

[0124] Unified processing

[0125] According to another aspect, the present disclosure is directed to Fig.19 The unified processing of the general prediction is shown in the example. Figure 3 In the system 1000 and more precisely in accordance with Figure 1 The elements 145, 170, 175 of the encoder 100 and in accordance with Figure 2 The components 230 and 275 of the video decoder 200 are implemented.

[0126] In at least one embodiment:

[0127] - For each predicted 0 and 1, create a mask. By construction, the sum of the mask values ​​at each position in the mask is 1.0.

[0128] -The predicted sample is then multiplied by the mask.

[0129] - Add the predictions multiplied by 2.

[0130] The mask is created as follows:

[0131] - For bi-prediction mode, the mask is uniform and has a value of 0.5.

[0132] - For GBi mode, the mask is uniform and the value of each mask is the value of the GBi weight, e.g., prediction 0 has a uniform mask with value 1 / 4, while prediction 1 has a uniform mask with value 3 / 4.

[0133] - For the multiple hypothesis mode, the mask is non-uniform and is processed by the multiple hypothesis as Figure 8 Describes the creation of a mask that depends on intra prediction.

[0134] - For triangle patterns, the mask is non-uniform and is composed of triangle patterns such as Figure 6 Describes the creation of a triangle. The mask depends on the triangle partitioning mode.

[0135] In a variant embodiment, using Fig. 20 The combining process is accomplished by the shift operation described in . We assume that the bit depth of the encoding used for the mask is N (typically N=3+1 bit for the sign).

[0136] exist Fig. 20 In , the >> symbol indicates a right arithmetic shift by N, the mask U is a uniform mask containing the values ​​(1<<(N-1)), and "X" is done using the internal bit depth precision M (M>N).

[0137] For a mask bit depth of N, the above mask values ​​are:

[0138] - For bi-prediction, the mask is uniformly equal to (1<<(N-1)).

[0139] -For GBi mode, the mask is uniform and the value of each mask is the value of the GBi weight, e.g., prediction 0 has a uniform mask value of 2 (corresponding to 1 / 4 if N=3), while prediction 1 has a uniform mask value of 6 (corresponding to 3 / 4 if N=3).

[0140] - For the multiple hypothesis mode, the mask is non-uniform and is processed by multiple hypothesis as Figure 8 Description Create.

[0141] - For triangle mode, the mask is non-uniform and is passed through the triangle mode as Figure 6 Description Create. The values ​​in the mask are derived from the weights.

[0142] grammar

[0143] The following table describes the proposed syntax elements.

[0144]

[0145]

[0146]

[0147]

[0148]

[0149]

[0150] The semantics of some elements of the grammar are described as follows:

[0151] - Partitioned_bipred[x0][y0] equal to 1 specifies that for the current coding unit, the weighting of the two predictions P0 and P1 is done by using at least two different weights for all samples. If this flag is equal to 0, only one weight is used.

[0152] - Partition_type[x0][y0] specifies the partition used to split the current coding unit into 2 prediction units of blocks, where x0, y0 specify the position (x0, y0) of the top left luma sample of the considered prediction block relative to the top left luma sample of the picture.

[0153] -Weighted_bipred[x0][y0] equal to 1 specifies that a unique weight other than 1 / 2 is used for the weighting of the 2 predictions. If the flag equals 0, a 1 / 2 weight is used.

[0154] - Has_intra_prediction[x0][y0] specifies that the current inter prediction for the current coding unit will be weighted using intra prediction.

[0155] CABAC context coding

[0156] By unifying the common prediction, more efficient CABAC contexts can be used for the following flags:

[0157] -partitioned_bipred: Neighboring blocks are either triangle or multi-hypothesis mode contexts.

[0158] - has_intra_prediction: Neighboring blocks are the context of the triangle.

[0159] -weighted_bipred: Neighboring blocks are contexts of GBi, triangle or multi-hypothesis mode.

[0160] Various implementations involve decoding. "Decoding" as used in this application may encompass, for example, all or part of the processing performed on a received coded sequence in order to produce a final output suitable for display. In various embodiments, such processing includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processing also or alternatively includes processing performed by a decoder of the various implementations described in this application, such as, Figure 4 and 8 Processing presented in .

[0161] As another example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or broadly to a more extensive decoding process will be clear based on the context of the particular description and is believed to be well understood by those skilled in the art.

[0162] Various implementations involve encoding. In a manner similar to the discussion above regarding "decoding," "encoding" as used in this application may encompass, for example, all or part of the processing performed on an input video sequence in order to produce an encoded bitstream. In various embodiments, such processing includes one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processing also or alternatively includes processing performed by an encoder of the various implementations described in this application, such as, Figure 4 and Figure 8 processing.

[0163] As another example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of entropy encoding and differential encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or broadly to a more extensive encoding process will be clear based on the context of the particular description and is believed to be well understood by those skilled in the art.

[0164] Note that the syntax elements used herein are descriptive terms. As such, they do not exclude the use of other syntax element names.

[0165] When the figures are presented as flow charts, it should be understood that they also provide block diagrams of corresponding devices. Similarly, when the figures are presented as block diagrams, it should be understood that they also provide flow charts of corresponding methods / processes.

[0166] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices, such as computers, cellular phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate information communication between end users.

[0167] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places throughout this document are not necessarily all referring to the same embodiment.

[0168] Additionally, this application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0169] Additionally, this application may refer to "accessing" various information. Accessing information may include, for example, receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0170] Additionally, the present application may refer to "receiving" various information. As with "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves, in one way or another, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information during an operation.

[0171] It is to be recognized that the use of " / ", "and / or", and "at least one of" below, for example, in the case of "A / B", "A and / or B", and "at least one of A and B", is intended to include selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such language is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, the language may be expanded for many of the items listed.

[0172] In addition, the word "signal" as used herein especially refers to indicating something to the corresponding decoder. For example, in some embodiments, the encoder sends a specific mode with a signal. In this way, in an embodiment, the same parameters are used on both the encoder side and the decoder side. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It is to be recognized that signaling can be completed in a variety of ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to send information to the corresponding decoder with a signal. Although the foregoing relates to the verb form of the word "signal", the word "signal" can also be used as a noun in this article.

[0173] It will be apparent to those skilled in the art that implementations may generate various signals formatted to carry information that may, for example, be stored or transmitted. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0174] We describe a number of embodiments. Features of these embodiments may be provided individually or in any combination. In addition, embodiments may include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types:

[0175] The unified processing of general inter prediction uses a unified syntax for video encoding and video decoding, which includes different modes that are combined together according to corresponding weights, the combination being a combination of two inter prediction modes or an inter prediction mode and an intra mode.

[0176] The video encoding method uses a unified process to process general inter-frame prediction using a unified syntax for video encoding and video decoding. The video encoding and video decoding include different modes combined together according to corresponding weights, and the combination is a combination of two inter-frame prediction modes or an inter-frame prediction mode and an intra-frame mode.

[0177] The video decoding method uses a unified process to process general inter-frame prediction using a unified syntax for video encoding and video decoding. The video encoding and video decoding include different modes combined together according to corresponding weights, and the combination is a combination of two inter-frame prediction modes or an inter-frame prediction mode and an intra-frame mode.

[0178] A video encoder, transcoder or other electronic device performs video encoding using a unified process that uses a unified syntax for video encoding and video decoding to handle general inter-frame prediction, where the video encoding and video decoding include different modes combined together according to corresponding weights, where the combination is a combination of two inter-frame prediction modes or an inter-frame prediction mode and an intra-frame mode.

[0179] A television, set-top box, mobile phone, tablet or other electronic device performs video decoding using a unified process that handles common inter-frame prediction using a unified syntax for video encoding and video decoding. Video encoding and video decoding include different modes that are combined together according to corresponding weights, where the combination is a combination of two inter-frame prediction modes or an inter-frame prediction mode and an intra-frame mode.

Claims

1. A method for video decoding, comprising: Obtaining video data representing an image block to be decoded; decoding the image block based on the obtained video data; Wherein, in the case where bidirectional prediction is applied to the image block predicted using a motion vector, decoding comprises decoding a first flag indicating whether weighted bidirectional prediction is applied to the image block, and in the case where the first flag indicates the application of weighted bidirectional prediction, decoding a second flag indicating whether weighted bidirectional prediction is performed by using at least two different weights within the image block; otherwise, conventional bidirectional prediction is applied to the image block; Wherein, in a case where the second flag indicates that weighted bidirectional prediction is performed by using at least two different weights within the image block, decoding includes decoding a third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, decoding an index in a weight pair table, and applying generalized bidirectional prediction to the image block based on the weight pair indexed by the index; and Wherein, when the third flag indicates that inter-frame prediction is combined with intra-frame prediction, decoding includes decoding the intra-frame direction and applying multi-hypothesis prediction to the image block based on the intra-frame direction; otherwise, decoding the partition type and applying triangle prediction to the image block based on the partition type.

2. The method according to claim 1, wherein: In the case where the merge mode is applied to the image block, the decoding includes decoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, decoding the third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, decoding the fourth flag indicating whether affine merging is applied, and decoding the merge or affine merge index based on the value of the fourth flag, and wherein, in the case where the third flag indicates that inter-frame prediction is combined with intra-frame prediction, the merge index of the intra-frame direction and the inter-frame prediction amount is decoded, and multi-hypothesis prediction is applied to the image block based on the intra-frame direction and the merge index, otherwise, the partition type and the triangle prediction amount index are decoded, and triangle prediction is applied to the image block based on the partition type and the triangle prediction amount index.

3. The method according to claim 1, wherein: In the case where the skip mode is applied to the image block, decoding includes decoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, decoding a triangle prediction index and applying triangle prediction to the image block based on the triangle prediction index, otherwise, decoding a fourth flag indicating whether to apply affine merging and decoding a merging or affine merging index based on the value of the fourth flag.

4. The method according to any one of claims 1 to 3, wherein: In context adaptive binary arithmetic coding, the context of the first flag is based on neighboring blocks decoded using generalized bidirectional prediction, triangle prediction, or multi-hypothesis prediction, the context of the second flag is based on neighboring blocks decoded using triangle prediction or multi-hypothesis prediction, and the context of the third flag is based on neighboring blocks decoded using triangle prediction.

5. A device for video decoding, comprising one or more processors, wherein: The one or more processors are configured to implement: Obtaining video data representing an image block to be decoded; Decoding the image block according to the obtained video data, Wherein, in the case where bidirectional prediction is applied to the image block predicted using a motion vector, decoding comprises decoding a first flag indicating whether weighted bidirectional prediction is applied to the image block, and in the case where the first flag indicates the application of weighted bidirectional prediction, decoding a second flag indicating whether weighted bidirectional prediction is performed by using at least two different weights within the image block; otherwise, conventional bidirectional prediction is applied to the image block; Wherein, in a case where the second flag indicates that weighted bidirectional prediction is performed by using at least two different weights within the image block, decoding includes decoding a third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, decoding an index in a weight pair table, and applying generalized bidirectional prediction to the image block based on the weight pair indexed by the index; and Wherein, when the third flag indicates that inter-frame prediction is combined with intra-frame prediction, decoding includes decoding the intra-frame direction and applying multi-hypothesis prediction to the image block based on the intra-frame direction; otherwise, decoding the partition type and applying triangle prediction to the image block based on the partition type.

6. The device according to claim 5, wherein: In the case where the merge mode is applied to the image block, the decoding includes decoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, decoding the third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, decoding the fourth flag indicating whether affine merging is applied, and decoding the merge or affine merge index based on the value of the fourth flag, and wherein, in the case where the third flag indicates that inter-frame prediction is combined with intra-frame prediction, the merge index of the intra-frame direction and the inter-frame prediction amount is decoded, and multi-hypothesis prediction is applied to the image block based on the intra-frame direction and the merge index, otherwise, the partition type and the triangle prediction amount index are decoded, and triangle prediction is applied to the image block based on the partition type and the triangle prediction amount index.

7. The device according to claim 5, wherein: In the case where the skip mode is applied to the image block, decoding includes decoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, decoding a triangle prediction index and applying triangle prediction to the image block based on the triangle prediction index, otherwise, decoding a fourth flag indicating whether to apply affine merging and decoding a merging or affine merging index based on the value of the fourth flag.

8. The device according to any one of claims 5 to 7, wherein: In context adaptive binary arithmetic coding, the context of the first flag is based on neighboring blocks decoded using generalized bidirectional prediction, triangle prediction, or multi-hypothesis prediction, the context of the second flag is based on neighboring blocks decoded using triangle prediction or multi-hypothesis prediction, and the context of the third flag is based on neighboring blocks decoded using triangle prediction.

9. A method for video encoding, comprising: Obtain the image block to be encoded; encoding the image block into encoded data; Wherein, in the case where bidirectional prediction is applied to the image block predicted using a motion vector, encoding comprises encoding a first flag indicating whether weighted bidirectional prediction is applied to the image block, and in the case where the first flag indicates the application of weighted bidirectional prediction, encoding a second flag indicating whether weighted bidirectional prediction is performed by using at least two different weights within the image block; otherwise, conventional bidirectional prediction is applied to the image block; Wherein, in a case where the second flag indicates that weighted bidirectional prediction is performed by using at least two different weights within the image block, encoding includes encoding a third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, encoding an index in a weight pair table, and applying generalized bidirectional prediction to the image block based on the weight pair indexed by the index; and In which, when the third flag indicates that inter-frame prediction is combined with intra-frame prediction, the encoding includes encoding the intra-frame direction and applying multi-hypothesis prediction to the image block based on the intra-frame direction; otherwise, the partition type is encoded and triangular prediction is applied to the image block based on the partition type.

10. The method according to claim 9, wherein: In the case where the merge mode is applied to the image block, the encoding includes encoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, encoding the third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, encoding the fourth flag indicating whether affine merging is applied, and encoding the merge or affine merge index based on the value of the fourth flag, and wherein, in the case where the third flag indicates that inter-frame prediction is combined with intra-frame prediction, the merge index of the intra-frame direction and the inter-frame prediction amount is encoded, and multi-hypothesis prediction is applied to the image block based on the intra-frame direction and the merge index, otherwise, the partition type and the triangle prediction amount index are encoded, and triangle prediction is applied to the image block based on the partition type and the triangle prediction amount index.

11. The method according to claim 9, wherein: In the case where the skip mode is applied to the image block, the encoding includes encoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, encoding a triangle prediction amount index and applying triangle prediction to the image block based on the triangle prediction amount index, otherwise, encoding a fourth flag indicating whether to apply affine merging and encoding a merging or affine merging index based on the value of the fourth flag.

12. The method according to any one of claims 9 to 11, wherein: In context adaptive binary arithmetic coding, the context of the first flag is based on neighboring blocks encoded using generalized bidirectional prediction, triangle prediction, or multi-hypothesis prediction, the context of the second flag is based on neighboring blocks encoded using triangle prediction or multi-hypothesis prediction, and the context of the third flag is based on neighboring blocks encoded using triangle prediction.

13. An apparatus for video encoding, comprising one or more processors, wherein: One or more processors are configured to implement: Obtain the image block to be encoded; encoding the image block into encoded data; Wherein, in the case where bidirectional prediction is applied to the image block predicted using a motion vector, encoding comprises encoding a first flag indicating whether weighted bidirectional prediction is applied to the image block, and in the case where the first flag indicates the application of weighted bidirectional prediction, encoding a second flag indicating whether weighted bidirectional prediction is performed by using at least two different weights within the image block; otherwise, conventional bidirectional prediction is applied to the image block; Wherein, in a case where the second flag indicates that weighted bidirectional prediction is performed by using at least two different weights within the image block, encoding includes encoding a third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, encoding an index in a weight pair table, and applying generalized bidirectional prediction to the image block based on the weight pair indexed by the index; and In which, when the third flag indicates that inter-frame prediction is combined with intra-frame prediction, the encoding includes encoding the intra-frame direction and applying multi-hypothesis prediction to the image block based on the intra-frame direction; otherwise, the partition type is encoded and triangular prediction is applied to the image block based on the partition type.

14. The device according to claim 13, wherein: In the case where the merge mode is applied to the image block, the encoding includes encoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, encoding the third flag indicating whether inter-frame prediction is combined with intra-frame prediction, otherwise, encoding the fourth flag indicating whether affine merging is applied, and encoding the merge or affine merge index based on the value of the fourth flag, and wherein, in the case where the third flag indicates that inter-frame prediction is combined with intra-frame prediction, the merge index of the intra-frame direction and the inter-frame prediction amount is encoded, and multi-hypothesis prediction is applied to the image block based on the intra-frame direction and the merge index, otherwise, the partition type and the triangle prediction amount index are encoded, and triangle prediction is applied to the image block based on the partition type and the triangle prediction amount index.

15. The device according to claim 13, wherein: In the case where the skip mode is applied to the image block, the encoding includes encoding the second flag, and in the case where the second flag indicates weighted bidirectional prediction by using at least two different weights within the image block, encoding a triangle prediction amount index and applying triangle prediction to the image block based on the triangle prediction amount index, otherwise, encoding a fourth flag indicating whether to apply affine merging and encoding a merging or affine merging index based on the value of the fourth flag.

16. The device according to any one of claims 13 to 15, wherein: In context adaptive binary arithmetic coding, the context of the first flag is based on neighboring blocks encoded using generalized bidirectional prediction, triangle prediction, or multi-hypothesis prediction, the context of the second flag is based on neighboring blocks encoded using triangle prediction or multi-hypothesis prediction, and the context of the third flag is based on neighboring blocks encoded using triangle prediction.

Images