Method, apparatus and computer program for signaling for multiple inter-prediction modes

Abstract

This disclosure relates generally to video coding, and more particularly to methods and systems for signaling composite inter-prediction modes. For example, to design an exemplary scheme for signaling an optical flow refinement flag and composite inter-prediction mode of a video block, the correlation between the use of optical flow refinement and various composite inter-prediction modes is explored. Specifically, the signaling of the inter-prediction mode of a block may depend on whether optical flow refinement is applied to the block.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on and claims the benefit of U.S. Provisional Patent Application No. 63 / 442,721, filed February 1, 2023, entitled "Method and Apparatus for Improved Signaling for Combined Inter-Prediction Modes," which is incorporated herein by reference in its entirety, and which claims the benefit of U.S. Non-Provisional Patent Application No. 18 / 463,508, filed September 8, 2023, entitled "Method and Apparatus for Signaling for Combined Inter-Prediction Modes."

[0002] FIELD This disclosure relates generally to video coding, and more particularly to methods and systems for signaling composite inter-prediction modes. [Background technology]

[0003] Uncompressed digital video can contain a series of pictures and may have specific bit rate requirements for transmission bandwidth in storage, data processing, and streaming applications. One goal of video coding and decoding can be to reduce redundancy in the uncompressed input video signal due to various compression techniques. Summary of the Invention [Means for solving the problem]

[0004] This disclosure relates generally to video coding, and more particularly to methods and systems for signaling composite inter-prediction modes. For example, to design an exemplary scheme for signaling an optical flow refinement flag and composite inter-prediction mode of a video block, the correlation between the use of optical flow refinement and various composite inter-prediction modes is explored. Specifically, the signaling of the inter-prediction mode of a block may depend on whether optical flow refinement is applied to the block.

[0005] In an example implementation, a method for decoding a video block in a video bitstream is disclosed. The method includes receiving a first syntax element signaled in the video bitstream, the first syntax element indicating whether optical flow refinement is applied to the video block, determining whether the optical flow refinement is applied to the video block based on a value of the first syntax element, receiving a second syntax element from the video bitstream after receiving the first syntax element, the second syntax element indicating a composite inter-prediction mode of the video block depending on whether the optical flow refinement is applied, determining the composite inter-prediction mode of the video block based on the value of the second syntax element, and predicting the video block based on the determined composite inter-prediction mode.

[0006] In the above example implementation, the value of the second syntax element is determined by decoding the second syntax element using a coding context that depends on whether optical flow refinement is applied.

[0007] In any one of the above implementations, if optical flow refinement is applied, a first context is used to decode a second syntax element indicating a composite inter-prediction mode of the video block, and if optical flow refinement is not applied, a second context different from the first context is used to decode the second syntax element indicating a composite inter-prediction mode of the video block.

[0008] In any one of the above implementations, when optical flow refinement is applied to a video block, only composite inter prediction modes that require signaling of one motion vector difference or no motion vector difference are allowed.

[0009] In any one of the above implementations, when optical flow refinement is applied to a video block, the syntax value space of the composite inter-prediction modes of the video block includes a subset of the NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV modes.

[0010] In any one of the above implementations, when optical flow refinement is applied to a video block, only hybrid inter prediction modes with one or no motion vector difference are allowed.

[0011] In any one of the above implementations, if optical flow refinement is applied to a video block, the syntax value space of the composite inter-prediction modes of the video block includes a subset of the NEAR_NEARMV, NEAR_NEWMV, and NEW_NEARMV modes.

[0012] In any one of the above implementation forms, with respect to the joint motion vector difference composite inter prediction mode, if optical flow refinement is not applied to the video block, two joint motion vector difference composite inter prediction modes are allowed, and if optical flow refinement is applied to the video block, only one of the two joint motion vector difference composite inter prediction modes is allowed.

[0013] In any one of the above implementations, the two joint motion vector difference composite inter prediction modes include JOINT_NEWMV mode and JOINT_AMVDNEWMV mode, and when optical flow refinement is applied to the video block, only one of the two allowed joint motion vector difference composite inter prediction modes is JOINT_AMVDNEWMV mode.

[0014] In any one of the above implementations, determining a composite inter prediction mode for the video block may include determining a mapping between possible values ​​of the second syntax element and multiple composite inter prediction modes based on whether optical flow refinement is applied to the video block, and determining a composite inter prediction mode for the video block based on the value of the second syntax element and the mapping.

[0015] In any one of the above implementations, the mapping between the possible values ​​of the second syntax element and the multiple composite inter-prediction modes is different when optical flow refinement is applied than when optical flow refinement is disabled.

[0016] In another example implementation, a method for decoding a video block in a video bitstream is disclosed. The method may include receiving a first syntax element signaled in a video bitstream, the first syntax element indicating a composite inter-prediction mode for the video block among a plurality of composite inter-prediction modes, determining the composite inter-prediction mode for the video block based on a value of the first syntax element, and determining whether a second syntax element for the video block is included in the video bitstream or receiving the second syntax element from the video bitstream in a manner dependent on the composite inter-prediction mode indicated by the first syntax element, the second syntax element indicating whether optical flow refinement is applied to the video block.

[0017] In the above example implementation, extracting the second syntax element includes decoding the second syntax element using a coding context that depends on the composite inter-prediction mode indicated by the first syntax element.

[0018] In any one of the above implementations, the second syntax element for the video block is included in the video bitstream only for a subset of the multiple composite inter-prediction modes.

[0019] In any one of the above implementations, the second syntax element for the video block is included in the video bitstream only if the composite inter-prediction mode of the video block requires at most one signaled motion vector difference.

[0020] In any one of the above implementations, the subset of the plurality of composite inter-prediction modes includes NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV modes.

[0021] In any one of the above implementations, the second syntax element for the video block is included in the video bitstream only if the composite inter prediction mode of the video block does not involve different motion vectors, or uses adaptive motion vector resolution, or has motion vector precision that is coarser than a predetermined precision threshold.

[0022] In any one of the above implementations, the second syntax element for the video block is included in the video bitstream only if the composite inter prediction mode of the video block involves at most one motion vector difference.

[0023] In any one of the above implementations, the subset of the plurality of composite inter-prediction modes includes NEAR_NEARMV, NEAR_NEWMV, and NEW_NEARMV modes.

[0024] Aspects of the present disclosure also provide an electronic device or apparatus that includes a circuit or processor configured to perform any of the above method implementations.

[0025] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by an electronic device, cause the electronic device to perform any one of the above method implementations.

[0026] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a schematic diagram illustrating a simplified block diagram of a communication system (100) according to an exemplary embodiment. [Figure 2] FIG. 2 is a schematic diagram illustrating a simplified block diagram of a communication system (200) according to an exemplary embodiment. [Figure 3] FIG. 2 is a schematic diagram illustrating a simplified block diagram of a video decoder according to an example embodiment. [Figure 4] FIG. 1 is a schematic diagram illustrating a simplified block diagram of a video encoder according to an example embodiment. [Figure 5] FIG. 2 is a block diagram of a video encoder according to another example embodiment. [Figure 6] FIG. 4 is a block diagram of a video decoder according to another example embodiment. [Figure 7] FIG. 1 illustrates a coding block partitioning scheme according to an exemplary embodiment of the present disclosure. [Figure 8] FIG. 10 illustrates another scheme for coding block partitioning according to an exemplary embodiment of the present disclosure. [Figure 9] FIG. 10 illustrates another scheme for coding block partitioning according to an exemplary embodiment of the present disclosure. [Figure 10] FIG. 10 illustrates an example logic flow for a method for signaling optical flow refinement flags and combined inter-prediction modes. [Figure 11] FIG. 1 is a schematic diagram of a computer system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0028] Throughout this specification and claims, terms may have subtle meanings that are suggested or implied in context beyond their explicitly stated meaning. The phrases "in one embodiment / implementation" or "in some embodiments / implementations" used in this disclosure do not necessarily refer to the same embodiment / implementation, and the phrases "in another embodiment / implementation" or "in other embodiments" used in this disclosure do not necessarily refer to different embodiments. For example, claimed subject matter is intended to include all or some combinations of the example embodiments / implementations.

[0029] Generally, terminology may be understood, at least in part, from its usage in context. For example, terms such as "and," "or," or "and / or" as used herein may have various context-dependent meanings. Typically, "or" when used to relate a list such as A, B, or C is intended to refer to A, B, and C, which are used herein in an inclusive sense, as well as A, B, or C, which are used herein in an exclusive sense. Furthermore, as used herein, the terms "one or more," "at least one," "a," "an," or "the" may be used in a singular or plural sense, depending at least in part on the context. In addition, the terms "based on" or "determined by" may be understood not to necessarily convey an exclusive set of factors, but instead may allow for the existence of additional factors not necessarily explicitly stated, again depending at least in part on the context.

[0030] FIG. 1 illustrates a simplified block diagram of a communication system (100) according to one embodiment of the present disclosure. The communication system (100) includes multiple terminal devices, e.g., 110, 120, 130, and 140, that can communicate with each other, e.g., via a network (150). In the example of FIG. 1, a first pair of terminal devices (110) and (120) may perform unidirectional data transmission. For example, the terminal device (110) may encode video data in the form of one or more coded bitstreams (e.g., streams of video pictures captured by the terminal device (110)) for transmission over the network (150). The terminal device (120) may receive the coded video data from the network (150), decode the coded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. The unidirectional data transmission may be implemented, for example, in media serving applications.

[0031] In another example, a second pair of terminal devices (130) and (140) may perform bidirectional transmission of coded video data, such as during video conferencing applications. For bidirectional transmission of data, in one example, each of the terminal devices (130) and (140) may code video data (e.g., of a stream of video pictures captured by the terminal device) for transmission to another of the terminal devices (130) and (140) for reconstructing and displaying the video pictures, and may receive coded video data from another of the terminal devices (130) and (140).

[0032] In the example of FIG. 1 , the terminal devices may be implemented as servers, personal computers, and smartphones, although the applicability of the underlying principles of the present disclosure may not be so limited. Embodiments of the present disclosure may be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing equipment, and the like. Network (150) represents any number or type of network that conveys coded video data between terminal devices, including, for example, wired (cabled) and / or wireless communication networks. Communication network (150) may exchange data over circuit-switched, packet-switched, and / or other types of channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0033] 2 illustrates the arrangement of a video encoder and a video decoder in a video streaming environment as an example of an application for the disclosed subject matter. The disclosed subject matter may be equally applicable to other video applications, including, for example, video conferencing, digital television broadcasting, gaming, virtual reality, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0034] As shown in Figure 2, a video streaming system may include a video source (201), e.g., a video capture subsystem (213), which may include a digital camera, for creating an uncompressed video picture or stream of pictures (202). In one example, the video picture stream (202) includes samples recorded by the digital camera of the video source 201. The video picture stream (202), depicted as a bold line to emphasize its high data volume compared to the encoded video data (204) (or coded video bitstream), may be processed by an electronic device (220) including a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination thereof, and may enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (204) (or encoded video bitstream (204)), depicted as a thin line to emphasize its small amount of data compared to the stream of uncompressed video pictures (202), can be stored on the streaming server (205) for future use or directly on a downstream video device (not shown). One or more streaming client subsystems, such as the client subsystems (206) and (208) of FIG. 2, can access the streaming server (205) to retrieve copies (207) and (209) of the encoded video data (204). The client subsystem (206) may include a video decoder (210), for example, within the electronic device (230). The video decoder (210) decodes the incoming copy of the encoded video data (207) and creates an outgoing stream of video pictures (211) that can be uncompressed and rendered on a display (212) (e.g., a display screen) or other rendering device (not shown).

[0035] 3 shows a block diagram of a video decoder (310) of an electronic device (330) according to any of the following embodiments of the present disclosure. The electronic device (330) may include a receiver (331) (e.g., receiving circuitry). The video decoder 310 may be used in place of the video decoder (210) of the example of FIG. 2.

[0036] As shown in FIG. 3, the receiver (331) may receive one or more coded video sequences from the channel (301). To address network jitter, a buffer memory (315) may be disposed between the receiver (331) and an entropy decoder / parser (320) (hereinafter, "parser (320)"). The parser (320) may be configured to reconstruct symbols (321) from the coded video sequence. Categories of these symbols include information used to manage the operation of the video decoder (310) and, potentially, information for controlling a rendering device such as the display (312) (e.g., a display screen). The parser (320) may parse / entropy decode the coded video sequence. The parser (320) may extract from the coded video sequence a set of subgroup parameters for at least one subgroup of pixels within the video decoder. Subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (320) may also extract information from the coded video sequence, such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc. Reconstruction of symbols (321) may involve multiple different processing or functional units. The units involved and how they participate may be controlled by subgroup control information parsed from the coded video sequence by the parser (320).

[0037] The first unit may include a scalar / inverse transform unit (351), which may receive quantized transform coefficients and control information from the parser (320), including information indicating which type of inverse transform to use, block size, quantization coefficients / parameters, quantization scaling matrices, and states as symbols (321). The scalar / inverse transform unit (351) may output blocks containing sample values, which may be input to an aggregator (355).

[0038] In some cases, the output samples of the scaler / inverse transform (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) may generate blocks of the same size and shape as the block being reconstructed using neighboring block information that has already been reconstructed and stored in a current picture buffer (358). The current picture buffer (358), for example, buffers partially reconstructed and / or fully reconstructed current pictures. In some implementations, the aggregator (355) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0039] In other cases, the output samples of the scaler / inverse transform unit (351) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (353) can access a reference picture memory (357) to fetch samples used for inter-picture prediction. After motion-compensating the fetched reference samples according to the symbols (321) associated with the block, these samples can be added by an aggregator (355) to the output of the scaler / inverse transform unit (351) (the output of unit 351 may be referred to as residual samples or a residual signal) to generate output sample information.

[0040] The output samples of the aggregator (355) may undergo various loop filtering techniques in a loop filter unit (356), including several types of loop filters. The output of the loop filter unit (356) may be a sample stream that can be output to the rendering device (312) as well as stored in a reference picture memory (357) for use in future inter-picture prediction.

[0041] 4 shows a block diagram of a video encoder (403) according to an exemplary embodiment of the present disclosure. The video encoder (403) may be included in an electronic device (420). The electronic device (420) may further include a transmitter (440) (e.g., a transmitting circuit). The video encoder (403) may be used in place of the video encoder (403) in the example of FIG.

[0042] The video encoder (403) may receive video samples from a video source (401). According to some exemplary embodiments, the video encoder (403) may code and compress pictures of a source video sequence into a coded video sequence (443) in real time or under other time constraints required by the application. Implementing an appropriate coding rate constitutes one function of the controller (450). In some embodiments, the controller (450) may be functionally coupled to and control other functional units, as described below. Parameters set by the controller (450) may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc.

[0043] In some exemplary embodiments, the video encoder (403) may be configured to operate in a coding loop. The coding loop may include a source coder (430) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs symbols to create sample data in a manner similar to that created by a (remote) decoder, even if the embedded decoder 433 processes the video bitstream coded by the source coder 430 without entropy coding (since any compression between the symbols and the coded video bitstream in entropy coding may be lossless in the video compression techniques considered in the disclosed subject matter). It should be noted at this point that any decoder technology, with the exception of parsing / entropy decoding, which may only exist in the decoder, may necessarily also need to exist in substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter may focus on decoder operation, which is similar to the decoding portion of the encoder. Therefore, a description of the encoder technology may be omitted, since it is the inverse of the decoder technology described generically. Only in certain areas or aspects will a more detailed description of the encoder be provided below.

[0044] During operation in some example implementations, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as "reference pictures."

[0045] The local video decoder (433) may decode coded video data for pictures that may be designated as reference pictures. The local video decoder (433) may replicate the decoding process that may be performed by a video decoder on the reference pictures and store the reconstructed reference pictures in a reference picture cache (434). In this way, the video encoder (403) can locally store copies of reconstructed reference pictures that have common content with reconstructed reference pictures obtained by a far-end (remote) video decoder (free of transmission errors).

[0046] The predictor (435) may perform the predictive search for the coding engine (432). That is, for a new picture to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that can serve as suitable prediction references for the new picture.

[0047] The controller (450) can manage the coding operations of the source coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0048] The output of all of the aforementioned functional units may be entropy coded in an entropy coder (445). The transmitter (440) may buffer the coded video sequence created by the entropy coder (445) and prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the coded video data. The transmitter (440) may merge the coded video data from the video coder (403) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0049] The controller (450) may manage the operation of the video encoder (403). During coding, the controller (450) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned as one of the following picture types: intra picture (I picture), predicted picture (P picture), or bidirectionally predicted picture (B picture), among multi-predictive pictures. A source picture may generally be spatially subdivided into multiple sample coding blocks, as described in further detail below.

[0050] 5 shows a diagram of a video encoder (503) according to another exemplary embodiment of this disclosure. The video encoder (503) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a current video picture in a sequence of video pictures and encode the processed block into a coded picture that is part of a coded video sequence. The exemplary video encoder (503) may be used in place of the example video encoder (403) of FIG. 4.

[0051] For example, the video encoder (503) receives a matrix of sample values ​​for a processing block. The video encoder (503) then determines whether the processing block is best coded using intra-mode, inter-mode, or bi-predictive mode, for example, using rate-distortion optimization (RDO).

[0052] In the example of Figure 5, the video encoder (503) includes an inter-encoder (530), an intra-encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), an overall controller (521), and an entropy encoder (525), coupled to each other as shown in the exemplary configuration of Figure 5.

[0053] The inter-encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in previous and subsequent pictures in display order), generate inter-prediction information (e.g., a description of redundant information due to inter-encoding techniques, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique.

[0054] The intra-encoder (522) is configured to receive samples of a current block (e.g., a processing block), compare the block with blocks already coded in the same picture, generate quantized coefficients after transformation, and possibly generate intra-prediction information (e.g., intra-prediction direction information according to one or more intra-encoding techniques).

[0055] The overall controller (521) may be configured to determine general control data and control other components of the video encoder (503) based on the general control data, for example, to determine a prediction mode for a block and provide a control signal to the switch (526) based on the prediction mode.

[0056] The residual calculator (523) may be configured to calculate the difference (residual data) between the received block and a prediction result of the block selected from the intra-encoder (522) or inter-encoder (530). The residual encoder (524) may be configured to encode the residual data to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various exemplary embodiments, the video encoder (503) also includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transform and generate decoded residual data. The entropy encoder (525) may be configured to format a bitstream to include the coded block and perform entropy coding.

[0057] 6 shows a diagram of an exemplary video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (610) may be used in place of the video decoder (410) of the example of FIG. 4.

[0058] In the example of FIG. 6, the video decoder (610) includes an entropy decoder (671), an inter-decoder (680), a residual decoder (673), a reconstruction module (674), and an intra-decoder (672), coupled together as shown in the exemplary configuration of FIG. 6.

[0059] The entropy decoder (671) may be configured to reconstruct, from a coded picture, certain symbols representing syntax elements that make up the coded picture. The inter decoder (680) may be configured to receive inter prediction information and generate inter prediction results based on the inter prediction information. The intra decoder (672) may be configured to receive intra prediction information and generate prediction results based on the intra prediction information. The residual decoder (673) may be configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The reconstruction module (674) may be configured to combine, in the spatial domain, the residual output by the residual decoder (673) with prediction results (output by the inter prediction module or the intra prediction module, as the case may be) to form reconstructed blocks that form part of the reconstructed picture as part of the reconstructed video.

[0060] It should be noted that the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using any suitable technology. In some embodiments, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more processors executing software instructions.

[0061] Focusing on block partitioning used in coding and decoding, a typical partitioning may start from a base block and may follow a predefined set of rules, a specific pattern, a partition tree, or some other partition structure or scheme. The partitioning may be hierarchical and recursive. After separating or dividing the base block according to any of the exemplary partitioning procedures described below or other procedures, or a combination thereof, a final set of partitions or coding blocks may be obtained. Each of these partitions may be at one of various partition levels in the partitioning hierarchy and may be partitions of various shapes. Each partition may be referred to as a coding block (CB). In various exemplary partitioning implementations described further below, each resulting CB may be a CB of any allowed size and partition level. Such partitions are referred to as coding blocks because they may form units for which some basic coding / decoding decisions may be made and for which coding / decoding parameters may be optimized, determined, and signaled in an encoded video bitstream. The highest or deepest level in the final partition represents the depth of the tree coding block partitioning structure. The coding block may be a luma coding block or a chroma coding block. The CB tree structure for each color may be referred to as a coding block tree (CBT). The coding blocks for all color channels may be collectively referred to as a coding unit (CU). The hierarchical structure for all color channels may be collectively referred to as a coding tree unit (CTU). The division pattern or structure of various color channels within a CTU may or may not be the same.

[0062] In some implementations, the partition tree schemes or structures used for the luma channel and the chroma channels may not need to be the same. In other words, the luma channel and the chroma channel may have separate coding tree structures or patterns. Furthermore, whether the luma channel and the chroma channel use the same or different coding partition tree structures, and the actual coding partition tree structure used, may depend on whether the slice being coded is a P, B, or I slice. For example, in an I slice, the chroma channel and the luma channel may have separate coding partition tree structures or coding partition tree structure modes, while in a P slice or a B slice, the luma channel and the chroma channel may share the same coding partition tree scheme. When separate coding partition tree structures or modes are applied, the luma channel may be partitioned into CBs by one coding partition tree structure, and the chroma channels may be partitioned into chroma CBs by another coding partition tree structure.

[0063] FIG. 7 illustrates ten exemplary predefined partitioning structures / patterns that enable recursive partitioning to form a partitioning tree. The root block can start from a predefined level (e.g., from the base block at the 128×128 level or the 64×64 level). The exemplary partitioning structure in FIG. 7 includes various 2:1 / 1:2 and 4:1 / 1:4 rectangular partitions. In some exemplary implementations, none of the rectangular partitions in FIG. 7 can be further subdivided. A coding tree depth may be further defined to indicate the partitioning depth from the root node or root block. For example, the coding tree depth for the root node or root block may be set to 0, and after the root block is further partitioned according to FIG. 7, the coding tree depth increases by 1. In some implementations, only 710 square partitions may enable recursive partitioning to the next level of the partitioning tree following the pattern in FIG. 7.

[0064] In some other exemplary implementations for coding block partitioning, a quadtree structure may be used. Such quadtree partitioning may be applied hierarchically and recursively to any square partition. Whether a base block or an intermediate block or partition is further quadtree partitioned can be adapted to various local characteristics of the base block or intermediate block / partition.

[0065] In yet some other examples, a ternary division scheme may be used to divide the base block or any intermediate blocks, as shown in FIG. 8. The ternary pattern may be implemented vertically, as shown at 802, or horizontally, as shown at 804. The exemplary division ratio in FIG. 8 is shown as 1:2:1, but other ratios may be predefined. In some implementations, two or more different ratios may be predefined. In some implementations, the width and height of the partitions of the exemplary ternary tree are always powers of two to avoid further transformations.

[0066] The above partitioning schemes may be combined in any manner at different partitioning levels. As an example, the above-described quadtree and bisection schemes may be combined to partition a base block into a quadtree-bisection (QTBT) structure. In such a scheme, the base block or intermediate blocks / partitions may be either quadtree-partitioned or bisectioned, if specified, according to a set of predefined conditions. A specific example is shown in FIG. 9, in which the base block is first quadtree-partitioned into four partitions, as shown by 902, 904, 906, and 908. Each of the resulting partitions is then either quadtree-partitioned into four further partitions (such as 908), or bisectioned at the next level into two further partitions (e.g., both symmetric, either horizontally or vertically, such as 902 or 906), or not divided (such as 904). Bisection or quadtree partitioning may be recursively enabled for square partitions, as shown by the overall exemplary partition pattern in 910 and the corresponding tree structure / representation in 920, where solid lines represent quadtree partitioning and dashed lines represent bisection. A flag may be used for each bisection node (non-leaf binary partition) to indicate whether the bisection is horizontal or vertical. For example, as shown in 920, which matches the partitioning structure in 910, a flag of "0" can represent horizontal bisection and a flag of "1" can represent vertical bisection. For quadtree partitioning, it is not necessary to specify the partition type, as quadtree partitioning always divides a block or partition both horizontally and vertically to generate four sub-blocks / partitions of equal size. In some implementations, a flag of "1" can represent horizontal bisection and a flag of "0" can represent vertical bisection.

[0067] In some example implementations of QTBT, the quadtree and bisection rule sets may be represented by the following predefined parameters and their associated corresponding functions: -CTU size: quadtree root node size (base block size) -MinQTSize: The minimum allowed quadtree leaf node size -MaxBTSize: Maximum allowed binary tree root node size -MaxBTDepth: Maximum allowed binary tree depth -MinBTSize: The minimum allowable binary tree leaf node size

[0068] In some example implementations of the QTBT partitioning structure, the CTU size may be set as 128x128 luma samples with corresponding 64x64 blocks of chroma samples (using typical chroma subsampling assumptions), MinQTSize may be set as 16x16, MaxBTSize may be set as 64x64, MinBTSize (both width and height) may be set as 4x4, and MaxBTDepth may be set as 4. Quad-tree partitioning may first be applied to the CTU to generate quad-tree leaf nodes. The quad-tree leaf nodes may have sizes from its minimum allowable size of 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). If a node is 128x128, it will not be initially split by the binary tree because its size exceeds MaxBTSize (i.e., 64x64). Otherwise, nodes that do not exceed MaxBTSize may be split by the binary tree. In the example of Figure 9, the base block is 128x128. The base block can only be quadtree partitioned according to a predefined set of rules. The base block has a partition depth of 0. Each of the four resulting partitions is 64x64, no larger than MaxBTSize, and may be further quadtree or binary tree partitioned at level 1. The process continues. When the binary tree depth reaches MaxBTDepth (i.e., 4), no further partitions may be considered. When the width of a binary tree node is equal to MinBTSize (i.e., 4), no further horizontal partitions may be considered. Similarly, when the height of a binary tree node is equal to MinBTSize, no further vertical partitions may be considered.

[0069] In some example implementations, the above QTBT scheme may be configured to support flexibility for luma and chroma to have the same or separate QTBT structures. For example, for P slices and B slices, the luma CTB and chroma CTB in one CTU may share the same QTBT structure. However, for I slices, the luma CTB may be divided into CBs by a QTBT structure, and the chroma CTB may be divided into chroma CBs by a different QTBT structure. This means that CUs may be used to refer to different color channels within an I slice; for example, an I slice may consist of a coding block for a luma component or coding blocks for two chroma components, and a CU in a P slice or B slice may consist of coding blocks for all three color components.

[0070] The various CB division schemes described above, as well as the further division of the CB into PBs, may be combined in any manner. The following specific implementations are provided as non-limiting examples.

[0071] Inter prediction may be performed, for example, in single reference mode or mixed reference mode. In some implementations, a skip flag may be initially included in the bitstream of the current block (or at a higher level) to indicate whether the current block is inter-coded and not skipped. If the current block is inter-coded, then another flag may be further included in the bitstream to signal whether single reference mode or mixed reference mode is used to predict the current block. In the single reference mode, one reference block may be used to generate the prediction block of the current block. In the mixed reference mode, two or more reference blocks may be used to generate the prediction block, for example, by weighted averaging. One or more reference blocks may be identified using one or more reference frame indices and one or more corresponding motion vectors indicating a shift between the one or more reference blocks and the current block in position relative to the frame, for example, in horizontal and vertical pixels. For example, the inter-predicted block of the current block may be generated from a single reference block identified by one motion vector in a reference frame as the predictive block in a single reference mode, but in a mixed reference mode, the predictive block may be generated by a weighted average of two reference blocks in two reference frames indicated by two reference frame indices and two corresponding motion vectors. The motion vectors may be coded and included in the bitstream in various ways.

[0072] In some example implementations, one or more reference picture lists including identification of short-term and long-term reference frames for inter prediction may be formed based on information in a reference picture set (RPS). For example, a single picture reference list may be formed for unidirectional inter prediction, denoted as L0 reference (or reference list 0), and two picture reference lists may be formed for bidirectional inter prediction, denoted as L0 (or reference list 0) and L1 (or reference list 1), for each of the two prediction directions. The reference frames included in the L0 and L1 lists may be ordered in various predetermined manners. The lengths of the L0 and L1 lists may be signaled in the video bitstream. Unidirectional inter prediction may be either a single reference mode or a mixed reference mode, provided that multiple references for generating a prediction block by weighted averaging in the mixed prediction mode are on the same side of the frame where the block to be predicted is located. Bidirectional inter prediction may only be a mixed mode, in that bidirectional inter prediction includes at least two reference blocks.

[0073] In some implementations, a merge mode (MM) for inter prediction may be implemented. Generally, in the merge mode, one or more of the motion vectors in single reference prediction or mixed reference prediction of the current PB may be derived from other motion vectors rather than being calculated and signaled independently. For example, in an encoding system, the current motion vector of the current PB may be represented by the difference between the current motion vector and one or more other already coded motion vectors (referred to as reference motion vectors). Such a motion vector difference, rather than the entire current motion vector, may be coded and included in the bitstream and linked to the reference motion vector. Correspondingly, in a decoding system, the motion vector corresponding to the current PB may be derived based on the decoded motion vector difference and the decoded reference motion vector linked thereto. As a specific form of the general merge mode (MM) inter prediction, such inter prediction based on the motion vector difference may be referred to as merge mode with motion vector difference (MMVD). Therefore, the general MM or the MMVD in particular may be implemented to exploit the correlation between motion vectors associated with different PBs to improve coding efficiency. For example, neighboring PBs may have similar motion vectors, and therefore the MVD may be small and can be coded efficiently. In another example, motion vectors may be correlated temporally (between frames) for similarly positioned / located blocks in space.

[0074] In some exemplary implementations of MMVD, a list of reference motion vector (RMV) or MV predictor candidates for motion vector prediction may be formed for a predicted block. The list of RMV candidates may include a predetermined number (e.g., two) of MV predictor candidate blocks whose motion vectors can be used to predict a current motion vector. The RMV candidate blocks may include blocks selected from neighboring blocks and / or temporal blocks within the same frame (e.g., blocks co-located in an ongoing or subsequent frame relative to the current frame). These options represent blocks at spatial or temporal positions relative to the current block that are likely to have similar or identical motion vectors to the current block. The size of the list of MV predictor candidates may be predetermined. For example, the list may include two or more candidates. To be on the list of RMV candidates, a candidate block may, for example, need to have the same reference frame (or frames) as the current block, must exist (e.g., if the current block is near an edge of a frame, a boundary check needs to be performed), must have already been coded during the encoding process, and / or must have already been decoded during the decoding process. In some implementations, the list of merge candidates may be first filled with spatially neighboring blocks (scanned in a certain predefined order) if they are available and satisfy the above conditions, and then filled with temporal blocks if space is still available in the list. Neighboring RMV candidate blocks may be selected, for example, from blocks to the left and above the current block. The list of RMV predictor candidates may be dynamically formed at various levels (sequence, picture, frame, slice, superblock, etc.) as a dynamic reference list (DRL). The DRL may be signaled in the bitstream.

[0075] In some implementations, the actual MV predictor candidate used as the reference motion vector for predicting the motion vector of the current block may be signaled. When the RMV candidate list includes two candidates, a one-bit flag called a merge candidate flag may be used to indicate the selection of the reference merge candidate. For a current block being predicted in combined mode, each of multiple motion vectors predicted using an MV predictor may be associated with a reference motion vector from the merge candidate list. The encoder can determine which RMV candidate more closely predicts the MV of the current coding block and signal the selection as an index to the DRL.

[0076] In some example implementations of MMVD, after an RMV candidate is selected and used as a base motion vector predictor for a motion vector to be predicted, a motion vector difference (MVD or delta MV representing the difference between the motion vector to be predicted and a reference candidate motion vector) may be calculated in the encoding system. Such MVD may include information representing the magnitude and direction of the MV difference, both of which may be signaled in the bitstream in various ways.

[0077] In some example implementations of MMVD, a distance index may be used to specify motion vector difference magnitude information and indicate one of a set of predefined offsets representing a predefined motion vector difference from a starting point (reference motion vector). The MV offset according to the signaled index may then be added to either the horizontal or vertical component of the starting (reference) motion vector. An example predefined relationship between the distance index and the predefined offset is specified in Table 1.

[0078] [Table 1]

[0079] In some example implementations of MMVD, a direction index may be further signaled and used to represent the direction of the MVD relative to the reference motion vector. In some implementations, the direction may be limited to either the horizontal or vertical direction. An example 2-bit direction index is shown in Table 2. In the example of Table 2, the interpretation of the MVD may change according to the information of the start / reference MV. For example, if the start / reference MV corresponds to a uni-predictive block or a bi-predictive block where both reference frame lists point to the same side of the current picture (i.e., the POCs of the two reference pictures are both greater than or both less than the POC of the current picture), the code in Table 2 may specify the sign (direction) of the MV offset added to the start / reference MV. If the start / reference MV corresponds to a bi-predictive block with two reference pictures on different sides of the current picture (i.e., the POC of one reference picture is larger than the POC of the current picture and the POC of the other reference picture is smaller than the POC of the current picture), and the difference between the reference POC in picture reference list 0 and the current frame is larger than the difference between the reference POC in picture reference list 1 and the current frame, then the code in Table 2 may specify the sign of the MV offset added to the reference MV corresponding to the reference picture in picture reference list 0, and the sign of the offset of the MV corresponding to the reference picture in picture reference list 1 may have the opposite value (opposite sign of the offset). Otherwise, if the difference between the reference POC in picture reference list 1 and the current frame is larger than the difference between the reference POC in picture reference list 0 and the current frame, then the code in Table 2 may specify that the sign of the MV offset added to the reference MV associated with picture reference list 1 and the sign of the offset to the reference MV associated with picture reference list 0 have opposite values.

[0080] [Table 2]

[0081] In some example implementations, the MVD may be scaled according to the difference in POC in each direction. If the difference in POC in both lists is the same, no scaling is required. Otherwise, if the difference in POC in Reference List 0 is greater than that of Reference List 1, the MVD of Reference List 1 is scaled. If the POC difference in Reference List 1 is greater than that of List 0, the MVD of List 0 may be scaled as well. If the starting MV is uni-predicted, the MVD is added to the available or reference MV.

[0082] In some example implementations of MVD coding and signaling for bidirectional mixed prediction, in addition to or instead of coding and signaling two MVDs separately, symmetric MVD coding may be implemented such that only one MVD needs to be signaled and the other MVD can be derived from the signaled MVD. In such implementations, motion information including reference picture indices for list 0 and list 1 are not signaled. Specifically, at the slice level, a flag called "mvd_l1_0_flag" may be included in the bitstream to indicate whether reference list 1 is not signaled in the bitstream. If this flag is 1, indicating that reference list 1 is equal to 0 (and therefore not signaled), a bidirectional prediction flag called "BiDirPredFlag" may be set to 0, meaning there is no bidirectional prediction. Otherwise, if mvd_l1_0_flag is 0, then BiDirPredFlag may be set to 1 if the closest reference picture in list 0 and the closest reference picture in list 1 form a forward and backward pair of reference pictures or a backward and forward pair of reference pictures, and the reference pictures in list 0 and list 1 are both short-term reference pictures. Otherwise, BiDirPredFlag is set to 0. A BiDirPredFlag of 1 may indicate that a symmetric mode flag is additionally signaled in the bitstream. A decoder can extract the symmetric mode flag from the bitstream when BiDirPredFlag is 1. The symmetric mode flag may be signaled, for example, at the CU level (if necessary) and may indicate whether symmetric MVD coding mode is used for the corresponding CU.When the symmetric mode flag is 1, it indicates the use of a symmetric MVD coding mode, indicating that only the reference picture indices for both list 0 and list 1 (called "mvp_l0_flag" and "mvp_l1_flag") are signaled by the MVD associated with list 0 (called "MVD0"), and that the other motion vector difference, "MVD1," should be derived rather than signaled. For example, MVD1 may be derived as -MVD0. Thus, in the exemplary symmetric MVD mode, only one MVD is signaled.

[0083] In some other example implementations for MV prediction, a harmonized scheme may be used to implement general merge-mode MMVD and some other types of MV prediction for both single-reference mode MV prediction and mixed-reference mode MV prediction. Various syntax elements may be used to signal how the MV of the current block is predicted. For example, for single-reference mode, the following MV prediction modes may be signaled:

[0084] NEARMV - Use one of the motion vector predictors (MVPs) in the list indicated by the DRL (Dynamic Reference List) index directly without MVD.

[0085] NEWMV - Use one of the motion vector predictors (MVP) in the list signaled by the DRL index as a reference and apply the delta to the MVP (eg, use MVD).

[0086] GLOBALMV - Use motion vectors based on frame-level global motion parameters.

[0087] Similarly, for a mixed reference inter prediction mode that uses two reference frames corresponding to the two MVs to be predicted, the following MV prediction modes may be signaled:

[0088] NEAR_NEARMV - For each of two of the MVs to predict, use one of the motion vector predictors (MVPs) in the list signaled by the DRL index without MVD.

[0089] NEAR_NEWMV - To predict the first of two motion vectors, one of the motion vector predictors (MVP) in the list signaled by the DRL index without MVD is used as a reference MV, and to predict the second of two motion vectors, one of the motion vector predictors (MVP) in the list signaled by the DRL index is used as a reference MV together with an additionally signaled delta MV (MVD).

[0090] NEW_NEARMV - To predict the second of two motion vectors, use one of the motion vector predictors (MVP) in the list signaled by the DRL index without MVD as a reference MV, and to predict the first of two motion vectors, use one of the motion vector predictors (MVP) in the list signaled by the DRL index as a reference MV together with an additionally signaled delta MV (MVD).

[0091] NEW_NEWMV - Use one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference MV and use it in conjunction with the additionally signaled delta MV to predict for each of the two MVs.

[0092] GLOBAL_GLOBALMV - Use MV from each reference based on frame-level global motion parameters.

[0093] Thus, the term "NEAR" above refers to MV prediction using a reference MV without any MVD, as in the general merge mode, whereas the term "NEW" refers to MV prediction using a reference MV and offsetting it with a signaled or derived MVD, as in the MMVD mode. In the case of hybrid inter prediction, both the reference-based motion vector and the motion vector delta described above may generally differ or be independent between the two references or between the two MVDs, even if the two MVDs are, for example, correlated, and such correlation can be utilized to reduce the amount of information required to signal the two motion vector deltas. To exploit such correlation, joint signaling of the two MVDs may be implemented and indicated in the bitstream, as described in further detail below.

[0094] In some implementations of the MVD, a predefined pixel resolution of the MVD may be allowed. For example, a motion vector accuracy (or precision) of 1 / 8 pixel may be allowed. The MVDs described above for various MV prediction modes can be constructed and signaled in various ways. In some implementations, various syntax elements can be used to signal the above motion vector differences in reference frame list 0 or list 1.

[0095] For example, a syntax element called "mv_joint" can specify which components of its associated motion vector difference are non-zero. For example, A mv_joint with a value of 0 can indicate that there is no non-zero MVD along either the horizontal or vertical direction, 1 can be shown that there is non-zero MVD only along the horizontal direction, 2 can be shown that there is non-zero MVD only along the vertical direction, 3 can be shown to have non-zero MVD along both the horizontal and vertical directions.

[0096] If the "mv_joint" syntax element for MVD signals that there are no non-zero MVD components, no further MVD information may be signaled. However, if the "mv_joint" syntax signals that there are one or two non-zero components, additional syntax elements may be further signaled for each of the non-zero MVD components, as described below.

[0097] For example, a syntax element called "mv_sign" may be used to further specify whether the corresponding motion vector difference component is positive or negative.

[0098] In another example, a syntax element called "mv_class" can be used to specify the class of motion vector differences between a predefined set of classes of corresponding non-zero MVD components. The predefined classes of motion vector differences can be used, for example, to divide a continuous magnitude space of motion vector differences into non-overlapping ranges of classes. Thus, the signaled MVD class indicates the magnitude range of the corresponding MVD component. In the example implementation shown in Table 3 below, higher classes correspond to motion vector differences with a larger magnitude range. The symbol (n,m) is used to represent a range of motion vector differences greater than n pixels and less than or equal to m pixels.

[0099] [Table 3]

[0100] In some other examples, a syntax element called “mv_bit” may be further used to specify the integer portion of the offset between a non-zero motion vector difference component and the magnitude of the start of the correspondingly signaled MV class magnitude range. In some other examples, a syntax element called “mv_fr” may be further used to specify the first two fractional bits of the motion vector difference of the corresponding non-zero MVD component, and a syntax element called “mv_hp” may be used to specify the third fractional bit (high-resolution bit) of the motion vector difference of the corresponding non-zero MVD component. The two bits of “mv_fr” essentially provide 1 / 4 pixel MVD resolution, while the “mv_hp” bit can further provide 1 / 8 pixel resolution. In some other implementations, two or more “mv_hp” bits may be used to provide MVD pixel resolution finer than 1 / 8 pixel. In some example implementations, an additional flag may be signaled at one or more of various levels to indicate whether MVD resolution of 1 / 8 pixel or greater is supported. If an MVD resolution does not apply to a particular coding unit, the above syntax elements for the corresponding unsupported MVD resolution may not be signaled.

[0101] However, in some other exemplary implementations, the resolution of the motion vector differences in various MVD magnitude classes may be differentiated or adaptive. Specifically, a high-resolution MVD for a large MVD magnitude of a higher MVD class may not result in a statistically significant improvement in compression efficiency or coding gain. Therefore, the MVD may be coded with reduced resolution (integer pixel resolution or fractional pixel resolution) or no resolution increase for a larger MVD magnitude range corresponding to a higher MVD magnitude class. The term "resolution" may also be referred to as "pixel resolution."

[0102] In some example implementations, each MVD class may be associated with a single allowed resolution. In some other implementations, one or more MVD classes may each be associated with two or more optional MVD pixel resolutions. For example, adaptively allowed MVD pixel resolutions may include, but are not limited to (in descending order of resolution): 1 / 64pel (pixel), 1 / 32pel, 1 / 16pel, 1 / 8pel, 1-4pel, 1 / 2pel, 1pel, 2pel, 4pel...

[0103] In some other example implementations, for MV classes equal to or greater than a threshold MV class, only a single MVD value may be allowed. For example, such a threshold MV class may be MV_CLASS 2. Thus, MV_CLASS 2 and above may only be allowed to have a single MVD value and no fractional pixel resolution.

[0104] Turning to various composite inter-prediction modes in which each MV is predicted by a reference motion vector and can be coded by an MVD, the two MVDs can be signaled separately or jointly in the bitstream, as described above. Thus, in some exemplary implementations, in addition to the above-mentioned NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, and GLOBAL_GLOBALMV modes, another inter-prediction mode called JOINT_NEWMV can be introduced for modes in which the MVD is for joint signaling of reference lists 0 and 1. Specifically, when the inter-prediction mode is indicated as NEW_NEWMV, the MVDs of reference lists 0 and 1 are signaled separately, but when the inter-prediction mode is indicated as JOINT_NEWMV mode, the MVDs of reference lists 0 and 1 are signaled together. In particular, for joint MVD, only one MVD, called joint_delta_mv, may need to be signaled and transmitted in the bitstream, and the MVDs for reference lists 0 and 1 may be derived from joint_delta_mv. The derived MVD can then be combined with a reference motion vector in reference list 0 or 1 to generate two motion vectors for locating reference blocks for hybrid inter prediction.

[0105] In some implementations of composite inter prediction, the JOINT_NEWMV mode may be signaled together with the NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, and GLOBAL_GLOBALMV modes. In such implementations, syntax may be included in the bitstream to indicate any one of these alternative composite inter prediction modes at any one of various signaling levels (e.g., sequence level, picture level, frame level, slice level, tile level, superblock level, etc.). Alternatively, the JOINT_NEWMV mode may be implemented as a sub-mode of the NEW_NEWMV mode. In other words, under the NEW_NEWMV mode, the two MVDs of the two reference blocks are either signaled together (hence, the JOINT_NEWMV sub-mode) or not signaled (another sub-mode of the NEW_NEWMV mode). In such an implementation, a first syntax element may be included in the bitstream to indicate any one of NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, and GLOBAL_GLOBALMV modes, and if the first syntax element indicates that NEW_NEWMV mode is selected for a coding block, then a second syntax element may be further included in the bitstream and may be extractable by a decoder to indicate whether the MVDs of the coding blocks are signaled separately or together.

[0106] In some example implementations, when JOINT_NEWMV mode is signaled and the POC distances between two reference frames and the current frame are different, the MVD may be scaled for reference list 0 or reference list 1 based on the POC distances. Specifically, the distance between reference frame list 0 and the current frame may be denoted as td0, and the distance between reference frame list 1 and the current frame may be denoted as td1. If td0 is greater than or equal to td1, joint_mvd may be used directly for reference list 0, and the MVD of reference list 1 may be derived from joint_mvd based on equation (1).

number

[0107] Otherwise, if td1 is greater than or equal to td0, then joint_mvd is used directly for reference list 1, and the MVD of reference list 0 is derived from joint_mvd according to equation (2).

number

[0108] In some example implementations, another inter-coding mode called AMVDMV may be added to the single reference case. When the AMVDMV mode is selected, it indicates that AMVD (adaptive motion vector differential) is applied to the signal MVD. To indicate whether AMVD is applied to the joint MVD coding mode, a flag, for example named amvd_flag, may be added under the JOINT_NEWMV mode. Then, when adaptive MVD resolution is applied to the joint MVD coding mode, the MVDs of the two reference frames are signaled together, and the accuracy of the MVD may be implicitly determined by the size of the MVD. Otherwise, the MVDs of the two (or more) reference frames may be signaled together, and conventional MVD coding without adaptive MVD resolution may be applied.

[0109] Turning to the composite inter-modes, as shown in Fig. 10, these modes are divided into two different reference frames F i-1 and F i+1 MV0 and MV1 from the current frame F i Therefore, for each block, two motion information components (e.g., motion vectors) may be signaled in the bitstream.

[0110] Alternatively, as shown in FIG. 11, an interpolation process can be used to find the two reference frames F i-1 and F i+1 The information of the current frame F i The interpolated frame may be projected at the same time as the current frame F to generate an interpolated frame. Multiple TIP modes may be supported. In one TIP mode, the interpolated frame may be used as an additional reference frame. i The coding blocks of F can directly reference the interpolated frame and take advantage of information coming from two different references with only the overhead cost of a single inter prediction mode. In another TIP mode, the interpolated frame is used to encode the current frame F while skipping any other conventional coding steps. i This mode can have significant coding and complexity advantages, especially for low bitrate applications.

[0111] In some implementations, adaptive motion vector resolution (AMVR) may be supported. In a particular example implementation, a total of seven MV precisions (8, 4, 2, 1, 1 / 2, 1 / 4, 1 / 8) may be supported. For each prediction block, the AMVR encoder may search all supported precision values ​​and signal the best precision to the decoder.

[0112] To reduce encoder execution time, two MV precision sets are supported. Each precision set may include four predefined precisions. The precision set may be adaptively selected at the frame level based on the frame's maximum precision value. The maximum precision may be signaled in the frame header. Table 4 summarizes exemplary supported precision values ​​based on exemplary frame-level maximum precisions.

[0113] [Table 4]

[0114] In some example implementations of AMVR, there may be a frame-level flag that indicates whether the MV of a frame includes sub-pel precision. AMVR may be enabled only if the value of the cur_frame_force_integer_mv flag is 0. In AMVR, if the precision of a block is less than the maximum precision, the motion model and interpolation filter may not be signaled. If the precision of a block is less than the maximum precision, the motion mode is inferred to be translational, and the interpolation filter is inferred to be a regular interpolation filter. Similarly, if the precision of a block is either 4 or 8 pixels, the inter-intra mode may remain unsignaled and may be inferred to be 0.

[0115] In some examples, an optical flow-based approach may be implemented to refine motion vectors (MVs) at the sub-block level for composite prediction. In particular, optical flow equations can be applied to formulate a least-squares problem, from which fine motions can be derived from gradients of composite inter-predicted samples. Using these fine motions, MVs for each sub-block can be refined within the prediction block, which can thus improve inter-prediction quality. This coding feature may be an extension of the well-known bidirectional optical flow (BDOF) concept, as it supports MV refinement when two reference blocks have any temporal distance to the current block. In some implementations, additional inter-composite modes, such as those listed below, are supported. · NEAR_NEARMV_OPTFLOW, · NEAR_NEWMV_OPTFLOW, · NEW_NEARMV_OPTFLOW, For example, four additional intercomposite modes may be added, such as NEW_NEWMV_OPTFLOW.

[0116] In some implementations, signaling of a composite inter-prediction mode depends on the optical flow refinement flag of the current block. For example, one flag called use_optflow may be signaled as a syntax element in the bitstream before the syntax element indicating the composite inter-prediction mode. The flag, for example, use_optflow, may indicate whether an optical flow-based composite mode is used. If use_optlow is set to 1 (or true), these composite inter-prediction modes are called optical flow modes, and the reference MV type is defined similarly to conventional composite modes (e.g., NEAR_NEWMV_OPTFLOW has the same reference MV type as NEAR_NEWMV), but composite prediction is based on sub-block-wise refined MVs rather than the original MVs. Otherwise, if use_optflow is set to 0 (false), optical flow refinement is not applied to the current block. Specifically, according to the disclosed method, a decoder may receive a first syntax element signaled in a video bitstream indicating whether optical flow refinement is applied for a video block, and determine whether optical flow refinement is applied for the video block based on the value of the first syntax element. After receiving the first syntax element, the decoder may receive a second syntax element from the video bitstream indicating a composite inter-prediction mode for the video block depending on whether optical flow refinement is applied, and determine the composite inter-prediction mode for the video block based on the value of the second syntax element. The decoder may then predict the video block based on the determined composite inter-prediction mode.

[0117] Therefore, in the above implementation, a flag is signaled as a syntax element in the bitstream before the signaling of a composite inter prediction mode among multiple composite inter prediction modes to indicate whether the composite inter prediction mode is a composite inter prediction mode without optical flow refinement or a composite inter prediction mode with optical flow refinement. The use and distribution / statistics of the composite inter prediction mode may be different when the optical flow refinement flag is on than when the optical flow refinement flag is off. Such correlation between the use of optical flow refinement and the composite inter prediction mode may be exploited in the signaling to improve coding efficiency.

[0118] The following various exemplary implementations for the signaling schemes of optical flow refinement and hybrid inter-prediction may be used separately or combined in any order. Furthermore, the methods, encoders, and decoders according to these implementations may each be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored in a non-transitory computer-readable medium. Hereinafter, the term block may be interpreted as a prediction block, a coding block, or a coding unit, i.e., a CU.

[0119] In this document, the orientation of a reference frame may be determined by whether the reference frame is before the current frame in display order or after the current frame in display order.

[0120] For these implementations, in mixed reference mode, if the POC of both reference frames of a motion vector pair is greater than or less than the POC of the current frame, then the orientations of the two reference frames may be the same. Alternatively, if the POC of one reference frame is greater than the POC of the current frame and the POC of the other reference frame is less than the POC of the current frame, then the orientations of the two reference frames are different.

[0121] In some example implementations, signaling of the composite inter-prediction mode in the bitstream may depend on the optical flow refinement flag of the current block, and / or the reference frame of the current block, and / or the inter-prediction mode of a neighboring block, and / or the reference frame of a neighboring block.

[0122] In some example implementations, the context for encoding / decoding the signaling of the composite inter-prediction mode may depend on the optical flow refinement flag of the current block, and / or the reference frame of the current block, and / or the inter-prediction mode of a neighboring block, and / or the reference frame of a neighboring block.

[0123] In some example implementations, if the optical flow refinement flag of the current block is false, one set of contexts may be used to code / decode the combined inter-prediction mode signaling, otherwise, another set of contexts is used instead.

[0124] In some example implementations, the context for signaling the combined inter-prediction mode may depend on whether the optical flow refinement mode is allowed for the current block, and / or the optical flow refinement flag of the current block, and / or the reference frame of the current block, and / or the inter-prediction mode of the neighboring block, and / or the reference frame of the neighboring block. If the corresponding flag is not to be signaled in the bitstream, optical flow refinement is not allowed. Whether optical flow refinement is allowed may be predefined or signaled at a higher level. When optical flow refinement is allowed, whether it is used for a particular block or at another level is signaled by the optical flow refinement flag mentioned above.

[0125] In some exemplary implementations, when the optical flow refinement flag is on for a block, only a subset of composite inter-prediction modes that require the signaling of at most one motion vector difference (MVD) are allowed. In other words, composite inter-prediction modes that require the signaling of more than one MVD are not allowed. Such a constraint improves signaling efficiency by reducing the signaling space when the optical flow refinement flag is on.

[0126] In some example implementations, when the optical flow refinement flag is on for a block, only a subset of the following composite inter prediction modes are allowed and signaled: NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV. These allowed composite inter prediction modes require, for example, that at most one MVD be signaled. Thus, when optical flow refinement is applied to a block, the syntax value space of composite inter prediction modes for a video block may not span all possible composite inter prediction modes, but instead may include a subset of the following modes: NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV. Such constraints improve signaling efficiency by reducing the signaling space when the optical flow refinement flag is on.

[0127] In some exemplary implementations, when the optical flow refinement flag is on for a block, only a subset of composite inter-prediction modes with at most one MVD for all reference frames is allowed. In other words, composite inter-prediction modes with more than one MVD are not allowed and therefore not signaled. Such a constraint improves signaling efficiency by reducing the signaling space when the optical flow refinement flag is on.

[0128] In some example implementations, when the optical flow refinement flag is on for a block, a subset of the NEAR_NEARMV, NEAR_NEWMV, and NEW_NEARMV composite inter prediction modes are allowed and signaled. Other composite inter prediction modes are not allowed and therefore not signaled. These allowed composite inter prediction modes include at most one MVD. Thus, when optical flow refinement is applied to a block, the syntax value space of composite inter prediction modes for a video block may not span all possible composite inter prediction modes, but instead may include a subset of the NEAR_NEARMV, NEAR_NEWMV, and NEW_NEARMV modes. Such constraints improve signaling efficiency by reducing the signaling space when the optical flow refinement flag is on.

[0129] In some example implementations, there may be two joint MVD coding modes allowed when the optical flow refinement flag is off: JOINT_NEWMV and JOINT_AMVDNEWMV. However, when the optical flow refinement flag is on for a block, at most one of these two joint MVD coding modes may be allowed. Such a constraint improves signaling efficiency by reducing the signaling space when the optical flow refinement flag is on.

[0130] In some example implementations, when the optical flow refinement flag is on for one block, the only allowed joint MVD coding mode may be JOINT_AMVDNEWMV. Such a constraint improves signaling efficiency by reducing the signaling space when the optical flow refinement flag is on.

[0131] In some example implementations, when the optical flow refinement flag of the current block is on, the mapping between the parsed syntax value and the composite inter prediction mode, i.e., the semantics of the syntax value, is different compared to the mapping when the optical flow refinement flag of the current block is off. For example, the same syntax value extracted for the composite inter prediction mode syntax element is mapped to different composite inter prediction modes when the optical flow refinement flag is on and when the optical flow refinement flag is off. In other words, the same syntax value can dynamically point to different composite inter prediction modes depending on the received / extracted value of the optical flow refinement flag from the bitstream. This implementation can be used to exploit the correlation between the optical flow flag and various composite inter prediction modes. For example, different inter prediction modes are likely to be invoked when the optical flow refinement flag is on and when the optical flow refinement flag is off. Therefore, more efficient syntax values ​​(eg, fewer signaling bits) can be used for coding to map to more likely composite inter-prediction modes for different optical flow refinement flag values.

[0132] In some example implementations, the optical flow refinement flag may be signaled after the combined inter-prediction mode, and the signaling of the optical flow refinement flag may depend on the combined inter-prediction mode. Such a constraint improves signaling efficiency.

[0133] In some example implementations, the context for signaling the optical flow refinement flag may depend on the combined inter-prediction mode. Such a constraint improves signaling efficiency.

[0134] In some example implementations, the optical flow refinement flag may be signaled only for a subset of the composite inter-prediction modes. In this way, the overall amount of signaling for the optical flow refinement flag can be reduced. The subset of composite inter-prediction modes can be determined so that the overall coding efficiency is not significantly impaired.

[0135] In some example implementations, the optical flow refinement flag may be signaled only for composite inter-prediction modes with at most one signaled MVD for multiple reference frames, such as a subset of the NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV composite inter-prediction modes. Such a constraint improves signaling efficiency.

[0136] In some example implementations, the optical flow refinement flag may be signaled only for composite inter-prediction modes involving at most one MVD, such as a subset of the NEAR_NEARMV, NEAR_NEWMV, and NEW_NEARMV composite inter-prediction modes. Such a constraint improves signaling efficiency.

[0137] In some example implementations, the optical flow refinement flag may only be signaled for composite inter-prediction modes that do not have MVD, or that use adaptive MVD resolution, or that have MV precision coarser than a threshold (e.g., MV precision coarser than 1 / 8 or 1 / 4 MV precision).

[0138]

[0033] Figure 10 shows an example logic flow 1000 according to the above implementation. Logic flow 1000 starts at S1001. At S1010, a first syntax element signaled in a video bitstream is received, indicating whether optical flow refinement is applied to the video block. At S1020, it is determined whether optical flow refinement is applied to the video block based on the value of the first syntax element. At S1030, after receiving the first syntax element, a second syntax element indicating a composite inter-prediction mode for the video block is received from the video bitstream depending on whether optical flow refinement is applied. At S1040, a composite inter-prediction mode for the video block is determined based on the value of the second syntax element. At S1050, the video block is predicted based on the determined composite inter-prediction mode. Logic flow 1000 stops at S1099.

[0139] Exemplary pseudocode for reading the above optical flow refinement and hybrid inter prediction syntax is provided below: #if CONFIG_OPTFLOW_REFINEMENT int use_optical_flow = 0; if (cm->features.opfl_refine_type == REFINE_SWITCHABLE && is_opfl_refine_allowed(cm, mbmi)) { use_optical_flow = aom_read_symbol(r, xd->tile_ctx->use_optflow_cdf[ctx], 2, ACCT_INFO(“use_optical_flow”)); } #endif / / CONFIG_OPTFLOW_REFINEMENT const int mode = #if CONFIG_OPTFLOW_REFINEMENT aom_read_symbol(r, xd->tile_ctx->inter_compound_mode_cdf[ctx], INTER_COMPOUND_REF_TYPES, ACCT_INFO(“inter_compound_mode_cdf”)); #else aom_read_symbol(r, xd->tile_ctx->inter_compound_mode_cdf[ctx], INTER_COMPOUND_MODES, ACCT_INFO(“inter_compound_mode_cdf”)); #endif / / CONFIG_OPTFLOW_REFINEMENT #if CONFIG_OPTFLOW_REFINEMENT if (use_optical_flow) { assert(is_inter_compound_mode(comp_idx_to_opfl_mode[mode])); return comp_idx_to_opfl_mode[mode]; } #endif / / CONFIG_OPTFLOW_REFINEMENT assert(is_inter_compound_mode(NEAR_NEARMV + mode)); return NEAR_NEARMV + mode; }

[0140] The above operations may be combined or arranged in any quantity or order as desired. Two or more of the steps and / or operations may be performed in parallel. The embodiments and implementations of the present disclosure may be used individually or combined in any order. Furthermore, each of the method (or embodiment), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium. The embodiments of the present disclosure may apply to luma blocks or chroma blocks. The term block may be interpreted as a prediction block, a coding block, or a coding unit (CU). The term block here may also be used to refer to a transform block. In the following sections, when referring to a block size, it may refer to either the width or height of the block, the maximum value of the width and height, the minimum value of the width and height, the size of the area (width * height), or the aspect ratio of the block (width:height or height:width).

[0141] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 11 illustrates a computer system (1100) suitable for implementing certain embodiments of the disclosed subject matter.

[0142] Computer software may be coded using any suitable machine code or computer language that can undergo mechanisms such as assembly, compilation, linking, etc. to create code containing instructions that can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or that can be executed via interpretation, microcode execution, etc.

[0143] The instructions may be executed on various types of computers or computer components including, for example, personal computers, tablet computers, servers, smartphones, gaming consoles, Internet of Things devices, and the like.

[0144] 11 for computer system (1100) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system (1100).

[0145] The computer system (1100) may include certain human interface input devices, which may include one or more (only one of each is depicted) of a keyboard (1101), a mouse (1102), a trackpad (1103), a touchscreen (1110), a data glove (not shown), a joystick (1105), a microphone (1106), a scanner (1107), and a camera (1108).

[0146] The computer system (1100) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of a human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1110), data gloves (not shown), or joystick (1105), although some haptic feedback devices may not function as input devices), audio output devices (such as speakers (1109), headphones (not shown)), visual output devices (such as screens (1110), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capabilities and with or without haptic feedback capabilities, some of which may output two-dimensional visual output or three- or more-dimensional output by means of stereoscopic output, virtual reality glasses (not shown), holographic displays, smoke tanks (not shown)), and printers (not shown).

[0147] The computer system (1100) may also include human-accessible storage devices and associated media such as optical media including CD / DVD ROM / RW (1120) with media such as CD / DVD (1121), thumb drives (1122), removable hard drives or solid state drives (1123), legacy magnetic media such as tape and floppy disks (not shown), and specialized ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0148] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves or other transitory signals.

[0149] The computer system (1100) may also include an interface (1154) to one or more communications networks (1155). The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet; cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc.; cable or wireless wide-area digital networks including cable, satellite, and terrestrial broadcast television; vehicular and industrial networks including CAN bus; etc.

[0150] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1140) of the computer system (1100).

[0151] The cores (1140) may include one or more central processing units (CPUs) (1141), graphics processing units (GPUs) (1142), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1143), task-specific hardware accelerators (1144), and graphics adapters (1150). These devices may be connected via a system bus (1148) along with read-only memory (ROM) (1145), random access memory (1146), and internal mass storage devices (1147) such as non-user-accessible internal hard drives or SSDs. In some computer systems, the system bus (1148) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1148) or via a peripheral bus (1149). In one example, a screen (1110) may be connected to the graphics adapter (1150). Architectures for peripheral buses include PCI, USB, and the like.

[0152] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0153] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that are within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure. [Explanation of symbols]

[0154] 0 picture reference list, 1 picture reference list, reference frame list, 100 communication system, 110, 120, 130, 140 terminal device, 150 network, 200 communication system, 201 video source, 202 stream, 203 video encoder, 204 encoded video data, 205 streaming server, 206, 208 client subsystem, 207, 209 copy, 210 video decoder, 211 video picture to be rendered, 212 display, 213 video capture system, 220, 230 electronic device, 301 channel, 310 video decoder, 312 display, 315 buffer memory, 319 symbols, 320 parser, 330 electronic device, 331 receiver, 351 scaler / inverse transform unit, 352 intra-picture prediction unit, 353 motion compensated prediction unit, 355 aggregator, 356 Loop filter, 357, reference picture memory, 358, current picture buffer, 401, video source, 403, video encoder, 420, electronic device, 430, source coder, 432, coding engine, 433, decoder, 434, reference picture memory, 435, predictor, 440, transmitter, 443, video sequence, 445, entropy coder, 450, controller, 460, channel, 503, video encoder, 521, general controller, 522, intra-encoder, 523, residual calculator, 524, residual encoder, 525, entropy encoder, 526, switch, 528, residual decoder, 530, inter-encoder, 610, video decoder, 671, entropy decoder, 672, intra-decoder, 673, residual decoder, 674, reconstruction module, 680, inter-decoder, 710, square partition, 802, vertical triangulation scheme, 804 Horizontal Tripartite Scheme, 910 Overall Exemplary Partition Pattern, 902, 904, 906, 908 Base Block, 920 Tree Structure, 1000 Logic Flow, 1100 Computer System, 1101 Keyboard, 1102 Mouse, 1103 Trackpad, 1105 Joystick, 1106 Microphone, 1107 Scanner, 1108 Camera, 1109 Speaker, 1110touch screen, 1120 CD / DVD ROM / RW, 1121 CD / DVD or other medium, 1122 thumb drive, 1123 removable hard drive or solid state drive, 1140 core, 1141 central processing unit, 1142 graphics processing unit, 1143 field programmable gate area, 1144 hardware accelerator, 1145 graphics adapter, 1146 random access memory, 1147 internal mass storage device, 1148 system bus, 1149 peripheral bus, 1150 graphics adapter, 1154 network interface, 1155 communication network

Claims

1. 1. A method for decoding video blocks in a video bitstream, comprising: receiving a first syntax element signaled in the video bitstream indicating whether optical flow refinement is applied to the video block; determining whether the optical flow refinement is applied to the video block based on a value of the first syntax element; receiving, after receiving the first syntax element, a second syntax element from the video bitstream that indicates a hybrid inter-prediction mode for the video block depending on whether the optical flow refinement is applied; determining the hybrid inter prediction mode for the video block based on a value of the second syntax element; predicting the video block based on the determined hybrid inter-prediction mode; A method comprising:

2. The method of claim 1 , wherein the value of the second syntax element is determined by decoding the second syntax element using a coding context that depends on whether the optical flow refinement is applied.

3. 3. The method of claim 2, wherein if the optical flow refinement is applied, a first context is used to decode the second syntax element indicating the composite inter-prediction mode of the video block, and if the optical flow refinement is not applied, a second context different from the first context is used to decode the second syntax element indicating the composite inter-prediction mode of the video block.

4. 2. The method of claim 1 , wherein only hybrid inter prediction modes that require signaling of one or no motion vector difference are allowed when the optical flow refinement is applied to the video block.

5. 5. The method of claim 4, wherein if the optical flow refinement is enabled for the video block, the syntax value space of the composite inter-prediction modes for the video block includes a subset of NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV modes.

6. 6. The method of claim 1, wherein only hybrid inter prediction modes with one or no motion vector difference are allowed when the optical flow refinement is applied to the video block.

7. 7. The method of claim 6, wherein the syntax value space of the hybrid inter-prediction modes for the video block includes a subset of NEAR_NEARMV, NEAR_NEWMV, and NEW_NEARMV modes when the optical flow refinement is applied to the video block.

8. Regarding the joint motion vector difference composite inter prediction mode, If the optical flow refinement is not applied to the video block, two joint motion vector difference composite inter prediction modes are allowed; 6. The method of claim 1, wherein only one of the two joint motion vector difference composite inter prediction modes is allowed when the optical flow refinement is applied to the video block.

9. The two joint motion vector difference composite inter prediction modes include a JOINT_NEWMV mode and a JOINT_AMVDNEWMV mode; when the optical flow refinement is applied to the video block, only one of the two allowed joint motion vector difference composite inter prediction modes is the JOINT_AMVDNEWMV mode. The method of claim 8.

10. Determining the composite inter prediction mode for the video block f includes: determining a mapping between possible values ​​of the second syntax element and a plurality of hybrid inter prediction modes based on whether the optical flow refinement for the video block is applied; and and determining the hybrid inter-prediction mode for the video block based on the value of the second syntax element and the mapping.

11. 11. The method of claim 10, wherein the mapping between the possible values ​​of the second syntax element and the plurality of hybrid inter prediction modes is different when the optical flow refinement is applied than when the optical flow refinement is not applied.

12. 1. A method for decoding video blocks in a video bitstream, comprising: receiving a first syntax element signaled in the video bitstream that indicates a composite inter-prediction mode for the video block among a plurality of composite inter-prediction modes; determining the hybrid inter-prediction mode for the video block based on a value of the first syntax element; determining whether a second syntax element for the video block is included in the video bitstream or receiving the second syntax element from the video bitstream in a manner dependent on the hybrid inter-prediction mode indicated by the first syntax element, the second syntax element indicating whether optical flow refinement is applied to the video block; A method comprising:

13. 13. The method of claim 12, wherein extracting the second syntax element comprises decoding the second syntax element using a coding context that depends on the hybrid inter-prediction mode indicated by the first syntax element.

14. The method of claim 12 , wherein the second syntax element for the video block is included in the video bitstream only for a subset of the plurality of composite inter-prediction modes.

15. 15. The method of claim 14, wherein the second syntax element for the video block is included in the video bitstream only if the composite inter prediction mode of the video block requires at most one signaled motion vector difference.

16. The method of claim 15 , wherein the subset of the plurality of composite inter-prediction modes includes NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, JOINT_NEWMV, and JOINT_AMVDNEWMV modes.

17. 15. The method of claim 14, wherein the second syntax element for the video block is included in the video bitstream only if the composite inter prediction mode for the video block involves at most one motion vector difference.

18. 15. The method of claim 14, wherein the second syntax element for the video block is included in the video bitstream only if the hybrid inter prediction mode for the video block does not involve distinct motion vectors, or uses adaptive motion vector resolution, or has motion vector precision coarser than a predetermined precision threshold.

19. In an electronic device, a memory for storing instructions and a processor, Executing the stored instructions, receiving a first syntax element signaled in a video bitstream indicating whether optical flow refinement is applied to video blocks within the video bitstream; determining whether the optical flow refinement is applied to the video block based on a value of the first syntax element; after receiving the first syntax element, receiving a second syntax element from the video bitstream that indicates a hybrid inter-prediction mode for the video block depending on whether the optical flow refinement is applied; determining the hybrid inter-prediction mode for the video block based on a value of the second syntax element; and a processor that predicts the video block based on the determined hybrid inter-prediction mode.

20. 13. An electronic device comprising: a memory for storing instructions; and a processor for executing the stored instructions to perform the method of claim 12.

Images