Video decoder, method performed by video decoder, computer program, and video encoder

Abstract

To provide a method for determining one or more scaling factors for a joint motion vector difference.SOLUTION: A method includes the steps of: receiving a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector difference (JMVD) associated with a block in the current picture; determining whether one or more scaling factors are used for an adaptive motion vector difference resolution (AMVD) based JMVD based on a predetermined condition; in response to determining the one or more scaling factors are used for the AMVD based JMVD, scaling the JMVD based on the one or more scaling factors; deriving a first MVD for the first reference picture based on the scaled JMVD; deriving a second MVD for the second reference picture based on the derived first MVD; and reconstructing the block based on the derived first MVD and second MVD.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] FIELD This disclosure relates generally to communication systems, and more particularly to methods and apparatus for inter-prediction mode dependent scaling factors. [Background technology]

[0002] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. This coding format was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 that includes semiconductor companies, video-on-demand providers, video content creators, software developers, and web browser vendors. Many of the AV1 project's components were sourced from previous research efforts by Alliance members. Individual contributors have been launching experimental technology platforms for years; Xipha and Mozilla's Daala released their code in 2010, Google's experimental VP9 evolution project, VP10, was announced on September 12, 2014, and Cisco's Thor was released on August 11, 2015. Building on the VP9 codebase, AV1 incorporates additional technologies, some of which were developed in these experimental formats. The first version, 0.1.0, of the AV1 reference codec was released on April 7, 2016. The Alliance announced the release of the AV1 bitstream specification on March 28, 2018, along with a reference software-based encoder and decoder. Validated version 1.0.0 of the specification was released on June 25, 2018. Validated version 1.0.0, including specification errata 1, was released on January 8, 2019. The AV1 bitstream specification includes a reference video codec. Current coding standards do not properly consider nonlinear motion between two frames when selecting the scaling factor for the motion vector difference (MVD) applied to those two frames. Summary of the Invention

[0003] The following presents a brief summary of one or more embodiments of the present disclosure in order to provide a basic understanding of such embodiments. This summary is not an exhaustive overview of all contemplated embodiments, and is not intended to identify key or essential elements of all embodiments or to delineate the scope of any and all embodiments. Its sole purpose is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] SUMMARY A method, apparatus, and non-transitory computer-readable medium for inter-prediction mode dependent scaling factors are disclosed in accordance with the present disclosure.

[0005] According to an example embodiment, a method performed by at least one processor of a video decoder includes receiving a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector difference (JMVD) associated with at least one block in the current picture. The method further includes determining, based on a predetermined condition, whether one or more scaling factors are used in a JMVD based on adaptive motion vector difference resolution (AMVD). The method further includes scaling the JMVD based on the one or more scaling factors in response to determining that one or more scaling factors are used in the JMVD based on AMVD. The method further includes deriving a first MVD of the first reference picture based on the scaled JMVD. The method further includes deriving a second MVD of the second reference picture based on the derived first MVD. The method further includes reconstructing at least one block based on the derived first MVD and the derived second MVD.

[0006] According to an exemplary embodiment, a video decoder includes at least one memory configured to store computer program code and at least one processor configured to access the computer program code and operate as directed by the computer program code. The computer program code includes receiving code configured to cause the at least one processor to receive a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector differential (JMVD) associated with at least one block in the current picture. The computer program code further includes decision code configured to cause the at least one processor to determine, based on a predetermined condition, whether one or more scaling factors are used in a JMVD based on adaptive motion vector differential decomposition (AMVD). The computer program code further includes scaling code configured to cause the at least one processor to scale the JMVD based on the one or more scaling factors in response to determining that one or more scaling factors are used in the JMVD based on the AMVD. The computer program code further includes first derivation code configured to cause the at least one processor to derive a first MVD for the first reference picture based on the scaled JMVD. The computer program code further includes second derivation code configured to cause the at least one processor to derive a second MVD of the second reference picture based on the derived first MVD. The computer program code further includes reconstruction code configured to cause the at least one processor to reconstruct at least one block based on the derived first MVD and the derived second MVD.

[0007] According to an example embodiment, a non-transitory computer-readable medium stores instructions that, when executed by a processor of a video decoder, cause the processor to perform a method including receiving a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector differential (JMVD) associated with at least one block in the current picture. The method further includes determining, based on a predetermined condition, whether one or more scaling factors are used in the JMVD based on adaptive motion vector differential decomposition (AMVD). The method further includes scaling the JMVD based on the one or more scaling factors in response to determining that one or more scaling factors are used in the JMVD based on the AMVD. The method further includes deriving a first MVD of the first reference picture based on the scaled JMVD. The method further includes deriving a second MVD of the second reference picture based on the derived first MVD. The method further includes reconstructing at least one block based on the derived first MVD and the derived second MVD.

[0008] Additional embodiments will be set forth in the description that follows, and in part will be obvious from the description, and / or may be learned by practice of presented embodiments of the present disclosure.

[0009] These and other aspects, features, and aspects of embodiments of the present disclosure will become apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic block diagram of a communication system according to various embodiments of the present disclosure. [Figure 2] FIG. 1 is a schematic block diagram of a communication system according to various embodiments of the present disclosure. [Figure 3] FIG. 2 is a schematic block diagram of a decoder according to various embodiments of the present disclosure. [Figure 4]FIG. 2 is a block diagram of an encoder according to various embodiments of the present disclosure. [Figure 5] 1 illustrates an example partition tree according to various embodiments of the present disclosure. [Figure 6] 1A and 1B depict an exemplary quadtree-binary tree (QTBT) structure according to various embodiments of the present disclosure. [Figure 7] (A) and (B) represent a ternary tree (TT) structure according to various embodiments of the present disclosure. [Figure 8] 1 illustrates exemplary merge mode motion vector difference (MMVD) search points according to various embodiments of the present disclosure. [Figure 9] 10 depicts a flowchart of an embodiment of a process for determining a joint motion vector differential (MVD) scaling factor, according to various embodiments of the present disclosure. [Figure 10] 1 illustrates an example computer system according to various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following detailed description of the example embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0012] The foregoing disclosure provides examples and descriptions and is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations. Moreover, one or more features or components of one embodiment may be incorporated into or combined with other embodiments (or one or more features of other embodiments). Moreover, in the flowcharts and operational descriptions provided below, it is understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) concurrently, and the order of one or more operations may be interchanged.

[0013] The systems and / or methods described herein may be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not limiting of implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code. It is understood that software and hardware may be designed to implement the systems and / or methods based on the description herein.

[0014] Although particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of these features may be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim within the scope of that claim.

[0015] No element, act, or instruction used herein should be construed as critical or essential unless expressly described as such. Also, as used herein, the indefinite article (“a” or “an”) is intended to include one or more items and may be used interchangeably with “one or more.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, terms such as “has,” “have,” “including,” and the like are intended to be open-ended terms. Furthermore, the phrase “based on” is intended to mean “based at least in part on,” unless expressly stated otherwise. Furthermore, phrases such as “at least one of [A] and [B]” or “at least one of [A] or [B]” should be understood to include A only, B only, or both A and B.

[0016] References herein to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with the indicated embodiment is included in at least one embodiment of the solution. Thus, appearances of the phrases "in one embodiment," "in an embodiment," and similar language herein may, but do not necessarily, all refer to the same embodiment.

[0017] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the present disclosure can be practiced without relying on one or more of the specific features or advantages of a particular embodiment. In other cases, additional features and advantages may be recognized in particular embodiments that may not be present in all embodiments of the present disclosure.

[0018] 1 illustrates a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. The system (100) may include at least two terminals (110, 120) interconnected via a network (150). In the case of unidirectional data transmission, a first terminal (110) may code video data at a local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the coded video data of the other terminal from the network (150), decode the coded data, and display the recovered video data. Unidirectional data transmission may be common in media serving applications, for example.

[0019] 1 depicts a second pair of terminals (130, 140) provided to support bidirectional transmission of coded video, such as may occur during a video conference. For bidirectional transmission of data, each terminal device (130, 140) may code video data captured at a local location for transmission to the other terminal over the network (150). Each terminal (130, 140) may also receive coded video data transmitted by the other terminal, decode the coded video data, and display the recovered video data on a local display device.

[0020] In FIG. 1 , the terminals (110-140) may be represented as servers, personal computers, smartphones, and / or any other type of terminal. For example, the terminals (110-140) may be laptop computers, tablet computers, media players, and / or dedicated video conferencing devices. The network (150) represents any number of networks that carry coded video data between the terminals (110-140), including, for example, wireline and / or wireless communication networks. The communication network (150) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of the network (150) may be irrelevant to the operation of the present disclosure unless otherwise described hereinafter.

[0021] 2 depicts the arrangement of a video encoder and decoder in a streaming environment as an example of an application of the disclosed subject matter, which may be similarly applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0022] As shown in FIG. 2 , the streaming system (200) may include a video source (201) and a capture subsystem (213) that may include an encoder (203). The video source (201) may be, for example, a digital camera and may be configured to generate an uncompressed video sample stream (202). The uncompressed video sample stream (202) may provide a high data volume compared to an encoded video bitstream and may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof for enabling or implementing aspects of the disclosed subject matter, as described in further detail below. The encoded video bitstream (204) may include a lower data volume compared to the sample stream and may be stored on a streaming server (205) for future use. One or more streaming clients (206) may access the streaming server (205) to retrieve a video bitstream (209), which may be a copy of the encoded video bitstream (204).

[0023] In embodiments, the streaming server (205) may also function as a Media-Aware Network Element (MANE). For example, the streaming server (205) may be configured to prune the encoded video bitstream (204) to tailor potentially different bitstreams to one or more streaming clients (206). In embodiments, a MANE may be separate from the streaming server (205) in the streaming system (200).

[0024] The streaming client (206) may include a video decoder (210) and a display (212). The video decoder (210) may, for example, decode a video bitstream (209), which may be an incoming copy of the encoded video bitstream (204), and generate an outgoing video sample stream (211) that may be rendered on a display (212) or other rendering device (not shown). In some streaming systems, the video bitstreams (204, 209) may be encoded according to a particular video coding / compression standard. Examples of such standards include, but are not limited to, ITU-T Recommendation H.265. A video coding standard commonly known as Versatile Video Coding (VVC) is under development. Embodiments of the present disclosure may be used in conjunction with VVC.

[0025] 3 illustrates an exemplary functional block diagram of a video decoder (210) attached to a display (212) according to an embodiment of the present disclosure. The video decoder (210) may include a channel (312), a receiver (310), a buffer memory (315), an entropy decoder / parser (320), a scaler / inverse transform unit (351), an intra-prediction unit (352), a motion-compensated prediction unit (353), an aggregator (355), a loop filter unit (356), a reference picture memory (357), and a current picture memory (358). In at least one embodiment, the video decoder (210) may include an integrated circuit, a series of integrated circuits, and / or other electronic circuitry. The video decoder (210) may also be embodied, in part or in whole, in software running on one or more CPUs with associated memory.

[0026] In this and other embodiments, the receiver (310) can receive one or more coded video sequences to be decoded by the video decoder (210), one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences can be received from a channel (312), which may be a hardware / software link to a storage device storing the coded video data. The receiver (310) may receive the coded video data along with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to their respective using entities (not shown). The receiver (310) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (315) may be coupled between the receiver (310) and the entropy decoder / parser (320) (hereinafter "parser"). When the receiver 310 is receiving data from a store-and-forward device with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer 315 may not be used or may be small. For use with best-effort packet networks such as the Internet, the buffer 315 may be required and may be relatively large and adaptively sized.

[0027] The video decoder (210) may include a parser (320) that reconstructs symbols (321) from the entropy-coded video sequence. These symbol categories include, for example, information used to manage the operation of the decoder (210) and, potentially, information for controlling a rendering device, such as a display (212), which may be coupled to the decoder (210) as shown in FIG. 2. Control information for the rendering device may take the form of, for example, a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow any video coding technique or standard, including variable length coding, Huffman coding, context-dependent or non-context-dependent arithmetic coding, and other principles well known to those skilled in the art. The parser (320) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (320) may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0028] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer (315) to generate symbols (321). The reconstruction of the symbols (321) may have many different units, depending on the type of coded video picture or portion thereof (e.g., inter and intra picture, inter and intra block) and other factors. Which units are included and how may be controlled by subgroup control information parsed by the parser (320) from the coded video sequence. The flow of such subgroup control information between the parser (320) and the following units is not shown for clarity.

[0029] Beyond the functional blocks already described, the decoder (210) may be conceptually subdivided into a number of functional units, which are described below. In an actual implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0030] One unit can be a scalar / inverse transform unit (351), which may receive quantized transform coefficients as symbols (321) from the parser (320) along with control information including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. The scalar / inverse transform unit (351) may output blocks containing sample values ​​that can be input to an aggregator (355).

[0031] In some cases, the output samples of the scaler / inverse transformer (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but may use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates blocks of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from the current (partially reconstructed) picture from the current picture memory (358). The aggregator (355), in some cases, adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transformer unit (351).

[0032] In other cases, the output samples of the scalar / inverse transform unit (351) may relate to an inter-coded, and potentially motion-compensated, block. In such cases, the motion-compensated prediction unit (353) may access the reference picture memory (357) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (321) related to the block, these samples may be added by the aggregator (355) to the output of the scalar / inverse transform unit (351) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (357) from which the motion-compensated prediction unit (353) fetches the prediction samples may be controlled by a motion vector. The motion vector may be available to the motion-compensated prediction unit (353) in the form of a symbol (321), which may have, for example, X, Y, and reference picture components. Motion compensation can also include interpolation of sample values ​​fetched from the reference picture memory (357) when sub-sample accurate motion vectors are in use, as well as motion vector prediction mechanisms.

[0033] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the coded video bitstream and made available to the loop filter unit (356) as symbols (321) from the parser (320). Note that video compression may also be responsive to meta-information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence, and may also be responsive to previously constructed loop-filtered sample values.

[0034] The output of the loop filter unit (356) can be a sample stream that can be output to the display (212) and further stored in a reference picture memory (357) for use in future inter-picture prediction.

[0035] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. Once a coded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (320)), the current reference picture can become part of the reference picture memory (357), and any unused current picture memory can be reallocated before beginning reconstruction of a subsequent coded picture.

[0036] The video decoder (210) may perform decoding operations according to a predetermined video compression technology, which may be documented in a standard such as ITU-T Recommendation H.265. A coded video sequence may conform to the syntax specified by the video compression technology or standard in use, in the sense that the coded video sequence conforms to the syntax of the video compression technology or standard as specified in the video compression technology document or standard, specifically in a profile document therein. Also, for compliance with some video compression technologies or standards, the complexity of the coded video sequence may be within boundaries defined by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further limited through a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0037] In embodiments, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0038] 4 illustrates an example functional block diagram of a video encoder (203) associated with a video source (201) according to an embodiment of the present disclosure. The video encoder (203) may include, for example, an encoder, such as a source coder (430), a coding engine (432), a (local) decoder (433), a reference picture memory (434), a predictor (435), a transmitter (440), an entropy coder (445), a controller (450), and a channel (460).

[0039] The encoder (203) may receive video samples from a video source (201) (not part of the encoder) that may capture video images to be coded by the encoder (203). The video source (201) may provide a source video sequence to be coded by the encoder (203) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCB, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (201) may be a storage device that stores prepared video. In a video conferencing system, the video source (201) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, impart the appearance of motion. The picture itself may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc. in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following description will focus on samples.

[0040] According to an embodiment, the encoder (203) may code and compress pictures of a source video sequence into a coded video sequence (443) in real time or under any other time constraints required by the application. Imposing an appropriate coding rate is one function of the controller (450). The controller (450) may control and be functionally coupled to other functional units, as described below. Coupling is not shown for clarity. Parameters set by the controller (450) may include parameters related to rate control (picture skip, quantizer, lambda value for rate-distortion optimization techniques, etc.), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. Those skilled in the art will readily identify the functions of the controller (450), as these functions may relate to a video encoder (303) optimized for a particular system design.

[0041] Some video encoders operate in what those skilled in the art will readily recognize as a "coding loop." As an overly simplified description, the coding loop may consist of a source coder (430) (responsible for generating symbols based on an input picture to be coded and a reference picture) and a (local) decoder (433) embedded in the encoder (203). The decoder (433) reconstructs the symbols to generate sample data that a (remote) decoder would also generate if the compression between the symbols and the coded video bitstream is lossless for a particular video compression technique. That reconstructed sample stream may be input to a reference picture memory (434). Because decoding the symbol stream yields bit-exact results independent of the location of the decoder (local or remote), the contents of the reference picture memory are also bit-perfect between the local and remote encoders. In other words, the predictive portion of the encoder "sees" exactly the same sample values ​​as the decoder would "see" when using prediction during decoding. This basic principle of reference picture synchronicity (and the resulting drift when synchronicity cannot be maintained, for example due to channel errors) is known to those skilled in the art.

[0042] The operation of the "local" decoder (433) can be the same as the "remote" decoder (210), already described in detail above in conjunction with Figure 3. However, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (445) and parser (320) can be lossless, the entropy decoding portion of the decoder (210), including the channel (312), receiver (310), buffer (315), and parser (320), may not be fully implemented in the local decoder (433).

[0043] An observation that can be made at this point is that any decoder technology, with the exception of parsing / entropy decoding, that is present in the decoder may need to be present in roughly the same functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on the operation of the decoder. Descriptions of encoder technologies may be omitted, as they may be the inverse of the decoder technology that is generically described. Only to certain extents are more detailed descriptions required, which are provided below.

[0044] As part of its operation, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from the video sequence designated as "reference frames." In this manner, the coding engine (432) codes differences between pixel blocks of the input frame and pixel blocks of the reference frames that may be selected as predictive references for the input frame.

[0045] The local video decoder (433) may decode coded video data of frames that may be designated as reference frames based on symbols generated by the source coder (430). The operation of the coding engine (432) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence is typically a copy of the source video sequence, with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder on the reference frames, causing the reconstructed reference frames to be stored in the reference picture memory (434). In this way, the encoder (403) may locally store copies of reconstructed reference frames that have content in common with the reconstructed reference frames that would be obtained by the far-end video decoder (without transmission errors).

[0046] The predictor (435) may perform a predictive search for the coding engine (432). That is, for a new frame to be coded, the predictor (435) may search the reference picture memory (434) for specific metadata, such as reference picture motion vectors, block shapes, or sample data (as candidate reference pixel blocks) that may serve as suitable prediction references for the new picture. The predictor (435) may operate on a sample block-by-pixel block basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0047] The controller (450) may manage the coding operations of the video coder (430), including, for example, setting parameters and subgroup parameters used to encode the video data. The output of all of the above functional units may undergo entropy coding in the entropy coder (445), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art as Huffman coding, variable length coding, arithmetic coding, etc.

[0048] The transmitter (440) may buffer the coded video sequence generated by the entropy coder (445) to prepare it for transmission over the communication channel (460). The communication channel (460) can be a hardware / software link to a storage device that stores the coded video data. The transmitter (440) may merge the coded video data from the video coder (430) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown). The controller (450) may manage the operation of the encoder (303). During coding, the controller (450) may assign a particular coding picture type to each coded picture, which may affect the coding technique that can be applied to each picture. For example, pictures may often be assigned as an Intra Picture (I-picture), a Predictive Picture (P-picture), or a Bi-directionally Predictive Picture (B-picture).

[0049] An intra picture (I-picture) can be a picture that can be coded and decoded without using any other picture in the sequence as a source of prediction. Some video codecs allow various types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of these variations of I-pictures and their respective applications and characteristics.

[0050] A predicted picture (P picture) can be a picture that can be coded and decoded by intra-prediction or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0051] Bidirectionally predicted pictures (B-pictures) can be pictures that can be coded and decoded using intra- or inter-prediction, which uses at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple-predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0052] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples, respectively) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to each picture of the blocks. For example, blocks of an I-picture may be coded non-predictively, or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be coded non-predictively, or with spatial prediction or temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be coded non-predictively, or with spatial prediction or temporal prediction with reference to one or two previously coded reference pictures.

[0053] The video coder (203) may perform coding operations in accordance with a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. During its operation, the video coder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the coded video data may conform to a syntax defined by the video coding technique or standard being used.

[0054] In embodiments, the transmitter (440) may transmit additional data along with the encoded video. The video coder (430) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0055] Before describing certain aspects of embodiments of the present disclosure in more detail, some terminology that will be referred to in the remainder of the specification is introduced below.

[0056] Hereinafter, "subpicture" refers to a rectangular arrangement of samples, blocks, macroblocks, coding units, or similar entities that may, in some cases, be semantically grouped and coded independently at varying resolutions. One or more subpictures may form a picture. One or more coded subpictures may form a coded picture. One or more subpictures may be assembled into a picture, and one or more subpictures may be extracted from a picture. In certain circumstances, one or more coded subpictures may be assembled into a coded picture in the compressed domain without transcoding to the sample level, and in the same or certain other cases, one or more coded subpictures may be extracted from a coded picture in the compressed domain.

[0057] Hereinafter, "Adaptive Resolution Change" (ARC) refers to a mechanism that enables changing the resolution of pictures or sub-pictures within a coded video sequence, e.g., by reference picture resampling. Hereinafter, "ARC parameters" refers to the control information needed to perform adaptive resolution change, which may include, e.g., filter parameters, scaling factors, output and / or reference picture resolutions, various control flags, etc.

[0058] VP9 uses a four-way partition tree from the 64x64 level down to the 4x4 level, with some additional restrictions for blocks 8x8 and smaller, as shown in the top half of Figure 5. Figure 5 shows the partitioning of a 64x64 block (500). The partition designated as R refers to recursive partitioning, where the same partition tree is repeated at lower scales until the lowest 4x4 level is reached.

[0059] AV1 not only extends the partition tree to the 10-way structure shown in Figure 5, but also increases the maximum size (called superblocks in VP9 / AV1) to start with 128x128 blocks (502). This partitioning includes 4:1 / 1:4 rectangular partitions, which did not exist in VP9. None of the rectangular partitions can be further subdivided. Furthermore, AV1 adds more flexibility to the use of partitions below the 8x8 level, in the sense that 2x2 chroma inter prediction is now possible in certain cases.

[0060] In HEVC, coding tree units (CTUs) may be divided into coding units (CUs) by using a quadtree structure, represented as a coding tree, to adapt to various local characteristics. The decision as to whether to code a picture area using inter-picture (temporal) prediction or intra-picture (spatial) prediction may be made at the CU level. Each CU may be further divided into one, two, or four prediction units (PUs) according to a PU partition type. The same prediction process may be applied within a PU, and related information may be transmitted to the decoder on a PU-by-PU basis. After obtaining a residual block by applying a prediction process based on the PU partition type, the CU may be partitioned into transform units (TUs) according to another quadtree structure, such as the CU's coding tree. One important feature of the HEVC structure is that it has multiple partition concepts, including CUs, PUs, and TUs. In HEVC, CUs or TUs can only have a square shape, while PUs can be square or rectangular in the case of inter-predicted blocks. In HEVC, a coding block may be further divided into four sub-blocks, and a transform may be performed on each sub-block (e.g., TU). Each TU may be further divided recursively (e.g., by quad-tree partitioning) into smaller TUs called Residual Quad-Tree (RQT). At picture boundaries, HEVC may use implicit quad-tree partitioning, whereby a block continues to be quad-tree partitioned until its size fits the picture boundary.

[0061] In HEVC, CTUs may be divided into CUs by using a quadtree structure represented as a coding tree to adapt to various local characteristics. A decision on whether to code a picture area using inter-picture (temporal) prediction or intra-picture (spatial) prediction may be made at the CU level. Each CU may be further divided into one, two, or four PUs according to a PU partition type. The same prediction process may be applied within one PU, and related information may be transmitted to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying a prediction process based on the PU partition type, the CU may be partitioned into transform units (TUs) according to another quadtree structure, such as a CU coding tree. One of the important features of the HEVC structure is that it has multiple partition concepts, including CUs, PUs, and TUs.

[0062] The QTBT structure can eliminate the concept of multiple partition types (e.g., the QTBT structure removes the separation of the concepts of CU, PU, ​​and TU), supporting more flexibility in CU partition shapes. In the QTBT block structure, CUs can be either square or rectangular. As shown in Figures 6(A) and 6(B), coding tree units (CTUs) can be initially partitioned using a quadtree structure. The quadtree leaf nodes can be further partitioned using a binary tree structure. Binary tree partitioning has two partition types: symmetric horizontal partitioning and symmetric vertical partitioning. The binary tree leaf nodes are sometimes called coding units (CUs), and their segmentation can be used for prediction and transform processing without any further partitioning. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. In JEM, CUs sometimes consist of coding blocks (CBs) of different color components. For example, in the case of P slices and B slices in 4:2:0 chroma format, one CU includes one luma CB and two chroma CBs, and sometimes consists of a single component CB. For example, in the case of an I slice, one CU includes only one luma CB or only two chroma CBs.

[0063] The following parameters are defined for the QTBT partitioning scheme: CTU size: Root node size of the quadtree, same concept as HEVC MinQTSize: The minimum allowed quadtree leaf node size MaxBTSize: Maximum allowed binary tree root node size MaxBTDepth: Maximum allowed binary tree depth MinBTSize: The minimum allowable binary tree leaf node size

[0064] In one example of a QTBT partitioning structure, the CTU size may be set as 128x128 luma samples with two corresponding 64x64 blocks of chroma samples, MinQTSize may be set as 16x16, MaxBTSize may be set as 64x64, MinBTSize (for both width and height) may be set as 4x4, and MaxBTDepth may be set as 4. Quad-tree partitioning may first be applied to the CTU to generate quad-tree leaf nodes. The quad-tree leaf nodes can have sizes from 16x16 (e.g., MinQTSize) to 128x128 (e.g., CTU size). If the leaf quad-tree node is 128x128, the node cannot be further divided by the binary tree because its size exceeds MaxBTSize (e.g., 64x64). Otherwise, the leaf quad-tree node may be further partitioned by the binary tree. Therefore, the quadtree leaf node becomes the root node of the binary tree with a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (e.g., 4), no further splits are considered. If a binary tree node has a width equal to MinBTSize (e.g., 4), no further horizontal splits are considered. Similarly, if a binary tree node has a height equal to MinBTSize, no further vertical splits are considered. The binary tree leaf node is further processed by the prediction and transform process without any further partitioning. In JEM, the maximum CTU size is 256x256 luma samples.

[0065] Figure 6(A) shows an example of block partitioning by using QTBT, and Figure 6(B) shows the corresponding tree representation. Solid lines indicate quadtree partitioning, and dotted lines indicate binary tree partitioning. At each partition (e.g., non-leaf) node of the binary tree, one flag can be signaled to indicate which partition type (e.g., horizontal or vertical) is used: 0 indicates horizontal partitioning, and 1 indicates vertical partitioning. In the case of quadtree partitioning, there is no need to indicate the partition type, since quadtree partitioning always divides a block both horizontally and vertically to generate four sub-blocks of equal size.

[0066] Furthermore, the QTBT scheme may support the flexibility for luma and chroma to have separate QTBT structures. Currently, for P slices and B slices, the luma CTB and chroma CTB in one CTU share the same QTBT structure. However, for I slices, the luma CTB may be partitioned into CUs by a QTBT structure, and the chroma CTB may be partitioned into chroma CUs by another QTBT structure. This means that a CU in an I slice can consist of a coding block of a luma component or a coding block of two chroma components, and a CU in a P slice or B slice can consist of coding blocks of all three color components.

[0067] In HEVC, inter prediction of small blocks may be restricted to reduce memory access for motion compensation, so that bi-prediction is not supported for 4x8 and 8x4 blocks, and inter prediction is not supported for 4x4 blocks. In QTBT implemented in JEM-7.0, these restrictions may be removed.

[0068] VVC includes a Multi-type-tree (MTT) structure, which adds horizontal and vertical center-side ternary trees in addition to the QTBT, as shown in Figures 7(A) and (B).

[0069] Important advantages of ternary tree partitioning include: (i) the complementarity of quadtree and binary tree partitioning: ternary tree partitioning can capture objects located at block centers, while quadtree and binary trees always split along block centers, and (ii) the width and height of the partitions in the proposed ternary tree are always powers of two, thereby eliminating the need for additional transformations. The design of the two-level tree is primarily motivated by reduced complexity: the complexity of traversing the tree is T D where T represents the number of split types and D is the tree depth.

[0070] In addition to the merge mode in which implicitly derived motion information is directly used to generate predicted samples for the current CU, a merge mode with motion vector differences (MMVD) is introduced in VVC. An MMVD flag can be signaled immediately after sending the skip flag and merge flag to specify whether the MMVD mode is used for the CU. In MMVD, after a merge candidate is selected, the merge candidate is further refined by the signaled MVD information. This further information can include a candidate flag, an index specifying the magnitude of the motion, and an index indicating the direction of the motion. In the MMVD mode, one of the first two candidates in the merge list can be selected to be used as the MV basis. A merge candidate flag can be signaled to specify which one is to be used.

[0071] The distance index specifies the motion magnitude information and indicates a predefined offset from the starting point. As shown in Figure 8, the offset is added to either the horizontal or vertical component of the starting MV in the L0 reference picture (800) and the L1 reference picture (802). The relationship between the distance index and the predefined offset is specified in Table 1. [Table 1]

[0072] The direction index can represent the direction of the MVD relative to the starting point. The direction index can represent one of four directions shown in Table 2, which indicates the sign of the MV offset specified by the direction index. [Table 2]

[0073] The meaning of the MVD code can vary according to the information of the starting MV. When the starting MV is a uni-predictive MV or a bi-predictive MV where both lists point to the same side of the current picture (e.g., the POCs of the two references are both greater than that of the current picture, or both less than that of the current picture), the code in Table 2 may specify the sign of the MV offset added to the starting MV. When the starting MV is a bi-predictive MV with two MVs pointing to different sides of the current picture (e.g., the POC of one reference is greater than that of the current picture, and the POC of the other reference is less than that of the current picture), and the difference of the POCs in list 0 is greater than that in list 1, the code in Table 2 may specify the sign of the MV offset added to the list 0 MV component of the starting MV, and the code of the list 1 MV has the opposite value. Otherwise, if the POC difference in List 1 is greater than List 0, then the code in Table 2 can specify the encoding of the MV offset added to the List 1 MV component of the starting MV, and the code of the List 0 MV has the opposite value.

[0074] The MVD may be scaled according to the difference in POC in each direction. If the difference in POC in both lists is the same, no scaling is necessary. Otherwise, if the difference in POC in list 0 is greater than that of list 1, the MVD of list 1 may be scaled. If the POC difference of L1 is greater than that of L0, the MVD of list 0 may be scaled in the same way. If the starting MV is unidirectionally predicted, the MVD may be added to the available MV.

[0075] In VVC, in addition to the normal unidirectional prediction and bidirectional prediction mode MVD signaling, a symmetric MVD mode for bidirectional MVD signaling can be applied. In the symmetric MVD mode, the motion information including the reference picture indexes of both list 0 and list 1 and the MVD of list 1 is not signaled, but can be derived instead. The decoding process of the symmetric MVD mode is as follows:

[0076] First, at the slice level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 can be derived as follows: If mvd_l1_zero_flag is 1, BiDirPredFlag is set equal to 0. Otherwise, if the closest reference picture in list 0 and the closest reference picture in list 1 form a forward and backward pair of reference pictures or a backward and forward pair of reference pictures, BiDirPredFlag is set to 1 and the reference pictures in both list 0 and list 1 are short-term reference pictures. Otherwise, BiDirPredFlag is set to 0.

[0077] Second, at the CU level, a symmetric mode flag indicating whether symmetric mode is used or not is explicitly signaled if the CU is bi-predictively coded and BiDirPredFlag is equal to 1.

[0078] When the symmetric mode flag is true, only mvp_l0_flag, mvp_l1_flag, and MVD0 may be explicitly signaled. The reference indexes for list 0 and list 1 may be set equal to a pair of reference pictures, respectively. MVD1 may be set equal to (-MVD0).

[0079] In AV1, for each coded block in an inter frame, if the mode of the current block is an inter coding mode rather than a skip mode, another flag may be signaled to indicate whether a single reference mode or a mixed reference mode is used for the current block. In a single reference mode, a predictive block may be generated by one motion vector, while in a mixed reference mode, a predictive block may be generated by a weighted average of two predictive blocks derived from two motion vectors.

[0080] In the case of a single reference, one or more of the following modes may be signaled: NEARMV-DRL (Dynamic Reference List) uses one of the motion vector predictors (MVP) in the list indicated by the index. · Use one of the motion vector predictors (MVPs) in the list signaled by the NEWMV-DRL index as a reference and apply the delta to the MVP. · GLOBALMV - Use motion vectors based on frame-level global motion parameters.

[0081] In case of mixed reference mode, the following modes can be signaled: · NEAR_NEARMV - Use one of the motion vector predictors (MVP) in the list signaled by the DRL index. · NEAR_NEWMV - Transmits a delta MV for the second MV using one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference. · NEW_NEARMV - Transmits the delta MV of the first MV using one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference. · NEW_NEWMV - Use one of the motion vector predictors (MVPs) in the list signaled by the DRL index as a reference and send the delta MVs for both MVs. · GLOBAL_GLOBALMV - Use the MV from each reference based on their frame-level global motion vectors.

[0082] AV1 allows for 1 / 8 pixel motion vector precision (or accuracy), and the following syntax can be used to signal motion vector differentials within reference frame list 0 or list 1: mv_joint specifies which components of the motion vector differential are non-zero 0 indicates that there is no non-zero MVD along either the horizontal or vertical direction. 1 indicates that there is non-zero MVD only along the horizontal direction. 2 indicates that there is non-zero MVD only along the vertical direction. 3 indicates that there is non-zero MVD along both the horizontal and vertical directions. · mv_sign specifies whether the motion vector differential is positive or negative. · mv_class specifies the class of the motion vector difference. · mv_bit specifies the integer part of the offset between the motion vector differential and the starting magnitude of each MV class. · mv_fr specifies the first two fractional bits of the motion vector differential. · mv_hp specifies the third fractional bit of the motion vector differential.

[0083] As shown in Table 3, Table 3 indicates the magnitude class of the MVD, the higher the class, the larger the magnitude of the motion vector difference will be. [Table 3]

[0084] For NEW_NEARMV and NEAR_NEWMV modes, the accuracy of the MVD may depend on the associated class and the magnitude of the MVD. First, fractional MVD is only allowed if the MVD magnitude is 1 pixel or less. Second, only one MVD value is allowed if the associated MV class value is MV_CLASS_1 or greater, and the MVD value for each MV class is derived as 4, 8, 16, 32, or 64 for MV class 1 (MV_CLASS_1), 2 (MV_CLASS_2), 3 (MV_CLASS_3), 4 (MV_CLASS_4), or 5 (MV_CLASS_5). Furthermore, if the current block is coded as NEW_NEWMV or NEAR_NEWMV mode, one context may be used to signal mv_joint or mv_class. Otherwise, another context may be used to signal mv_joint or mv_class.

[0085] The permitted MVD values ​​for each MV class are presented in Table 4. [Table 4]

[0086] A new inter-coding mode called JOINT_NEWMV may be applied to indicate whether the MVDs of two reference lists are signaled together. When the inter-prediction mode is equal to JOINT_NEWMV mode, the MVDs of reference list 0 and reference list 1 may be signaled together. Therefore, under this mode, only one MVD called joint_mvd is signaled and transmitted to the decoder, and the delta MVs of reference list 0 and reference list 1 may be derived from joint_mvd. The JOINT_NEWMV mode may be signaled together with NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, and GLOBAL_GLOBALMV modes. No additional context is added.

[0087] When JOINT_NEWMV mode is signaled and the POC distances between two reference frames and the current frame are different, the MVD may be scaled for reference list 0 or reference list 1 based on the POC distances. Specifically, the distance between reference frame list 0 and the current frame is denoted as td0, and the distance between reference frame list 1 and the current frame is denoted as td1. If td0 is greater than or equal to td1, joint_mvd may be used directly for reference list 0, and the mvd of reference list 1 may be derived from joint_mvd as follows:

number

[0088] Otherwise, if td1 is greater than or equal to td0, then joint_mvd is used directly for reference list 1, and the mvd for reference list 0 is derived from joint_mvd as follows:

number

[0089] A new inter-coding mode called AMVDMV is added to the single-reference case. When the AMVDMV mode is selected, this mode may indicate that AMVD is applied to a single MVD. A flag called amvd_flag may be added under the JOINT_NEWMV mode to indicate whether AMVD is applied to the joint MVD coding mode. When adaptive MVD decomposition is applied to the joint MVD coding mode, also called the joint AMVD coding mode, the MVDs of two reference frames may be signaled together, and the accuracy of the MVD may be implicitly determined by the size of the MVD. Otherwise, the MVDs of two (or more) reference frames may be signaled together, and conventional MVD coding is applied.

[0090] AMVR was first proposed in CWG-C012, where a total of seven MV precisions (8, 4, 2, 1, 1 / 2, 1 / 4, 1 / 8) are supported. For each precision block, the AVM encoder explores all supported precision values ​​and signals the best precision to the decoder. To reduce encoder runtime, two precision sets are supported. Each precision set may contain four predefined precisions. The precision set may be adaptively selected at the frame level based on the frame's maximum precision value. As with AV1, the maximum precision may be signaled in the frame header. Table 5 summarizes the supported precision values ​​based on the frame-level maximum precision. [Table 5]

[0091] In the current AMV software (similar to AV1), there is a frame-level flag to indicate whether the MV of a frame contains sub-pel precision or not. AMVR is only enabled if the value of the cur_frame_force_integer_mv flag is 0. In AMVR, if the block precision is less than the maximum precision, the motion model and interpolation filter are not signaled. If the block precision is less than the maximum precision, the motion mode can be inferred as translation motion and the interpolation filter can be inferred as a regular interpolation filter. Similarly, if the block precision is either 4-pel or 8-pel, the inter-intra mode is not signaled and can be inferred as 0.

[0092] In the joint MVD coding mode (JMVD), there is an assumption that linear motion exists between the backward reference frame and the forward reference frame. When the joint MVD coding mode is selected for a block, one joint MVD can be signaled for both reference frames, and the MVDs of the two reference frames can be derived from the joint MVD based on the distance between the reference frame and the current frame. However, the motion between the two reference frames may not always be linear. For example, the motion may be slower or faster from the backward reference frame to the forward reference frame.

[0093] In the case of AMVD-based JMVD, MVD can be applied only along either the horizontal or vertical direction. However, in the case of conventional JMVD, MVD can be applied along both the horizontal and vertical directions. Therefore, sharing the same scaling factor between AMVD-based JMVD and conventional JMVD is suboptimal.

[0094] Embodiments of the present disclosure are directed to a set of advanced image and video coding techniques. More specifically, embodiments of the present disclosure are directed to inter-prediction mode-dependent scaling factors. The embodiments of the present disclosure may be used separately or combined in any order. Furthermore, each of the embodiments utilizing an encoder or decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium. Hereinafter, the term block may be interpreted as a prediction block, a coding block, or a coding unit (i.e., CU). The term block may also be used to refer to a transform block.

[0095] The orientation of the reference frame may be determined by whether the reference frame precedes the current frame in standard order or follows the current frame in standard order. The terms x-axis and y-axis may refer to the horizontal and vertical components of a two-dimensional value, and may be replaced by two other axes along two predefined directions that are orthogonal to each other, and the same embodiments are applicable to these other axes. For example, the x-axis and y-axis may be replaced by a 45-degree axis and a 135-degree axis, respectively. Conventional JMVD may refer to JMVD using normal full MV decomposition or JMVD using AMVR.

[0096] In some embodiments, when MVDs are jointly signaled for multiple reference frames in one block, flexible scaling factors are used to derive the MVDs for reference frame lists 0 and / or 1 from the signaled joint MVD based on the signaled scaling factors in the bitstream or the coded information of the current block (or neighboring blocks). The allowed / supported scaling factors may be different for JMVD based on AMVD and conventional JMVD with different MV precisions.

[0097] In some embodiments, the values ​​of the scaling factors are restricted to powers of two, such as 1, 2, 4, 1 / 2, 1 / 4, etc.

[0098] In some embodiments, for JMVD modes based on AMVD, the same scaling factor is applied to both the vertical and horizontal components of the MVD of reference frame lists 0 and / or 1.

[0099] In some embodiments, one predefined lookup table may be used to store the supported / allowed scaling factors, and the associated entry index for the selected scaling factor in the lookup table is signaled in the bitstream. Table 6 represents an example of a lookup table that specifies the scaling factor for each index. [Table 6]

[0100] In some embodiments, the AMVD JMVD scaling factors may be derived from the conventional JMVD scaling factors, i.e., if AMVD is applied along the x-axis, the x-axis scaling factor of the conventional JMVD is used, otherwise, if AMVD is applied along the y-axis, the y-axis scaling factor of the conventional JMVD is used.

[0101] In some embodiments, the AMVD JMVD scaling factors may be a subset of the scaling factors extracted from conventional JMVD. For example, if conventional JMVD mode is selected for a block, different scaling factors may be applied separately to the vertical or horizontal components of the MVD of reference frame lists 0 and / or 1.

[0102] In some embodiments, one predefined lookup table may be used to store supported / allowed scaling factor pairs for the vertical and horizontal components of MVD in conventional JMVD mode, and the associated entry index for a selected scaling factor pair in the lookup table may be signaled in the bitstream. Table 7 shows an example of a table where separate scaling factors are specified for each of the x-axis and y-axis. [Table 7]

[0103] In some embodiments, the scaling factor of one component of the MVD (either the vertical or horizontal component) may be restricted to be 1, while the scaling factor of the other component of the MVD can be other values ​​such as 2 or 1 / 2. The supported scaling factor pairs are also stored in one lookup table, and the associated entry index for the selected scaling factor pair in the lookup table is signaled in the bitstream. Table 8 shows an example of a lookup table where the scaling factor for one of the x-axis and y-axis is restricted to 1. [Table 8]

[0104] In some embodiments, the context for signaling the scaling factor may depend on the inter-prediction mode information of the current block. For example, if the current block is coded as a JMVD mode based on AMVD, one context may be used to signal the scaling factor of the current block. Otherwise, another context may be used to signal the scaling factor of the current block. In another example, if the current block is coded as a conventional JMVD mode, one context may be used to signal the scaling factor of the current block when AMVR is applied, and another context is used when AMVR is not applied.

[0105] In some embodiments, the context for signaling the scaling factor depends on the distance between the current frame and its reference frame. For example, if the distance between the reference frame and the current frame is equal to or greater than a threshold, one context may be used. Otherwise, another context may be used. As an example, the threshold is set to 4 or 8.

[0106] In some embodiments, the MV precision of the current block may be used as a context for signaling the scaling factor. For example, one context may be used to signal the index of the scaling factor if the MV precision is equal to or greater than a threshold value. Otherwise, another context may be used instead. As an example, the threshold value may be set to 1 pel.

[0107] In some embodiments, the indices of the scaling factors may be signaled only if the current block is coded as a JMVD mode based on AMVD. In some embodiments, the indices of the scaling factors are signaled only if optical flow refinement is applied to the current block. In some embodiments, if the indices of the scaling factors indicate that at least one of the scaling factors is not equal to 1, the interpolation filter type is not signaled in the bitstream and is derived as one default interpolation filter type. For example, the default interpolation filter type may be one of the REGULAR, SMOOTH, or SHARP filters defined in AV1.

[0108] In some embodiments, if the scaling factor index indicates that at least one of the scaling factors is not equal to 1, only a subset of the MV precisions may be allowed or signaled in the bitstream. For example, if the scaling factor index indicates that at least one of the scaling factors is not equal to 1, the MV precision of the current block is finer than one threshold, such as 2 pels. In another example, if the scaling factor index indicates that at least one of the scaling factors is not equal to 1, the MV precision of the current block is not signaled but is derived as one default value. In another example, if the scaling factor index indicates that at least one of the scaling factors is not equal to 1, the MV precision of the current block is not signaled but may be derived as a frame-level MV precision or a superblock-level MV precision. In some embodiments, the scaling factor index may be used as a context for signaling the MV precision of the current block when the current block is coded as a joint MVD mode.

[0109] 9 depicts a flowchart of an embodiment of a process (900) for determining a scaling factor for a joint MVD. The process (900) may be performed by a decoder, such as the decoder (210). The process may begin at operation (902) in which a coded video bitstream is received. The bitstream may include a current picture, a first reference picture, a second reference picture, and a joint MVD associated with at least one block in the current picture.

[0110] The process may proceed to operation (904), where it is determined whether one or more scaling factors are to be used for the JMVD based on the AMVD based on a predetermined condition, such as a flag included in the received bitstream. The process may proceed to operation (906), where, in response to determining that one or more scaling factors are to be used for the AMVD JMVD, the JMVD is scaled based on the one or more scaling factors. As an example, the scaling factors are determined based on index values ​​included in the bitstream that index into a lookup table. The lookup table may specify scaling factors to be applied to both the x-axis and the y-axis, or the lookup table may specify separate scaling factors to be applied to both the x-axis and the y-axis.

[0111] The process proceeds to operation (908), where a first MVD of the first reference picture is derived based on the scaled joint MVD. As an example, the first MVD may be determined as shown in equation (1). The process proceeds to operation (910), where a second MVD of the second reference picture is derived based on the derived first MVD. As an example, the second MVD may be determined as shown in equation (2). The process proceeds to operation (912), where at least one block is decoded based on the derived first MVD and the derived second MVD.

[0112] The techniques of the embodiments of the present disclosure described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 10 illustrates a computer system (1000) suitable for implementing embodiments of the disclosed subject matter.

[0113] The computer software may be coded in any suitable machine code or computer language that may be amenable to mechanisms such as assembly, compilation, linking, etc. to generate code containing instructions that may be executed by a central processing unit (CPU), graphics processing unit (GPU), etc. directly or through interpretation, microcode execution, etc.

[0114] The instructions may be executed by various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming consoles, Internet of Things devices, and the like.

[0115] 10 for computer system 1000 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement regarding any one or combination of components described in the exemplary embodiment of computer system 1000.

[0116] The computer system 1000 may include certain human interface input devices. Such human interface input devices may respond to input by one or more users through, for example, tactile input (e.g., keyboard, swipe, dataglove motion), audio input (e.g., voice, claps), visual input (e.g., gestures), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0117] The input human interface devices may include one or more of a keyboard (1001), a mouse (1002), a trackpad (1003), a touchscreen (1010), a data glove, a joystick (1005), a microphone (1006), a scanner (1007), and a camera (1008) (only one of each is shown).

[0118] The computer system 1000 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen 1010, data gloves, or joystick 1005; however, haptic feedback devices that do not function as input devices may also exist), audio output devices (e.g., speakers 1009, headphones (not shown)), visual output devices (e.g., CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capabilities and each with or without haptic feedback capabilities, some of which may provide two-dimensional visual output or output in more than three dimensions via means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0119] The computer system (1000) may also include human-accessible storage devices and their associated media, such as CD / DVD ROM / RW (1020), including CD / DVD or similar media (1021), thumb drives (1022), removable hard disks or solid state drives (1023), legacy magnetic media, such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices, such as security dongles (not shown), and the like.

[0120] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0121] The computer system 1000 may also include interfaces to one or more communication networks. Networks may be, for example, wireless, wireline, or optical. Networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet; wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, and the like; TV wireline or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and vehicle and factory networks including CAN bus. Certain networks generally require an external network interface adapter attached to a particular general-purpose digital port or peripheral bus 1049 (e.g., a USB port on the computer system 1000). Others are generally built into the core of the computer system 1000 by attachment to a system bus, as described below (e.g., an Ethernet network interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 1000 can communicate with other entities. Such communications may be one-way receive-only (e.g., broadcast TV) or one-way transmit-only (e.g., a CAN bus to a particular CAN bus device), or may be two-way to other computer systems, for example, using local or wide-area digital networks. Such communications may include communications to a cloud computing environment (1055). Specific protocols or protocol stacks may be used with each of the networks and network interfaces described above.

[0122] The above-mentioned human interface devices, human-accessible storage devices, and network interface (1054) may be attached to the core (1040) of the computer system (1000).

[0123] The core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1043), task-specific hardware accelerators (1044), etc. These devices may be connected through a system bus (1048), along with read-only memory (ROM) (1045), random access memory (RAM) (1046), internal mass storage devices such as internal non-user-accessible hard drives, SSDs, etc. (1047). In some computer systems, the system bus (1048) may be accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1048) or through a peripheral bus (1049). Architectures for peripheral buses include PCI, USB, etc. A graphics adapter (1050) may be included in the core (1040).

[0124] The CPU (1041), GPU (1042), FPGA (1043), and accelerator (1044) can execute specific instructions that, in combination, can constitute the above-mentioned computer code. The computer code can be stored in ROM (1045) or RAM (1046). Temporary data can also be stored in RAM (1046), while persistent data can be stored, for example, in an internal mass storage device (1047). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory. Cache memory can be closely associated with one or more of the CPU (1041), GPU (1042), mass storage device (1047), ROM (1045), RAM (1046), etc.

[0125] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those skilled in the computer software arts.

[0126] By way of example, and not limitation, a computer system having the architecture (1000), and in particular the core (1040), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with the user-accessible mass storage devices previously introduced, in addition to specific storage of the core (1040) that is non-transitory in nature, such as the core's internal mass storage device (1047) or ROM (1045). Software implementing various embodiments of the present disclosure can be stored in such devices and executable by the core (1040). The computer-readable media can include one or more memory devices or chips, depending on particular needs. Software can cause the core (1040), and specifically the processor (including CPU, GPU, FPGA, etc.) therein, to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM (1046) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system can provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1044)) that can operate in place of or in conjunction with software to perform particular processes or particular portions of particular processes described herein. References to software can encompass logic, where appropriate, and vice versa. References to computer-readable media can encompass circuitry (e.g., integrated circuits (ICs)) storing software for execution, circuitry embodying logic for execution, or both, where appropriate. The present disclosure encompasses any appropriate combination of hardware and software.

[0127] The above disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

[0128] It is understood that the specific order or hierarchy of blocks within the processes / flowcharts disclosed herein is a description of example approaches. Based on design preferences, it is understood that the specific order or hierarchy of blocks within the processes / flowcharts may be rearranged. Additionally, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in a sample order and are not meant to be limited to the specific order or hierarchy presented.

[0129] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail of integration. Furthermore, one or more of the above components may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or medium) having computer-readable program instructions for causing a processor to perform operations.

[0130] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded thereon, and any suitable combination thereof. As used herein, a computer-readable storage medium should not be construed as being a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0131] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0132] The computer-readable program code / instructions for carrying out operations may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, such as a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform aspects or operations.

[0133] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to create a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, generate means for implementing the functions / acts specified in the flowcharts and / or one or more blocks of the block diagrams. These computer-readable program instructions can be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored constitutes an article of manufacture including instructions that implement aspects of the functions / acts specified in the flowcharts and / or one or more blocks of the block diagrams.

[0134] The computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps, whereby the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowcharts and / or one or more blocks of the block diagrams.

[0135] The flowcharts and block diagrams in the figures represent the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks than those shown in the figures. In some alternative implementations, the functions shown in the blocks may be performed out of the order shown in the figures. For example, two blocks shown in succession may actually be executed concurrently or nearly concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and a computer program.

[0136] It will be apparent that the systems and / or methods described herein can be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specific control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. Thus, although the operation and behavior of the systems and / or methods have been described herein without reference to specific software code, it will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0137] Appendix A: Acronyms AV1:AOMedia Video 1 HEVC:High Efficiency Video Coding MVD: Motion vector difference MVP: Motion vector prediction VVC: Versatile Video Coding CfL: Chroma from Luma SDT: Semi Decoupled Tree SDP: Semi Decoupled Partitioning SST: Semi Separate Tree SB: Super Block IBC(or IntraBC): Intra Block Copy CDF: Cumulative Density Function SCC: Screen Content Coding GBI: Generalized Bi-prediction BCW: Bi-prediction with CU-level weights CIIP: Combined intra-inter prediction POC: Picture Order Count DRL: Dynamic Reference List OBMC: Overlapped block motion compensation AMVD: adaptive MVD resolution AMVR: adaptive motion vector resolution JMVD: Joint coding of motion vector difference(or joint MVD coding) MVP0: motion vector prediction for reference frame list 0 MVP1: motion vector prediction for reference frame list 1

[0138] The above disclosure also includes the embodiments listed below.

[0139] (1) A method performed by at least one processor of a video decoder, comprising: receiving a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector differential (JMVD) associated with at least one block in the current picture; determining whether one or more scaling factors are used in adaptive motion vector differential decomposition (AMVD)-based JMVD based on a predetermined condition; scaling the JMVD based on the one or more scaling factors in response to determining that the one or more scaling factors are to be used in a JMVD based on the AMVD; deriving a first MVD of the first reference picture based on the scaled JMVD; deriving a second MVD of the second reference picture based on the derived first MVD; reconstructing the at least one block based on the derived first MVD and the derived second MVD; A method having the following.

[0140] (2) in response to determining that the one or more scaling factors are to be used in a Joint Multiplication and Decomposition (JMVD) based on the AMVD, the one or more scaling factors are limited to powers of two; The method according to feature (1).

[0141] (3) in response to determining that the one or more scaling factors are to be used in a Joint Multimedia Video Display (JMVD) based on the AMVD, the one or more scaling factors include the same scaling factor for a vertical component and a horizontal component of the JMVD; The method according to feature (1).

[0142] (4) the coded video bitstream further includes an index value; the one or more scaling factors are determined by using a lookup table that specifies one or more predetermined scaling factors based on the index value. The method according to feature (3).

[0143] (5) the one or more predetermined scaling factors include a first scaling factor for a horizontal component of the JMVD and a second scaling factor for a vertical component of the JMVD; The method according to feature (4).

[0144] (6) the first scaling factor is different from the second scaling factor; The method according to feature (5).

[0145] (7) one of the first scaling coefficient and the second scaling coefficient is limited to 1, and the other of the first scaling coefficient and the second scaling coefficient is a value other than 1; The method according to feature (6).

[0146] (8) determining that the one or more scaling factors are to be used in the JMVD based on the AMVD based on a flag in the coded video bitstream indicating that the one or more scaling factors are to be used in the JMVD based on the AMVD; and in response to determining that the one or more scaling factors are to be used in a Joint Music Video Decoder (JMVD) based on the AMVD, applying the same one or more predetermined scaling factors to both horizontal and vertical components of the JMVD. The method according to feature (5).

[0147] (9) determining that the one or more scaling factors are to be used in the AMVD-based JMVD based on a flag in the coded video bitstream indicating that the AMVD-based JMVD is applied to a horizontal component of the JMVD; the JMVD based on the AMVD utilizes the one or more scaling factors applied to the horizontal component of the JMVD; The method according to feature (1).

[0148] (10) determining that the one or more scaling factors are to be used in the AMVD-based JMVD based on a flag in the coded video bitstream indicating that the AMVD-based JMVD is applied to a vertical component of the JMVD; the JMVD based on the AMVD utilizes the one or more scaling factors applied to the vertical component of the JMVD; The method according to feature (1).

[0149] (11) A video decoder, at least one memory configured to store computer program code; at least one processor configured to access the computer program code and to operate as directed by the computer program code; and The computer program code receiving code configured to cause the at least one processor to receive a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector differential (JMVD) associated with at least one block in the current picture; decision code configured to cause the at least one processor to determine, based on a predetermined condition, whether one or more scaling factors are used in adaptive motion vector differential decomposition (AMVD)-based JMVD; scaling code configured to cause the at least one processor to scale the JMVD based on the one or more scaling factors in response to determining that the one or more scaling factors are to be used in a JMVD based on the AMVD; first derivation code configured to cause the at least one processor to derive a first MVD for the first reference picture based on the scaled JMVD; second derivation code configured to cause the at least one processor to derive a second MVD of the second reference picture based on the derived first MVD; reconstruction code configured to cause the at least one processor to reconstruct the at least one block based on the derived first MVD and the derived second MVD; a video decoder.

[0150] (12) in response to determining that the one or more scaling factors are to be used in a Joint Multiplication and Decomposition (JMVD) based on the AMVD, the one or more scaling factors are limited to powers of two; A video decoder according to feature (11).

[0151] (13) in response to determining that the one or more scaling factors are to be used in a Joint Multimedia Video Display (JMVD) based on the AMVD, the one or more scaling factors include the same scaling factor for a vertical component and a horizontal component of the JMVD; A video decoder according to feature (11).

[0152] (14) The coded video bitstream further includes an index value; the one or more scaling factors are determined by using a lookup table that specifies one or more predetermined scaling factors based on the index value. A video decoder according to feature (13).

[0153] (15) The one or more predetermined scaling factors include a first scaling factor for a horizontal component of the JMVD and a second scaling factor for a vertical component of the JMVD. A video decoder according to feature (14).

[0154] (16) the first scaling factor is different from the second scaling factor; A video decoder according to feature (15).

[0155] (17) One of the first scaling coefficient and the second scaling coefficient is limited to 1, and the other of the first scaling coefficient and the second scaling coefficient is a value other than 1. 16. A video decoder according to claim 15.

[0156] (18) determining that the one or more scaling factors are to be used in the JMVD based on the AMVD based on a flag in the coded video bitstream indicating that the one or more scaling factors are to be used in the JMVD based on the AMVD; and in response to determining that the one or more scaling factors are to be used in a Joint Music Video Decoder (JMVD) based on the AMVD, applying the same one or more predetermined scaling factors to both horizontal and vertical components of the JMVD. A video decoder according to feature (15).

[0157] (19) Determining that the one or more scaling factors are used in the AMVD-based JMVD is based on a flag in the coded video bitstream indicating that the AMVD-based JMVD is applied to a horizontal component of the JMVD; the JMVD based on the AMVD utilizes the one or more scaling factors applied to the horizontal component of the JMVD; 12. A video decoder according to claim 11.

[0158] (20) A non-transitory computer-readable medium storing instructions, The instructions, when executed by a processor of a video decoder, cause the processor to: receiving a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector differential (JMVD) associated with at least one block in the current picture; determining whether one or more scaling factors are used in adaptive motion vector differential decomposition (AMVD)-based JMVD based on a predetermined condition; scaling the JMVD based on the one or more scaling factors in response to determining that the one or more scaling factors are to be used in a JMVD based on the AMVD; deriving a first MVD of the first reference picture based on the scaled JMVD; deriving a second MVD of the second reference picture based on the derived first MVD; reconstructing the at least one block based on the derived first MVD and the derived second MVD; 10. A non-transitory computer-readable medium for performing a method having the steps of:

[0159] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Patent Application No. 63 / 390,910, filed July 20, 2022, and U.S. Patent Application No. 17 / 982,946, filed November 8, 2022, the disclosures of which are incorporated herein by reference in their entireties.

Claims

[Claim 1] 1. A method performed by at least one processor of a video decoder, comprising: receiving a coded video bitstream including a current picture, a first reference picture, a second reference picture, and a joint motion vector differential (JMVD) associated with at least one block in the current picture; determining whether one or more scaling factors are used in adaptive motion vector differential decomposition (AMVD)-based JMVD based on a predetermined condition; scaling the JMVD based on the one or more scaling factors in response to determining that the one or more scaling factors are to be used in a JMVD based on the AMVD; deriving a first MVD of the first reference picture based on the scaled JMVD; deriving a second MVD of the second reference picture based on the derived first MVD; reconstructing the at least one block based on the derived first MVD and the derived second MVD; A method having the following.

Images