Method for performing sub-block-based inter prediction and non-transitory computer-readable storage medium

Abstract

The present disclosure provides a video decoding method using one-subblock temporal motion prediction (sbAmvp mode) for a conventional inter mode. The method comprises: receiving a bitstream, the bitstream comprising one or more syntax elements signaling coding unit (CU) level motion information associated with a coding unit (CU), where the CU is encoded using a conventional inter-frame mode; dividing the CU into a plurality of sub-blocks; and performing the sbAmvp mode on the plurality of sub-blocks. Performing the sbAmvp mode includes deriving motion information for a sub-block of the plurality of sub-blocks based on the signaled CU level motion information.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This disclosure claims priority to U.S. Provisional Application No. 63 / 381,146, filed on October 27, 2022; U.S. Provisional Application No. 63 / 477,569, filed on December 29, 2022; U.S. Provisional Application No. 63 / 493,301, filed on March 30, 2023; and U.S. Provisional Application No. 63 / 587,190, filed on October 2, 2023, and U.S. Patent Application No. 18 / 487,387, filed on October 16, 2023. The entire contents of all of the foregoing applications are incorporated herein by reference. Technical Field

[0002] The present disclosure relates generally to video processing and, more particularly, to methods and systems for performing sub-block based inter prediction. Background Art

[0003] A video is a set of static images (or "frames") that capture visual information. To reduce storage memory and transmission bandwidth, videos can be compressed before storage or transmission, and decompressed before display. The compression process is often referred to as encoding, while the decompression process is often referred to as decoding. There are various video coding formats that use standardized video coding techniques, the most common of which are based on prediction, transform, quantization, entropy coding, and in-loop filtering. Video coding standards, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Versatile Video Coding (VVC / H.266) standard, and the AVS standard, are developed by standardization organizations to specify specific video coding formats. As video standards adopt more and more advanced video coding technologies, the coding efficiency of new video coding standards is getting higher and higher. Summary of the invention

[0004] The present disclosure provides a video decoding method using a sub-block temporal motion prediction (sbAmvp mode) for a conventional inter-frame mode. The method includes: receiving a bitstream, the bitstream including one or more syntax elements that signal coding unit (CU) level motion information associated with a CU, wherein the CU is encoded using a conventional inter-frame mode; dividing the CU into a plurality of sub-blocks; and performing the sbAmvp mode on the plurality of sub-blocks. Performing the sbAmvp mode includes: deriving motion information of a sub-block among the plurality of sub-blocks based on the signaled CU-level motion information.

[0005] In some embodiments, a method for video decoding using a sub-block temporal motion prediction (sbAmvp mode) for a conventional inter-frame mode is provided. The method includes: receiving a video sequence; encoding one or more images of the video sequence; and generating a bitstream. The encoding includes: encoding a coding unit (CU) using a conventional inter-frame mode; signaling CU-level motion information associated with the CU; dividing the CU into a plurality of sub-blocks; and performing the sbAmvp mode on the plurality of sub-blocks. Performing the sbAmvp mode includes deriving motion information of a sub-block of the plurality of sub-blocks based on the signaled CU-level motion information.

[0006] In some embodiments, a non-transitory computer-readable storage medium stores a bitstream generated by the following operations, including: encoding a coding unit (CU) using a conventional inter-frame mode; signaling CU-level motion information associated with the CU; dividing the CU into a plurality of sub-blocks; and performing an sbAmvp mode on the plurality of sub-blocks. Performing the sbAmvp mode includes deriving motion information of a sub-block of the plurality of sub-blocks based on the signaled CU-level motion information.

[0007] In some embodiments, a video encoding device is provided, which includes: a memory configured to store an instruction set; and one or more processors configured to execute the instruction set, so that the device performs the method according to the above embodiment.

[0008] In some embodiments, a video decoding device is provided. The video compression device comprises: a memory configured to store an instruction set; and one or more processors configured to execute the instruction set, so that the device performs the method according to the above embodiment.

[0009] In some embodiments, a computer program product is provided, wherein the computer program product comprises: a computer program instruction set, wherein the computer program instruction set enables a computer to execute the method according to the above embodiments.

[0010] In some embodiments, a computer program is provided, wherein the computer program comprises: a computer program, wherein the computer program enables a computer to execute the method according to the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Embodiments and aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings. The various features illustrated in the drawings are not drawn to scale.

[0012] Figure 1 An example diagram of an exemplary system for preprocessing and encoding image data according to some embodiments of the present disclosure is shown.

[0013] Figure 2A A schematic diagram showing an exemplary encoding process of a hybrid video coding system consistent with an embodiment of the present disclosure is shown.

[0014] Figure 2B A schematic diagram showing another exemplary encoding process of a hybrid video encoding system consistent with an embodiment of the present disclosure is shown.

[0015] Figure 3A A schematic diagram showing an exemplary decoding process of a hybrid video coding system consistent with an embodiment of the present disclosure is shown.

[0016] Figure 3B A schematic diagram showing another exemplary decoding process of a hybrid video coding system consistent with an embodiment of the present disclosure is shown.

[0017] Figure 4 A block diagram of an exemplary apparatus for preprocessing or encoding image data according to some embodiments of the present disclosure is shown.

[0018] Figure 5 is a schematic diagram illustrating a bitstream structure according to some embodiments of the present disclosure.

[0019] Figure 6 is a diagram illustrating an exemplary process for deriving a motion vector for a sub-block in a sub-block based temporal motion vector prediction (SbTMVP) mode according to some embodiments of the present disclosure.

[0020] Figure 7 Multiple concatenated blocks for temporal motion are shown according to some embodiments of the present disclosure.

[0021] Figure 8 is an exemplary schematic diagram illustrating conventional inter-frame prediction according to some embodiments of the present disclosure.

[0022] Fig. 9 is an exemplary schematic diagram illustrating sub-blocks in a coding unit (CU) according to some embodiments of the present disclosure.

[0023] Fig.10 is an exemplary schematic diagram illustrating motion derivation of a sub-block according to some embodiments of the present disclosure.

[0024] Fig.11 A flowchart illustrating an exemplary method for using sub-block temporal motion prediction for conventional inter mode (sbAmvp mode) according to some implementations of the present disclosure is shown.

[0025] Fig.12 A flowchart of an exemplary method for executing an sbAmvp mode based on a collocated block according to some embodiments of the present disclosure is shown.

[0026] Fig.13 An example diagram for template matching according to some embodiments of the present disclosure is shown.

[0027] Fig.14 An example diagram showing a collocated image using non-inter mode encoding according to some embodiments.

[0028] Fig.15 An example graph showing average motion of top and left sub-blocks according to some embodiments of the present disclosure.

[0029] Fig.16 An example graph showing average motion of surrounding sub-blocks according to some embodiments of the present disclosure is shown.

[0030] Fig.17 An exemplary schematic diagram of multiple sub-blocks in a CU with motion shift according to some embodiments of the present disclosure is shown.

[0031] Fig.18 An exemplary schematic diagram showing motion derivation of a sub-block based on motion shifting according to some embodiments of the present disclosure is shown.

[0032] Fig.19 An exemplary flow chart of a method for performing an sbAmvp mode based on a motion shift according to some embodiments of the present disclosure is shown.

[0033] Fig. 20 An exemplary flow chart of a method for MVP derivation according to some embodiments of the present disclosure is shown.

[0034] Fig.21 An example diagram of a prediction process of reference samples according to some embodiments of the present disclosure is shown.

[0035] Fig. 22 An example diagram showing multiple directions of motion according to some embodiments of the present disclosure is shown.

[0036] Fig.23 An example diagram of candidates with fixed motion directions according to some embodiments of the present disclosure is shown.

[0037] Fig.24 An example diagram showing two sets of motion directions according to some embodiments of the present disclosure.

[0038] Fig.25 Some embodiments of the present disclosure are shown Fig.24 An example diagram with two sets of motion direction candidates is shown.

[0039] Fig.26A first set of example graphs of motion magnitude and motion direction according to some embodiments of the present disclosure are shown.

[0040] Fig. 27 A second set of example graphs of motion magnitude and motion direction according to some embodiments of the present disclosure are shown.

[0041] Fig.28 A third set of example graphs of motion magnitude and motion direction according to some embodiments of the present disclosure are shown.

[0042] Fig.29 A flow chart of an example method for determining which set of motion magnitude and motion direction to use is shown in accordance with some embodiments of the present disclosure.

[0043] Fig.30 Another flow chart of an exemplary method for determining which set of motion magnitude and motion direction to use according to some embodiments of the present disclosure is shown.

[0044] Fig.31 An exemplary process for deriving α and β according to some embodiments of the present disclosure is shown.

[0045] Figure 32-34 Different LIC scaling and offset according to some embodiments of the present disclosure are shown respectively. DETAILED DESCRIPTION

[0046] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, wherein, unless otherwise specified, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with aspects related to the present invention as described in the appended claims. Specific aspects of the present disclosure are described in more detail below. In the event of a conflict with a term or definition incorporated by reference, the terms and definitions provided herein shall prevail.

[0047] The Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (ITU-T VCEG) and the ISO / IEC Moving Picture Experts Group (ISO / IEC MPEG) is currently developing the Versatile Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, VVC aims to achieve the same subjective quality as (HEVC / H.265) using half the bandwidth.

[0048] To achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET has been developing technologies beyond HEVC using the Joint Exploration Model (JEM) reference software. As coding technologies are incorporated into JEM, JEM achieves significantly higher coding performance than HEVC.

[0049] The VVC standard has been developed recently and continues to include more coding techniques that provide better compression performance. VVC is based on the same hybrid video coding system that has been used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, H.263, etc.

[0050] A video is a set of static images (or "frames") arranged in a time sequence to store visual information. A video capture device (e.g., a camera) can be used to capture and store those images in time sequence, and a video playback device (e.g., a television, a computer, a smart phone, a tablet computer, a video player, or any end-user terminal with a display function) can be used to display such images in time sequence. In addition, in some applications, the video capture device can transmit the captured video to the video playback device (e.g., a computer with a monitor) in real time, such as for monitoring, conferencing, or live broadcasting.

[0051] In order to reduce the storage space and transmission bandwidth required for these applications, the video can be compressed before storage and transmission, and decompressed before display. The compression and decompression can be implemented by software or dedicated hardware executed by a processor (e.g., a processor of a general-purpose computer). The module for compression is generally referred to as an "encoder", and the module for decompression is generally referred to as a "decoder". The encoder and decoder can be collectively referred to as "codecs". The encoder and decoder can be implemented as any of various suitable hardware, software, or combinations thereof. For example, the hardware implementation of the encoder and decoder can include circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, or any combination thereof. The software implementation of the encoder and decoder can include program code, computer executable instructions, firmware, or any suitable computer-implemented algorithm or process fixed in a computer-readable medium. Video compression and decompression can be implemented by various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, H.26x series, etc. In some applications, the codec may decompress the video from a first coding standard and recompress the decompressed video using a second coding standard, in which case the codec may be referred to as a "transcoder."

[0052] The video encoding process can identify and retain useful information that can be used to reconstruct the image, and ignore unimportant information for the reconstruction. If the ignored, unimportant information cannot be fully reconstructed, such an encoding process can be called "lossy". Otherwise, it can be called "lossless". Most encoding processes are lossy, which is a trade-off to reduce the required storage space and transmission bandwidth.

[0053] Useful information for a certain image being encoded (referred to as the "current image") includes changes relative to a reference image (e.g., an image previously encoded and reconstructed). Such changes may include changes in pixel position, brightness, or color, with the position changes being of most interest. Changes in the position of a group of pixels representing an object may characterize the motion of an object between the reference image and the current image.

[0054] A picture that is encoded without reference to another picture (i.e., it is its own reference picture) is called an "I-picture." If some or all blocks in a picture (e.g., a plurality of blocks that are typically part of a video picture) are predicted using intra-prediction or inter-prediction (e.g., uni-prediction) with one reference picture, the picture is called a "P-picture." If at least one block in the picture is predicted using two reference pictures (e.g., bi-directional prediction), the picture is called a "B-picture."

[0055] Figure 1 1 is a block diagram of a system 100 for preprocessing and encoding image data according to some embodiments of the present disclosure. The image data may include an image (also referred to as an "image" or "frame"), multiple images, or a video. An image is a static image. Multiple images may be spatially or temporally related or unrelated. A video is a set of images arranged in a time series.

[0056] like Figure 1 As shown, system 100 includes a source device 120 that provides encoded video data that is subsequently decoded by a target device 140. Consistent with the disclosed embodiments, each of source device 120 and target device 140 may include any of a variety of devices, including: a desktop computer, a notebook (e.g., laptop) computer, a server, a tablet, a set-top box, a mobile phone, a vehicle, a camera, an image sensor, a robot, a television, a wearable device (e.g., a smart watch or a wearable camera), a display device, a digital media player, a video game console, a video streaming device, and the like. Source device 120 and target device 140 may be configured for wireless or wired communication.

[0057] refer to Figure 1, the source device 120 may include an image / video preprocessor 122, an image / video encoder 124, and an output interface 126. The target device 140 may include an input interface 142, an image / video decoder 144, and one or more machine vision applications 146. The image / video preprocessor 122 preprocesses image data, i.e., one or more images or one or more videos, and generates an input bit stream for the image / video encoder 124. The image / video encoder 124 encodes the input bit stream and outputs an encoded bit stream 162 via the output interface 126. The encoded bit stream 162 is sent via the communication medium 160 and is received by the input interface 142. The image / video decoder 144 then decodes the encoded bit stream 162 to generate decoded data, which can be utilized by the machine vision application 146.

[0058] More specifically, the source device 120 may further include various devices (not shown) for providing source image data to be preprocessed by the image / video preprocessor 122. The device for providing source image data may include an image / video capture device, such as a camera, an image / video archiving or storage device containing previously captured images / videos, or an image / video feed interface for receiving images / videos from an image / video content provider.

[0059] The image / video encoder 124 and the image / video decoder 144 can each be implemented in any of a variety of suitable encoder or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When encoding or decoding is partially implemented in software, the image / video encoder 124 or the image / video decoder 144 can store multiple instructions for the software in a suitable, non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform techniques consistent with the present disclosure. Each of the image / video encoder 124 or the image / video decoder 144 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC) in a corresponding device.

[0060] The image / video encoder 124 and the image / video decoder 144 may operate according to any video coding standard, such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), AOMedia Video 1 (AV1), Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG), etc. Alternatively, the image / video encoder 124 and the image / video decoder 144 may be custom devices that do not conform to existing standards. Figure 1 Not shown, but in some embodiments, the image / video encoder 124 and the image / video decoder 144 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units (MUX-DEMUX), or other hardware and software, to handle the encoding of audio and video in a common data stream or in separate data streams.

[0061] Output interface 126 may include any type of medium or device capable of transmitting encoded bitstream 162 from source device 120 to target device 140. For example, output interface 126 may include a transmitter or transceiver configured to transmit encoded bitstream 162 directly from source device 120 to target device 140 in real-time. Encoded bitstream 162 may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to target device 140.

[0062] The communication medium 160 may include a transient medium, such as a wireless broadcast or a wired network transmission. For example, the communication medium 160 may include a radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). The communication medium 160 may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. In some embodiments, the communication medium 160 may include a router, a switch, a base station, or any other device that can be used to facilitate communication from the source device 120 to the target device 140. For example, a network server (not shown) may receive an encoded bit stream 162 from the source device 120 and provide the encoded bit stream 162 to the target device 140, such as via a network transmission.

[0063] Communication medium 160 may also take the form of a storage medium (e.g., a non-transitory storage medium) such as a hard disk, a flash drive, an optical disk, a digital video disk, a Blu-ray disk, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded image data. In some embodiments, a computer device of a media generation device (e.g., an optical disk pressing device) may receive the encoded image data from source device 120 and generate a disk containing the encoded video data.

[0064] Input interface 142 may include any type of medium or device capable of receiving information from communication medium 160. The received information includes encoded bitstream 162. For example, input interface 142 may include a receiver or transceiver configured to receive encoded bitstream 162 in real time.

[0065] The machine vision application 146 includes various hardware and / or software for utilizing the decoded image data generated by the image / video decoder 144. For example, the machine vision application 146 may include a display device that displays the decoded image data to a user and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices. As another example, the machine vision application 146 may include one or more processors configured to use the decoded image data to perform various machine vision applications, such as object recognition and tracking, facial recognition, image matching, image / video search, augmented reality, robotic vision and navigation, autonomous driving, three-dimensional structure construction, stereo correspondence, motion tracking, etc.

[0066] Next, combine Figure 2A-2B and Figure 3A-3B Exemplary image data encoding and decoding techniques (such as those used by image / video encoder 124 and image / video decoder 144) are described.

[0067] Figure 2A Schematic diagram of an example encoding process 200A consistent with an embodiment of the present disclosure is shown. For example, the encoding process 200A may be performed by Figure 1 The image / video encoder 124 in FIG. Figure 2A As shown, the encoder can encode the video sequence 202 into a video bit stream 228 according to process 200A. The video sequence 202 may include a group of images (referred to as "original images") arranged in time order. Each original image of the video sequence 202 may be divided by the encoder into a plurality of basic processing units, a plurality of basic processing sub-units, or a plurality of regions for processing. In some embodiments, the encoder may perform process 200A at the basic processing unit level of each original image of the video sequence 202. For example, the encoder may perform process 200A in an iterative manner, wherein the encoder may encode the basic processing unit in one iteration of process 200A. In some embodiments, the encoder may perform process 200A in parallel for a plurality of regions of each original image of the video sequence 202.

[0068] exist Figure 2A2, an encoder may feed a basic processing unit (referred to as an "original BPU") of an original image of a video sequence 202 to a prediction stage 204 to produce prediction data 206 and a prediction BPU 208. The encoder may subtract the prediction BPU 208 from the original BPU to produce a residual BPU 210. The encoder may feed the residual BPU 210 to a transform stage 212 and a quantization stage 214 to produce quantized transform coefficients 216. The encoder may feed the prediction data 206 and the quantized transform coefficients 216 to a binary encoding stage 226 to produce a video bitstream 228. The components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 may be referred to as a "forward path." During process 200A, after the quantization stage 214, the encoder may feed the quantized transform coefficients 216 to an inverse quantization stage 218 and an inverse transform stage 220 to generate a reconstructed residual BPU 222. The encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224, which is used in the prediction stage 204 for the next iteration of the process 200A. The components 218, 220, 222, and 224 of the process 200A may be referred to as a "reconstruction path." The reconstruction path may be used to ensure that both the encoder and the decoder use the same reference data for prediction.

[0069] The encoder can iteratively perform process 200A to encode each original BPU of the original image (in the forward path) and generate a prediction reference 224 (in the reconstruction path) for encoding the next original BPU of the original image. After encoding all the original BPUs of the original image, the encoder can continue to encode the next image in the video sequence 202.

[0070] Referring to process 200A, the encoder may receive a video sequence 202 generated by a video capture device (eg, a camera). As used herein, the term "receiving" may refer to any action of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or in any way inputting data.

[0071] In the prediction stage 204, in the current iteration, the encoder may receive an original BPU and a prediction reference 224, and perform a prediction operation to generate prediction data 206 and a prediction BPU 208. The prediction reference 224 may be generated from a reconstruction path in a previous iteration of the process 200A. The purpose of the prediction stage 204 is to reduce information redundancy by extracting prediction data 206 that can be used to reconstruct the original BPU into a prediction BPU 208 from the prediction data 206 and the prediction reference 224.

[0072] Ideally, the predicted BPU 208 can be the same as the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is usually slightly different from the original BPU. In order to record such differences, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate a residual BPU 210. For example, the encoder can subtract the pixel value (e.g., grayscale value or RGB value) of the predicted BPU 208 from the pixel value corresponding to the original BPU. Each pixel value of the residual BPU 210 can have a residual value as a result of this subtraction between the corresponding pixel values ​​of the original BPU and the predicted BPU 208. Compared with the original BPU, the predicted data 206 and the residual BPU 210 can have fewer bits, but they can be used to reconstruct the original BPU without obvious quality degradation. Thus, the original BPU is compressed.

[0073] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing the residual BPU 210 into a set of two-dimensional "basic patterns", each of which is associated with a "transform coefficient". The basic patterns can have the same size (e.g., the size of the residual BPU 210). Each basic pattern can represent a changing frequency (e.g., the frequency of brightness change) component of the residual BPU 210. No basic pattern can be reproduced from any combination (e.g., linear combination) of any other basic patterns. In other words, the decomposition can decompose the multiple changes of the residual BPU 210 into the frequency domain. This decomposition is similar to the discrete Fourier transform of a function, where the basic patterns are similar to the basis functions (e.g., trigonometric functions) of the discrete Fourier transform, and the transform coefficients are similar to the coefficients associated with the basis functions.

[0074] Different transform algorithms may use different basic modes. Various transform algorithms may be used in the transform stage 212, such as discrete cosine transform, discrete sine transform, and the like. The transform at the transform stage 212 is reversible. That is, the encoder may restore the residual BPU 210 by an inverse operation of the transform (referred to as an "inverse transform"). For example, in order to restore a certain pixel of the residual BPU 210, the inverse transform may be to multiply the corresponding pixel values ​​of a plurality of basic modes by their respective associated coefficients, and to add the above products to produce a weighted sum. For video coding standards, both the encoder and the decoder may use the same transform algorithm (and therefore have the same basic mode). Therefore, the encoder may only record the transform coefficients, and the decoder may reconstruct the residual BPU 210 based on these transform coefficients without receiving the basic mode from the encoder. Compared with the residual BPU 210, the transform coefficients may have fewer bits, but they may be used to reconstruct the residual BPU 210 without significant quality degradation. Therefore, the residual BPU 210 is further compressed.

[0075] The encoder may further compress the transform coefficients in the quantization stage 214. During the transform process, different basic patterns may represent different frequencies of change (e.g., frequencies of brightness changes). Since the human eye is generally better at recognizing low-frequency changes, the encoder may ignore information about high-frequency changes without causing a significant decrease in decoding quality. For example, in the quantization stage 214, the encoder may generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (called a "quantization parameter") and rounding the quotient to its nearest integer. After such an operation, some of the transform coefficients of the high-frequency basic patterns may be converted to zero, while the transform coefficients of the low-frequency basic patterns may be converted to smaller integers. The encoder may ignore the zero-valued quantized transform coefficients 216, thereby further compressing the transform coefficients. The quantization process is also reversible, where the quantized transform coefficients 216 can be reconstructed as transform coefficients in the inverse operation of quantization (called "inverse quantization").

[0076] Because the encoder ignores the remainder of this division in the rounding operation, the quantization stage 214 may be lossy. Generally, the quantization stage 214 may cause the greatest information loss in the process 200A. The greater the information loss, the fewer bits may be required to quantize the transform coefficients 216. To achieve different levels of information loss, the encoder may use different values ​​for the quantization parameter or any other parameter in the quantization process.

[0077] At the binary encoding stage 226, the encoder may encode the prediction data 206 and the quantized transform coefficients 216 using a binary encoding technique, such as entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the encoder may also encode other information at the binary encoding stage 226, such as the prediction mode used at the prediction stage 204, the parameters of the prediction operation, the transform type at the transform stage 212, the parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., rate control parameters), etc. The encoder may use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packaged for network transmission.

[0078] Referring to the reconstruction path of process 200A, at the inverse quantization stage 218, the encoder may perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. At the inverse transform stage 220, the encoder may generate a reconstructed residual BPU 222 based on the reconstructed transform coefficients. The encoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224 to be used in the next iteration of process 200A.

[0079] It should be noted that other variations of process 200A may be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed by the encoder in a different order. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may omit Figure 2A one or more stages in a process.

[0080] Figure 2B A schematic diagram of another exemplary encoding process 200B consistent with an embodiment of the present disclosure is shown. Process 200B can be modified from process 200A. For example, process 200B can be used by an encoder that conforms to a hybrid video coding standard (e.g., H.26x series). Compared to process 200A, the forward path of process 200B additionally includes a mode decision stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B additionally includes a loop filtering stage 232 and a buffer 234.

[0081] In general, prediction techniques can be divided into two categories: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or "intra-frame prediction") can use pixels from one or more encoded adjacent BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in the spatial prediction may include the multiple adjacent BPUs. The spatial prediction can reduce the spatial redundancy inherent in the image. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") can use multiple regions from one or more encoded images to predict the current BPU. That is, the prediction reference 224 in the temporal prediction may include multiple encoded images. Temporal prediction can reduce the temporal redundancy inherent in the multiple images.

[0082] Referring to process 200B, in the forward path, the encoder performs the prediction operation in the spatial prediction stage 2042 and the temporal prediction stage 2044. For example, in the spatial prediction stage 2042, the encoder may perform intra-frame prediction. For a certain original BPU of a certain image being encoded, the prediction reference 224 may include one or more neighboring BPUs that have been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder may generate a predicted BPU 208 by extrapolating multiple neighboring BPUs. Extrapolation techniques may include, for example, linear extrapolation or interpolation, polynomial extrapolation or interpolation, etc. In some embodiments, the encoder may perform the extrapolation at the pixel level, for example, by extrapolating to obtain the corresponding pixel value of each pixel of the predicted BPU 208. The multiple neighboring BPUs used for extrapolation can be positioned from various directions relative to the original BPU, for example, in the vertical direction (for example, at the top of the original BPU), the horizontal direction (for example, on the left side of the original BPU), the diagonal direction (for example, the lower left, lower right, upper left or upper right of the original BPU), or any direction defined in the video coding standard used. For intra-frame prediction, the prediction data 206 may include, for example, the positions (for example, coordinates) of the multiple neighboring BPUs used, the sizes of the multiple neighboring BPUs used, the parameters of the extrapolation, the direction of the neighboring BPUs used relative to the original BPU, and the like.

[0083] For another example, in the temporal prediction stage 2044, the encoder may perform the inter-frame prediction. For a certain original BPU of the current image, the prediction reference 224 may include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference image may be encoded and the BPU may be reconstructed by the BPU. For example, the encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a reconstructed BPU. After all reconstructed BPUs of the same image are generated, the encoder may generate a reconstructed image as a reference image. The encoder may perform a "motion estimation" operation to search for a matching area within a range of the reference image (referred to as a "search window"). The position of the search window in the reference image may be determined based on the position of the original BPU in the current image. For example, the search window may be centered at a position in the reference image having the same coordinates as the original BPU in the current image, and may extend outward by a predetermined distance. When the encoder identifies an area similar to the original BPU in the search window (e.g., by using a pixel recursive algorithm, a block matching algorithm, etc.), the encoder may determine such an area as a matching area. The matching area may have a different size than the original BPU (e.g., less than, equal to, greater than, or having a different shape). Since the reference image and the current image are temporally separated on the time axis, it can be considered that the matching area "moves" to the position of the original BPU over time. The encoder may record the direction and distance of this movement as a "motion vector". When multiple reference images are used, the encoder may search for a matching area and determine the associated motion vector for each reference image. In some embodiments, the encoder may assign weights to the pixel values ​​of the matching areas of each matching reference image.

[0084] Motion estimation may be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, the prediction data 206 may include, for example, the location (e.g., coordinates) of the matching region, a plurality of motion vectors associated with the matching region, the number of reference images, a plurality of weights associated with the plurality of reference images, and the like.

[0085] In order to generate the predicted BPU 208, the encoder may perform a "motion compensation" operation. The motion compensation may be used to reconstruct the predicted BPU 208 based on the prediction data 206 (e.g., the motion vector) and the prediction reference 224. For example, the encoder may move the matching region of the reference image according to the motion vector to predict the original BPU of the current image. When multiple reference images are used, the encoder may move the matching regions of the multiple reference images according to the respective motion vectors and average pixel values ​​of the matching regions. In some embodiments, if the encoder has assigned weights to the pixel values ​​of the matching regions of the respective matching reference images, the encoder may add the weighted sum of the pixel values ​​of the multiple moved matching regions.

[0086] In some embodiments, inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction relative to the current image. Unidirectional inter-frame prediction uses a reference image before the current image. Bidirectional inter-frame prediction can use one or more reference images in both temporal directions relative to the current image.

[0087] Still referring to the forward path of process 200B, after spatial prediction 2042 and temporal prediction stage 2044, at mode decision stage 230, the encoder may select a prediction mode (e.g., one of intra prediction or inter prediction) for the current iteration of process 200B. For example, the encoder may perform rate-distortion optimization techniques, wherein the encoder may select a prediction mode to minimize a value of a cost function based on the bit rate of a candidate prediction mode and the distortion of a reconstructed reference image under the candidate prediction mode. Based on the selected prediction mode, the encoder may generate a corresponding prediction BPU 208 and prediction data 206.

[0088] In the reconstruction path of process 200B, if the intra-frame prediction mode has been selected in the forward path, after generating the prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image), the encoder can directly feed the prediction reference 224 to the spatial prediction stage 2042 for subsequent use (e.g., for extrapolation of the next BPU of the current image). If the inter-frame prediction mode has been selected in the forward path, after generating the prediction reference 224 (e.g., in the current image, all BPUs have been encoded and reconstructed), the encoder can feed the prediction reference 224 to the loop filtering stage 232, where the encoder can apply loop filtering to the prediction reference 224 to reduce or eliminate distortion (e.g., blocking effects) introduced by inter-frame prediction. The encoder can apply various loop filtering techniques in the loop filtering stage 232, such as deblocking, sample adaptive offset, adaptive loop filtering, etc. The loop-filtered reference pictures may be stored in a buffer 234 (or "decoded picture buffer") for subsequent use (e.g., as an inter-prediction reference picture for a future picture of the video sequence 202). The encoder may store one or more reference pictures in the buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the encoder may encode the parameters of the loop filter (e.g., loop filter strength) as well as the quantized transform coefficients 216, the prediction data 206, and other information in a binary encoding stage 226.

[0089] Figure 3A FIG. 3 is a schematic diagram of an exemplary decoding process 300A consistent with an embodiment of the present disclosure. The process 300A may be Figure 2A In some embodiments, process 300A may be similar to the reconstruction path of process 200A. A decoder (e.g., Figure 1 The image / video decoder 144 in the embodiment of the present invention can decode the video bitstream 228 into a video stream 304 according to the process 300A. The video stream 304 can be very similar to the video sequence 202. However, due to the compression and decompression process (e.g., Figure 2A-2B The information in the quantization stage 214 in the video stream 304 is lost, and typically the video stream 304 is not identical to the video sequence 202. Figure 2A-2B 200A and 200B in the decoder, the decoder may perform process 300A at a basic processing unit (BPU) level for each picture encoded in the video bitstream 228. For example, the decoder may perform process 300A in an iterative manner, wherein the decoder may decode a basic processing unit in one iteration of process 300A. In some embodiments, the decoder may perform process 300A in parallel for regions of each picture encoded in the video bitstream 228.

[0090] exist Figure 3A In the process 300A, the decoder may feed a portion of the video bitstream 228 associated with a basic processing unit (referred to as a "coded BPU") of an encoded picture to the binary decoding stage 302. In the binary decoding stage 302, the decoder may decode the portion into prediction data 206 and quantized transform coefficients 216. The decoder may feed the quantized transform coefficients 216 to the inverse quantization stage 218 and the inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may feed the prediction data 206 to the prediction stage 204 to generate the prediction BPU 208. The decoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded picture buffer in a computer memory). The decoder may feed the prediction reference 224 to the prediction stage 204 for performing a prediction operation in the next iteration of the process 300A.

[0091] The decoder may iteratively perform process 300A to decode each coded BPU of the coded picture and generate a prediction reference 224 for encoding the next coded BPU of the coded picture. After decoding all coded BPUs of the coded picture, the decoder may output the picture to a video stream 304 for display and continue decoding the next coded picture in the video bitstream 228.

[0092] In the binary decoding stage 302, the decoder may perform the inverse of the binary coding technique used by the encoder (e.g., entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the decoder may also decode other information in the binary decoding stage 302, such as the prediction mode, parameters of the prediction operation, the transform type, parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., rate control parameters), etc. In some embodiments, if the video bitstream 228 is transmitted in the form of packets over the network, the decoder may unpack the video bitstream 228 before feeding the video bitstream 228 to the binary decoding stage 302.

[0093] Figure 3BA schematic diagram of another exemplary decoding process 300B consistent with an embodiment of the present disclosure is shown. Process 300B can be modified from process 300A. For example, process 300B can be used by a decoder that complies with a hybrid video coding standard (e.g., H.26x series). Compared to process 300A, process 300B additionally divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and also includes a loop filtering stage 232 and a buffer 234.

[0094] In process 300B, for a coded basic processing unit (referred to as "current BPU") of an encoded image (referred to as "current image") being decoded, the prediction data 206 decoded by the decoder from the binary decoding stage 302 may contain various types of data, depending on what prediction mode the encoder used to encode the current BPU. For example, if the encoder uses intra-frame prediction to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating that intra-frame prediction is used, parameters of the intra-frame prediction operation, and the like. The parameters of the intra-frame prediction operation may include, for example, the position (e.g., coordinates) of one or more neighboring BPUs used as reference, the size of multiple neighboring BPUs, extrapolation parameters, directions of multiple neighboring BPUs relative to the original BPU, and the like. For another example, if the encoder uses inter-frame prediction to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating that inter-frame prediction is used, parameters of the inter-frame prediction operation, and the like. The parameters of the inter-frame prediction operation may include, for example, the number of multiple reference images associated with the current BPU, the weights respectively associated with the multiple reference images, the positions (e.g., coordinates) of one or more matching regions in the respective reference images, one or more motion vectors respectively associated with the matching regions, and the like.

[0095] Based on the prediction mode indicator, the decoder can decide whether to perform spatial prediction (e.g., intra prediction) in the spatial prediction stage 2042 or temporal prediction (e.g., inter prediction) in the temporal prediction stage 2044. Details of performing such spatial prediction or temporal prediction are described in Figure 2B After performing such spatial prediction or temporal prediction, the decoder may generate a prediction BPU 208. The decoder may add the prediction BPU 208 and the reconstructed residual BPU 222 to generate a prediction reference 224, such as Figure 3A as shown in .

[0096] In process 300B, the decoder may feed the prediction reference 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 for performing a prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intra-frame prediction in the spatial prediction stage 2042, then after generating the prediction reference 224 (e.g., the decoded current BPU), the decoder may feed the prediction reference 224 directly to the spatial prediction stage 2042 for subsequent use (e.g., for extrapolation of the next BPU of the current image). If the current BPU is decoded using inter-frame prediction in the temporal prediction stage 2044, then after generating the prediction reference 224 (e.g., in a reference image, all BPUs have been decoded), the encoder may feed the prediction reference 224 to the loop filtering stage 232 to reduce or eliminate distortion (e.g., blocking effects). The decoder may perform the following steps: Figure 2B In-loop filtering is applied to the prediction reference 224 in the manner described in . The loop filtered reference picture may be stored in a buffer 234 (e.g., a decoded picture buffer in a computer memory) for subsequent use (e.g., as an inter-frame prediction reference picture for a subsequent encoded picture of the video bitstream 228). The decoder may store one or more reference pictures in the buffer 234 for use in the temporal prediction stage 2044. In some embodiments, when the prediction mode indicator of the prediction data 206 indicates that inter-frame prediction is used to encode the current BPU, the prediction data may further include parameters of the loop filtering (e.g., loop filtering strength).

[0097] Return to reference Figure 1 Each of the image / video preprocessor 122, the image / video encoder 124 and the image / video decoder 144 may be implemented using any suitable hardware, software or a combination thereof. Figure 4 4 is a block diagram of an example apparatus 400 for processing image data consistent with an embodiment of the present disclosure. For example, the apparatus 400 may be a preprocessor, an encoder, or a decoder. Figure 4As shown, the device 400 may include a processor 402. When the processor 402 executes the instructions described herein, the device 400 may become a special-purpose machine for preprocessing, encoding, or decoding image data. The processor 402 may be any type of circuit capable of manipulating or processing information. For example, the processor 402 may include any number of central processing units (or "CPUs"), graphics processing units (or "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, intellectual property cores (IP cores), programmable logic arrays (PLAs), programmable array logic (PALs), general array logic (GALs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or other similar combinations. In some embodiments, the processor 402 may also be a group of processors grouped into a single logical component. For example, as Figure 4 As shown, processor 402 may include multiple processors, including processor 402a, processor 402b, and processor 402n.

[0098] The apparatus 400 may also include a memory 404 configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). Figure 4 As shown, the stored data may include program instructions (e.g., program instructions for implementing the stages in process 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 may access the program instructions and data for processing (e.g., via bus 410), and execute the program instructions to perform operations or manipulations on the data for processing. Memory 404 may include a high-speed random access storage device or a non-transitory storage device. In some embodiments, memory 404 may include any number of random access memories (RAMs), read-only memories (ROMs), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, secure digital (SD) cards, memory sticks, compact flash (CF) cards, etc. or any combination of the like. Memory 404 may also be a group of memories (e.g., multiple memory cards) grouped into a single logical component. Figure 4 not shown).

[0099] The bus 410 may be a communication device that transmits data between components inside the apparatus 400 , such as an internal bus (eg, a CPU-memory bus), an external bus (eg, a universal serial bus port, a peripheral component interconnect express port), and the like.

[0100] For ease of explanation and without ambiguity, the processor 402 and other data processing circuits are collectively referred to as "data processing circuits" in this disclosure. The data processing circuits may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. In addition, the data processing circuits may be a single independent module or may be fully or partially combined into any other component of the device 400.

[0101] The device 400 may further include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (“NFC”) adapters, cellular network chips, or any combination of other similar items.

[0102] In some embodiments, the apparatus 400 may further include a peripheral interface 408 to provide a connection to one or more peripheral devices. Figure 4 As shown, the peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, touchpad, or touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), a video input device (e.g., a camera or input interface coupled to a video archive), and the like.

[0103] It should be noted that the video codec (e.g., a codec that performs process 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules in device 400. For example, some or all stages of process 200A, 200B, 300A, or 300B can be implemented as one or more software modules of device 400, such as program instructions that can be loaded into memory 404. For another example, some or all stages of process 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of device 400, such as dedicated data processing circuits (e.g., FPGA, ASIC, NPU, etc.).

[0104] Figure 5 5 is a schematic diagram showing a bitstream structure according to some embodiments of the present disclosure. In some embodiments, the structure of the bitstream 500 can be applied to Figure 1 The video bitstream 162 is shown. Figure 5, a bitstream 500 includes a video parameter set (VPS) 510, a sequence parameter set (SPS) 520, a picture parameter set (PPS) 530, a picture header 540, and slices 550 to 570, which are separated by synchronization markers M1-M7. The slices 550 to 570 each include a corresponding header block (e.g., header 552) and a data block (e.g., data 554), each of which includes one or more CTUs (e.g., CTU1-CTUn in data 554). In addition, each CTU further includes a plurality of CUs (e.g., Figure 5 CU1-CUn in the .

[0105] According to some embodiments, the bitstream 500 is a sequence of bits in the form of a network abstraction layer (NAL) unit or a byte stream, forming one or more coded video sequences (CVS). A CVS includes one or more coded layer video sequences (CLVS). In some embodiments, a CLVS is a sequence of multiple picture units (PUs), and each PU includes a coded picture. In particular, a PU includes zero or one picture header NAL unit and a coded picture, the picture header NAL unit (e.g., picture header 540) including a picture header syntax structure as a payload), and the coded picture includes one or more video coding layer (VCL) NAL units. Optionally, one or more other non-VCL NAL units. In some embodiments, a VCL NAL unit is a collective term for a subset of multiple coded slice NAL units (e.g., slices 550 to 570) and multiple reserved values ​​of NAL unit types classified as VCL NAL units. The coded slice NAL unit contains a slice header and a slice data block (e.g., header 552 and data 554).

[0106] In some embodiments of the present disclosure, a layer may be a set of video coding layer (VCL) NAL units with a specific NAL layer ID value and one or more associated non-VCL NAL units. In these layers, inter-layer prediction may be applied between different layers to achieve high compression performance.

[0107] As described above, in a general video coding (e.g., VVC / H.266) standard, an image may be partitioned into a set of CTUs, and each CTU may be further partitioned into multiple coding units (CUs) using a quadtree, a binary tree, or a ternary tree. Multiple CTUs may form a tile, a strip, or a sub-image. When an image includes three sample arrays for storing three color components (e.g., a luminance component and two chrominance components), a CTU may include N×N (N is an integer) luminance sample blocks, each luminance sample block being associated with two chrominance sample blocks. In some embodiments, an output layer set (OLS) may be specified to support decoding of some but not all layers. An OLS is a layer set that includes a specified layer set, wherein one or more layers in the layer set are specified as multiple output layers. Thus, an OLS may include one or more output layers and other layers required to decode one or more output layers for inter-layer prediction.

[0108] The Enhanced Compression Model (ECM) has been proposed and used as a new software basis for developing tools beyond the VVC standard.

[0109] In ECM, a mode called sub-block based temporal motion vector prediction (SbTMVP) is supported. In SbTMVP mode, a CU is partitioned into 4×4 sub-blocks, and each sub-block obtains its own motion vector from the motion field in the collocated image. When the motion vector of each sub-block is obtained, a motion shift is applied using the motion vectors of multiple lower left neighboring blocks of the CU. Figure 6 1 is a schematic diagram illustrating an exemplary process of deriving multiple motion vectors of multiple sub-blocks in a sub-block based temporal motion vector prediction (SbTMVP) mode according to some embodiments of the present disclosure. Figure 6 As shown, a motion shift MV is obtained from the neighboring block A1 of the CU of the current image. A1 , and shift the motion by MV A1 Used to derive a motion field in the juxtaposed image. For each sub-block, the motion information of its corresponding sub-block in the juxtaposed image is used to derive the motion information of the sub-block in the current image. After the motion information of a juxtaposed sub-block 602 is identified, it is converted into multiple motion vectors and multiple reference indexes of the current sub-block 602'. The reference index of the sub-block is selected from any one of a plurality of reference images in a reference image list. The selected reference image is the reference image whose scaling factor is closest to 1. Temporal motion scaling is applied after identifying the reference index. Note that when the corresponding sub-block in the juxtaposed image is non-inter-coded (e.g., intra-coded or intra-block copy (IBC) coded), the motion information of the center sub-block of the CU is used. As Figure 6As shown, a corresponding sub-block 601 is non-inter-coded, that is, the sub-block 601 does not include any motion information. Therefore, the motion information of the central sub-block 602 is used. As a result, the motion information of a sub-block 601' in the current image corresponding to the sub-block 601 is set to be the same as the motion information of a sub-block 602' corresponding to the sub-block 602 in the collocated image.

[0110] In VVC, temporal motion is used as one of the candidates for merge mode and advanced motion vector prediction (AMVP) mode. Figure 7 FIG. 4 shows a plurality of juxtaposed blocks for temporal motion according to some embodiments of the present disclosure. Figure 7 As shown, the temporal motion is derived from a collocated picture and is obtained from the lower right C0 or the center C1 of the collocated block 701. Typically, only one collocated picture is allowed in VVC. The collocated picture is selected at the encoder and signaled in the picture and slice headers in the bitstream.

[0111] In ECM, in order to further improve the prediction accuracy of temporal motion, another collocated image is introduced. Two collocated images are used, which are two reference frames with the smallest picture order count (POC) different from the frame to be encoded.

[0112] In ECM, the reference picture list of each block is reordered based on the template matching (TM) cost. For a unidirectionally predicted block, all reference pictures in reference picture list 0 and reference picture list 1 are interleaved to produce a joint list. Then, the TM cost is calculated for each reference picture. The joint list is reordered based on the ascending order of TM cost. For a bidirectionally predicted block, a list of multiple reference picture pairs from list 0 and list 1 is generated and similarly reordered based on the TM cost. The index of the selected pair is signaled.

[0113] In the current design, the sbTMVP is only allowed in merge mode. However, it cannot be applied to the conventional inter mode which allows the motion vector difference to be signaled. Therefore, the advantage of using temporal motion in multiple sub-blocks cannot be fully utilized. Figure 8 is an exemplary schematic diagram showing conventional inter-frame prediction according to some embodiments of the present disclosure. Figure 8 As shown in FIG. 8 , in a conventional inter mode, the motion vector 801 is used to predict the entire CU 802 , and the CU 802 is not partitioned into a plurality of sub-blocks.

[0114] In the present disclosure, sub-block temporal motion prediction for a conventional inter mode (hereinafter referred to as sbAmvp mode) is proposed.

[0115] According to some embodiments, temporal motion information associated with the plurality of sub-blocks may be obtained from a collocated block. Specifically, a CU is predicted using sub-block temporal motion and a signaled motion. Fig. 9 is an exemplary schematic diagram showing multiple sub-blocks in a CU according to some embodiments of the present disclosure. Fig.10 FIG. 1 is an exemplary schematic diagram showing motion derivation of a sub-block according to some embodiments of the present disclosure. Fig. 9 and Fig.10 , a CU 910 is partitioned into a plurality of sub-blocks. Each sub-block in the current CU 910 derives its motion information from a collocated block 920 and a signaled CU-level motion information. For a sub-block 901, its corresponding collocated sub-block 901' is first identified. Then, the motion information of the corresponding collocated sub-block (hereinafter referred to as collocated motion) is scaled to a reference image 930 (eg, Fig.10 ). The scaling motion is added to the signaled CU-level motion and used as the motion of the sub-block 901. Note that the reference images of each sub-block may be different from or the same as each other, and the signaled CU-level motion information may include any subset of the set {inter-prediction direction (hereinafter referred to as interDir), reference image index (hereinafter referred to as refIdx), motion vector predictor (hereinafter referred to as MVP), motion vector difference (hereinafter referred to as MVD)}.

[0116] Fig.11 1 shows a flowchart of an example method 1100 using sub-block temporal motion prediction for conventional inter mode (sbAmvp mode) according to some embodiments of the present disclosure. The method 1100 may be performed by a decoder (e.g., by Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1100 may be performed by a processor 402 of the computer system. In some embodiments, the method 1100 may be implemented by a computer program product included in a computer-readable medium, the computer program product including a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.11 , method 1100 may include the following steps 1102 to 1106.

[0117] At step 1102, coding unit (CU) level motion information associated with a CU is received. The CU is encoded using a conventional inter mode.

[0118] In step 1104, the CU is divided into a plurality of sub-blocks. For example, a CU is divided into 4×4 sub-blocks.

[0119] At step 1106, the sbAmvp mode is performed on the plurality of sub-blocks. Performing the sbAmvp mode includes deriving motion information of a sub-block among the plurality of sub-blocks based on the signaled CU-level motion information. The signaled CU-level motion information may include any subset of {inter prediction direction (hereinafter referred to as interDir), reference picture index (hereinafter referred to as refIdx), motion vector predictor (hereinafter referred to as MVP), motion vector difference (hereinafter referred to as MVD)}.

[0120] In this disclosure, several examples are provided below for deriving temporal motion information associated with multiple sub-blocks from a collocated block. The collocated block is in a collocated picture.

[0121] Fig.12 1 shows a flowchart of an exemplary method 1200 for performing the sbAmvp mode based on a concatenated block according to some embodiments of the present disclosure. The method 1200 may be performed by a decoder (e.g., by Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, a processor (e.g., Figure 4 The method 1200 may be performed by a processor 402 of the computer system. In some embodiments, the method 1200 may be implemented by a computer program product included in a computer-readable medium, the computer program product including a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.12 , method 1200 may include the following steps 1202 to 1206.

[0122] At step 1202, a corresponding collocated sub-block in the collocated image is identified for the sub-block.

[0123] In step 1204, the motion information of the collocated sub-block is scaled to a reference image. Therefore, a scaled motion can be obtained.

[0124] At step 1206, a scaled motion is added to a CU-level motion to obtain a motion for the sub-block.

[0125] At step 1208, motion information of the sub-block including the motion of the sub-block is obtained. The motion information of the sub-block includes the obtained motion of the sub-block, the identified reference image of the sub-block, and the inter-frame prediction direction inherited from a CU-level motion information.

[0126] In some embodiments, the reference image of each sub-block is the same as the other sub-blocks. That is, the multiple reference images of all the multiple sub-blocks are the same. In some embodiments, the reference image of each sub-block is the same as the other sub-blocks, and the signaled CU-level motion information includes interDir, refIdx, MVP and MVD. The derivation of motion information of multiple sub-blocks may further include: determining a reference image of a sub-block using refIdx of the signaled CU-level motion information; determining a CU-level motion by adding the MVD to the MVP; obtaining a scaled motion by scaling the collocated motion to the reference image; and adding the scaled motion to the CU-level motion to obtain a sub-block motion.

[0127] In some embodiments, the reference image of each sub-block is the same as that of other sub-blocks, and the signaled CU-level motion information includes interDir, MVP and MVD, and the signaled CU-level motion information points to a predefined reference image. The predefined reference image may be a collocated reference image. This means that the reference image is a collocated image. The predefined reference image may also be signaled at a slice level, a picture level, a PPS level or an SPS level. The derivation of motion information of multiple sub-blocks may further include: determining the CU-level motion by adding the MVD to the MVP; obtaining the scaled motion by scaling the collocated motion to the predefined reference image; and adding the scaled motion to the CU-level motion.

[0128] In some embodiments, when the signaled CU-level motion information does not include MVD, the CU-level motion vector is set to a motion vector prediction value. For example, the reference image of each sub-block is the same as the other multiple sub-blocks, and the signaled CU-level motion information includes interDir, refIdx and MVP. In this example, the derivation of motion information of multiple sub-blocks may further include: determining the reference image of multiple sub-blocks using refIdx of the signaled CU-level motion information; setting the CU-level motion to the MVP; obtaining the scaled motion by scaling the collocated motion to the reference image; and adding the scaled motion to the CU-level motion.

[0129] In some embodiments, the reference image of each sub-block is the same as the other multiple sub-blocks, and the CU-level motion information signaled includes interDir and MVD, and the reference image and MVP can be determined using a (TM) cost. More specifically, for each reference image, a motion vector prediction value candidate is derived in a manner similar to the AMVP process of conventional inter-frame prediction. Fig.13 An example diagram for template matching according to some embodiments of the present disclosure is shown. Fig.13 As shown, a TM cost is calculated for each motion vector predictor candidate. Template 1310 includes reconstructed samples adjacent to the left 1311 and / or above 1312 of CU 1320. Reference samples 1330 of the template are generated by each motion vector predictor candidate. Therefore, the reference image and MVP of the signaled CU-level motion information are set to the image with the minimum TM cost. In this example, the derivation of motion information of multiple sub-blocks may further include: determining the reference image and MVP of multiple sub-blocks using TM cost; determining the CU-level motion by adding the MVD to the MVP; obtaining the scaled motion by scaling the collocated motion to the reference image; and adding the scaled motion to the CU-level motion.

[0130] In some embodiments, continuing from the above example, when calculating TM cost, instead of using only motion vector predictor candidates, the MVD is also involved in TM cost calculation. Specifically, the plurality of reference samples of the template are generated by the motion vector obtained by the motion vector predictor candidate plus the MVD.

[0131] In some embodiments, the reference image of each sub-block is different from other sub-blocks. In other words, the reference images of the sub-blocks are different.

[0132] In some embodiments, the reference image of each sub-block is different from other sub-blocks, and the signaled CU-level motion information includes interDir, refIdx, MVP and MVD. Deriving the motion information of multiple sub-blocks may further include: determining the collocated motion and the reference image of each sub-block, wherein the reference image is selected from any one of multiple reference images in the reference image list, and the selected reference image is a reference image with a scaling factor closest to 1; obtaining the scaled motion by scaling the collocated motion to the determined reference image; determining the CU-level motion by adding the MVD to the MVP; scaling the CU-level motion to the determined reference image; and adding the scaled motion to the scaled CU-level motion.

[0133] In some embodiments, the reference image of each sub-block is different from other sub-blocks, and the signaled CU-level motion information includes interDir, MVP and MVD, and the signaled CU-level motion information points to a predefined reference image. The predefined reference image may be a collocated reference image. The predefined reference image may be signaled at a slice level, a picture level, a PPS level or an SPS level. The derivation of motion information for multiple sub-blocks may further include: determining the collocated motion and the reference image of each sub-block, wherein the reference image is selected from multiple reference images in a reference image list, and the selected reference image is a reference image with a scaling factor closest to 1; obtaining the scaled motion by scaling the collocated motion to the determined reference image; determining the CU-level motion by adding the MVD to the MVP; scaling the CU-level motion to the determined reference image; and adding the scaled motion to the scaled CU-level motion.

[0134] In some embodiments, the collocated sub-block may be encoded using a non-inter mode and the collocated motion may not be present. Fig.14 An example diagram showing collocated images encoded using non-inter mode according to some embodiments. Fig.15 An example graph showing average motion of top and left sub-blocks according to some embodiments of the present disclosure. Fig.16 An example diagram showing the average motion of multiple sub-blocks around the device according to some embodiments of the present disclosure is shown. Fig.14 As shown, the collocated sub-block 1401 in the collocated image is non-inter-coded. It is proposed to set the motion of the collocated sub-block to zero motion, the center sub-block of the CU (e.g., Fig.14 The motion of the sub-block 1402 in the middle), the average motion of the top and left sub-blocks, that is, (MV T +MV L ) / 2(such as Fig.15 As shown), the average motion of the surrounding multiple sub-blocks ( Fig.16 ), that is, (MV T +MV L +MV R +MV B ) / 4(such as Fig.15 In some embodiments, when calculating an average motion of multiple sub-blocks, the motion of each sub-block is first scaled to the same reference image, and the reference image may be indicated by the CU-level motion information signaled, or may be a collocated reference image.

[0135] In some embodiments, the collocated motion is subtracted from the average motion before being added to the signaled CU-level motion information. Specifically, the derivation of the motion information of the plurality of sub-blocks may further include: determining a reference image of the plurality of sub-blocks using refIdx of the signaled CU-level motion information; determining the CU-level motion by adding the MVD to the MVP; obtaining the scaled motion by scaling the collocated motion to the reference image; calculating an average motion of all scaled motions; obtaining a difference motion by subtracting the scaled collocated motion from the average motion; and adding the difference motion to the CU-level motion.

[0136] In this disclosure, several examples for deriving temporal motion information associated with multiple sub-blocks from a motion shift are provided below. Specifically, it is proposed to signal motion information at the CU level and use it as a motion shift to derive sub-block temporal motion information. Fig.17 is an exemplary schematic diagram showing multiple sub-blocks in a CU with a motion shift according to some embodiments of the present disclosure. Fig.18 FIG. 4 is an exemplary schematic diagram showing the motion derivation of a sub-block based on the motion shift according to some embodiments of the present disclosure. Fig.17 , similar to the SbTMVP mode, a CU 1710 is partitioned into multiple sub-blocks (e.g., sub-block 1711), and each sub-block obtains its own motion vector from a motion field 1721 in a collocated image 1720. When the motion vector of each sub-block is obtained, a motion shift is applied. The motion shift is signaled at the CU level 1730. Reference Fig.18 Then, the motion in the collocated image 1740 is converted into the motion information of the sub-block 1711. The signaled CU-level motion information may include interDir, refIdx, MVP, and MVD.

[0137] Fig.19 1 shows a flowchart of a method 1900 for performing a sbAmvp mode based on a motion shift according to some embodiments of the present disclosure. The method 1900 may be performed by a decoder (e.g., by Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1900 may be performed by a processor 402 of the computer system. In some embodiments, the method 1900 may be implemented by a computer program product included in a computer-readable medium, the computer program product including a computer (e.g., Figure 4Computer executable instructions, such as program codes, executed by the device 400. Fig.19 , method 1900 may include the following steps 1902-1906.

[0138] In step 1902, a motion vector of a sub-block is obtained based on the motion shift.

[0139] At step 1904, a corresponding collocated sub-block is identified in a collocated image of the sub-block.

[0140] In step 1906, the motion information of a corresponding collocated sub-block is converted into the motion information of the sub-block.

[0141] In some embodiments, the signaled CU-level motion information includes MVP and MVD, and the MVP points to the collocated reference image. The derivation of motion information of multiple sub-blocks may further include: determining the motion shift by adding MVD to MVP; identifying the multiple corresponding sub-blocks in the collocated image with the motion shift; and converting the motion information of the corresponding sub-blocks to the motion information of the sub-blocks.

[0142] In some embodiments, the signaled CU-level motion information contains only MVP, and the MVP points to the collocated reference image. The derivation of motion information of multiple sub-blocks may further include: setting the motion shift to MVP; identifying the multiple corresponding sub-blocks in the collocated image with the motion shift; and converting the motion information of the corresponding sub-blocks to the motion information of the sub-blocks.

[0143] In some embodiments, when converting the motion information of the corresponding sub-block into the motion information of the sub-block, a reference index of the sub-block is selected from any one of a plurality of reference images in the reference image list. The selected reference image is a reference image having a scaling factor closest to 1.

[0144] In some embodiments, the signaled CU-level motion information includes interDir, refIdx, MVP and MVD, and the MVP points to a collocated reference image. The MVP and MVD are used to derive motion shifts, and the interDir and refIdx are used to derive the reference image for multiple sub-blocks. The derivation of motion information for multiple sub-blocks may further include: determining the motion shift by adding MVD to MVP; identifying the multiple corresponding sub-blocks in the collocated image with the motion shift; determining the reference image for multiple sub-blocks using interDir and refIdx; and scaling the motion information of the corresponding sub-blocks to the reference image.

[0145] The sbAmvp mode with various instances proposed above may be combined with a local illumination compensation (LIC) mode, a multiple hypothesis prediction (MHP) mode, an overlapped block motion compensation (OBMC) mode, a bidirectional prediction with CU level weights (BCW) mode, and / or an adaptive motion vector resolution (AMVR) mode.

[0146] The present disclosure also provides embodiments for deriving and signaling an MVP.

[0147] In some embodiments, the MVP of a CU can be derived using the same process as the AMVP MVP derivation process in the ECM. That is, first, a MVP candidate list is constructed using multiple motions from multiple spatially adjacent blocks, temporally juxtaposed blocks, multiple non-adjacent adjacent blocks, and a history-based motion vector prediction (HMVP) table. An HMVP table includes multiple historical motion candidates that can be used to help predict multiple future motions and movements. By using these various motion sources and the HMVP table, the candidate list is able to capture a comprehensive range of multiple potential MVPs. Then, the template matching (TM) cost is calculated for each MVP candidate, and the one with the smallest TM cost is selected and further refined by the template matching process. The TM cost is calculated using the multiple adjacent reconstructed samples and the multiple reference samples predicted using the multiple MVP candidates.

[0148] Fig. 20 2000 for MVP derivation according to some embodiments of the present disclosure. The method 2000 may be performed by a decoder (e.g., by Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 2000 may be performed by a processor 402 of the computer system. In some embodiments, the method 2000 may be implemented by a computer program product included in a computer-readable medium, the computer program product including a computer (e.g., Figure 4 A computer executable instruction set, such as a program code, executed by a device 400. Fig. 20 , method 2000 may include the following steps 2002-2008.

[0149] In step 2002, a MVP candidate list is constructed using motions from a plurality of spatially neighboring blocks, a temporally collocated block, a plurality of non-neighboring neighboring blocks, and a history-based motion vector prediction (HMVP) table.

[0150] In step 2004, a TM cost is calculated for each MVP candidate in the MVP candidate list.

[0151] In step 2006, a MVP candidate with a minimum TM cost is selected.

[0152] In step 2008, a template matching process is used to refine the MVP candidates.

[0153] In some embodiments, the plurality of reference samples are predicted using corresponding collocated temporal motions instead of the MVP candidates. Fig.21 An example diagram of a prediction process of multiple reference samples according to some embodiments of the present disclosure is shown. Fig.21 As shown, the MVP 2110 points to the collocated image 2120 and identifies the plurality of corresponding neighboring sub-blocks 2121. For each corresponding neighboring sub-block 2121, a temporal motion is derived and the plurality of reference samples of the template 2131 of a current CU 2130 is predicted using the temporal motion.

[0154] For the signaling of MVD, in one example, the MVD is signaled in the same manner as the AMVP MVD in the ECM. That is, multiple absolute values ​​of the MVD in the horizontal and vertical directions are binarized and signaled, respectively. The symbols of the MVD can be reordered using the TM cost and indicated by a signaled index.

[0155] In some embodiments, the MVD is constructed using an index indicating the motion amplitude and an index indicating the motion direction. The motion amplitude and the motion direction are selected from multiple predefined sets respectively. In one instance, a predefined set of motion amplitudes is {1 / 4-pixel, 1 / 2-pixel, 1-pixel, 2-pixel, 4-pixel, 8-pixel}, which is similar to that used in the MMVD (merged with motion vector difference) mode. In another example, considering that the TMVP (temporal motion vector prediction) is stored in a 4×4 block grid, in order to obtain different TMVPs, the MVD is a multiple of 4 pixels. Therefore, the predefined set of motion amplitudes can be {4-pixels, 8-pixels, 16-pixels, 32-pixels, 64-pixels, 128-pixels}. Fig. 22 An example of multiple directions of motion according to some embodiments of the present disclosure is shown. The predefined set of directions of motion may be Fig. 22 Any subset of the multiple directions shown.

[0156] The predefined set of motion amplitudes may be modified based on sequence resolution, motion direction, etc. In one example, for a sequence resolution greater than 4096000 luma samples, the predefined set of motion amplitudes is set to {8-pixel, 16-pixel, 32-pixel, 64-pixel}. For a sequence resolution less than or equal to 4096000 luma samples, the predefined set of motion amplitudes is set to {4-pixel, 8-pixel, 16-pixel, 32-pixel}. In another example, for a horizontal motion direction, the predefined set of motion amplitudes is set to {4-pixel, 8-pixel, 16-pixel, 32-pixel}. For a vertical motion direction, the predefined set of motion amplitudes is set to {4-pixel, 8-pixel, 12-pixel, 16-pixel}.

[0157] The two indices for the motion magnitude and the motion direction may be signaled separately. Alternatively, the two indices are merged and signaled.

[0158] In some embodiments, the predefined set of motion magnitudes is set to {4-pixels, 8-pixels, 16-pixels, 32-pixels, 64-pixels, 128-pixels}. Fig. 22 The plurality of motion directions shown, the predefined set of motion directions is set to {0, 4, 8, 12}. A parameter, for example, an index or variable, whose value ranges from 0 to 23, is signaled to indicate the motion magnitude and motion direction, and the index of the motion magnitude and the index of the motion direction are derived as follows:

[0159] Index of the motion amplitude = the signaled parameter / (number of motion directions = 4),

[0160] Index of the direction of motion = the signaled parameter % (number of directions of motion = 4).

[0161] In another example, all possible combinations of motion magnitude and motion direction are reordered according to the TM cost, and a parameter (eg, index) is signaled to indicate which combination to use.

[0162] In some embodiments, a high-level flag is signaled to indicate whether to use the proposed method, wherein the high-level flag may be an SPS flag, a PPS flag, a picture header level flag, or a stripe level flag. In addition, the high-level flag may be signaled only when TMVP is enabled.

[0163] In some embodiments, a CU level flag is signaled to indicate whether the proposed sbAmvp mode is used. The CU level flag is signaled when the CU is not encoded using merge mode.

[0164] Note that all proposed methods can be combined freely. For example, for a CU encoded using non-merge mode. A CU level flag is signaled to indicate whether the proposed method is used. When the proposed method is used, a first parameter is signaled to indicate whether the MVD is zero. If the MVD is non-zero, a second parameter indicating the motion magnitude and the motion direction of the MVD is signaled. The index of the motion magnitude and the index of the motion direction are derived as follows:

[0165] Index of motion amplitude = the second parameter / number of motion directions,

[0166] Index of the direction of motion = the second parameter % the number of directions of motion.

[0167] In some embodiments, if the interDir in the CU-level motion information is not signaled, then the interDir in the motion information of the sub-block is inferred to contain the reference direction of the collocated image. In some embodiments, if the refIdx in the CU-level motion information is not signaled, then a refIdx in the motion information of the sub-block is set to the index of the collocated image. Furthermore, AMVR, LIC, BCW, MHP parameters are not signaled and are inferred to be disabled. The OBMC flag is signaled to indicate whether the OBMC is applied to the proposed method.

[0168] In some embodiments, when a CU is encoded using the proposed sbAmvp mode, the CU is partitioned into a plurality of sub-blocks, and each sub-block obtains its own motion vector from the motion field in the collocated image. The derivation of motion information of a plurality of sub-blocks may further include: determining motion shift by adding MVD to MVP; identifying the plurality of corresponding sub-blocks in the collocated image with the motion shift; and converting the motion information of the corresponding sub-blocks into the motion information of the sub-blocks.

[0169] In some embodiments, when the motion information of the corresponding sub-block is not available, the motion information of the central sub-block is used. If the motion information of the central sub-block is also not available, the motion shift is used.

[0170] In some embodiments, the plurality of candidates for motion directions may be changed according to an index of the motion magnitude. Fig.23 An example diagram of multiple candidates with fixed motion directions according to some embodiments of the present disclosure is shown. Fig.24 An example diagram of two sets of motion directions according to some embodiments of the present disclosure is shown. Fig.23An example is shown), such as Fig.24 As shown, two groups of motion directions are proposed. In the first group 2410, horizontal and vertical directions are supported. In the second group 2420, four diagonal directions are supported. When the index of the motion amplitude is an even number, the first group 2410 of motion directions is used. When the index of the motion amplitude is an odd number, the second group 2420 of motion directions is used. Fig.25 Some embodiments of the present disclosure are shown. Fig.24 An example diagram of multiple candidates for two sets of motion directions is shown. Fig.25 As shown, multiple MVD candidates in two sets of motion directions are shown.

[0171] Instead of using the same number of motion magnitudes and motion directions for all images in a sequence, it may vary from one image to another, since the characteristics of each image may be different from each other. Therefore, in the present disclosure, it is proposed to use an implicit or explicit method to determine the number of multiple motion magnitudes and multiple motion directions for each image. The proposed method can also be applied to existing ECM coding modes, such as MMVD, SbTMVP.

[0172] Fig.26 A first set of example graphs of motion magnitude and motion direction according to some embodiments of the present disclosure are shown. Fig. 27 A second set of example graphs of motion magnitude and motion direction according to some embodiments of the present disclosure are shown. Fig.28 A third set of motion amplitudes and motion directions according to some embodiments of the present disclosure is shown. Fig.26 As shown, the first set of motion amplitudes and motion directions 2600 includes 4 motion amplitudes with 2 sets of motion directions. Fig. 27 As shown, the second set of motion amplitudes and motion directions 2700 includes 8 motion amplitudes with 2 sets of motion directions. Fig.28 As shown, the third set of motion amplitudes and motion directions 2800 includes 4 motion amplitudes with 8 fixed motion directions.

[0173] In some embodiments, the number of the plurality of motion amplitudes and the plurality of motion directions is determined according to whether the current image or the current strip is a low-latency image. For example, if the current image or the current strip is a low-latency image, 8 motion amplitudes with 2 sets of motion directions are used ( Fig. 27 If the current image / strip is a non-low-delay image, 8 motion amplitudes with 8 fixed motion directions are used ( Fig.28 The third group 2800).

[0174] In some embodiments, the number of the plurality of motion magnitudes and the plurality of motion directions is determined according to the POC difference between the current image and its collocated image. For example, if the POC difference between the current image / strip and its collocated image is greater than a positive integer value (e.g., 2), 4 motion magnitudes with 2 sets of motions ( Fig.26 If the POC difference between the current picture / strip and its collocated picture is equal to or less than a positive integer value (e.g., 2), 8 motion amplitudes with 8 fixed motion directions are used ( Fig.28 The third group 2800).

[0175] In some embodiments, the number of the plurality of motion magnitudes and the plurality of motion directions is determined according to the temporal layer. For example, if the temporal layer of the current image / strip is greater than a non-negative integer value (e.g., 3), then 4 motion magnitudes with 2 sets of motion directions are used ( Fig.26 If the temporal level of the current picture / strip is equal to or less than a non-negative integer value (e.g., 3), 8 motion amplitudes with 8 fixed motion directions are used ( Fig.28 The third group in ).

[0176] In some embodiments, the number of the plurality of motion amplitudes and the plurality of motion directions is determined according to the parity of the number of the POC of the current image / strip. For example, if the POC of the current image / strip is an even number, 8 motion amplitudes with 8 fixed motion directions are used ( Fig.28 If the POC of the current picture / strip is an odd number, 8 motion amplitudes with 2 sets of motion directions are used ( Fig. 27 The second group 2700).

[0177] and Figure 26-28 The relevant multiple motion amplitudes are listed in the following Tables 1-3 respectively. Table 1: 4 movement amplitudes with 2 sets of movement directions Index of range of motion 0 1 2 3 Amplitude of motion (in luma samples) 4 8 12 16 Table 2: 8 movement amplitudes with 2 sets of movement directions Index of range of motion 0 1 2 3 4 5 6 7 Amplitude of motion (in luma samples) 4 8 12 16 20 24 28 32 Table 3: 8 movement amplitudes with 8 fixed movement directions Index of range of motion 0 1 2 3 4 5 6 7 Amplitude of motion (in luma samples) 4 8 12 16 20 24 28 32

[0178] In some embodiments, the number of the plurality of motion magnitudes and the plurality of motion directions is determined according to a quantization parameter (QP) of the current picture / slice.

[0179] The disclosed methods for determining the number of the plurality of motion amplitudes and the plurality of motion directions may be freely combined.

[0180] In some embodiments, the number of the plurality of motion magnitudes and the plurality of motion directions is determined according to the POC difference between the current image and its juxtaposed image and whether the current image is a low-latency image. Fig.29 2900 is shown for determining the set of multiple motion magnitudes and multiple motion directions to use according to some embodiments of the present disclosure. Fig.29 As shown, in step 2902, it is determined whether the POC difference between the current image and a collocated image corresponding to the current image is greater than a positive integer. If the POC difference between the current image and its collocated image is greater than the positive integer value (e.g., 2), then in step 2904, 4 motion amplitudes ( Fig.26 The first group 2600 in FIG. 2600). If the POC difference between the current image and its collocated image is equal to or less than the positive integer value (e.g., 2), proceed to step 2906 to determine whether the current image is a low-delay image. If the current image is a low-delay image, then in step 2908, 8 motion amplitudes ( Fig. 27 If the current image is a non-low-delay image, then in step 2910, 8 motion amplitudes ( Fig.28 The third group 2800).

[0181] In some embodiments, the number of the plurality of motion amplitudes and the plurality of motion directions is determined according to a time layer and whether the current image is a low-latency image. Fig.30 Another flow chart 3000 for determining the set of multiple motion magnitudes and multiple motion directions to use is shown in accordance with some embodiments of the present disclosure. Fig.30 As shown, in step 3002, it is determined whether the current image is a low-delay image. If the current image is a low-delay image in step 3004, 8 motion amplitudes ( Fig. 27 2700 in the second group). If the current image is a non-low-delay image, go to step 3006 to determine whether a temporal layer of the current image is greater than a non-negative integer (e.g., 4). If the temporal layer of the current image is greater than the non-negative integer (e.g., 4), then in step 3008, use 4 motion amplitudes ( Fig.26 If the temporal layer of the current image is equal to or less than the non-negative integer, then in step 3010, 8 motion amplitudes ( Fig.23The third group 2800).

[0182] In some embodiments, instead of implicitly determining the number of the multiple motion magnitudes and multiple motion directions for each picture, the settings may be explicitly signaled at a picture header or a slice header.

[0183] In some embodiments, two flags are signaled to indicate the number of the plurality of motion magnitudes and the number of the plurality of motion directions, respectively. For example, a first flag is signaled to indicate whether the number of the plurality of motion magnitudes is equal to 4 or 8, and a second flag is signaled to indicate whether the number of the plurality of motion directions is fixed to 8 or adaptively switched between two sets of motion directions.

[0184] In some embodiments, only three settings are supported for the number of multiple motion amplitudes and multiple motion directions ( Fig.26 , Fig. 27 , Fig.28 ). A first flag is signaled to indicate whether 8 motion amplitudes with 8 fixed motion directions are used ( Fig.28 If 8 motion amplitudes with 8 fixed motion directions are not used, a second flag is signaled to indicate the use of 4 motion amplitudes with 2 sets of motion directions ( Fig.26 The first group 2600) and 8 motion amplitudes with 2 sets of motion directions ( Fig. 27 Which one of the second group 2700).

[0185] In some embodiments, the first flag and the second flag are signaled only for non-low-latency images. For low-latency images, they are always fixed to 8 motion amplitudes with 2 sets of motion directions ( Fig. 27 The second group 2700).

[0186] In some embodiments, for the encoder side, a fast algorithm may be applied. Basically, the proposed sbAmvp mode may not be tested according to the best coding mode of a current block or the best coding mode of a historical block located at the same position and having the same size as the current block.

[0187] For example, when the best coding mode of a current block is not the SbTMVP mode, and the historical blocks are not encoded using the SbTMVP mode or the proposed method, the number of MVDs to be tested for the proposed sbAmvp mode is reduced by half.

[0188] In order to reduce the complexity of the encoder and the decoder, in some embodiments, a slice header flag is signaled to indicate whether the proposed sbAmvp mode is enabled for the current slice. The slice header flag is determined based on the enabled area of ​​the proposed sbAmvp mode and the SbTMVP mode in multiple images that have been previously decoded. When the enabled area is greater than a predefined threshold, the proposed mode is enabled for the current slice and the slice header flag is set to true. The enabled area refers to the number of samples / pixels encoded using the sbAmvp / SbTMVP mode. Otherwise, if the enabled area is not greater than the predefined threshold, the proposed mode is disabled for the current slice and the slice header flag is set to false. In some embodiments, the predefined threshold can be changed based on one or more of the QP, temporal layer, low delay conditions, etc. of the current slice.

[0189] In some embodiments, the proposed sbAmvp mode is disabled for a block whose width or height is smaller than a predefined positive value N. For example, the value N may be set equal to 8.

[0190] In some embodiments, the proposed sbAmvp mode is disabled for a block whose width or height is larger than a predefined positive value M. For example, the value M may be set equal to 64.

[0191] According to some embodiments, multiple collocated images may be used to derive temporal motion. As described above, with respect to the temporal motion derivation used in VVC, an additional collocated image is introduced in the ECM. The additional collocated image is different from the original collocated image used in VVC. Several methods are proposed in this disclosure to select a collocated image for the proposed sbAmvp.

[0192] In some embodiments, the collocated image used for the sbAmvp mode is always fixed to the original collocated image used in VVC.

[0193] In some embodiments, when encoding a block using sbAmvp mode, a flag is signaled to indicate which of the two collocated images is used to derive temporal motion. When the flag is equal to 0, the original collocated image is used. Otherwise, when the flag is equal to 1, the additional collocated image is used.

[0194] In some embodiments, when a block is located in a low-delay picture, a flag is signaled to indicate which of the two collocated pictures is used to derive temporal motion. When a block is located in a non-low-delay picture, the collocated picture is always fixed to the original collocated picture used in VVC.

[0195] In some embodiments, the above-mentioned block-level reference picture list reordering technique may be applied to reorder the two collocated pictures (the original collocated picture and the additional collocated picture). A flag is signaled to indicate which one of the multiple collocated pictures (with index 0 or with index 1) is used to derive the temporal motion of the proposed sbAmvp mode.

[0196] In some embodiments, two motion vector predictors for the two collocated images are first derived. Then, the TM cost of the two motion vector predictors is calculated. The one with the minimum TM cost is selected, and its corresponding collocated image is used to derive the temporal motion of the proposed sbAmvp mode.

[0197] In some embodiments, when encoding a block using the proposed sbAmvp mode, a flag is signaled to indicate the inter prediction direction of the block. The collocated image used to derive temporal motion is determined according to the inter prediction direction of the block. The collocated image is selected from a list of reference images to which the inter prediction direction of the block points.

[0198] In some embodiments, the number of the plurality of collocated images is determined according to whether the current image is a low-delay image. If the current image is a low-delay image, two collocated images (the original collocated image used in VVC and the additional collocated image) are used, and a flag is signaled for an sbAmvp coding block to indicate which of the plurality of collocated images is used. If the current image is a non-low-delay image, only the original collocated image is used.

[0199] According to some embodiments, various methods may be used to select the reference image. In the design of the sbTMVP mode, the reference index of the sub-block is selected from any one of a plurality of reference images in a reference image list. The selected reference image is the reference image whose scaling factor is closest to 1. The temporal motion scaling is applied after identifying the reference index.

[0200] In some embodiments, in the proposed sbAmvp mode, the reference indexes of the sub-blocks are derived in the same manner as in the sbTMVP mode. That is, the reference indexes of the sub-blocks may be different from each other.

[0201] In some embodiments, in order to reduce discontinuity between multiple sub-blocks, it is proposed to fix the reference index of the sub-block within a block, and the block is encoded using the sbAmvp mode. As an example, the reference index of the sub-block is fixed to 0. As another example, the reference index of each sub-block is first calculated. For each sub-block, a reference index with a scaling factor closest to 1 is selected. A histogram is constructed using the multiple reference indexes of the selected multiple sub-blocks. Then, the reference index with the highest amplitude is selected for all sub-blocks, and the temporal motion scaling is applied to scale all temporal motions of multiple sub-blocks to the reference index with the highest amplitude.

[0202] According to some embodiments, the proposed sbAmvp mode can be combined with an adaptive motion vector resolution (AMVR) mode. In the conventional inter-frame mode, the motion vector difference can be signaled with a difference resolution including {1 / 4-pixel, 1-pixel, 4-pixel, 1 / 2-pixel}. In order to fully utilize the advantages of the AMVR mode, the present disclosure proposes to combine the proposed sbAmvp mode with the AMVR mode.

[0203] In some embodiments, when a block is encoded using the sbAmvp mode, a parameter (e.g., an index) is signaled to indicate the resolution of the MVD amplitude. The parameter can be signaled in the same manner as the conventional inter-frame mode. Considering that the TMVP is stored in a 4×4 block grid, the MVD amplitude can be a multiple of 4-pixels in order to obtain different TMVPs. Therefore, when the proposed sbAmvp mode is combined with AMVR, the resolution of the MVD amplitude of sbAmvp is {4-pixels, 16-pixels, 64-pixels, 8-pixels}.

[0204] In some embodiments, the resolution of the MVD amplitude of the sbAmvp can be any subset of {4-pixel, 16-pixel, 64-pixel, 8-pixel}. For example, the resolution of the MVD amplitude of the sbAmvp is {4-pixel, 16-pixel}. In another example, the resolution of the MVD amplitude of the sbAmvp is {4-pixel, 64-pixel}.

[0205] In some embodiments, the combination of the sbAmvp and AMVR modes is enabled for larger sequence resolutions because the AMVR mode achieves higher gains for those sequences. For example, when the sequence resolution is greater than or equal to 720p (i.e., 1280×720), the combination of the sbAmvp and AMVR modes is enabled. When a block is encoded using the sbAmvp mode, a parameter is signaled to indicate the resolution of the MVD amplitude. When the sequence resolution is less than 720p, the combination of the sbAmvp and AMVR modes is disabled. The resolution of the MVD amplitude of the sbAmvp is always fixed to 4-pixel.

[0206] According to some embodiments, the proposed sbAmvp mode can be combined with the multiple hypothesis prediction (MHP) mode. In the ECM design, when a block is encoded using a bidirectional prediction mode with unequal weights, the multiple hypothesis prediction (MHP) mode can be applied. When the MHP mode is applied, one or two additional motion information is signaled to improve the prediction quality.

[0207] In some embodiments, the MHP is applied to the sbAmvp mode. When a block is encoded using the sbAmvp mode, a flag is signaled to indicate whether the MHP is applied. When the MHP is applied to the sbAmvp encoded block, a first set of prediction samples is generated using the motion information of each sub-block, and a second set of prediction samples is generated using the additional motion information. The first set of prediction samples and the second set of prediction samples are then mixed together.

[0208] According to some embodiments, the proposed sbAmvp mode can be combined with local illumination compensation (LIC). LIC is used in ECM to adjust for illumination changes between a reference image and a current image. The multiple prediction samples of a LIC coding block are modified according to a linear formula (i.e., α*p[x]+β), where α is a scale, β is an offset, and p[x] is a prediction sample. Fig.31 An example diagram of a process for deriving α and β according to some embodiments of the present disclosure is shown. Fig.31 , using the neighboring reconstruction template 3120 of the current CU 3110 and its corresponding reference template 3130 to derive α and β.

[0209] In order to adjust the illumination changes of the sbAmvp mode, the present disclosure proposes to apply the LIC mode to sbAmvp. In some embodiments, a flag is signaled to indicate whether the LIC mode is applied to an sbAmvp coding block. When the LIC is applied, a linear formula is derived at the sub-block level. More specifically, in the sbAmvp mode, a block is divided into multiple sub-blocks. Only multiple sub-blocks located at the CU boundary can be encoded using the LIC mode. For each sub-block located at the CU boundary, the LIC scaling and offset of the sub-block are derived using the adjacent reconstruction template of the sub-block itself and the corresponding reference template of the sub-block. That is, the LIC scaling and offset of each sub-block can be different from each other. Figure 32-34 Different LIC scaling and offset according to some embodiments of the present disclosure are shown respectively. Fig.32 As shown, for the upper left sub-block 3201, the top and left reconstructed samples 3202 are used to derive multiple LIC parameters. Fig.33 As shown, for the upper right sub-block 3301, only the top reconstruction sample 3302 is used because the left adjacent sample of the upper right sub-block 3301 has not been constructed. Fig.34 As shown, for the lower left sub-block 3401, only the left reconstructed samples 3402 are used because the top neighboring samples of the lower left sub-block 3401 have not been constructed yet.

[0210] According to some embodiments, the merge mode with MVD (MMVD) may be applied to SbTMVP candidates. In ECM, the merge mode with MVD (MMVD) is applied to multiple affine merge candidates, but not to multiple SbTMVP candidates. In the present disclosure, it is proposed to extend the MMVD to the SbTMVP mode.

[0211] In some embodiments, when a CU is encoded using SbTMVP with MMVD mode, an index is signaled to indicate the MVD. Then, the MVD is added to the CU by adding the MVD to the CU from multiple neighboring blocks (e.g., a left neighboring block (e.g., Figure 6 The motion shift of the SbTMVP mode is derived from the motion derived from A1)) in .

[0212] Note that all proposed methods can be freely combined.

[0213] In some embodiments, a non-transitory computer-readable storage medium is provided, storing a bitstream of a video. The bitstream includes one or more syntax elements that signal the coding unit (CU) level motion information as described above. In some embodiments, the bitstream includes one or more syntax elements that signal multiple parameters (e.g., multiple flags or multiple indices) included in the sbAmvp mode as proposed above. In some embodiments, the bitstream includes one or more syntax elements that signal multiple parameters (e.g., multiple flags or multiple indices) included in the above method.

[0214] The embodiments may be further described using the following terms: 1. A video decoding method using sub-block temporal motion prediction for conventional inter-frame mode (sbAmvp mode), comprising: Receiving a bitstream comprising one or more syntax elements for signaling coding unit (CU) level motion information associated with a coding unit (CU), wherein the CU is encoded using a conventional inter mode; Divide the CU into a plurality of sub-blocks; and Executing the sbAmvp mode on the plurality of sub-blocks, wherein executing the sbAmvp mode comprises: Motion information of a sub-block of the plurality of sub-blocks is derived based on the signaled CU-level motion information. 2. A method according to clause 1, wherein the CU-level motion information notified by the signal comprises one or more of an inter-frame prediction direction interDir, a reference picture index refIdx, a motion vector predictor MVP or a motion vector difference MVD. 3. The method of clause 2, wherein the motion information for the sub-block is further derived based on a motion shift. 4. The method of clause 3, wherein executing the sbAmvp mode further comprises: Obtaining a motion vector of a sub-block based on the motion shift; identifying a corresponding collocated sub-block in a collocated image of the sub-block; and The motion information of a corresponding collocated sub-block is converted into the motion information of the sub-block. 5. The method of clause 4, wherein the signaled CU-level motion information comprises MVP and MVD, and the MVP points to a collocated reference picture, performing the sbAmvp mode further comprising: The motion shift is determined by adding the MVD to the MVP. 6. The method of clause 4, wherein the signaled CU-level motion information includes only MVP, and the MVP points to a collocated reference picture, and performing the sbAmvp mode further comprises: The MVP is set to the motion shift. 7. A method according to clause 4, wherein the reference index of the sub-block is selected from any one of a plurality of reference pictures in a reference picture list. 8. The method of clause 4, wherein the signaled CU-level motion information includes interDir, refIdx, MVP, and MVD, and the MVP points to a collocated reference picture, and performing the sbAmvp mode further comprises: determining the motion shift by adding the MVD to the MVP; and The reference image of the sub-block is determined using the interDir and refIdx. 9. A method according to clause 4, wherein the sbAmvp mode is combined with one or more of a local illumination compensation (LIC) mode, a multiple hypothesis prediction (MHP) mode, an overlapped block motion compensation (OBMC) mode, a bidirectional prediction with CU level weights (BCW) mode, or an adaptive motion vector resolution (AMVR) mode. 10. The method according to clause 4, further comprising: In response to an interDir not being signaled, the interDir is determined as a reference direction including the collocated image. 11. The method according to clause 4, further comprising: In response to the refldx not being signaled, the refldx is determined as an index of the collocated image. 12. The method according to clause 3, further comprising: determining a motion shift by adding the MVD to the MVP; identifying a plurality of corresponding sub-blocks in a collocated image having the motion shift; and The motion information of the corresponding sub-block is converted into the motion information of the sub-block. 13. The method of clause 4, wherein, before obtaining a motion vector for the sub-block based on the motion shift, performing the sbAmvp mode further comprises: identifying a corresponding collocated sub-block in a collocated image of the sub-block; determining whether motion information of the corresponding collocated sub-block is available; and In response to the motion information of the corresponding collocated sub-block being unavailable, determining whether motion information of a center sub-block is available; and In response to the motion information of the central sub-block being available, the motion information of the central sub-block is converted into the motion information of the sub-block. 14. The method of clause 2, wherein deriving motion information for the sub-block is further based on a collocated block of the CU, wherein the collocated block is in a collocated picture. 15. The method of clause 14, wherein executing the sbAmvp mode further comprises: identifying a corresponding collocated sub-block in the collocated image for the sub-block; Scaling the motion information of the collocated sub-blocks to a reference image; Adding a scaled motion to a CU level motion to obtain a certain motion of the sub-block; and Motion information of the sub-block including motion of the sub-block is obtained. 16. A method according to clause 15, wherein all reference images of the plurality of sub-blocks are the same. 17. The method of clause 16, wherein the signaled CU-level motion information includes the interDir, the refIdx, the MVP, and the MVD, and performing the sbAmvp mode further comprises: Determining the reference picture using the refldx of the CU-level motion information; and The CU-level motion is determined by adding the MVD to the MVP. 18. The method of clause 16, wherein the signaled CU-level motion information comprises interDir, MVP, and MVD, and the signaled CU-level motion information points to a predefined reference picture, performing the sbAmvp mode further comprising: setting the reference image to the predefined reference image; and The CU-level motion is determined by adding the MVD to the MVP. 19. The method of clause 16, wherein the signaled CU-level motion information comprises interDir, refIdx, and MVP, the reference picture being determined using the refIdx; and performing the sbAmvp mode further comprises: Set the CU-level motion as the MVP. 20. The method of clause 16, wherein the signaled CU-level motion information comprises interDir and MVD, and performing the sbAmvp mode further comprises: Determine the reference image and the MVP using a template matching (TM) cost; and The CU-level motion is determined by adding the MVD to the MVP. 21. A method according to clause 20, wherein the TM cost is calculated using a motion vector prediction candidate. 22. A method according to clause 20, wherein the TM cost is generated by adding a motion vector prediction candidate to the MVD. 23. The method of clause 20, wherein the reference picture of the signaled CU-level motion information and the MVP are set to a picture with a minimum TM cost. 24. The method of clause 15, wherein the reference images of the plurality of sub-blocks are different. 25. The method of clause 24, wherein the signaled CU-level motion information includes interDir, refIdx, MVP, and MVD, and performing the sbAmvp mode further comprises: determining the motion information of the collocated sub-block and the reference image of each sub-block, wherein the reference image is selected from one of a plurality of reference images in a reference image list, and the selected reference image is a reference image having a scaling factor closest to 1; and The CU-level motion is determined by adding the MVD to the MVP. 26. The method of clause 24, wherein the signaled CU-level motion information comprises interDir, MVP, and MVD, and the signaled CU-level motion information points to a predefined reference picture, and performing the sbAmvp mode further comprises: Setting a collocated reference image as the predefined reference image; determining motion information of the collocated sub-blocks and a reference image for each sub-block, wherein the reference image is selected from any one of a plurality of reference images in a reference image list, and the selected reference image has a scaling factor closest to 1; and The CU-level motion is determined by adding the MVD to the MVP. 27. The method according to clause 26, wherein the predefined reference picture is signaled at slice level, picture level, PPS level or SPS level. 28. A method according to clause 26, wherein the collocated sub-blocks are encoded using a non-inter mode, and the motion information of the collocated sub-blocks is selected from one of zero motion, a motion of a center sub-block of the CU, an average motion of a top sub-block and a left sub-block, an average motion of multiple surrounding sub-blocks, or an average motion of the CU. 29. A method according to clause 28, wherein the average motion is obtained by scaling a motion of each sub-block to a same reference image, and the reference image is indicated by the signaled CU-level motion information or is the collocated reference image. 30. The method of clause 28, wherein executing the sbAmvp mode further comprises: Determine the reference picture of a plurality of sub-blocks using a refIdx of the signaled CU-level motion information; determining the CU-level motion by adding the MVD to the MVP; Obtaining a scaled motion by scaling the motion information of the collocated sub-block to the reference image; Calculating an average motion of all said scaling motions; obtaining a difference motion by subtracting the average motion from the scaled motion; and The difference motion is added to the CU level motion. 31. The method of clause 1, wherein a header flag is signaled to indicate whether the sbAmvp mode is enabled or disabled. 32. A method according to clause 1, wherein enabling or disabling the sbAmvp mode for the current slice is determined based on the sbAmvp mode in a previously decoded image and an enabled area of ​​a sub-block based temporal motion vector prediction (SbTMVP) mode. 33. The method according to clause 32, further comprising: determining whether the enabled area is greater than a predefined threshold; and In response to the enabled area being greater than the predefined threshold, enabling the sbAmvp mode of the current stripe; or In response to the enabled area being equal to or less than the predefined threshold, disabling the sbAmvp mode of the current slice. 34. The method of clause 33, wherein the predefined threshold is determined based on one or more of a quantization parameter QP of the current slice, a temporal layer of the current slice, and a low delay condition. 35. The method of clause 1, wherein the sbAmvp mode is disabled for a block when a width or a height of the block is less than a first predefined positive value. 36. The method of clause 1, wherein the sbAmvp mode is disabled for a block when a width or a height of the block is greater than a second predefined positive value. 37. The method of clause 1, wherein a collocated picture used to derive motion information for the sub-block is an original collocated picture used in VVC. 38. The method according to clause 1, further comprising: decoding a flag indicating a collocated image for deriving motion information of the sub-block; When the flag is equal to a first value, determining that the collocated image is a first collocated image used in VVC; and When the flag is equal to a second value, the juxtaposed image is determined to be a second juxtaposed image different from the first juxtaposed image. 39. The method according to clause 1, further comprising: A flag is decoded, the flag indicating which one of a first collocated image for VVC and a second collocated image different from the first collocated image is used to derive motion information of the sub-block, wherein the first collocated image and the second collocated image are reordered based on a block-level reference image re-encoding method. 40. The method according to clause 1, further comprising: deriving a first motion vector prediction value for a first collocated picture for Versatile Video Coding (VVC); deriving a second motion vector prediction value for a second collocated image different from the first collocated image; calculating template matching (TM) costs of the first motion vector predictor and the second motion vector predictor respectively; and A collocated picture for deriving the motion information of the sub-block is determined as a corresponding collocated picture having a minimum TM cost. 41. The method according to clause 1, further comprising: decoding a flag indicating an inter-prediction direction of a block; and A collocated image for deriving motion information of the sub-block is determined according to the inter-prediction direction of the block. 42. A method according to clause 41, wherein the collocated picture is selected from a list of reference pictures to which the inter-prediction direction of the block points. 43. The method according to clause 1, further comprising: Determining the number of the plurality of juxtaposed images according to whether the current image is a low-latency image; In response to the current image being a low-delay image, determining that the plurality of collocated images include a first collocated image used in a VVC and a second collocated image different from the first collocated image; and decoding a flag indicating which of the plurality of collocated images is used to derive motion information of the sub-block; or In response to the current picture not being a low-delay picture, a collocated picture for deriving motion information of the sub-block is determined as the first collocated picture used in the VVC. 44. The method according to clause 1, further comprising: identifying a reference index of the sub-block from one of a plurality of reference pictures in a reference picture list, wherein the selected one of the reference pictures has a scaling factor closest to 1; and A temporal motion scaling is performed after identifying the reference index. 45. A method according to clause 44, wherein the plurality of reference indices of the plurality of sub-blocks are different. 46. ​​The method of clause 1, wherein a reference index of the sub-blocks within a CU is fixed. 47. A method according to clause 46, wherein the reference index is fixed to 0. 48. The method according to clause 46, further comprising: determining a plurality of reference indexes for the plurality of sub-blocks, one reference index for each sub-block, wherein a scaling factor of the reference index is closest to 1; constructing a histogram using the determined plurality of reference indices of the plurality of sub-blocks; selecting a reference index having the highest magnitude for all sub-blocks; and Temporal motion scaling is performed to scale all temporal motions of the plurality of sub-blocks to the selected reference index. 49. The method of clause 1, wherein the sbAmvp mode is combined with an adaptive motion vector resolution (AMVR) mode. 50. The method of clause 49, wherein the resolution of the MVD amplitude is {4-pixel, 16-pixel, 64-pixel, 8-pixel}. 51. A method according to clause 49, wherein the sbAmvp mode is combined with an adaptive motion vector resolution AMVR mode and the resolution of the MVD amplitude is selected from a subset of {4-pixel, 16-pixel, 64-pixel, 8-pixel}. 52. The method of clause 49, wherein whether to enable a combination of the sbAmvp mode and the AMVR mode is determined based on a sequence resolution. 53. The method according to clause 52, further comprising: When the sequence resolution is greater than or equal to 720p, enabling a combination of the sbAmvp mode and the AMVR mode, and decoding a parameter indicating a resolution of the MVD amplitude; or When the sequence resolution is equal to or less than or equal to 720p, the combination of the sbAmvp mode and the AMVR mode is disabled, and the resolution of the MVD amplitude is fixed to 4-pixel. 54. The method of clause 1, wherein the sbAmvp mode is combined with a multiple hypothesis prediction (MHP) mode. 55. The method according to clause 54, further comprising: decoding a flag, the flag indicating whether the MHP mode is applied; In response to applying the MHP mode, generating a first set of prediction samples using the motion information of each sub-block and generating a second set of prediction samples using additional motion information; and The first set of prediction samples and the second set of prediction samples are mixed. 56. The method of clause 1, wherein the sbAmvp mode is combined with a local illumination compensation (LIC) mode. 57. The method according to clause 56, further comprising: decoding a flag indicating whether the LIC is applied; and In response to applying the LIC, a linear formula is derived at a sub-block level, where the linear formula is α*p[x]+β, where α is a scale, β is an offset, and p[x] is a prediction sample. 58. The method of clause 56, wherein the LIC mode is applied to a sub-block located at a CU boundary. 59. A method according to clause 58, wherein a LIC scale and an offset for the sub-block are derived using a corresponding neighbouring reconstruction template and a corresponding reference template for the sub-block. 60. A method of decoding a bitstream to output one or more images of a video stream, the method comprising: receiving a bit stream; and Decoding one or more images using the encoding information of the bitstream, wherein the decoding comprises: A number of a plurality of motion magnitudes and a number of a plurality of motion directions of the image are determined. 61. The method of clause 60, wherein the number of the plurality of motion amplitudes and the number of the plurality of motion directions are determined based on whether the current image is a low-latency image. 62. A method according to clause 61, wherein, when the current image is the low-latency image, 8 motion amplitudes with 2 sets of motion directions are used; and, when the current image is a non-low-latency image, 8 motion amplitudes with 8 fixed motion directions are used. 63. The method of clause 60, wherein the number of the plurality of motion magnitudes and the number of the plurality of motion directions are determined based on a picture order count (POC) difference between the current image and the corresponding collocated image. 64. A method according to clause 63, wherein, when the POC difference between the current image and the corresponding juxtaposed image is greater than a positive integer value, 4 motion amplitudes with 2 sets of motion directions are used; and, when the POC difference between the current image and the corresponding juxtaposed image is equal to or less than a positive integer value, 8 motion amplitudes with 8 fixed motion directions are used. 65. The method of clause 60, wherein the number of motion magnitudes and the number of motion directions are determined based on a temporal layer of the current image. 66. A method according to clause 65, wherein, when the temporal layer of the current image is greater than a non-negative integer value, 4 motion amplitudes with 2 sets of motion directions are used; and, when the temporal layer of the current image is equal to or less than the non-negative integer value, 8 motion amplitudes with 8 fixed motion directions are used. 67. The method of clause 60, wherein the number of the plurality of motion magnitudes and the number of the plurality of motion directions are determined based on the parity of the number of POCs of the current image. 68. A method according to clause 67, wherein when the POC of the current image is odd, 4 motion amplitudes with 2 sets of motion directions are used; and when the POC of the current image is even, 8 motion amplitudes with 8 fixed motion directions are used. 69. The method of clause 60, wherein the number of the plurality of motion magnitudes and the number of the plurality of motion directions are determined based on a QP of the current picture. 70. A method according to clause 60, wherein the number of the multiple motion amplitudes and the number of the multiple motion directions are determined based on a POC difference between a current image and a corresponding juxtaposed image of the current image and whether the current image is a low-delay image. 71. The method according to clause 70, further comprising: Determine whether a POC difference between a current image and a corresponding concatenated image of the current image is greater than a positive integer; In response to a POC difference between the current image and the corresponding collocated image being greater than the positive integer, using 4 motion amplitudes with 2 sets of motion directions; and In response to a POC difference between the current image and the corresponding collocated image being equal to or less than the positive integer, determining whether the current image is a low-delay image; In response to the current image being a low-delay image, using 8 motion amplitudes with 2 sets of motion directions; and In response to the current image being a non-low-delay image, 8 motion amplitudes with 8 fixed motion directions are used. 72. The method of clause 60, wherein the number of the plurality of motion amplitudes and the number of the plurality of motion directions are determined based on a temporal layer of a current image and whether the current image is a low-delay image. 73. The method according to clause 72, further comprising: Determining whether the current image is a low-latency image; In response to the current image being a low-delay image, using 8 motion amplitudes with 2 sets of motion directions; or In response to the current image not being a low-delay image, determining whether the temporal layer of the current image is greater than a non-negative integer; In response to the temporal level of the current image being greater than a non-negative integer, using 4 motion magnitudes with 2 sets of motion directions; and In response to the temporal level of the current image being equal to or less than a non-negative integer, 8 motion amplitudes with 8 fixed motion directions are used. 74. The method according to clause 60, further comprising: decoding a first flag indicating a number of a plurality of motion amplitudes; and A second flag is decoded, the second flag indicating the number of the plurality of motion directions. 75. A method according to clause 74, wherein the first flag indicates whether the number of multiple motion amplitudes is equal to 4 or 8, and the second flag indicates whether the number of multiple motion directions is fixed to 8 or adaptively switched between two groups of motion directions. 76. The method according to clause 60, further comprising: decoding a first flag indicating whether to use 8 motion amplitudes with 8 fixed motion directions; and In response to not using the 8 motion amplitudes with the 8 fixed motion directions, a second flag is decoded, the second flag indicating which of the 4 motion amplitudes with the 2 sets of motion directions or the 8 motion amplitudes with the 2 sets of motion directions is used. 77. A method according to clause 76, wherein the first flag and the second flag are signaled for multiple non-low-latency images. 78. A method according to clause 60, wherein 8 motion amplitudes with 2 sets of motion directions are used for multiple low-latency images. 79. A video decoding method, comprising: decoding an index indicating a motion vector difference MVD; and A motion shift for sub-block based temporal motion vector prediction (SbTMVP) mode is derived by adding the MVD to a motion derived from a plurality of neighboring blocks. 80. A video encoding method using sub-block temporal motion prediction for conventional inter-frame mode (sbAmvp mode), comprising: receiving a video sequence; encoding one or more images of the video sequence; and Generates a bit stream; wherein the encoding comprises: Encoding a coding unit CU using the conventional inter-frame mode; signaling CU-level motion information associated with the CU; Divide the CU into a plurality of sub-blocks; and Executing the sbAmvp mode on the plurality of sub-blocks, wherein executing the sbAmvp mode comprises: Motion information of a sub-block of the plurality of sub-blocks is derived based on the signaled CU-level motion information. 81. A method according to clause 80, wherein the signaled CU-level motion information includes one or more of an inter-frame prediction direction interDir, a reference picture index refIdx, a motion vector predictor MVP or a motion vector difference MVD. 82. A method according to clause 81, wherein deriving the motion information of the sub-block is further based on motion shifting. 83. The method of clause 82, wherein executing the sbAmvp mode further comprises: Obtaining a motion vector of a sub-block based on the motion shift; identifying a corresponding collocated sub-block in a collocated image of the sub-block; and The motion information of a corresponding collocated sub-block is converted into the motion information of the sub-block. 84. The method of clause 83, wherein the signaled CU-level motion information comprises MVP and MVD, and the MVP points to a collocated reference picture, performing the sbAmvp mode further comprising: The motion shift is determined by adding the MVD to the MVP. 85. The method of clause 83, wherein the signaled CU-level motion information includes only MVP, and the MVP points to a collocated reference picture, and performing the sbAmvp mode further comprises: The MVP is set to the motion shift. 86. A method according to clause 83, wherein the reference index of the sub-block is selected from any one of a plurality of reference pictures in a reference picture list. 87. The method of clause 83, wherein the signaled CU-level motion information includes interDir, refIdx, MVP, and MVD, and the MVP points to a collocated reference picture, and performing the sbAmvp mode further comprises: determining the motion shift by adding the MVD to the MVP; and The reference image of the sub-block is determined using the interDir and refIdx. 88. A method according to clause 83, wherein the sbAmvp mode is combined with one or more of a local illumination compensation (LIC) mode, a multiple hypothesis prediction (MHP) mode, an overlapped block motion compensation (OBMC) mode, a bidirectional prediction with CU level weights (BCW) mode, or an adaptive motion vector resolution (AMVR) mode. 89. The method according to clause 83, further comprising: In response to an interDir not being signaled, the interDir is determined to include the reference direction of the collocated image. 90. The method according to clause 83, further comprising: In response to the refldx not being signaled, the refldx is determined as an index of the collocated image. 91. The method according to clause 82, further comprising: determining a motion shift by adding the MVD to the MVP; identifying a plurality of corresponding sub-blocks in a collocated image having the motion shift; and The motion information of the corresponding sub-block is converted into the motion information of the sub-block. 92. The method of clause 83, wherein, before obtaining a motion vector for the sub-block based on the motion shift, performing the sbAmvp mode further comprises: identifying a corresponding collocated sub-block in a collocated image of the sub-block; determining whether motion information of the corresponding collocated sub-block is available; and In response to the motion information of the corresponding collocated sub-block being unavailable, determining whether motion information of a center sub-block is available; and In response to motion information of a central sub-block being available, the motion information of the central sub-block is converted into the motion information of the sub-block. 93. A non-transitory computer-readable storage medium storing a bit stream, wherein the operation of generating the bit stream comprises: Encode a coding unit (CU) using a conventional inter-frame mode; signaling CU-level motion information associated with the CU; Divide the CU into a plurality of sub-blocks; and Executing an sbAmvp mode on the plurality of sub-blocks, wherein executing the sbAmvp mode comprises: Motion information of a sub-block of the plurality of sub-blocks is derived based on the signaled CU-level motion information. 94. A non-transitory computer-readable storage medium according to clause 93, wherein the signaled CU-level motion information includes one or more of an inter-frame prediction direction interDir, a reference image index refIdx, a motion vector prediction value MVP, or a motion vector difference MVD. 95. The non-transitory computer-readable storage medium of clause 94, wherein deriving the motion information of the sub-block is further based on a motion shift. 96. The non-transitory computer-readable storage medium of clause 95, wherein executing the sbAmvp mode further comprises: Obtaining a motion vector of a sub-block based on the motion shift; identifying a corresponding collocated sub-block in a collocated image of the sub-block; and The motion information of a corresponding collocated sub-block is converted into the motion information of the sub-block. 97. The non-transitory computer-readable storage medium of clause 96, wherein the signaled CU-level motion information comprises MVP and MVD, and wherein the MVP points to a collocated reference picture, and performing the sbAmvp mode further comprises: The motion shift is determined by adding the MVD to the MVP. 98. The non-transitory computer-readable storage medium of clause 96, wherein the signaled CU-level motion information includes only MVP, and the MVP points to a collocated reference picture, and executing the sbAmvp mode further comprises: The MVP is set to the motion shift. 99. The non-transitory computer-readable storage medium of clause 96, wherein the reference index of the sub-block is selected from any one of a plurality of reference pictures in a reference picture list. 100. The non-transitory computer-readable storage medium of clause 96, wherein the signaled CU-level motion information includes interDir, refIdx, MVP, and MVD, and the MVP points to a collocated reference picture, and executing the sbAmvp mode further comprises: determining the motion shift by adding the MVD to the MVP; and The reference image of the sub-block is determined using the interDir and refIdx. 101. A non-transitory computer-readable storage medium as described in clause 83, wherein the sbAmvp mode is combined with one or more of a local illumination compensation (LIC) mode, a multiple hypothesis prediction (MHP) mode, an overlapped block motion compensation (OBMC) mode, a bidirectional prediction with CU level weights (BCW) mode, or an adaptive motion vector resolution (AMVR) mode. 102. The non-transitory computer-readable storage medium of clause 96, wherein the operations further comprise: In response to an interDir not being signaled, the interDir is determined to include the reference direction of the collocated image. 103. The non-transitory computer-readable storage medium of clause 96, wherein the operations further comprise: In response to the refldx not being signaled, the refldx is determined as an index of the collocated image. 104. The non-transitory computer-readable storage medium of clause 95, wherein the operations further comprise: determining a motion shift by adding the MVD to the MVP; identifying a plurality of corresponding sub-blocks in a collocated image having the motion shift; and The motion information of the corresponding sub-block is converted into the motion information of the sub-block. 105. The non-transitory computer-readable storage medium of clause 96, wherein before obtaining a motion vector for the sub-block based on the motion shift, performing the sbAmvp mode further comprises: identifying a corresponding collocated sub-block in a collocated image of the sub-block; determining whether motion information of the corresponding collocated sub-block is available; and In response to the motion information of the corresponding collocated sub-block being unavailable, determining whether motion information of a center sub-block is available; and In response to motion information of a central sub-block being available, the motion information of the central sub-block is converted into the motion information of the sub-block. 106. A video encoding method, comprising: receiving a video sequence; encoding one or more images of the video sequence; and Generate a bit stream; wherein the encoding comprises: A number of a plurality of motion magnitudes and a number of a plurality of motion directions of the image are determined. 107. A non-transitory computer-readable storage medium storing a bit stream, wherein the operation of generating the bit stream comprises: A number of a plurality of motion magnitudes and a number of a plurality of motion directions of the image are determined. 108. A video encoding method, comprising: receiving a video sequence; encoding one or more images of the video sequence; and Generate a bit stream; wherein the encoding comprises: deriving a motion shift for sub-block based temporal motion vector prediction SbTMVP mode by adding a motion vector difference MVD to a motion derived from a plurality of left neighboring blocks; and An index indicating the MVD is encoded. 109. A non-transitory computer-readable storage medium storing a bit stream, wherein the operation of generating the bit stream comprises: deriving a motion shift for sub-block based temporal motion vector prediction SbTMVP mode by adding a motion vector difference MVD to a motion derived from a plurality of left neighboring blocks; and An index indicating the MVD is encoded. 110. A video decoding device, comprising: a memory configured to store a set of instructions; and One or more processors configured to execute the set of instructions to cause the apparatus to perform a video compression method according to any one of clauses 1 to 79. 111. A video encoding device, comprising: a memory configured to store a set of instructions; and One or more processors configured to execute the set of instructions to cause the apparatus to perform a video decoding method according to any of clauses 80 to 92, 106 and 108. 112. A computer program product comprising: a set of computer program instructions, the set of computer program instructions causing a computer to perform the video decoding method according to any one of clauses 1 to 79. 113. A computer program product comprising: a set of computer program instructions, the set of computer program instructions causing a computer to perform the video encoding method according to any one of clauses 80 to 92, 106 and 108. 114. A computer program, and the computer program enables a computer to perform the video decoding method according to any one of clauses 1 to 79. 115. A computer program, and the computer program enables a computer to perform the video encoding method according to any of clauses 80 to 92, 106 and 108.

[0215] In some embodiments, a non-transitory computer-readable storage medium is also provided. In some embodiments, the medium may store all or part of a video bitstream containing one or more flags indicating the resampling applied, for example, temporal resampling and spatial resampling. In some embodiments, the medium may store all or part of a video bitstream containing an index indicating a resampling factor. In some embodiments, the medium may store a set of instructions executable by a device (such as the disclosed encoder and decoder) for performing the above method. Common forms of non-transitory media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a pattern of holes, RAM, PROM and EPROM, FLASH-EPROM or any other flash memory, NVRAM, cache, registers, any other storage chip or cassette and networked versions thereof. The device may include one or more processors (CPUs), input / output interfaces, network interfaces or memories.

[0216] It should be noted that relational terms such as "first" and "second" herein are used only to distinguish one entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "comprise", "have", "contain", and "include" and other similar forms are intended to be synonymous and open-ended, in that one or more items following any of these words are not intended to be an exhaustive list of such one or more items, or to be limited to the one or more items listed.

[0217] As used herein, unless otherwise specifically stated, the term "or" encompasses all possible combinations unless not feasible. For example, if a database is specified to include either A or B, then unless otherwise specifically stated or not feasible, the database may include A, or B, or A and B. As a second example, if a database is specified to include A, B, or C, then unless otherwise specifically stated or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0218] It should be understood that the above embodiments can be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-mentioned computer readable medium. When executed by a processor, the software can execute the disclosed method. The computer unit and other functional units described in the present disclosure can be implemented by hardware, or software, or a combination of hardware and software. It should also be understood by those of ordinary skill in the art that the above-mentioned multiple modules / units can be combined into one module / unit, and each of the above-mentioned modules / units can be further divided into multiple sub-modules / sub-units.

[0219] In the foregoing description, embodiments have been described with reference to many specific details, which may vary depending on the implementation. Certain adaptations and modifications may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art by considering the description and practice disclosed herein. The description and embodiments are intended to be considered merely exemplary, and the true scope and nature of the invention are indicated by the appended claims. The order of steps shown in the figures is for illustrative purposes only and is not intended to be limited to any particular order of steps. Therefore, it will be appreciated by those skilled in the art that these steps may be performed in different orders while implementing the same method.

[0220] In the drawings and the specification, exemplary embodiments have been disclosed. However, many variations and modifications may be made to these embodiments. Accordingly, although specific terms are employed, they are used only in a general and descriptive sense and not for the purpose of limitation.

Claims

1. A video decoding method using a sub-block temporal motion prediction for a conventional inter-frame mode (sbAmvp mode), comprising: Receiving a bitstream including one or more syntax elements, the one or more syntax elements signaling CU-level motion information associated with a coding unit (CU), wherein the CU is encoded in a conventional inter mode; Dividing the CU into a plurality of sub-blocks; as well as Executing the sbAmvp mode on the plurality of sub-blocks, wherein executing the sbAmvp mode comprises: Motion information of a sub-block of the plurality of sub-blocks is derived based on the signaled CU-level motion information.

2. The method according to claim 1, wherein: The signaled CU-level motion information includes one or more of an inter-prediction direction interDir, a reference picture index refIdx, a motion vector predictor MVP, or a motion vector difference MVD.

3. The method according to claim 2, wherein: Motion information for the sub-block is further derived based on a motion shift.

4. The method according to claim 3, wherein: Executing the sbAmvp mode further includes: Obtaining a motion vector of a sub-block based on the motion shift; identifying a corresponding collocated sub-block in a collocated image of the sub-block; and The motion information of a corresponding collocated sub-block is converted into the motion information of the sub-block.

5. The method according to claim 4, wherein: The signaled CU-level motion information includes MVP and MVD, and the MVP points to a collocated reference picture, and executing the sbAmvp mode further includes: The motion shift is determined by adding the MVD to the MVP.

6. The method according to claim 4, wherein: The signaled CU-level motion information includes only MVP, and the MVP points to a collocated reference picture, and performing the sbAmvp mode further includes: The MVP is set to the motion shift.

7. The method according to claim 4, wherein: The reference index of the sub-block is selected from any one of a plurality of reference pictures in a reference picture list.

8. The method according to claim 4, wherein: The signaled CU-level motion information includes interDir, refIdx, MVP, and MVD, and the MVP points to a collocated reference picture, and performing the sbAmvp mode further includes: determining the motion shift by adding the MVD to the MVP; and The reference image of the sub-block is determined using the interDir and refIdx.

9. The method according to any one of claims 4 to 8, wherein: The sbAmvp mode is combined with one or more of a local illumination compensation (LIC) mode, a multiple hypothesis prediction (MHP) mode, an overlapped block motion compensation (OBMC) mode, a bidirectional prediction with CU level weights (BCW) mode, or an adaptive motion vector resolution (AMVR) mode.

10. The method according to claim 4, further comprising: In response to the interDir not being signaled, the interDir is determined as a reference direction including the collocated image.

11. The method according to claim 4, further comprising: In response to the refldx not being signaled, the refldx is determined as an index of the collocated image.

12. The method according to claim 3, further comprising: determining the motion shift by adding the MVD to the MVP; identifying a plurality of corresponding sub-blocks in a collocated image having the motion shift; as well as The motion information of the corresponding sub-block is converted into the motion information of the sub-block.

13. The method according to claim 4, wherein: Before obtaining a motion vector of the sub-block based on the motion shift, executing the sbAmvp mode further comprises: identifying a corresponding collocated sub-block in a collocated image of the sub-block; determining whether motion information of the corresponding collocated sub-block is available; and In response to the motion information of the corresponding collocated sub-block being unavailable, determining whether motion information of a center sub-block is available; and In response to the motion information of the central sub-block being available, the motion information of the central sub-block is converted into the motion information of the sub-block.

14. A video encoding method using sub-block temporal motion prediction for conventional inter-mode (sbAmvp mode), comprising: receiving a video sequence; encoding one or more images of the video sequence; as well as Generate a bit stream; The encoding includes: Encoding a coding unit CU using the conventional inter-frame mode; signaling CU-level motion information associated with the CU; Divide the CU into a plurality of sub-blocks; and Executing the sbAmvp mode on the plurality of sub-blocks, wherein executing the sbAmvp mode comprises: Motion information of a sub-block of the plurality of sub-blocks is derived based on the signaled CU-level motion information.

15. The method according to claim 14, wherein: The signaled CU-level motion information includes one or more of an inter-prediction direction interDir, a reference picture index refIdx, a motion vector predictor MVP, or a motion vector difference MVD.

16. The method according to claim 15, wherein: Deriving the motion information of the sub-block is further based on a motion shift.

17. The method according to claim 16, wherein: Executing the sbAmvp mode further includes: Obtaining a motion vector of a sub-block based on the motion shift; identifying a corresponding collocated sub-block in a collocated image of the sub-block; and The motion information of a corresponding collocated sub-block is converted into the motion information of the sub-block.

18. A non-transitory computer-readable storage medium storing a bit stream, the operation of generating the bit stream comprising: Encode a coding unit (CU) using a conventional inter-frame mode; signaling CU-level motion information associated with the CU; Dividing the CU into a plurality of sub-blocks; as well as Executing an sbAmvp mode on the plurality of sub-blocks, wherein executing the sbAmvp mode comprises: Motion information of a sub-block of the plurality of sub-blocks is derived based on the signaled CU-level motion information.

19. The non-transitory computer readable storage medium of claim 18, wherein: The signaled CU-level motion information includes one or more of an inter-frame prediction direction interDir, a reference picture index refIdx, a motion vector predictor MVP, or a motion vector difference MVD.

20. The non-transitory computer readable storage medium of claim 19, wherein: Deriving the motion information of the sub-block is further based on a motion shift.

21. A video decoding device, comprising: a memory configured to store a set of instructions; as well as One or more processors, wherein the one or more processors are configured to execute the instruction set to enable the apparatus to perform the video compression method according to any one of claims 1 to 13.

22. A video encoding device, comprising: a memory configured to store a set of instructions; and One or more processors, wherein the one or more processors are configured to execute the instruction set so that the apparatus performs the video decoding method according to any one of claims 14 to 17.

23. A computer program product comprising: A computer program instruction set, and the computer program instruction set enables a computer to execute the video compression method according to any one of claims 1 to 13.

24. A computer program product comprising: A computer program instruction set, and the computer program instruction set enables a computer to execute the video decoding method according to any one of claims 14 to 17.

25. A computer program, and the computer program enables a computer to execute the video compression method according to any one of claims 1 to 13.

26. A computer program, and the computer program enables a computer to execute the video decoding method according to any one of claims 14 to 17.

Images