Method for temporal motion vector prediction

Abstract

The present disclosure provides a computer-implemented method for video processing. The method comprises receiving a video sequence; and encoding the video sequence by determining whether a co-located block in a co-located picture of a current block has a block vector (BV); in response to the co-located block having the BV, determining the BV; determining a motion vector for temporal motion vector prediction (TMVP) of the current block based on the BV of the co-located block; and determining a reference index of the TMVP as a reference index of the co-located picture.

Description

[0001] Cross-reference to related applications This disclosure claims priority to U.S. Provisional Application No. 63 / 774,822, filed March 20, 2025, and U.S. Patent Application No. 19 / 555,211, filed March 3, 2026, both of which are incorporated herein by reference in their entirety. Technical Field

[0002] This disclosure relates generally to video processing, and more specifically, to a method for predicting temporal motion vectors. Background Technology

[0003] Video consists of a set of still images (or "frames") that capture visual information. To reduce storage memory and transmission bandwidth, video can be compressed before storage or transmission and decompressed before display. This compression process is typically called encoding, while the decompression process is typically called decoding. There are many video coding formats that use standardized video coding techniques, the most common being based on prediction, transform, quantization, entropy coding, and loop filtering. Standardization organizations have developed video coding standards that specify particular video coding formats, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Universal Video Coding (VVC / H.266) standard, and the AVS standard. As video standards adopt increasingly advanced video coding techniques, the coding efficiency of new video coding standards also increases. Summary of the Invention

[0004] Embodiments of this disclosure provide a method for encoding a video sequence. The method includes: receiving a video sequence; and encoding the video sequence by: determining whether a co-location block in a co-location image of a current block has a block vector (BV); determining the BV in response to the co-location block having the BV; determining a motion vector for temporal motion vector prediction (TMVP) of the current block based on the BV of the co-location block; and determining a reference index of the TMVP as a reference index of the co-location image.

[0005] Embodiments of this disclosure provide a method for decoding a bitstream. The method includes receiving a bitstream and decoding the bitstream to generate a video sequence. The decoding includes: determining whether a co-location block in a co-location image of a current block has a block vector (BV); determining the BV in response to the co-location block having the BV; determining a motion vector for temporal motion vector prediction (TMVP) of the current block based on the BV of the co-location block; and determining a reference index of the TMVP as a reference index of the co-location image.

[0006] Embodiments of this disclosure provide a method for transmitting a bitstream. The method includes: receiving a video sequence; encoding the video sequence by: determining whether a co-location block in a co-location image of a current block has a block vector (BV); determining the BV in response to the co-location block having the BV; determining a motion vector for temporal motion vector prediction (TMVP) of the current block based on the BV of the co-location block; and determining a reference index of the TMVP as a reference index of the co-location image; and transmitting a bitstream generated based on the encoding. Attached Figure Description

[0007] Embodiments and aspects of this disclosure are illustrated in the following detailed description and accompanying drawings. Various features shown in the figures are not drawn to scale.

[0008] Figure 1 This is a schematic diagram illustrating the structure of an exemplary video sequence according to some embodiments of the present disclosure.

[0009] Figure 2A This is a schematic diagram illustrating an exemplary encoding process of a hybrid video encoding system consistent with embodiments of this disclosure.

[0010] Figure 2B This is a schematic diagram illustrating another exemplary encoding process of a hybrid video encoding system consistent with embodiments of this disclosure.

[0011] Figure 3A This is a schematic diagram illustrating an exemplary decoding process of a hybrid video coding system consistent with embodiments of this disclosure.

[0012] Figure 3B This is a schematic diagram illustrating another exemplary decoding process of a hybrid video coding system consistent with embodiments of the present disclosure.

[0013] Figure 4 This is a block diagram of an exemplary apparatus for encoding or decoding video according to some embodiments of the present disclosure.

[0014] Figure 5Exemplary motion vector scaling for time merging candidates is shown according to some embodiments of this disclosure.

[0015] Figure 6 Exemplary locations of colocation blocks according to some embodiments of this disclosure are shown.

[0016] Figure 7 This is a flowchart of an exemplary method for deriving motion vectors for time-based motion vector prediction (TMVP) from block vectors (BV) according to some embodiments of this disclosure.

[0017] Figure 8A This is a flowchart of exemplary sub-steps of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure.

[0018] Figure 8B Some embodiments according to this disclosure are shown, and Figure 8A An exemplary illustration consistent with the method described above.

[0019] Figure 9A This is a flowchart of another exemplary sub-step of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure.

[0020] Figure 9B Some embodiments according to this disclosure are shown, and Figure 9A An exemplary illustration consistent with the method described above.

[0021] Figure 10A This is a flowchart of another exemplary sub-step of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure.

[0022] Figure 10B Some embodiments according to this disclosure are shown, and Figure 10A An exemplary illustration consistent with the method described above.

[0023] Figure 11 This is a flowchart of another exemplary sub-step of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure.

[0024] Figure 12 This is a flowchart of an exemplary method for constructing a TMVP candidate list according to some embodiments of this disclosure.

[0025] Figure 13 This illustrates the need to store time motion according to existing technology.

[0026] Figure 14 Exemplary time motions requiring storage are illustrated according to some embodiments of this disclosure. Detailed Implementation

[0027] Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, in which, unless otherwise stated, the same numbers in different figures characterize the same or similar elements. The embodiments set forth in the following description of the exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the invention as described in the appended claims. Specific aspects of this disclosure are described in more detail below. In case of conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall prevail.

[0028] The Joint Video Experts Team (JVET) of the ITU-T Video Coding Expert Group (ITU-T VCEG) and the ISO / IEC Moving Picture Expert Group (ISO / IEC MPEG) is currently developing the Universal Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, VVC aims to achieve the same subjective quality as HEVC / H.265 using half the bandwidth.

[0029] To achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET has been using the Joint Exploration Model (JEM) reference software development techniques other than HEVC. With coding techniques incorporated into JEM, JEM achieves significantly higher coding performance than HEVC.

[0030] The VVC standard has recently been finalized and continues to incorporate more coding technologies to provide better compression performance. VVC adopts the hybrid video coding system used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, and H.263.

[0031] Video is a collection of still images (or "frames") arranged in chronological order to store visual information. Video capture devices (e.g., cameras) can be used to capture and store those images in chronological order, and video playback devices (e.g., televisions, computers, smartphones, tablets, video players, or any end-user terminal with a display) can be used to display these images in chronological order. Furthermore, in some applications, video capture devices can transmit the captured video in real time to the video playback device (e.g., a computer with a display), for example, for video observation, conferencing, or live streaming.

[0032] To reduce the storage space and transmission bandwidth required by these applications, the video can be compressed before storage and transmission, and decompressed before display. This compression and decompression can be implemented by software executed by a processor (e.g., a processor in a general-purpose computer) or by dedicated hardware. The module used for compression is typically called an "encoder," while the module used for decompression is typically called a "decoder." The encoder and decoder can be collectively referred to as a "codec." The encoder and decoder can be implemented as any of a variety of suitable hardware, software, or combinations thereof. For example, the hardware implementation of the encoder and decoder can include circuit systems such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. The software implementation of the encoder and decoder can include program code, computer-executable instructions, firmware, or any suitable computer-implemented algorithm or process embedded in a computer-readable medium. Video compression and decompression can be implemented using various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, H.26x series, etc. In some applications, the codec can decompress the video according to a first encoding standard and recompress the decompressed video using a second encoding standard. In this case, the codec can be called a "transcoder".

[0033] Video coding processes identify and retain useful information that can be used to reconstruct an image, while ignoring information that is not important for reconstruction. If the ignored, unimportant information cannot be fully reconstructed, such a coding process can be called "lossy." Otherwise, it can be called "lossless." Most coding processes are lossy, a trade-off between reducing required storage space and transmission bandwidth.

[0034] Useful information about an image being encoded (referred to as the "current image") includes changes relative to a reference image (e.g., a previously encoded and reconstructed image). These changes can include variations in pixel position, brightness, or color, with positional changes being of primary interest. The positional changes of a set of pixels characterizing an object can reflect the movement of that object between the reference image and the current image.

[0035] An image encoded without referencing another image (i.e., whose reference image is itself) is called an "I-image". If some or all blocks in an image (e.g., these blocks typically refer to different parts of a video image) are predicted using intra-frame prediction or inter-frame prediction with a reference image (e.g., one-way prediction), the image is called a "P-image". If at least one block in an image is predicted using two reference images (e.g., two-way prediction), the image is called a "B-image".

[0036] Figure 1 The structure of an exemplary video sequence 100 according to some embodiments of the present disclosure is shown. The video sequence 100 may be live video or video that has already been captured and archived. The video sequence 100 may be real video, computer-generated video (e.g., computer game video), or a combination thereof (e.g., real video with augmented reality effects). The video sequence 100 may originate from a video capture device (e.g., a camera), a video archive containing previously captured video (e.g., a video file stored on a storage device), or a video feed interface (e.g., a video broadcast transceiver) that receives video from a video content provider.

[0037] like Figure 1 As shown, video sequence 100 may include a series of images arranged chronologically along a timeline, including images 102, 104, 106, and 108. Images 102-106 are consecutive, and there are more images between images 106 and 108. Figure 1 In this diagram, image 102 is an I-image, and its reference image is image 102 itself. Image 104 is a P-image, and its reference image is image 102, as indicated by the arrow. Image 106 is a B-image, and its reference images are images 104 and 108, as indicated by the arrow. In some embodiments, the reference image of an image (e.g., image 104) may not be immediately preceding or following that image. For example, the reference image of image 104 may be an image preceding image 102. It should be noted that the reference images of images 102-106 are merely examples, and this disclosure does not limit the embodiments of the reference images to specific cases. Figure 1 The example shown.

[0038] Typically, due to the computational complexity of this task, video codecs do not encode or decode the entire image at once. Instead, they segment the image into multiple basic segments and encode or decode the image segment by segment. Such basic segments are referred to herein as basic processing units (BPUs). For example, Figure 1 Structure 110 illustrates an example structure of an image (e.g., any one of images 102-108) from video sequence 100. In structure 110, the image is divided into 4×4 basic processing units, the boundaries of which are shown as dashed lines. In some embodiments, the basic processing unit may be referred to as a "macroblock" in some video coding standards (e.g., MPEG series, H.261, H.263, or H.264 / AVC), or as a "coding tree unit" (CTU) in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit may have a variable size in the image, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or any arbitrary shape and size of pixels. The size and shape of the basic processing units for the image can be selected based on a balance between coding efficiency and the level of detail to be maintained in the basic processing units.

[0039] The basic processing unit can be a logical unit that may include a set of different types of video data stored in computer memory (e.g., in a video frame buffer). For example, a basic processing unit for a color image may include a luminance component (Y) representing non-color luminance information, one or more chrominance components (e.g., Cb and Cr) representing color information, and associated syntax elements, wherein the size of the luminance and chrominance components may be the same as that of the basic processing unit. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), the luminance and chrominance components may be referred to as a "coding tree block" (CTB). Any operation performed on the basic processing unit can be repeated on the luminance and chrominance components of the basic processing unit separately.

[0040] Video encoding involves multiple operational stages, examples of which are shown in Figure 2A-2B and Figures 3A-3BFor each stage, the size of the basic processing unit may still be too large to process, and therefore can be further divided into segments referred to herein as "basic processing subunits". In some embodiments, the basic processing subunit may be referred to as a "block" in some video coding standards (e.g., MPEG series, H.261, H.263, or H.264 / AVC), or as a "coding unit" (CU) in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing subunit may have the same or smaller size as the basic processing unit. Similar to the basic processing unit, the basic processing subunit is also a logical unit that may include a set of different types of video data (e.g., Y, Cb, Cr, and associated syntax elements) stored in computer memory (e.g., stored in a video frame buffer). Any operation performed on the basic processing subunit can be repeated on the luma and chroma components of the basic processing subunit separately. It should be noted that this division can be performed at deeper levels as needed for processing. It should also be noted that different schemes can be used to divide the basic processing units at different stages.

[0041] For example, in the pattern decision-making stage (examples of which are in...) Figure 2B As shown in the diagram, the encoder can decide which prediction mode (e.g., intra-image prediction or inter-image prediction) to use for the basic processing unit, which may be too large to make such a decision. The encoder can divide the basic processing unit into multiple basic processing subunits (e.g., CUs in H.265 / HEVC or H.266 / VVC) and determine the prediction type for each individual basic processing subunit.

[0042] For another example, in the prediction phase (the example is in...) Figure 2A-2B As shown, the encoder can perform prediction operations at the level of a basic processing subunit (e.g., a CU). However, in some cases, the basic processing subunit may still be too large to process. The encoder can further divide the basic processing subunit into smaller segments (e.g., referred to as "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC), at which the prediction operations can be performed.

[0043] For another example, in the transformation phase (the example of which is in...) Figure 2A and Figure 2BAs shown in the diagram, the encoder can perform transform operations on residual basic processing subunits (e.g., CUs). However, in some cases, the basic processing subunits may still be too large to process. The encoder can further divide the basic processing subunits into smaller segments (e.g., referred to as "transform blocks" or "TBs" in H.265 / HEVC or H.266 / VVC), at which point the transform operation can be performed. It should be noted that the partitioning scheme of the same basic processing subunit can differ between the prediction and transform phases. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same coding unit (CU) can have different sizes and numbers.

[0044] exist Figure 1 In structure 110, the basic processing unit 112 is further divided into 3×3 basic processing sub-units, the boundaries of which are shown as dashed lines. Different basic processing units of the same image can be divided into basic processing sub-units using different schemes.

[0045] In some implementations, to provide parallel processing and fault tolerance for video encoding and decoding, an image can be divided into multiple regions for processing, such that for a particular region of the image, the encoding or decoding process can be independent of information from any other region of the image. In other words, each region of the image can be processed independently. Through this approach, the codec can process different regions of the image in parallel, thereby improving encoding efficiency. Furthermore, when data in one region is corrupted during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same image without relying on the corrupted or lost data, thus providing fault tolerance. In some video coding standards, images can be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: "slices" and "tiles." It should also be noted that different images in the video sequence 100 can have different partitioning schemes for dividing an image into multiple regions.

[0046] For example, in Figure 1 In the diagram, structure 110 is divided into three regions 114, 116, and 118, the boundaries of which are shown as solid lines within structure 110. Region 114 comprises four basic processing units. Regions 116 and 118 each comprise six basic processing units. It should be noted that... Figure 1 The basic processing unit, basic processing subunit, and region of the intermediate structure 110 are merely examples, and this disclosure does not limit its embodiments.

[0047] Figure 2A A schematic diagram of an exemplary encoding process 200A consistent with embodiments of this disclosure is shown. For example, the encoding process 200A may be performed by an encoder. Figure 2A As shown, the encoder can encode the video sequence 202 into a video bitstream 228 according to process 200A. Similar to... Figure 1 Video sequence 100 and video sequence 202 may include a set of images (referred to as "original images") arranged in chronological order. Similar to... Figure 1 In structure 110, the encoder can divide each raw image of video sequence 202 into multiple basic processing units, multiple basic processing sub-units, or multiple regions for processing. In some embodiments, the encoder can perform process 200A at the level of basic processing units for each raw image of video sequence 202. For example, the encoder can perform process 200A iteratively, wherein the encoder can encode one basic processing unit in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel for multiple regions (e.g., regions 114-118) of each raw image of video sequence 202.

[0048] exist Figure 2A In this process, the encoder feeds the basic processing unit (referred to as the "raw BPU") of the original image of video sequence 202 to prediction stage 204 to generate prediction data 206 and prediction BPU 208. The encoder can subtract prediction BPU 208 from the raw BPU to generate residual BPU 210. The encoder can feed residual BPU 210 to transform stage 212 and quantization stage 214 to generate quantization transform coefficients 216. The encoder can feed prediction data 206 and quantization transform coefficients 216 to binary encoding stage 226 to generate video bitstream 228. Components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 can be referred to as the "forward path". During process 200A, after quantization stage 214, the encoder can feed quantization transform coefficients 216 to inverse quantization stage 218 and inverse transform stage 220 to generate reconstructed residual BPU 222. The encoder can add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224, which is used in the next iteration of process 200A during prediction phase 204. Components 218, 220, 222, and 224 of process 200A can be referred to as a "reconstruction path." The reconstruction path can be used to ensure that both the encoder and the decoder use the same reference data for prediction.

[0049] The encoder can iteratively execute process 200A to encode each raw BPU of the raw image (in the forward path) and generate prediction reference 224 for encoding the next raw BPU of the raw image (in the reconstruction path). After encoding all raw BPUs of the raw image, the encoder can continue to encode the next image in the video sequence 202.

[0050] Referring to process 200A, the encoder can receive a video sequence 202 generated by a video acquisition device (e.g., a camera). As used herein, the term "receive" can refer to any action of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or otherwise inputting data.

[0051] In prediction phase 204, during the current iteration, the encoder may receive the original BPU and prediction reference 224, and perform prediction operations to generate prediction data 206 and prediction BPU 208. Prediction reference 224 can be generated from the reconstruction path of the previous iteration of process 200A. The purpose of prediction phase 204 is to reduce information redundancy by extracting prediction data 206 from prediction data 206 and prediction reference 224 that can be used to reconstruct the original BPU into prediction BPU 208.

[0052] Ideally, the predicted BPU 208 can be identical to the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is typically slightly different from the original BPU. To record this difference, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate the residual BPU 210. For example, the encoder can subtract the pixel value (e.g., grayscale or RGB value) of the predicted BPU 208 from the value of the corresponding pixel in the original BPU. Each pixel in the residual BPU 210 can have a residual value generated by this subtraction between the corresponding pixel values ​​of the original BPU and the predicted BPU 208. Compared to the original BPU, the predicted data 206 and the residual BPU 210 can have fewer bits, but they can be used to reconstruct the original BPU without significant quality degradation. Thus, the original BPU is compressed.

[0053] To further compress the residual BPU 210, in the transform phase 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing it into a two-dimensional set of "fundamental patterns," each fundamental pattern being associated with "transform coefficients." The fundamental patterns can have the same size (e.g., the size of the residual BPU 210). Each fundamental pattern can characterize a frequency-varying component of the residual BPU 210 (e.g., the frequency of brightness variation). No fundamental pattern can be reproduced by any combination of any other fundamental patterns (e.g., a linear combination). In other words, the decomposition decomposes the variation of the residual BPU 210 into the frequency domain. This decomposition is analogous to the discrete Fourier transform of a function, where the fundamental patterns are analogous to the basis functions of the discrete Fourier transform (e.g., trigonometric functions), and the transform coefficients are analogous to the coefficients associated with the basis functions.

[0054] Different transform algorithms can use different base modes. Various transform algorithms can be used in transform stage 212, such as discrete cosine transform, discrete sine transform, etc. The transform in transform stage 212 is reversible. That is, the encoder can recover the residual BPU 210 through the inverse operation of the transform (called the "inverse transform"). For example, to recover a pixel of the residual BPU 210, the inverse transform can be to multiply the value of the corresponding pixel in the base mode by the corresponding correlation coefficient and sum the products to produce a weighted sum. For video coding standards, both the encoder and decoder can use the same transform algorithm (and thus the same base mode). Therefore, the encoder can only record the transform coefficients, and the decoder can reconstruct the residual BPU 210 based on the transform coefficients without receiving the base mode from the encoder. Compared to the residual BPU 210, the transform coefficients can have fewer bits, but they can be used to reconstruct the residual BPU 210 without significant quality degradation. Therefore, the residual BPU 210 is further compressed.

[0055] The encoder can further compress the transform coefficients in quantization phase 214. During the transform process, different fundamental modes can represent different frequencies of change (e.g., brightness change frequency). Because the human eye is generally better at identifying low-frequency changes, the encoder can ignore information about high-frequency changes without causing a significant degradation in decoding quality. For example, in quantization phase 214, the encoder can generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (called a "quantization scaling factor") and performing rounding on the quotient. After this operation, some transform coefficients of the high-frequency fundamental modes can be converted to zero, while the transform coefficients of the low-frequency fundamental modes can be converted to smaller integers. The encoder can ignore the zero-valued quantized transform coefficients 216, thereby further compressing the transform coefficients. The quantization process is also reversible, wherein the quantized transform coefficients 216 can be reconstructed into the transform coefficients in the inverse operation of quantization (called "inverse quantization").

[0056] Because the encoder ignores the remainder of such division in the rounding operation, the quantization stage 214 may be lossy. Typically, the quantization stage 214 may constitute the primary source of information loss in process 200A. The greater the information loss, the fewer bits the quantization transform coefficients 216 may require. To obtain different levels of information loss, the encoder can use different values ​​of the quantization syntax elements, or different values ​​of any other syntax elements of the quantization process.

[0057] In the binary encoding stage 226, the encoder may use binary encoding techniques to encode the prediction data 206 and the quantization transform coefficients 216, such as entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantization transform coefficients 216, the encoder may also encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the syntax elements of the prediction operation, the transform type of the transform stage 212, the syntax elements of the quantization process (e.g., quantization syntax elements), encoder control syntax elements (e.g., bitrate control syntax elements), etc. The encoder may use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packaged for network transmission.

[0058] Referring to the reconstruction path of process 200A, in the inverse quantization stage 218, the encoder can perform inverse quantization on the quantization transform coefficients 216 to generate reconstructed transform coefficients. In the inverse transform stage 220, the encoder can generate a reconstruction residual BPU 222 based on the reconstructed transform coefficients. The encoder can add the reconstruction residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224 to be used in the next iteration of process 200A.

[0059] It should be noted that other variations of process 200A can be used to encode video sequence 202. In some embodiments, the stages of process 200A can be performed by the encoder in different orders. In some embodiments, one or more stages of process 200A can be combined into a single stage. In some embodiments, a single stage of process 200A can be divided into multiple stages. For example, transform stage 212 and quantization stage 214 can be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may be omitted. Figure 2A One or more stages in the process.

[0060] Figure 2B A schematic diagram of another exemplary encoding process 200B consistent with embodiments of the present disclosure is shown. Process 200B can be derived from process 200A. For example, process 200B can be used by an encoder conforming to a hybrid video coding standard (e.g., H.26x series). Compared to process 200A, the forward path of process 200B additionally includes a mode decision stage 230, and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B additionally includes a loop filtering stage 232 and a buffer 234.

[0061] Generally, prediction techniques can be categorized into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or "intra-frame prediction") uses pixels from one or more encoded neighboring BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in the spatial prediction can include the neighboring BPUs. Spatial prediction can reduce the inherent spatial redundancy of the image. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") uses regions from one or more encoded images to predict the current BPU. That is, the prediction reference 224 in the temporal prediction can include the encoded images. Temporal prediction can reduce the inherent temporal redundancy of the image.

[0062] Referring to process 200B, in the forward path, the encoder performs prediction operations in spatial prediction phase 2042 and temporal prediction phase 2044. For example, in spatial prediction phase 2042, the encoder may perform intra-frame prediction. For a given original BPU of an image being encoded, prediction reference 224 may include one or more adjacent BPUs that have already been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder can generate a predicted BPU 208 by extrapolating adjacent BPUs. The extrapolation technique may include, for example, linear extrapolation or interpolation, polynomial extrapolation or interpolation, etc. In some embodiments, the encoder may perform the extrapolation at the pixel level, for example, extrapolating the corresponding pixel value for each pixel of the predicted BPU 208. The adjacent BPU used for extrapolation can be positioned relative to the original BPU from various directions, such as vertical (e.g., at the top of the original BPU), horizontal (e.g., to the left of the original BPU), diagonal (e.g., at the lower left, lower right, upper left, or upper right of the original BPU), or any direction defined in the video coding standard used. For intra-frame prediction, prediction data 206 may include, for example, the location (e.g., coordinates) of the adjacent BPU used, the size of the adjacent BPU used, the extrapolation syntax elements, the orientation of the adjacent BPU used relative to the original BPU, etc.

[0063] In another example, during the temporal prediction phase 2044, the encoder may perform the inter-frame prediction. For a given original BPU of the current image, the prediction reference 224 may include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference images may be encoded and reconstructed on a BPU-by-BPU basis. For example, the encoder may add a reconstructed residual BPU 222 to the prediction BPU 208 to generate a reconstructed BPU. After all reconstructed BPUs of the same image have been generated, the encoder may generate a reconstructed image as the reference image. The encoder may perform a "motion estimation" operation to search for a matching region within a range (referred to as a "search window") of the reference image. The position of the search window in the reference image may be determined based on the position of the original BPU in the current image. For example, the search window may be centered at a position in the reference image that has the same coordinates as the original BPU in the current image and may extend outward by a predetermined distance. When the encoder identifies a region similar to the original BPU in the search window (e.g., using a pixel recursive algorithm, block matching algorithm, etc.), the encoder can determine such a region as a matching region. The matching region may have different specifications than the original BPU (e.g., less than, equal to, or greater than the original BPU, or have a different shape). Because the reference image and the current image are temporally separated in the timeline (e.g., as...), Figure 1 As shown), the matching region can be considered to have "moved" to the location of the original BPU over time. The encoder can record the direction and distance of this movement as a "motion vector". When using multiple reference images (e.g., Figure 1 When using image 106 in the reference image, the encoder can search for a matching region for each reference image and determine the associated motion vector of the matching region. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching region for each matching reference image.

[0064] The motion estimation can be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, the prediction data 206 may include, for example, the position (e.g., coordinates) of the matching region, the motion vector associated with the matching region, the number of reference images, the weights associated with the reference images, etc.

[0065] To generate the predicted BPU 208, the encoder can perform a "motion compensation" operation. This motion compensation can be used to reconstruct the predicted BPU 208 based on the predicted data 206 (e.g., motion vectors) and the predicted reference 224. For example, the encoder can move the matching region of the reference image according to the motion vectors, thereby allowing the encoder to predict the original BPU of the current image. When using multiple reference images (e.g., ... Figure 1 When the image (106) is being processed, the encoder can move the matching region of the reference image according to the motion vectors and average pixel values ​​of the matching region. In some embodiments, if the encoder has already assigned weights to the pixel values ​​of the matching regions of each matching reference image, the encoder can perform a weighted summation of the pixel values ​​of the moved matching region.

[0066] In some embodiments, the inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction relative to the current image. For example, Figure 1 Image 104 in the image is a unidirectional inter-frame predicted image, where the reference image (e.g., image 102) precedes image 104. Bidirectional inter-frame prediction can use one or more reference images in two temporal directions relative to the current image. For example, Figure 1 Image 106 is a bidirectional inter-frame predicted image, wherein the reference images (e.g., images 104 and 108) are relative to image 104 in two time directions.

[0067] Still referring to the forward path of process 200B, after spatial prediction stage 2042 and temporal prediction stage 2044, in mode decision stage 230, the encoder can select a prediction mode (e.g., one of intra-frame prediction or inter-frame prediction) for the current iteration of process 200B. For example, the encoder can perform rate-distortion optimization techniques, wherein the encoder selects a prediction mode based on the bit rate of a candidate prediction mode and the distortion of the reference image reconstructed under the candidate prediction mode to minimize the value of the cost function. Based on the selected prediction mode, the encoder can generate a corresponding prediction BPU 208 and prediction data 206.

[0068] In the reconstruction path of process 200B, if an intra-prediction mode has already been selected in the forward path, the encoder can directly feed the prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image) to the spatial prediction stage 2042 for subsequent use (e.g., for extrapolation of the next BPU of the current image) after generating the prediction reference 224. The encoder can also feed the prediction reference 224 to the loop filtering stage 232, where the encoder can apply loop filtering to the prediction reference 224 to reduce or eliminate distortions (e.g., blockiness) introduced during the encoding of the prediction reference 224. The encoder can apply various loop filtering techniques in the loop filtering stage 232, such as deblocking, sample adaptive shifting, adaptive loop filtering, etc. The loop-filtered reference image can be stored in buffer 234 (or “decoded picture buffer (DPB)”) for subsequent use (e.g., as an inter-frame prediction reference image for future images of video sequence 202). The encoder may store one or more reference images in buffer 234 for use in the time prediction stage 2044. In some embodiments, the encoder may encode the syntax elements of the loop filter (e.g., loop filter strength), as well as the quantization transform coefficients 216, prediction data 206, and other information in the binary encoding stage 226.

[0069] Figure 3A A schematic diagram of an exemplary decoding process 300A consistent with embodiments of this disclosure is shown. Process 300A may be corresponding to Figure 2A The compression process 200A described above is followed by the decompression process. In some embodiments, process 300A may be similar to the reconstruction path described in process 200A. The decoder can decode the video bitstream 228 into video stream 304 according to process 300A. Video stream 304 may be very similar to video sequence 202. However, due to information loss during compression and decompression (e.g., Figure 2A and Figure 2B In the quantization stage 214, video stream 304 typically differs from video sequence 202. Similar to... Figure 2A and Figure 2B In processes 200A and 200B, the decoder can perform process 300A at the level of a basic processing unit (BPU) for each image encoded in the video bitstream 228. For example, the decoder can perform process 300A iteratively, wherein the decoder can decode one basic processing unit in one iteration of process 300A. In some embodiments, the decoder can perform process 300A in parallel for multiple regions (e.g., regions 114-118) of each image encoded in the video bitstream 228.

[0070] exist Figure 3A In this process, the decoder may feed a portion of the video bitstream 228 associated with a basic processing unit (referred to as a "coded BPU") of the encoded image to the binary decoding stage 302. In the binary decoding stage 302, the decoder may decode the portion into prediction data 206 and quantization transform coefficients 216. The decoder may feed the quantization transform coefficients 216 to the inverse quantization stage 218 and the inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may feed the prediction data 206 to the prediction stage 204 to generate a prediction BPU 208. The decoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded image buffer in computer memory). The decoder may feed the prediction reference 224 to the prediction stage 204 for performing a prediction operation in the next iteration of process 300A.

[0071] The decoder can iteratively execute process 300A to decode each encoded BPU of the encoded image and generate a prediction reference 224 for encoding the next encoded BPU of the encoded image. After decoding all encoded BPUs of the encoded image, the decoder can output the image to video stream 304 for display and continue decoding the next encoded image in video bitstream 228.

[0072] In binary decoding stage 302, the decoder may perform the inverse operation of the binary encoding technique used by the encoder (e.g., entropy coding, variable-length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and quantization transform coefficients 216, the decoder may also decode other information in binary decoding stage 302, such as, for example, prediction mode, syntax elements of prediction operation, transform type, syntax elements of quantization process (e.g., quantization syntax elements), encoder control syntax elements (e.g., bitrate control syntax elements), etc. In some embodiments, if the video bitstream 228 is transmitted over the network in the form of data packets, the decoder may unpack the video bitstream 228 before feeding it to binary decoding stage 302.

[0073] Figure 3BA schematic diagram of another exemplary decoding process 300B consistent with embodiments of the present disclosure is shown. Process 300B can be derived from process 300A. For example, process 300B can be used by a decoder conforming to a hybrid video coding standard (e.g., H.26x series). Compared to process 300A, process 300B further divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and further includes a loop filtering stage 232 and a buffer 234.

[0074] In process 300B, for the encoded basic processing unit (referred to as the "current BPU") of the encoded image being decoded (referred to as the "current image"), the prediction data 206 decoded by the decoder from binary decoding stage 302 can include various types of data depending on the prediction mode used by the encoder to encode the current BPU. For example, if the encoder uses intra-frame prediction to encode the current BPU, the prediction data 206 can include a prediction mode indicator (e.g., a flag value) indicating intra-frame prediction, syntax elements of the intra-frame prediction operation, etc. The syntax elements of the intra-frame prediction operation can include, for example, the positions (e.g., coordinates) of one or more neighboring BPUs used as references, the size of the neighboring BPUs, extrapolation syntax elements, the orientation of the neighboring BPUs relative to the original BPU, etc. For another example, if the encoder uses inter-frame prediction to encode the current BPU, the prediction data 206 can include a prediction mode indicator (e.g., a flag value) indicating inter-frame prediction, syntax elements of the inter-frame prediction operation, etc. The syntax elements of the inter-frame prediction operation may include, for example, the number of reference images associated with the current BPU, the weights associated with the reference images respectively, the positions (e.g., coordinates) of one or more matching regions in each reference image, and one or more motion vectors associated with the matching regions respectively.

[0075] Based on the prediction mode indicator, the decoder can determine whether to perform spatial prediction (e.g., intra-frame prediction) in the spatial prediction phase 2042 or temporal prediction (e.g., inter-frame prediction) in the temporal prediction phase 2044. Figure 2B The details of performing such spatial or temporal predictions are described herein and will not be repeated below. After performing such spatial or temporal predictions, the decoder can generate a prediction BPU 208. The decoder can then add the prediction BPU 208 and the reconstruction residual BPU 222 to generate a prediction reference 224, as shown below. Figure 3A As described.

[0076] In process 300B, the decoder can feed prediction reference 224 to spatial prediction stage 2042 or temporal prediction stage 2044 for performing prediction operations in the next iteration of process 300B. For example, if the current BPU is decoded using the intra-frame prediction in spatial prediction stage 2042, the decoder can feed prediction reference 224 directly to spatial prediction stage 2042 for subsequent use (e.g., for extrapolation of the next BPU of the current image) after generating prediction reference 224 (e.g., the decoded current BPU). If the current BPU is decoded using inter-frame prediction in temporal prediction stage 2044, the decoder can feed prediction reference 224 to loop filtering stage 232 to reduce or eliminate distortion (e.g., blockiness) after generating prediction reference 224 (e.g., a reference image where all BPUs have been decoded). The decoder can then... Figure 2B The method described herein applies loop filtering to prediction reference 224. The loop-filtered reference image may be stored in buffer 234 (e.g., a decoded image buffer (DPB) in computer memory) for subsequent use (e.g., as an inter-frame prediction reference image used as a future encoded image of video bitstream 228). The decoder may store one or more reference images in buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the prediction data may also include loop filtering syntax elements (e.g., loop filtering strength). In some embodiments, the prediction data includes loop filtering syntax elements when the prediction mode indicator of prediction data 206 indicates that inter-frame prediction has been used to encode the current BPU.

[0077] Figure 4 This is a block diagram of an exemplary apparatus 400 for encoding or decoding video, consistent with embodiments of this disclosure. Figure 4As shown, device 400 may include processor 402. When processor 402 executes the instructions described herein, device 400 may become a dedicated machine for video encoding or decoding. Processor 402 may be any type of circuit system capable of manipulating or processing information. For example, processor 402 may include any combination of any number of central processing units (or “CPU”), graphics processing units (or “GPU”), neural processing units (“NPU”), microcontroller units (“MCU”), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLAs), programmable array logic (PALs), generic array logic (GALs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), system-on-chip (SoCs), application-specific integrated circuits (ASICs), etc. In some embodiments, processor 402 may also be a set of processors grouped into individual logic components. For example, such as Figure 4 As shown, processor 402 may include multiple processors, including processor 402a, processor 402b and processor 402n.

[0078] The device 400 may also include a memory 404 configured to store data (e.g., instruction sets, computer code, intermediate data, etc.). For example, such as Figure 4As shown, the stored data may include program instructions (e.g., program instructions for implementing the stages in processes 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 can access the program instructions and the data for processing (e.g., via bus 410) and execute the program instructions to perform operations or manipulations on the data for processing. Memory 404 may include a high-speed random access memory device or a non-volatile memory device. In some embodiments, memory 404 may include any combination of any number of random-access memory (RAM), read-only memory (ROM), optical discs, magnetic disks, hard disks, solid-state drives, flash drives, security digital (SD) cards, memory sticks, compact flash (CF) cards, etc. Memory 404 may also be a group of memories grouped into a single logical component. Figure 4 (Not shown in the image).

[0079] Bus 410 may be a communication device for transmitting data between components within device 400, such as an internal bus (e.g., CPU-memory bus), an external bus (e.g., a Universal Serial Bus port, a Peripheral Component Interconnect Fast Port), etc.

[0080] For ease of explanation and to avoid ambiguity, processor 402 and other data processing circuitry are collectively referred to as "data processing circuitry" in this disclosure. The data processing circuitry may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. Furthermore, the data processing circuitry may be a single, independent module, or may be wholly or partially integrated into any other component of device 400.

[0081] Device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, intranet, local area network, mobile communication network, etc.). In some embodiments, network interface 406 may include any combination of any number of network interface controllers (NICs), radio frequency (RF) modules, transceivers, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near-field communication (NFC) adapters, cellular network chips, etc.

[0082] In some embodiments, optionally, the device 400 may further include a peripheral interface 408 to provide connectivity to one or more peripheral devices. Figure 4 As shown, the peripheral devices may include, but are not limited to, cursor control devices (e.g., mouse, touchpad, or touchscreen), keyboards, displays (e.g., cathode ray tube displays, liquid crystal displays, or light-emitting diode displays), video input devices (e.g., cameras or input interfaces coupled to video archives), etc.

[0083] It should be noted that the video codec (e.g., the codec for executing processes 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules in device 400. For example, some or all stages of processes 200A, 200B, 300A, or 300B can be implemented as one or more software modules of device 400, such as program instructions that can be loaded into memory 404. As another example, some or all stages of processes 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of device 400, such as dedicated data processing circuitry (e.g., FPGA, ASIC, NPU, etc.).

[0084] An Enhanced Compression Model (ECM) has been proposed and is being used as a new software foundation for developing tools that go beyond the VVC standard.

[0085] The subblock-based temporal motion vector predictor is described below. In VVC, motion information is extracted from the center or lower right corner of the co-location block in the signaled co-location image to derive the temporal motion vector prediction (TMVP) and merging pattern for Advanced Motion Vector Prediction (AMVP). First, the lower right corner of the co-location block is checked; if there is no valid motion from the lower right corner position, the center block is then checked. Similarly, for the subblock-based temporal motion vector prediction (SbTMVP) pattern, motion information from the left adjacent position is used as a motion offset, and then the motion offset is used to obtain the sub-CU level TMVP.

[0086] In ECM, three aspects were modified to further improve the coding efficiency of TMVP or SbTMVP. First, the lower right corner and center position of the co-location block are checked. If both positions contain valid motion information and there are available vacancies in the candidate list, motion vectors from these two positions are included as TMVP candidates. Second, for the SbTMVP mode, two co-location images are used, which are two reference frames with the minimum picture order count (POC) distance relative to the frame to be encoded. Third, motion offsets for SbTMVP used to locate the TMVP are adaptively determined from multiple positions based on template cost. More specifically, two motion offset candidate lists are constructed for the two co-location frames respectively. The SbTMVP is derived using the motion offset with the minimum template matching cost. The merged list based on sub-blocks includes a maximum of four SbTMVP candidates. The SbTMVP candidate with the minimum template matching cost derived from the first co-location frame is placed in the first entry without reordering, while other SbTMVP candidates are ordered together with affine candidates. In addition, the prediction direction of the template for each sub-block is determined based on the center sub-block.

[0087] Chained motion vector prediction (CMVP) is described below. CMVP candidates are introduced as one of the merging candidates and inserted after history-based motion vector prediction (HMVP) candidates for both regular merging and template matching (TM) merging modes. CMVP candidates are derived by accumulating the motion vectors (MVs) and block vectors (BVs) obtained through recursive tracking, based on pre-derived motion vectors (MVs). In a typical design, CMVP candidates can be derived for each merging index and each list of reference images. Trackable reference images are limited to those in the list of reference images.

[0088] In the initial merge candidate list, more additional CMVP candidates are added before the zero-padded candidates. Simultaneously, the maximum size of the initial merge candidate list is increased by 4. The maximum number of candidates preceding the original CMVP candidates remains unchanged.

[0089] When exporting additional CMVP candidates, only the center position of the current block is checked to find the tracked MV or BV.

[0090] If an existing TMVP candidate has a non-zero reference index, an additional TMVP candidate is added, scaled to the reference image at index 0. This additional TMVP candidate is used to derive the CMVP candidate. After exporting the TMVP and related CMVP candidates, an adaptive reordering of merge candidates (ARMC) is performed. CMVP candidates are also exported in Bilateral Matching (BM) merging mode.

[0091] The merge list construction in ECM is described below. In ECM, the merge candidate list is constructed by sequentially including the following seven categories of candidates: - Spatial MVP from adjacent CUs; - Time MVP from co-located blocks; - Non-adjacent spatial candidates; - A history-based MVP derived from a first-in-first-out (FIFO) table; - Chain motion vector prediction; - Paired average MVP; and - Zero MV.

[0092] Furthermore, after constructing the merge candidate list, the merge candidates are reordered based on the cost of template matching (hereinafter referred to as ARMC). The template matching cost of the merge candidate is measured by the sum of absolute differences (SAD) between the samples of the template of the current block and the reference samples corresponding to the template. The template includes a set of reconstructed samples adjacent to the current block. The reference samples of the template are located using the motion information of the merge candidates.

[0093] Next, the temporal candidate export is described. When exporting temporal merge candidates, scaling motion vectors are exported based on co-location blocks belonging to the co-location reference images. The list of reference images and reference indices used to export these co-location blocks are explicitly signaled in the title sequence. Figure 5 Exemplary motion vector scaling for time merging candidates is illustrated according to some embodiments of this disclosure. For example... Figure 5 As shown by the dashed lines, a scaled motion vector for temporal merging candidates is obtained. This scaled motion vector is obtained by scaling the motion vector of the co-location block using the POC distance, tb, and td, where tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the co-location image and the co-location image. The reference image index of the temporal merging candidate is set to the reference image with the scaling factor closest to 1.

[0094] It should be noted that when deriving time merging candidates, the scaled motion vector is derived from one of the motions of the co-location block, wherein one of the motions of the co-location block is determined according to the following rules: - If the motion of the co-location block is bidirectional predictive motion and the current image is a low-latency image, then the L0 motion of the TMVP is scaled from the L0 motion of the co-location block, and the L1 motion of the TMVP is scaled from the L1 motion of the co-location block.

[0095] Otherwise, if the motion of the co-location block is a bidirectional predicted motion and the current image is not a low-latency image, then, based on the reference image list of the co-location block, it is determined which of the two motions of the co-location block will be used to perform scaling. More specifically, if the co-location block comes from the L0 reference image list, then both the L0 and L1 motions of the TMVP are scaled from the L1 motion of the co-location block. Similarly, if the co-location block comes from the L1 reference image list, then both the L0 and L1 motions of the TMVP are scaled from the L0 motion of the co-location block.

[0096] Otherwise, if the motion of the co-location block is L0 predicted motion, then the L0 and L1 motions of the TMVP are scaled from the L0 motion of the co-location block, regardless of whether the current image is a low-latency image. Similarly, if the motion of the co-location block is L1 predicted motion, then the L0 and L1 motions of the TMVP are scaled from the L1 motion of the co-location block.

[0097] When a co-location block is not inter-coded, it means that motion is unavailable, and the scaled motion vector of this co-location block is considered unavailable. Furthermore, in traditional ECM designs, the TMVP can only be derived from a single co-location image, and TMVP candidates can be L0 prediction, L1 prediction, or bidirectional prediction.

[0098] Next, we describe the construction of the AMVP list in ECM. In ECM, the AMVP list is constructed sequentially using the following four candidate categories: - Spatial MVP from adjacent CUs; - Time MVP from co-located blocks; - Non-adjacent spatial candidates; and - A history-based MVP derived from a FIFO table.

[0099] Apply the TM process to the AMVP list to reorder the candidates.

[0100] The TMVP is restricted to being derived from only one co-located image. However, SbTMVP and CMVP allow temporal motion to be obtained from multiple reference images. Furthermore, TMVP cannot be obtained from BV. These limitations may restrict the predictive efficiency of TMVP.

[0101] This disclosure provides solutions to one or more of the aforementioned problems in next-generation video coding.

[0102] Embodiments of this disclosure provide a method for deriving a TMVP from multiple reference images. Consistent with this disclosure, it is proposed to allow the TMVP to be derived from multiple reference images, rather than from a single co-located image.

[0103] In some embodiments, TMVP candidates are derived from two co-located images, which are two reference frames with the minimum point of convergence (POC) distance relative to the frame to be encoded. These two reference frames can be the same as the reference frames used in SbTMVP mode.

[0104] In some embodiments, when a TMVP candidate is added to a merge list or an AMVP list, multiple co-location blocks are checked. Figure 6 Exemplary locations of co-location blocks according to some embodiments of this disclosure are shown. When deriving TMVP candidates using two reference frames, the motion of the two reference frames can be added in different orders.

[0105] In one example, motion from the first frame and motion from the second reference frame are added sequentially. For example, motion from the co-location blocks of the first reference image is added first (e.g., motion from positions C0, C1, C2, ...), and then motion from the co-location blocks of the second reference image is added (e.g., motion from positions C0, C1, C2, ...). In this example, as... Figure 6 As shown, a co-location block comprises 4×4 sub-blocks, and a reference image comprises 6×6 blocks. It is understood that in some embodiments, the sizes of the co-location block and the reference image may be different.

[0106] In another example, motion from the first image and motion from the second reference image are added alternately. For example, motion from position C0 of the first and second reference frames is added, then motion from position C1 of the first and second reference frames is added, and so on.

[0107] In some embodiments, TMVP candidates are derived from any reference image in the list of reference images.

[0108] In some embodiments, the number of reference images used in TMVP export may depend on the coding mode. For example, for regular merging mode, inter-frame intra-frame joint prediction (CIIP) mode, BM mode, TM mode, merging with motion vector difference (MMVD) mode, affine mode, SbTMVP mode, and AMVP mode, the number of reference images is set to N1, N2, N3, N4, N5, N6, N7, and N8, respectively. The values ​​of N1, N2, N3, N4, N5, N6, N7, and N8 are positive integers and may be the same as or different from each other.

[0109] Embodiments of this disclosure provide a method for deriving a TMVP from a BV (Body Vector). In this disclosure, it is proposed to obtain the TMVP from the BV rather than from a motion vector.

[0110] Figure 7 This is a flowchart of an exemplary method for deriving motion vectors for a TMVP from a BV according to some embodiments of this disclosure. Method 700 can be generated by an encoder (e.g., via...). Figure 2A Process 200A or Figure 2B Process 200B), decoder (e.g., via Figure 3A Process 300A or Figure 3B The process 300B) is performed, or is carried out by a device (e.g., Figure 4 The device 400) is executed by one or more software or hardware components. For example, a processor (e.g., Figure 4 The processor 402 can execute method 700. In some embodiments, method 700 can be implemented by a computer program product included in a computer-readable medium, the computer program product comprising components developed by a computer (e.g., a processor 402). Figure 4 The device 400 executes computer-executable instructions, such as program code. (See reference) Figure 7 Method 700 may include the following steps 702 to 706.

[0111] In step 702, it is determined whether the co-location block in the co-location image of the current block has a block vector (BV); for example, when the co-location block is encoded using Intra Block Copy (IBC) or Intra Template Matching Prediction (IntraTMP) modes, the co-location block has a BV. In some embodiments, it is determined whether to encode the co-location block using BV.

[0112] In step 704, in response to the co-location block having the BV, the BV is determined.

[0113] In step 706, based on the BV of the co-location block, a motion vector for the TMVP of the current block is determined, and the reference index of the TMVP is determined as the reference index of the co-location image. A TMVP candidate may include the motion vector and the reference index.

[0114] In some embodiments, the BV of the co-located block is directly used as a TMVP candidate for the current block. Figure 8A This is a flowchart illustrating exemplary sub-steps of a method for deriving motion vectors for a TMVP from a BV according to some embodiments of this disclosure. Figure 8A As shown, step 706 of method 700 further includes step 802. In step 802, the BV of the co-located block is directly used as the motion vector for the TMVP of the current block. Figure 8B Some embodiments according to this disclosure are shown, and Figure 8A An exemplary illustration consistent with the method described above. (See reference...) Figure 8B Block 821 in co-located image 820 is a co-located block of current block 811 in current image 810. BV 822 is the BV of co-located block 821. Block 811' shows the corresponding position of current block 811 in co-located image 820. In this example, BV 822 of co-located block 821 is directly used as the motion vector for TMVP of current block 811. Therefore, based on the determined motion vector (e.g., BV 822), prediction block 829 of current block 811 is obtained through TMVP.

[0115] In some embodiments, when obtaining the TMVP from the BV-encoded co-location block, the relative position of the current block and the co-location block is taken into account. That is, the TMVP of the current block is executed based on the BV of the co-location block and the relative position between the current block and the co-location block. Figure 9A This is a flowchart of another exemplary sub-step of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure. Figure 9A As shown, step 706 of method 700 further includes steps 902 and 904. In step 902, the relative position between the current block and the co-located block is determined. In step 904, the sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block. The sum is a vector. Figure 9B Some embodiments according to this disclosure are shown, and Figure 9A An exemplary illustration consistent with the method described above. (See reference...) Figure 9BBlock 921 in co-located image 920 is a co-located block of current block 911 in current image 910. BV 922 is the BV of co-located block 921. Block 911' shows the corresponding position of current block 911 in co-located image 920. The relative position 923 between current block 911' and co-located block 921 is determined. The relative position 923 is a vector. The sum 924 of relative position 923 and BV 922 is obtained. In this example, the sum 924 is used as the motion vector of the TMVP of current block 911. Therefore, based on the determined motion vector (i.e., the sum 924), the predicted block 929 of current block 911 is obtained through TMVP. In this example, the position of the block refers to the top left corner of the block.

[0116] In a conventional implementation, the temporal information of the reference image is stored on a 4×4 sub-block basis. When obtaining the TMVP from a co-located block, the CU size of the co-located block is not considered. In some embodiments, it is proposed that when obtaining the TMVP from a BV-encoded co-located block, not only the relative position between the current block and the co-located block is considered, but also the CU size of the co-located block. That is, the TMVP of the current block is performed based on the BV of the co-located block, the relative position between the current block and the co-located block, and the CU size of the co-located block. Figure 10A This is a flowchart of another exemplary sub-step of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure. Figure 10A As shown, step 706 of method 700 further includes steps 1002 to 1006. In step 1002, a first relative position between the current block and the co-located block is determined. In step 1004, a second relative position between the co-located block and its co-located CU is determined. In step 1006, the sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block. The sum is a vector. Figure 10B Some embodiments according to this disclosure are shown, and Figure 10A An exemplary illustration consistent with the method described above. (See reference...) Figure 10BBlock 1021 in co-location image 1020 is a co-location block of current block 1011 in current image 1010. BV1022 is the BV of co-location block 1021. Block 1011' shows the corresponding position of current block 1011 in co-location image 1020. Co-location block 1021 is contained in co-location CU 1023. A first relative position 1024 between current block 1011' and co-location CU 1023 is determined, and a second relative position 1025 between co-location block 1021 and co-location CU 1023 is determined. Relative positions 1024 and 1025 are vectors. Then, the sum 1026 of BV 1022, the first relative position 1024, and the second relative position 1025 is obtained. In this example, the sum 1026 is used as the motion vector for the TMVP of current block 1011. Therefore, based on the determined motion vectors (i.e., and 1026), the predicted block 1029 of the current block 1011 is obtained via TMVP. In this example, the position of a block or CU refers to the top-left corner of the block or CU.

[0117] In some embodiments, the proposed TMVP candidate derived from BV-encoded co-location blocks can be used in one of the following modes: regular merge mode, CIIP mode, BM mode, TM mode, MMVD mode, affine mode, SbTMVP mode, and AMVP mode. It is not required that all encoding modes in the codec employ the same TMVP derivation method. In one example, regular merge mode, TM mode, CIIP mode, and MMVD mode are allowed to derive TMVP candidates from BV-encoded co-location blocks. In another example, only regular merge mode is allowed to derive TMVP candidates from BV-encoded co-location blocks.

[0118] In traditional ECM implementations, both IBC and intraTMP modes use BV encoding. However, in intraTMP mode, more than two BVs are used to predict intraTMP-encoded blocks. When these intraTMP-encoded blocks are used as co-placed blocks for TMVP, the BVs may be inaccurate. Therefore, in some embodiments, it is proposed to derive TMVP candidates from IBC-encoded co-placed blocks. That is, TMVP candidates cannot be derived from intraTMP-encoded co-placed blocks. Figure 11 This is a flowchart of another exemplary sub-step of a method for deriving motion vectors for TMVP from BV according to some embodiments of this disclosure. Figure 11 As shown, step 704 of method 700 further includes steps 1102 and 1104.

[0119] In step 1102, in response to the colocation block having the BV, it is determined whether to use intraTMP mode or IBC mode to encode the colocation block.

[0120] In step 1104, in response to encoding the colocation block using the IBC mode, the BV of the colocation block is obtained. Then, step 706 of method 700 is performed.

[0121] Embodiments of this disclosure also provide a method for TMVP derivation. In a conventional ECM merge list design, three TMVP candidates (e.g., bidirectional predictive TMVP, L0 predictive TMVP, and L1 predictive TMVP) can be derived for each co-location block. However, due to the derivation process described above, these three TMVP candidates may be identical to each other. In this disclosure, it is proposed to check whether the two predictive values ​​(MVs) of the bidirectional predictive TMVP are identical to each other. If the two MVs of the bidirectional predictive TMVP are identical to each other, the L0 predictive TMVP and L1 predictive TMVP are not added to the list.

[0122] Figure 12 This is a flowchart of an exemplary method for constructing a TMVP candidate list according to some embodiments of this disclosure. Method 1200 may be performed by an encoder (e.g., via...) Figure 2A Process 200A or Figure 2B Process 200B), decoder (e.g., via Figure 3A Process 300A or Figure 3B The process 300B) is performed, or is carried out by a device (e.g., Figure 4 The device 400) is executed by one or more software or hardware components. For example, a processor (e.g., Figure 4 The processor 402 can execute method 1200. In some embodiments, method 1200 can be implemented by a computer program product included in a computer-readable medium, the computer program product comprising components implemented by a computer (e.g., processor 402). Figure 4 The device 400 executes computer-executable instructions, such as program code. (See reference) Figure 12 Method 1200 may include the following steps 1202 to 1206.

[0123] In step 1202, a first TMVP candidate is derived using bidirectional prediction, and the first TMVP candidate is added to the TMVP candidate list. The TMVP candidate comprises two motion vectors.

[0124] In step 1204, it is determined whether the two motion vectors of the TMVP candidate are the same.

[0125] In step 1206, in response to the two motion vectors being identical, the second TMVP candidate (derived using L0 prediction) and the third TMVP candidate (derived using L1 prediction) are not added to the TMVP candidate list. In some embodiments, the process of deriving the second and third TMVP candidates using L1 and L0 predictions is skipped.

[0126] Embodiments of this disclosure also provide a method for constructing a merge list or an AMVP list. In conventional ECM designs, the merge list or AMVP list includes spatial candidates, temporal candidates, non-adjacent spatial candidates, and history-based motion vector prediction (HMVP) candidates, and the order in which candidate types are added to the merge list or AMVP list is fixed. The spatial candidates are always added before the temporal candidates, and the non-adjacent spatial candidates and HMVP candidates are always added after the temporal candidates. However, when the co-located image is close to the current image, the temporal candidate is more accurate than the spatial candidate, and when the co-located image is far from the current image, the spatial candidate is more accurate than the temporal candidate. In this disclosure, an adaptive reordering of the candidate list construction is proposed based on the POC distance between the current image and its co-located images.

[0127] In some embodiments, an image sequence count (POC) distance between the current image and its co-occurring images is determined, and a candidate merging list is constructed based on the POC distance. In some embodiments, it is determined whether the POC distance is less than a threshold N1, where the threshold N1 is a positive number. The threshold N1 can be a predefined value. The threshold N1 can also be signaled at the SPS, PPS, image header, slice header, coding tree unit (CTU), or CU level. When the POC distance between the current image and its co-occurring images is less than the threshold N1, a temporal candidate is added before a spatial candidate. When the POC distance between the current image and its co-occurring images is greater than or equal to the threshold N1, a spatial candidate is added before a temporal candidate. In one example, the threshold N1 is set to 1.

[0128] In some embodiments, it is determined whether the POC distance is less than a threshold N2, where the threshold N2 is a positive number. The threshold N2 can be a predefined value. The threshold N2 can also be signaled at the SPS, PPS, image header, title sequence, CTU, or CU level. When the POC distance between the current image and its co-located images is less than the threshold N2, a temporal candidate is added before the non-adjacent spatial candidate. When the POC distance between the current image and its co-located images is greater than or equal to the threshold N2, a non-adjacent spatial candidate is added before the temporal candidate. In one example, the threshold N2 is set to 2.

[0129] In some embodiments, it is determined whether the POC distance is less than a threshold N3, where the threshold N3 is a positive number. The threshold N3 can be a predefined value. The threshold N3 can also be signaled at the SPS, PPS, image header, title sequence, CTU, or CU level. When the POC distance between the current image and its co-located images is less than the threshold N3, the temporal candidate is added before the non-adjacent spatial candidate and the HMVP candidate. When the POC distance between the current image and its co-located images is greater than or equal to the threshold N3, the temporal candidate is added after the non-adjacent spatial candidate and the HMVP candidate. In one example, the threshold N3 is set to 4.

[0130] In some embodiments, the construction of the candidate list is adaptively reordered based on temporal layers. For example, the temporal layer of the current image is determined, and a candidate merging list is constructed based on said temporal layer. In some embodiments, it is determined whether the temporal layer is less than a threshold M. The threshold M is zero or a positive integer. The threshold M can be a predefined value. The threshold M can be signaled at the SPS, PPS, image header, title sequence, CTU, or CU level. When the temporal layer of the current image is less than the threshold M (i.e., the probability that the co-located image is far from the current image is high), non-adjacent spatial candidates are added before the temporal candidates. When the temporal layer of the current image is greater than or equal to the threshold M, non-adjacent spatial candidates are added after the temporal candidates. In one example, the threshold M is set to 5.

[0131] In some embodiments, signaling sends flags to indicate how the candidate list is constructed, such as the order in which the candidates are constructed. These flags can be signaled at the SPS, PPS, image header, title sequence, CTU, or CU level. In one example, flags are signaled at the CU level to indicate whether a temporal candidate is added before a spatial candidate. In another example, flags are signaled at the title sequence level to indicate whether a temporal candidate is added after a non-adjacent spatial candidate.

[0132] Embodiments of this disclosure provide a method for applying a co-location region to a Time-Based MVP. With the help of a Time-Based MVP, temporal motion can be derived from either reference image. Figure 13 This illustrates the time motion that needs to be stored according to existing technology. For example... Figure 13 As shown, gray area 1310 is used to derive TMVP candidates. That is, all CTU temporal motions are stored. However, this design would require a large amount of memory to store the temporal motions, which is impractical for real-world hardware implementations. In this disclosure, it is proposed to store only a small amount of CTU temporal motions for each reference image.

[0133] Figure 14 Exemplary time motions requiring storage are illustrated according to some embodiments of this disclosure. For example... Figure 14As shown, for each reference image, only one CTU time motion 1421 is stored. The position of the CTU time motion 1411 in reference image 1410 corresponds to the position of the current CTU 1421 in the current image 1420. The current CTU 1421 contains a current CU 1422.

[0134] In some embodiments, the number of CTU time movements to be stored is determined based on the POC distance between the current image and each reference image. More time movements are stored when a reference image is closer to the current image. For example, a first number of CTU time movements are stored when the POC distance between the current image and its co-located images is less than a threshold N. A second number of CTU time movements are stored when the POC distance between the current image and its co-located images is greater than or equal to the threshold N. The first number is greater than the second number. The threshold N is a positive number and can be a predefined value. In some embodiments, the position of the CTU time movement in the reference image corresponds to the position of the current CTU. If more than one CTU time movement is stored, the region of the CTU time movement in the reference image is expanded centered on the position corresponding to the current CTU.

[0135] The embodiments described in this disclosure can be freely combined.

[0136] In some embodiments, a non-transitory computer-readable medium is provided that stores a bitstream. The bitstream is generated by receiving a video sequence and encoding the video sequence to generate encoded information included in the bitstream. The bitstream can be transmitted to a decoder for decoding. The video sequence is encoded using the method described above.

[0137] In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and the instructions can be executed by a device (e.g., the disclosed encoder and decoder) to perform the methods described above. Common forms of non-transitory media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs or any other flash memory, NVRAMs, caches, registers, any other memory chips or cassette tapes, and their networking versions. The device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0138] It should be noted that the relational terms such as "first," "second," etc., used in this document are only used to distinguish one entity or operation from another, and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the words "comprising," "having," "containing," and "including," as well as other similar forms, are intended to be identical in meaning and open-ended, because one or more items following any of these words do not imply an exhaustive list of such one or more items, nor do they imply limitation to only the listed one or more items.

[0139] As used herein, unless otherwise specified, the term "or" covers all possible combinations unless impractical. For example, if it is specified that a database may include A or B, then unless otherwise specified or impractical, the database may include A, or B, or A and B. As a second example, if it is specified that a database may include A, B, or C, then unless otherwise specified or impractical, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0140] It should be understood that the above embodiments can be implemented by hardware, software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-described computer-readable medium. When executed by the processor, the software can perform the disclosed methods. The computing units and other functional units described in this disclosure can be implemented by hardware, software, or a combination of hardware and software. Those skilled in the art should also understand that multiple modules / units described above can be combined into one module / unit, and each module / unit described above can be further divided into multiple sub-modules / sub-units.

[0141] The embodiments may be further described using the following terms: 1. A method for encoding a video sequence, the method comprising: Receive video sequences; and The video sequence is encoded using the following operations: Determine whether the co-location blocks in the co-location image of the current block have a block vector (BV); In response to the co-location block having the BV, the BV is determined; Based on the BV of the co-located block, determine the motion vector for the temporal motion vector prediction (TMVP) of the current block; and The reference index of the TMVP is determined as the reference index of the co-located image.

[0142] 2. The method according to Clause 1, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: The BV of the co-located block is used as the motion vector for the TMVP of the current block.

[0143] 3. The method according to Clause 1, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: Determine the relative position between the current block and the co-located block; and The sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block.

[0144] 4. The method according to Clause 1, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: Determine the first relative position between the current block and the co-located block; Determine a second relative position between the co-location block and its co-location coding unit, wherein the co-location coding unit includes the co-location block; and The sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block.

[0145] 5. The method according to Clause 1, wherein determining the BV in response to the co-location block having the BV further comprises: In response to the co-location block having the BV, determine whether to encode the co-location block using Intra-Template Matching Prediction (IntraTMP) mode or Intra-Block Copy (IBC) mode; and In response to encoding the co-location block using the IBC mode, the BV of the co-location block is obtained.

[0146] 6. The method according to Clause 1, wherein the co-occurring block is encoded using IntraTMP mode or IntraBlock Copy (IBC) mode.

[0147] 7. The method according to Clause 1, wherein the current block is encoded using at least one of the following: regular merging mode, combined inter and intra prediction (CIIP) mode, bi-directional motion (BM) mode, template matching (TM) mode, merge mode with motion vector difference (MMVD), affine mode, sub-block-based temporal motion vector prediction (SbTMVP) mode, or advanced motion vector prediction (AMVP) mode.

[0148] 8. The method according to Clause 1, further comprising: Store the bit stream generated based on the encoding.

[0149] 9. A method for encoding a video sequence, the method comprising: Receive video sequences; and The video sequence is encoded using the following operations: An adaptive candidate merging list is constructed, which includes spatial candidates, temporal candidates, non-adjacent spatial candidates, and history-based motion vector prediction (HMVP) candidates.

[0150] 10. The method according to Clause 9, wherein adaptively constructing the candidate merge list comprises: Determine the image order count (POC) distance between the current image and its co-located images; and The candidate merging list is constructed based on the POC distance.

[0151] 11. The method according to Clause 10, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the spatial candidate; or In response to the POC being greater than or equal to the threshold, the spatial candidate is added before the temporal candidate.

[0152] 12. The method according to Clause 10, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the non-adjacent spatial candidate; or In response to the POC being greater than or equal to the threshold, the non-adjacent spatial candidate is added before the temporal candidate.

[0153] 13. The method according to Clause 10, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the non-adjacent spatial candidate and the HMVP candidate; or In response to the POC being greater than or equal to the threshold, the non-adjacent spatial candidate and the HMVP candidate are added before the temporal candidate.

[0154] 14. The method according to Clause 11, wherein the encoding further comprises: The threshold is encoded at the SPS, PPS, image header, title sequence, coding tree unit (CTU), or coding unit level.

[0155] 15. The method according to Clause 9, wherein adaptively constructing the candidate merge list further comprises: Determine the time layer of the current image; and The candidate merging list is constructed based on the time layer.

[0156] 16. The method according to Clause 15, wherein constructing the candidate merge list based on the time layer further comprises: Determine whether the time layer is less than a threshold; In response to the time layer being less than the threshold, the non-adjacent spatial candidate is added before the time candidate; or In response to the time layer being greater than or equal to the threshold, the time candidate is added before the non-adjacent spatial candidate.

[0157] 17. The method according to Clause 16, wherein the encoding further comprises: The threshold is encoded at the SPS, PPS, image header, title sequence, coding tree unit (CTU), or coding unit level.

[0158] 18. The method according to Clause 9, wherein adaptively constructing the candidate merge list further comprises: Encode a flag bit that indicates whether a temporal candidate is added before a spatial candidate; In response to the flag bit being a first value, the temporal candidate is added before the spatial candidate; or In response to the flag bit being a second value, the spatial candidate is added before the temporal candidate.

[0159] 19. The method according to Clause 18, wherein the flag bit is encoded at the coding unit level.

[0160] 20. The method according to Clause 9, wherein adaptively constructing the candidate merge list further comprises: Encode a flag bit that indicates whether to add a time candidate before a non-adjacent spatial candidate; In response to the flag bit being a first value, the temporal candidate is added before the spatial candidate; or In response to the flag bit being a second value, the non-adjacent spatial candidate is added before the temporal candidate.

[0161] 21. The method according to Clause 20, wherein the flag bit is encoded at the beginning of the sequence.

[0162] 22. The method according to Clause 9, further comprising: Store the bit stream generated based on the encoding.

[0163] 23. A method for decoding a bitstream, the method comprising: Receive bitstream; and Decoding the bitstream to generate a video sequence, the decoding includes: Determine whether the co-location blocks in the co-location image of the current block have a block vector (BV); In response to the co-location block having the BV, the BV is determined; Based on the BV of the co-located block, determine the motion vector for the temporal motion vector prediction (TMVP) of the current block; and The reference index of the TMVP is determined as the reference index of the co-located image.

[0164] 24. The method according to Clause 23, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: The BV of the co-located block is used as the motion vector for the TMVP of the current block.

[0165] 25. The method according to Clause 23, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: Determine the relative position between the current block and the co-located block; and The sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block.

[0166] 26. The method according to Clause 23, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: Determine the first relative position between the current block and the co-located block; Determine a second relative position between the co-location block and its co-location coding unit, wherein the co-location coding unit includes the co-location block; and The sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block.

[0167] 27. The method according to Clause 23, wherein determining the BV in response to the co-location block having the BV further comprises: In response to the co-location block having the BV, determine whether to encode the co-location block using Intra-Template Matching Prediction (IntraTMP) mode or Intra-Block Copy (IBC) mode; and In response to encoding the co-location block using the IBC mode, the BV of the co-location block is obtained.

[0168] 28. The method according to Clause 23, wherein the co-occurring block is encoded using IntraTMP mode or IntraBlock Copy (IBC) mode.

[0169] 29. A method for transmitting a bit stream, the method comprising: Receive video sequences; The video sequence is encoded using the following operations: Determine whether the co-location blocks in the co-location image of the current block have a block vector (BV); In response to the co-location block having the BV, the BV is determined; Based on the BV of the co-located block, determine the motion vector for the temporal motion vector prediction (TMVP) of the current block; and The TMVP reference index is determined as the reference index of the co-located image; and Transmit the bit stream generated based on the encoding.

[0170] 30. The method according to Clause 29, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: The BV of the co-located block is used as the motion vector for the TMVP of the current block.

[0171] 31. The method according to Clause 29, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: Determine the relative position between the current block and the co-located block; and The sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block.

[0172] 32. The method according to Clause 29, wherein determining the motion vector for the TMVP of the current block based on the BV of the co-located block further comprises: Determine the first relative position between the current block and the co-located block; Determine a second relative position between the co-location block and its co-location coding unit, wherein the co-location coding unit includes the co-location block; and The sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block.

[0173] 33. The method according to clause 29, wherein determining the BV in response to the co-location block having the BV further comprises: In response to the co-location block having the BV, determine whether to encode the co-location block using Intra-Template Matching Prediction (IntraTMP) mode or Intra-Block Copy (IBC) mode; and In response to encoding the co-location block using the IBC mode, the BV of the co-location block is obtained.

[0174] 34. The method according to Clause 29, wherein the co-occurring block is encoded using IntraTMP mode or IntraBlock Copy (IBC) mode.

[0175] 35. A method for decoding a bitstream, the method comprising: Receive bitstream; and Decoding the bitstream to generate a video sequence, the decoding includes: An adaptive candidate merging list is constructed, which includes spatial candidates, temporal candidates, non-adjacent spatial candidates, and history-based motion vector prediction (HMVP) candidates.

[0176] 36. The method according to clause 35, wherein adaptively constructing the candidate merge list comprises: Determine the image order count (POC) distance between the current image and its co-located images; and The candidate merging list is constructed based on the POC distance.

[0177] 37. The method according to clause 36, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the spatial candidate; or In response to the POC being greater than or equal to the threshold, the spatial candidate is added before the temporal candidate.

[0178] 38. The method according to clause 36, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the non-adjacent spatial candidate; or In response to the POC being greater than or equal to the threshold, the non-adjacent spatial candidate is added before the temporal candidate.

[0179] 39. The method according to clause 36, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the non-adjacent spatial candidate and the HMVP candidate; or In response to the POC being greater than or equal to the threshold, the non-adjacent spatial candidate and the HMVP candidate are added before the temporal candidate.

[0180] 40. The method according to clause 39, wherein the decoding further comprises: The threshold is decoded at the SPS, PPS, image header, title sequence, coding tree unit (CTU), or coding unit level.

[0181] 41. The method according to clause 35, wherein adaptively constructing the candidate merge list further comprises: Determine the time layer of the current image; and The candidate merging list is constructed based on the time layer.

[0182] 42. The method according to clause 41, wherein constructing the candidate merge list based on the time layer further comprises: Determine whether the time layer is less than a threshold; In response to the time layer being less than the threshold, the non-adjacent spatial candidate is added before the time candidate; or In response to the time layer being greater than or equal to the threshold, the time candidate is added before the non-adjacent spatial candidate.

[0183] 43. The method according to clause 42, wherein the decoding further comprises: The threshold is decoded at the SPS, PPS, image header, title sequence, coding tree unit (CTU), or coding unit level.

[0184] 44. The method according to clause 35, wherein adaptively constructing the candidate merge list further comprises: Encode a flag bit that indicates whether a temporal candidate is added before a spatial candidate; In response to the flag bit being a first value, the temporal candidate is added before the spatial candidate; or In response to the flag bit being a second value, the spatial candidate is added before the temporal candidate.

[0185] 45. The method according to clause 44, wherein the flag bits are encoded at the encoding unit level.

[0186] 46. ​​The method according to clause 35, wherein adaptively constructing the candidate merge list further comprises: Decode a flag bit that indicates whether to add a time candidate before a non-adjacent spatial candidate; In response to the flag bit being a first value, the temporal candidate is added before the spatial candidate; or In response to the flag bit being a second value, the non-adjacent spatial candidate is added before the temporal candidate.

[0187] 47. The method according to Clause 46, wherein the flag bit is encoded at the beginning of the sequence.

[0188] 48. A method for transmitting a bit stream, the method comprising: Receive video sequences; The video sequence is encoded using the following operations: An adaptive candidate merging list is constructed, comprising spatial candidates, temporal candidates, non-adjacent spatial candidates, and history-based motion vector prediction (HMVP) candidates; and Transmit the bit stream generated based on the encoding.

[0189] 49. The method according to clause 48, wherein adaptively constructing the candidate merge list comprises: Determine the image order count (POC) distance between the current image and its co-located images; and The candidate merging list is constructed based on the POC distance.

[0190] 50. The method according to clause 49, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the spatial candidate; or In response to the POC being greater than or equal to the threshold, the spatial candidate is added before the temporal candidate.

[0191] 51. The method according to clause 49, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the non-adjacent spatial candidate; or In response to the POC being greater than or equal to the threshold, the non-adjacent spatial candidate is added before the temporal candidate.

[0192] 52. The method according to clause 49, wherein constructing the candidate merge list based on the POC distance further comprises: Determine whether the POC distance is less than a threshold; In response to the POC being less than the threshold, the temporal candidate is added before the non-adjacent spatial candidate and the HMVP candidate; or In response to the POC being greater than or equal to the threshold, the non-adjacent spatial candidate and the HMVP candidate are added before the temporal candidate.

[0193] 53. The method according to clause 50, wherein the encoding further comprises: The threshold is encoded at the SPS, PPS, image header, title sequence, coding tree unit (CTU), or coding unit level.

[0194] 54. The method according to clause 48, wherein adaptively constructing the candidate merge list further comprises: Determine the time layer of the current image; and The candidate merging list is constructed based on the time layer.

[0195] 55. The method according to clause 54, wherein constructing the candidate merge list based on the time layer further comprises: Determine whether the time layer is less than a threshold; In response to the time layer being less than the threshold, the non-adjacent spatial candidate is added before the time candidate; or In response to the time layer being greater than or equal to the threshold, the time candidate is added before the non-adjacent spatial candidate.

[0196] 56. The method according to clause 55, wherein the encoding further comprises: The threshold is encoded at the SPS, PPS, image header, title sequence, coding tree unit (CTU), or coding unit level.

[0197] 57. The method according to clause 48, wherein adaptively constructing the candidate merge list further comprises: Encode a flag bit that indicates whether a temporal candidate is added before a spatial candidate; In response to the flag bit being a first value, the temporal candidate is added before the spatial candidate; or In response to the flag bit being a second value, the spatial candidate is added before the temporal candidate.

[0198] 58. The method according to Clause 57, wherein the flag bit is encoded at the coding unit level.

[0199] 59. The method according to clause 48, wherein adaptively constructing the candidate merge list further comprises: Encode a flag bit that indicates whether to add a time candidate before a non-adjacent spatial candidate; In response to the flag bit being a first value, the temporal candidate is added before the spatial candidate; or In response to the flag bit being a second value, the non-adjacent spatial candidate is added before the temporal candidate.

[0200] 60. The method according to Clause 59, wherein the flag bit is encoded at the beginning of the sequence.

[0201] In the foregoing specification, numerous specific details have been described with reference to embodiments, which may vary depending on the implementation. Certain adjustments and modifications may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the appended claims. The sequence of steps shown in the figures is also to be considered for illustrative purposes only and is not intended to limit one to any particular sequence of steps. Therefore, those skilled in the art will understand that these steps may be performed in a different order while implementing the same method.

[0202] Exemplary embodiments have been disclosed in the accompanying drawings and description. However, many variations and modifications can be made to these embodiments. Accordingly, although specific terminology has been used, it is used in a general and descriptive sense only and is not intended to be limiting.

Claims

1. A method for encoding a video sequence, the method comprising: Receive video sequences; as well as The video sequence is encoded using the following operations: Determine whether the co-location blocks in the co-location image of the current block have a block vector BV; In response to the co-location block having the BV, the BV is determined; Based on the BV of the co-located block, determine the motion vector for the time motion vector prediction TMVP of the current block; as well as The reference index of the TMVP is determined as the reference index of the co-located image.

2. The method according to claim 1, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: The BV of the co-located block is used as the motion vector for the TMVP of the current block.

3. The method according to claim 1, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: Determine the relative position between the current block and the co-located block; and The sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block.

4. The method according to claim 1, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: Determine the first relative position between the current block and the co-located block; Determine a second relative position between the co-location block and its co-location coding unit, wherein the co-location coding unit includes the co-location block; and The sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block.

5. The method according to claim 1, wherein, In response to the co-location block having the BV, determining the BV further includes: In response to the co-location block having the BV, determine whether to encode the co-location block using IntraTMP mode or IntraBlockCopy (IBC) mode; and In response to encoding the co-location block using the IBC mode, the BV of the co-location block is obtained.

6. The method according to claim 1, wherein, The co-location block is encoded using either IntraTMP mode (Intra Template Matching Prediction) or IBC mode (Intra Block Copying).

7. The method according to claim 1, wherein, The current block is encoded using at least one of the following modes: regular merging mode, inter-frame and intra-frame joint prediction (CIIP) mode, bidirectional motion (BM) mode, template matching (TM) mode, merging (MMVD) mode with motion vector difference, affine mode, sub-block-based temporal motion vector prediction (SbTMVP) mode, or advanced motion vector prediction (AMVP) mode.

8. The method according to claim 1, further comprising: Store the bit stream generated based on the encoding.

9. A method for decoding a bitstream, the method comprising: Receive bit stream; as well as Decoding the bitstream to generate a video sequence, the decoding includes: Determine whether the co-location blocks in the co-location image of the current block have a block vector BV; In response to the co-location block having the BV, the BV is determined; Based on the BV of the co-located block, determine the motion vector for the temporal motion vector prediction (TMVP) of the current block; and The reference index of the TMVP is determined as the reference index of the co-located image.

10. The method according to claim 9, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: The BV of the co-located block is used as the motion vector for the TMVP of the current block.

11. The method according to claim 9, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: Determine the relative position between the current block and the co-located block; and The sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block.

12. The method according to claim 9, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: Determine the first relative position between the current block and the co-located block; Determine a second relative position between the co-location block and its co-location coding unit, wherein the co-location coding unit includes the co-location block; and The sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block.

13. The method according to claim 9, wherein, In response to the co-location block having the BV, determining the BV further includes: In response to the co-location block having the BV, determine whether to encode the co-location block using IntraTMP mode or IntraBlockCopy (IBC) mode; and In response to encoding the co-location block using the IBC mode, the BV of the co-location block is obtained.

14. The method according to claim 9, wherein, The co-location block is encoded using either IntraTMP mode (Intra Template Matching Prediction) or IBC mode (Intra Block Copying).

15. A method for transmitting a bit stream, the method comprising: Receive video sequences; The video sequence is encoded using the following operations: Determine whether the co-location blocks in the co-location image of the current block have a block vector BV; In response to the co-location block having the BV, the BV is determined; Based on the BV of the co-located block, determine the motion vector for the time motion vector prediction TMVP of the current block; as well as The reference index of the TMVP is determined as the reference index of the co-located image; as well as Transmit the bit stream generated based on the encoding.

16. The method according to claim 15, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: The BV of the co-located block is used as the motion vector for the TMVP of the current block.

17. The method according to claim 15, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: Determine the relative position between the current block and the co-located block; and The sum of the BV of the co-located block and the relative position is used as the motion vector for the TMVP of the current block.

18. The method according to claim 15, wherein, Determining the motion vector for the TMVP of the current block based on the BV of the co-located block further includes: Determine the first relative position between the current block and the co-located block; Determine a second relative position between the co-location block and its co-location coding unit, wherein the co-location coding unit includes the co-location block; and The sum of the BV of the co-located block, the first relative position, and the second relative position is used as the motion vector for the TMVP of the current block.

19. The method according to claim 15, wherein, In response to the co-location block having the BV, determining the BV further includes: In response to the co-location block having the BV, determine whether to encode the co-location block using IntraTMP mode or IntraBlockCopy (IBC) mode; and In response to encoding the co-location block using the IBC mode, the BV of the co-location block is obtained.

20. The method of claim 15, wherein, The co-location block is encoded using either IntraTMP mode (Intra Template Matching Prediction) or IBC mode (Intra Block Copying).

Images