Spatial prediction order of scanning

Abstract

To encode a coding unit in inter mode, there are multiple inter modes that use a motion vector predictor (MVP) list to signal motion information. The MVP list for different coding modes includes spatial predictors. In one embodiment, for all coding modes that use the MVP list, the top spatial predictor is added to the motion vector predictor list first before the left spatial predictor. In another embodiment, a subset of coding modes add the top spatial predictor before the left spatial predictor, and another subset of coding modes add the left spatial predictor before the top spatial predictor. For example, all merge modes add the top spatial predictor first, and all AMVP modes add the left spatial predictor first. In another example, all non-subblock modes add the top spatial predictor first, and all subblock modes add the left spatial predictor first.

Description

Technical Field

[0001] This implementation generally relates to a method and apparatus for video encoding or decoding, and more specifically, to a method and apparatus for adjusting motion vector predictor factors in a predictor factor list. Background Technology

[0002] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transform to utilize spatial and temporal redundancy in video content. Generally, intra-frame or inter-frame prediction is used to leverage intra- or inter-frame correlations, followed by transforming, quantizing, and entropy coding of the differences between the original and predicted blocks (usually represented as prediction error or prediction residual). To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0003] According to an implementation scheme, a video decoding method is provided, the method comprising: generating a list of motion vector predictor factors for blocks of images in a video sequence, wherein the blocks are decoded using one of a plurality of encoding modes, wherein the list of motion vector predictor factors includes a plurality of spatial predictor factors, wherein for each of the plurality of encoding modes, a first spatial predictor factor in the list comes from the top adjacent block, and a second spatial predictor factor following the first spatial predictor factor comes from the left adjacent block; and decoding motion vectors associated with the blocks based on the list of motion vector predictor factors.

[0004] According to another embodiment, a video coding method is provided, the method comprising: accessing blocks of images in a video sequence; generating a list of motion vector predictors for the blocks, wherein the blocks are encoded using one of a plurality of coding modes, wherein the list of motion vector predictors includes a plurality of spatial predictors, wherein for each of the plurality of coding modes, a first spatial predictor in the list comes from the top adjacent block, and a second spatial predictor following the first spatial predictor comes from the left adjacent block; and encoding a motion vector associated with the blocks based on the list of motion vector predictors.

[0005] According to another embodiment, a video decoding method is provided, the method comprising: generating a list of motion vector predictor factors for the block, wherein the block is decoded using one of a plurality of coding modes, wherein the list of motion vector predictor factors includes a plurality of spatial predictor factors, wherein for each of one or more merging modes of the plurality of coding modes, a first spatial predictor factor in the list comes from the top adjacent block, and a second spatial predictor factor following the first spatial predictor factor comes from the left adjacent block, and wherein for each of one or more AMVP modes of the plurality of coding modes, the first spatial predictor factor in the list comes from the left adjacent block, and the second spatial predictor factor following the first spatial predictor factor comes from the top adjacent block; and decoding motion vectors associated with the block based on the list of motion vector predictor factors.

[0006] According to another embodiment, a video coding method is provided, the method comprising: accessing blocks of images in a video sequence; generating a list of motion vector predictors for the blocks, wherein the blocks are encoded using one of a plurality of coding modes, wherein the list of motion vector predictors includes a plurality of spatial predictors, wherein for each of one or more merging modes of the plurality of coding modes, a first spatial predictor in the list comes from the top adjacent block, and a second spatial predictor following the first spatial predictor comes from the left adjacent block, and wherein for each of one or more AMVP modes of the plurality of coding modes, the first spatial predictor in the list comes from the left adjacent block, and the second spatial predictor following the first spatial predictor comes from the top adjacent block; and encoding motion vectors associated with the blocks based on the list of motion vector predictors.

[0007] According to another embodiment, a video decoding method is provided, the method comprising: generating a list of motion vector predictor factors for the block, wherein the block is decoded using one of a plurality of coding modes, wherein the list of motion vector predictor factors includes a plurality of spatial predictor factors, wherein for each of one or more non-sub-block modes of the plurality of coding modes, a first spatial predictor factor in the list comes from the top adjacent block, and a second spatial predictor factor following the first spatial predictor factor comes from the left adjacent block, and wherein for each of one or more sub-block modes of the plurality of coding modes, the first spatial predictor factor in the list comes from the left adjacent block, and the second spatial predictor factor following the first spatial predictor factor comes from the top adjacent block; and decoding motion vectors associated with the block based on the list of motion vector predictor factors.

[0008] According to another embodiment, a video coding method is provided, the method comprising: accessing blocks of images in a video sequence; generating a list of motion vector predictors for the blocks, wherein the blocks are encoded using one of a plurality of coding modes, wherein the list of motion vector predictors includes a plurality of spatial predictors, wherein for each of one or more non-sub-block modes of the plurality of coding modes, a first spatial predictor in the list comes from the top adjacent block, and a second spatial predictor following the first spatial predictor comes from the left adjacent block, and wherein for each of one or more sub-block modes of the plurality of coding modes, the first spatial predictor in the list comes from the left adjacent block, and the second spatial predictor following the first spatial predictor comes from the top adjacent block; and encoding a motion vector associated with the blocks based on the list of motion vector predictors.

[0009] According to another embodiment, an apparatus for video encoding is provided, wherein the apparatus includes one or more processors configured to: access blocks of images in a video sequence; generate a list of motion vector predictors for the blocks, wherein the blocks are encoded using one of a plurality of encoding modes, wherein the list of motion vector predictors includes a plurality of spatial predictors, wherein for each of the plurality of encoding modes, a first spatial predictor in the list comes from the top adjacent block, and a second spatial predictor following the first spatial predictor comes from the left adjacent block; and encode a motion vector associated with the blocks based on the list of motion vector predictors.

[0010] According to another embodiment, an apparatus for video decoding is provided, wherein the apparatus includes one or more processors configured to: generate a list of motion vector predictor factors for blocks of images in a video sequence, wherein the blocks are decoded using one of a plurality of encoding modes, wherein the list of motion vector predictor factors includes a plurality of spatial predictor factors, wherein for each of the plurality of encoding modes, a first spatial predictor factor in the list comes from the top adjacent block, and a second spatial predictor factor following the first spatial predictor factor comes from the left adjacent block; and decode motion vectors associated with the blocks based on the list of motion vector predictor factors.

[0011] One embodiment provides a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the above embodiments. One or more embodiments of this embodiment also provide a computer-readable storage medium storing instructions for encoding or decoding video data according to the above methods. One or more embodiments of this embodiment also provide a computer-readable storage medium storing a bitstream generated according to the above methods. One or more embodiments of this embodiment also provide a method and apparatus for transmitting or receiving a bitstream generated according to the above methods. Attached Figure Description

[0012] Figure 1 A block diagram of the system in which various aspects of this implementation scheme can be realized is shown.

[0013] Figure 2 A block diagram illustrating an implementation scheme for a video encoder is shown.

[0014] Figure 3 A block diagram illustrating an implementation scheme for a video decoder is shown.

[0015] Figure 4 Examples are shown to represent the concepts of Coding Tree Unit (CTU) and Coding Tree (CT) for compressing HEVC images.

[0016] Figure 5 An example is shown where a coding tree unit (CTU) is divided into coding units (CU), prediction units (PU), and transform units (TU).

[0017] Figure 6 This shows the partitions used in VVC.

[0018] Figure 7 This shows an example of partitioning an image using quadtrees and binary / ternary trees in VVC.

[0019] Figure 8 The locations of spatial and temporal predictors are shown.

[0020] Figure 9 An example of an affine motion vector field based on a 4×4 sub-CU is shown in VVC.

[0021] Figure 10 An example of affine model inheritance is shown.

[0022] Figure 11 An example of SbTMVP prediction is shown.

[0023] Figure 12This illustrates a method for adjusting the spatial predictor scan order based on whether the encoding mode is a merge mode or an AMVP mode, according to the implementation scheme.

[0024] Figure 13 This illustrates a method for adjusting the spatial predictor scan order based on whether the encoding pattern is a sub-block pattern, according to the implementation scheme. Detailed Implementation

[0025] Figure 1 A block diagram illustrating examples of systems in which various aspects and embodiments can be implemented. System 100 may be embodied as a device comprising the various components described below and configured to perform one or more aspects described in this patent application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 100 may be embodied individually or in combination in a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more aspects described in this patent application.

[0026] System 100 includes at least one processor 110 configured to execute instructions loaded thereon for implementing various aspects, such as those described in this patent application. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry as known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140 that may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 140 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0027] System 100 includes an encoder / decoder module 130 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both of an encoding module and a decoding module. Alternatively, the encoder / decoder module 130 may be implemented as a separate element of system 100 or may be incorporated within processor 110 as a combination of hardware and software known to those skilled in the art.

[0028] Program code to be loaded onto processor 110 or encoder / decoder 130 to execute the various aspects described in this patent application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more items from various projects during execution of the processes described in this patent application. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results of processing equations, formulas, operations, and operational logic.

[0029] In several embodiments, the memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be processor 110 or encoder / decoder module 130) is used for one or more of these functions. External memory may be memory 120 and / or storage device 140, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[0030] As shown in block 105, input to the components of system 100 can be provided through various input devices. Such input devices include, but are not limited to: (i) an RF section that receives RF signals, for example, transmitted by a broadcaster via radio; (ii) a composite input terminal; (iii) a USB input terminal; and / or (iv) an HDMI input terminal.

[0031] In various embodiments, the input device of block 105 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various of these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box implementation, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various implementations rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF section includes an antenna.

[0032] Furthermore, USB and / or HDMI terminals may include corresponding interface processors for connecting system 100 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 110. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, within a separate interface IC or within processor 110. Demodulated streams, error-corrected streams, and demultiplexed streams are provided to various processing elements, including, for example, processor 110 and encoder / decoder 130, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0033] Various components of system 100 can be housed within an integrated housing. Within the integrated housing, various components can be interconnected using a suitable connection arrangement 115 (e.g., internal buses as known in the art, including I2C buses, wiring, and printed circuit boards) and transmit data therebetween.

[0034] System 100 includes a communication interface 150 capable of communicating with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0035] In various implementations, data is streamed to system 100 using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signal in these implementations is received via a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other implementations use a set-top box to provide streaming data to system 100, delivering data via an HDMI connection to input block 105. Still other implementations use an RF connection to input block 105 to provide streaming data to system 100.

[0036] System 100 can provide output signals to various output devices, including display 165, speaker 175, and other peripheral devices 185. In various embodiments, the other peripheral devices 185 include one or more of the following: a standalone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, control signals are transmitted between system 100 and display 165, speaker 175, or other peripheral devices 185 using signaling (such as AV.Link, CEC, or other communication protocols), which enable device-to-device control with or without user intervention. Output devices can be communicatively coupled to system 100 via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices can be connected to system 100 via communication interface 150 using communication channel 190. Display 165 and speaker 175 can be integrated into a single unit with other components of system 100 in electronic devices (e.g., television sets). In various implementations, the display interface 160 includes a display driver, such as a timing controller (T Con) chip.

[0037] Alternatively, if the RF portion of input 105 is part of a separate set-top box, the display 165 and speaker 175 may be separately located from one or more other components. In various embodiments where the display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0038] Figure 2 An example of a video encoder 200, such as a VVC (Video Coding Universal) encoder, is shown. In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "encoding" or "coded," and the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstruction" is used at the encoding end, while "decoding" is used at the decoding end.

[0039] Before encoding, the video sequence may undergo pre-encoding processes, such as applying color transformations to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a more resilient signal distribution to compression (e.g., histogram equalization using one of the color components). Metadata may be associated with pre-processing and appended to the bitstream.

[0040] In encoder 200, the image is encoded by encoder elements as described below. The input signal is mapped (201). The mapping in 201 may correspond to the forward mapping in 291, or may further include other mappings for preprocessing. The image to be encoded is processed in units of CUs (202). Each CU is encoded using either an intra-frame mode or an inter-frame mode. When a CU is encoded in intra-frame mode, intra-frame prediction is performed (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The forward mapping (291) is applied to the prediction signal. The encoder determines (205) which of the intra-frame or inter-frame modes is used to encode the CU, and indicates the intra / inter-frame decision by a prediction mode label. The prediction residual is calculated by subtracting (210) the mapped prediction block (from step 291) from the mapped original image block (from step 201).

[0041] Then, the prediction residual is transformed (225) and quantized (230). The quantized transform coefficients, motion vectors, and other syntax elements are entropy encoded (245) to output a bitstream. The encoder can also skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., encode the residual directly without applying the transform or quantization process. In direct PCM encoding, no prediction is applied and the coding unit samples are directly encoded into the bitstream.

[0042] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct an image block. An inverse map (290) and a loop filter (265) are applied to the reconstructed signal to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coded artifacts. The filtered image is stored in a reference image buffer (280).

[0043] Figure 3 A block diagram illustrating an example video decoder 300, such as a VVC decoder, is shown. In decoder 300, the bitstream is decoded by decoder elements, as described below. Video decoder 300 typically performs operations similar to... Figure 2 The described decoding channel is the inverse of the encoding channel, which performs video decoding as part of the encoded video data.

[0044] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy-decoded (330) to obtain transform coefficients, motion vectors, picture partitioning information, and other encoded information. The picture partitioning information indicates the size of the CTU and how the CTU is divided into CUs (and possibly PUs where applicable). Therefore, the decoder can partition (335) the picture into CTUs based on the decoded picture partitioning information, and further partition each CTU into CUs. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals.

[0045] The decoded prediction residual and prediction block are combined (355) to reconstruct the image block. The prediction block (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). In addition, a forward mapping (395) is applied to the prediction signal. In the case of dual prediction, the two motion-compensated predictions can be combined with a weighted sum. An inverse mapping (396) and a loop filter (365) are applied to the reconstructed signal. The filtered image is stored in a reference image buffer (380).

[0046] The output from the loop filter can be inversely mapped (390), which reverses the mapping process (201) performed in preprocessing. The decoded image can then undergo further post-decoding processing, such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4). Post-decoding processing can utilize metadata derived in the pre-encoding process and signaled in the bitstream.

[0047] In HEVC, motion-compensated temporal prediction is used to exploit the redundancy that exists between consecutive frames in a video, where motion vectors are associated with each prediction unit (PU). Each PU is represented by a coding tree in the compressed domain. Figure 4 As shown, this is the quadtree partitioning of the CTU, where each leaf is called a coding unit (CU). Each CU is then given some intra-frame or inter-frame prediction parameters as prediction information. For this purpose, the CU can be spatially partitioned into one or more prediction units (PUs), and some prediction information can be assigned to each PU. Intra-frame or inter-frame coding modes are then assigned at the CU level. Figure 5 These concepts are further illustrated in the text.

[0048] The new video compression tools in VVC include a coding tree unit representation in the compression domain, which allows for a more flexible representation of image data. In VVC, a quadtree with a nested multi-type tree (MTT) using binary and ternary segmentation structures replaces the concept of multiple partition unit types; that is, VVC eliminates the distinction between CU, PU, ​​and TU concepts, except in some special cases. In the VVC coding tree structure, the shape of a CU can be square or rectangular. First, the coding tree unit (CTU) is partitioned using a quadtree structure. Then, the leaf nodes of the quadtree can be further partitioned using a multi-type tree structure.

[0049] Specifically, the tree decomposition of CTU is divided into different stages: first, the CTU is partitioned into a quadtree structure; then, each quadtree leaflet is further divided into binary or ternary branches. For example... Figure 6 As shown, there are four partitioning types in the multi-type tree structure: Vertical Binary Partition (VER), Horizontal Binary Partition (HOR), Vertical Tripartite Partition (VER_TRIPLE), and Horizontal Tripartite Partition (HOR_TRIPLE). HOR_TRIPLE partitioning or VER_TRIPLE partitioning (horizontal or vertical tripartite partitioning) involves dividing a coding unit (CU) into three sub-coding units (sub-CUs), where the corresponding sizes in the considered spatial partitioning directions are equal to 1 / 4, 1 / 2, and 1 / 4 of the size of the parent CU, respectively. Multi-type tree leaf nodes are called coding units (CUs), and except in some special cases, this segmentation is used for prediction and transform processing without further partitioning. Figure 7 The example shown is a CTU that is divided into multiple CUs with quadtree and ternary / binary tree coding block structures.

[0050] List of predictors in VTM (VVC test model)

[0051] In VVC, predictor lists exist in different inter-frame coding modes: regular merging, MMVD (merging mode with MVD), CIIP (intra-inter joint prediction), TPM / GEO, IBC (intra-block copying), regular AMVP (Advanced Motion Vector Prediction), affine AMVP, and sub-block merging. These predictor lists contain spatial predictors selected from spatially adjacent blocks at positions {A0, A1, A2, B0, B1, B2, B3}, such as... Figure 8 As shown. Predictors from the left neighboring block are considered left spatial predictors, and predictors from the top neighboring block are considered top spatial predictors. Some blocks (such as the top-left block B2) are considered both left and top neighboring blocks.

[0052] In VTM-6.0 (see “Algorithm Description for Universal Video Coding and Testing Model 6 (VTM 6)”, JVET-O2002, 15th Session: July 3-12, 2019, Gothenburg, Sweden), spatial candidates for each list are typically scanned from left to top. For example, for the regular merge mode, the first spatial predictor comes from A1 (left), and if it differs from the predictor of A1, the second spatial predictor comes from B1 (top). For the regular AMVP mode, the first spatial predictor is selected from either A0 or A1 (left), and the second spatial predictor comes from B0, B1, or B2 (top). The different lists are described in Table 1.

[0053] Table 1: VTM-6.0 list of predictor candidates (cands) for each coding scheme

[0054]

[0055]

[0056]

[0057]

[0058] Standard merge mode

[0059] In VTM-6.0, a list of predictors for the regular merge pattern is constructed by including the following types of predictors (also known as candidates): This list is also called the merge candidate list, merge list, or regular merge list.

[0060] 1) Spatial MVP (Motion Vector Predictor) from the spatial neighborhood CU;

[0061] 2) Time MVP from co-located CUs;

[0062] 3) History-based MVP from FIFO table;

[0063] 4) Paired average MVP; and

[0064] 5) Zero MVP.

[0065] For each CU encoded in the merge pattern, the index of the selected predictor is encoded. The generation process for merge candidates for each category is described below.

[0066] Spatial candidate export

[0067] The derivation of space merge candidates in VVC is the same as in HEVC. (The last part, "located in...", appears to be a fragment and doesn't translate directly. It's unclear what it refers to.) Figure 8 Up to four merge candidates are selected from the candidates at the locations depicted. The derivation order is A1, B1, B0, A0, and B2. Location B2 is considered only if any CU at locations A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another slice or tile) or is intra-coded. After adding the candidate at location A1, the addition of the remaining candidates is subject to a redundancy check, which ensures that candidates with the same motion information are excluded from the list, thus improving coding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check. Instead, only pairs as shown in Table 1 are considered, and a candidate is added to the list only if the corresponding candidate used for the redundancy check does not have the same motion information. For example, in Table 1, “Spatial B1 + Pruned A1” indicates that the candidate at location B1 is added to the merge list only if it is different from the candidate at location A1.

[0068] Time candidate export ("TMVP C0 / C1")

[0069] Only one temporal candidate is added to the merge list. Specifically, when exporting this temporal merge candidate, scaled motion vectors are exported based on the co-localized CUs belonging to the juxtaposed reference image. For example... Figure 8 As depicted, the position for selecting a temporal candidate between candidate C0 and C1 is used. If the CU at position C0 is unavailable, already intra-coded, or located outside the current line of the CTU, position C1 is used. Otherwise, position C0 is used when deriving temporal merge candidates.

[0070] Historical Merge Candidate Derivation

[0071] Historically based MVP (HMVP) merge candidates are added to the merge list after the spatial MVP and TMVP. To use an HMVP, motion information for previously encoded blocks is stored in a table and used as the MVP for the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-coded CU exists, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0072] HMVP candidates can be used during the merge candidate list construction process. The latest several HMVP candidates in the table are checked sequentially and inserted into the candidate list following the TMVP candidates. Redundancy checks are performed on some spatial merge candidates among the HMVP candidates. Each HMVP candidate in the last two (most recent) HMVP candidates in the FIFO is compared with spatial candidates A1 and B1, and added to the merge list only if it does not have the same motion information. The HMVP merge candidate list construction process terminates once the total number of available merge candidates reaches the maximum allowed number of merge candidates minus one.

[0073] Pairwise average merge candidate derivation

[0074] Paired average candidates are generated by averaging the first pair of candidates in the existing merge candidate list. When adding paired average merge candidates, if the merge list is not full, a zero MVP is inserted at the end until the maximum number of merge candidates is reached.

[0075] Merge pattern with MVD (MMVD)

[0076] In addition to using the implicitly derived motion information directly for the merging pattern of the current CU's prediction sample generation, VVC also introduces a merging pattern with motion vector difference (MMVD).

[0077] In MMVD, after selecting a merge candidate, it is further refined using MVD information signaled via a semaphore. Additional information includes merge candidate markers, an index specifying the motion amplitude, and an index indicating the motion direction. In MMVD mode, one of the top two candidates in the merge list is selected as the MV basis. The merge candidate marker is signaled to specify which one to use.

[0078] Triangular Partitioning (TPM) for Inter-Frame Prediction

[0079] When using the triangular partitioning mode, the CU is uniformly divided into two triangular partitions using either diagonal or anti-diagonal partitioning. Each triangular partition in the CU uses its own motion for inter-frame prediction; only a single prediction is allowed for each partition. The single prediction candidate list is derived directly from the merge candidate list constructed according to the merging mode as described above.

[0080] If the triangular partitioning pattern is used for the current CU, then a marker indicating the direction of the triangular partition (diagonal or anti-diagonal) is further signaled, along with two merge indexes (one for each partition).

[0081] Intra-block copy mode (IBC)

[0082] Intra-Block Copy (IBC) mode is derived from the screen content extension of HEVC. This mode simulates inter-frame prediction using the decoded current frame. It only supports IBC AMVP and IBC merge modes, which have the same list of predictors as their corresponding regular modes but with a simpler list.

[0083] In VVC, the IBC pooled predictor list (also known as the IBC pooled candidate list or IBC pooled list) is constructed by including the following types of candidates:

[0084] 1) Spatial MVP from the spatial neighborhood CU;

[0085] 2) Historical MVP from IBC FIFO table; and

[0086] 3) Zero MVP.

[0087] Spatial candidate export

[0088] The derivation of spatial IBC merging candidates is the same as that of regular merging. Only a maximum of two spatial IBC merging candidates are selected from those located at positions A1 and B1. After adding the candidate at position A1, the addition of the remaining candidates is subject to redundancy checks. This ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency.

[0089] Historical Merge Candidate Derivation

[0090] Historically based MVP (HMVP) IBC merge candidates are added to the IBC merge list following the spatial IBC MVP in the same manner as the regular merge list. However, a separate FIFO is used for the IBC pattern.

[0091] When an HMVP candidate is added, if the IBC merge list is not full, insert a zero IBC MVP at the end until the maximum number of IBC merge candidates is reached.

[0092] In IBC AMVP mode, since the only available reference frame is the current reference frame, only one list of predictors needs to be constructed. This list of predictors is then reused as a merged list of two candidate IBCs without pruning.

[0093] Standard AMVP mode

[0094] The AMVP mode in VVC (also known as regular AMVP) is similar to the AMVP mode in HEVC. A list of predictors for the AMVP mode (also known as the AMVP candidate list, AMVP list, or regular AMVP list) is constructed for each reference frame in the reference frame list by including the following types of candidates:

[0095] 1) Spatial MV from the spatial neighborhood CU;

[0096] 2) Time MV from the co-located CU;

[0097] 3) Historical MV from the FIFO table; and

[0098] 4) Zero MV.

[0099] The accuracy of the motion storage device (1 / 16-pel) is higher than the allowable motion vector difference accuracy, which, depending on the AMVR (Adaptive Motion Vector Resolution) mode, can be 1 / 4-pel, 1 / 2-pel, 1-pel, or 4-pel. To comply with such constraints, each predictor is rounded to the allowable motion vector difference accuracy.

[0100] For each CU encoded in AMVP mode and for each list of reference frames, the index of the selected reference frame, the index of the selected predictor (also known as an AMVP candidate), and the motion vector difference (the difference between the selected motion vector estimated by motion estimation and the predictor) are encoded. The following section encodes reference frame x (denoted as ref) for reference frame list y. xy The process of generating AMVP candidates for each category is described.

[0101] Spatial candidate export

[0102] ref in VVC xy The derivation of the spatial AMVP candidates is the same as in HEVC. In the location... Figure 8 Choose up to two AMVP candidates from the candidates at the positions depicted. One candidate is selected from the left position A0 or A1, and the other is selected from the top position B. 0, B1 or B2. For a candidate to be considered, a reference frame (ref) must also be used. xySpecifically, when A0 is not inter-frame coded or does not use a reference frame (ref)... xy If the left-hand predictor is selected from position A1, then the addition of the top-hand candidate is subject to a redundancy check after the left-hand candidate is added. This ensures that candidates with the same motion vector are excluded from the list, thereby improving coding efficiency.

[0103] Time candidate export ("TMVP C0 / C1")

[0104] This is achieved by using the same method as a regular merge list, but with the restriction of using refs. xy The export process adds only one time candidate to the AMVP list.

[0105] Candidate output based on history

[0106] If using ref xy If there is no redundancy check, then several HMVP candidates in the FIFO table are checked in reverse order and inserted into the candidate list after the TMVP candidates. The HMVP AMVP candidate list construction process terminates once the total number of available AMVP candidates reaches the maximum allowed number of AMVP candidates.

[0107] When adding an HMVP candidate, if the AMVP list is not full, insert a zero MV at the end until the maximum number of AMVP candidates is reached.

[0108] Affine AMVP mode

[0109] Affine AMVP and sub-block patterns are similar to regular AMVP, but use... Figure 9 The model presented is an affine motion model rather than a translation model.

[0110] One flag signals that the AMVP mode is affine, and another flag indicates whether a 4-parameter affine model or a 6-parameter affine model is used. Then, for each list of reference frames, the index of the selected reference frame, the index of the selected predictor (also known as the affine AMVP candidate), and the two motion vector differences (the difference between the best CPMV estimated by affine motion estimation and the CPMV predictor) are encoded.

[0111] Select predictors from the predictor list, which is constructed for each reference frame of the reference frame list by including the following types of candidates:

[0112] 1) Inherited spatial affine CPMV (control point motion vector) from the spatial affine neighborhood CU;

[0113] 2) Affine CPMV constructed from the spatial neighborhood CU;

[0114] 3) Translational CPMV constructed from spatial and temporal neighborhood CUs; ​​and

[0115] 4) Zero-affine CPMV.

[0116] The accuracy of the motion storage device (1 / 16-pel) is higher than the allowable motion vector difference accuracy, which, depending on the AMVR (Adaptive Motion Vector Resolution) mode, can be 1 / 16-pel, 1 / 4-pel, or 1-pel. To comply with such constraints, each predictor is rounded to the allowable motion vector difference accuracy.

[0117] Inherited Spatial Affine Candidate Derivation

[0118] Reference frame in VVC xy The derivation of the space affine candidate of inheritance is based on the position of Figure 10 Affine candidates are selected from the positions depicted. One candidate is chosen from the left positions A0 and A1, and another candidate is chosen from the top positions B0, B1, and B2. For consideration, the candidates must be affine coded, and each candidate must also use a reference frame. xy .

[0119] The affine model (CPMV) for the current CU is derived from the storage model of the affine neighborhood under consideration.

[0120] Constructed affine candidate export

[0121] By extracting CPMV from the top left (LT) position (A2, B2, B3), the left bottom (LB) position (A0, A1), and the right top (RT) position (B0, B1), a ref is constructed. xy The affine model is then defined as a combination of these extracted CPMVs.

[0122] Constructed translation candidate export

[0123] When adding constructed affine candidates, if the affine AMVP list is not full, the translation model is inserted into the list. All CPMVs of the current affine model are successively set as translation motion vectors from the top position, left position, top-left position, and time position.

[0124] If the affine AMVP list is not full after this point, insert a zero CPMV at the end until the maximum number of affine AMVP candidates is reached.

[0125] Sub-block merging mode

[0126] The sub-block merging pattern includes two distinct sets of predictor candidates: SbTMVP and affine merging. For the affine AMVP pattern, a flag signals that the merging pattern is sub-block. Then, for the regular merging pattern, the index of the selected predictor is encoded for each CU encoded in the sub-block merging pattern.

[0127] A list of predictors for the sub-block merging pattern (also known as the sub-block merging candidate list or affine merging list) is constructed by including the following types of candidates:

[0128] 1) SbTMVP candidate;

[0129] 2) Inherited spatial affine CPMV from the spatial affine neighborhood CU;

[0130] 3) Affine CPMV constructed from spatial and temporal neighborhood CUs; ​​and

[0131] 4) Zero CPMV.

[0132] Sub-block temporal motion vector predictor (SbTMVP) export

[0133] The SbTMVP candidate uses the left spatial candidate A1 as input (or zero MV if A1 is unavailable). This input motion vector is used to juxtapose reference frames to retrieve motion information of the current CU based on an 8×8 matrix, such as... Figure 11 As shown.

[0134] Inherited Spatial Affine Candidate Derivation

[0135] The derivation of the inherited spatial affine candidate is the same as that of the affine AMVP pattern, where the affine model (CPMV) for the current CU is derived from the storage model of the left affine neighborhood and then to the top affine neighborhood.

[0136] Constructed affine candidate export

[0137] An affine model is constructed by extracting CPMVs from the top left position (A2, B2, B3), left position (A0, A1), top position (B0, B1), and time position C0. The current affine model is then constrained to be a combination of these extracted CPMVs.

[0138] If the sub-block merge list is not full after this point, insert zero CPMV at the end until the maximum number of sub-block merge candidates is reached.

[0139] In VTM-7.0 (see “Algorithm Description for Universal Video Coding and Test Model 7 (VTM 7)”, JVET-P2002, 16th Session: October 1-11, 2019, Gothenburg, Sweden), the regular merge mode, MMVD mode, and TPM / GEO mode employ top-left scanning, while other coding modes maintain left-top scanning, as shown in Table 2. For the regular merge mode, Table 3 shows the differences in merge list construction.

[0140] Table 2: Candidate predictors modified by VTM-7.0

[0141]

[0142] Table 3: Differences between VTM-6.0 and VTM-7.0 regular merge lists

[0143]

[0144] The top-left scan in VTM-7.0 only affects merge mode, MMVD mode, and triangle mode. Therefore, the concept of top-left scan can be extended to all other coding modes, such as IBC mode, AMVP mode, affine AMVP mode, and sub-block merge mode. In one implementation, it is proposed to consider top-left scan of spatial predictors in each applicable coding mode.

[0145] Extending to IBC

[0146] It is proposed to use a top-left scan for the spatial predictors constructed from the IBC list as presented in Table 4. As shown in Table 4, spatial predictor B1 is considered before spatial predictor A1 in the predictor list construction. This will affect both IBC merging and IBC AMVP, as both modalities use the same list.

[0147] Table 4: Top left-side scan extends to the IBC list

[0148]

[0149] Extending to regular AMVP

[0150] In the case of AMVP, a set of spatial neighborhoods is explored. However, in VTM-7.0, the left group is scanned before the top group. Therefore, it is proposed to explore the top positions (B0 / B1 / B2) before exploring the left positions (A0 / A1), as shown in Table 5.

[0151] Table 5: Top left-side scan extends to the regular AMVP list

[0152]

[0153] In the variant, B2 (i.e., the top left candidate) in the left candidate group can also be moved, as shown in Table 6.

[0154] Table 6: Top left-side scan extends to the regular AMVP list using B2 as the left-side candidate.

[0155]

[0156] Extending to Affine AMVP

[0157] In affine AMVP, the same left-hand candidate group is explored before exploring the top candidate group, just as in the case of AMVP. It is also possible to revert to the scan order presented in Table 7.

[0158] Table 7: Top left-side scan extends to the affine AMVP list

[0159]

[0160]

[0161] In the variant, B2 (i.e., the top left candidate) in the left candidate group can also be moved, as shown in Table 8.

[0162] Table 8: Top left-side scan extends to the affine AMVP list using B2 as the left-side candidate.

[0163]

[0164] Extend to sub-block merging

[0165] The sub-block merging list in VTM-7.0 includes five candidates with different properties. These can be SbTMVP candidates, inherited or constructed affine models. The proposed top-left scan of the spatial candidates can be extended to the derived affine model, as presented in Table 9.

[0166] Table 9: Top left-side scan extends to sub-block merge list

[0167]

[0168]

[0169] In the variant, B2 (i.e., the top left candidate) in the left candidate group can also be moved, as shown in Table 10.

[0170] Table 10: Top left scan extends to the sub-block merge column using B2 as the left candidate. surface

[0171]

[0172] In another variation, the modification can also affect the SbTMVP candidates by using the top candidate of the space as input instead of the left candidate, as shown in Table 11.

[0173] Table 11: Top left scan extends to the sub-block merge list including SbTMVP

[0174]

[0175]

[0176] In the preceding text, it was proposed to use a top-left scan for spatial neighborhoods across all encoding patterns. In another implementation, it was proposed to apply this order only to subgroups of encoding patterns.

[0177] As mentioned above, when using a list of motion vector predictors, there are typically two types of encoding modes: merge mode and AMVP mode. For encoding units using merge mode, a list of predictors is generated and the indices in the list (e.g., mvp_idx) are transmitted to the decoder. The decoder generates the same list of predictors and selects the motion vector predictors for the current encoding unit based solely on the decoded indices of the selected motion vector predictors. Typically, no other motion information, such as the reference image index (ref_idx) or motion vector difference (MVD), needs to be transmitted.

[0178] For coding units using the AMVP mode, a list of predictors is generated, and the indices in the list (e.g., mvp_idx) are transmitted to the decoder. Additionally, other motion information, such as the reference image index (ref_idx) and motion vector difference (MVD), is also transmitted. The decoder generates the same list of predictors and selects motion vector predictors from the list as the motion vector predictors for the current coding unit. Then, the motion vectors are decoded based on the motion vector predictors, the decoded MVD, and the reference image index.

[0179] In VVC, MMVD is usually treated as a merge mode, but it signals MVD information.

[0180] In one implementation, we propose using a top-left scan (1210, e.g., the top spatial predictor is the first predictor in the predictor list, followed by the left spatial predictor) for all coding units with merging patterns (1205, e.g., regular merging, MMVD, TPM, IBC, sub-block merging), and a left-top scan (1220, e.g., the left spatial predictor is the first predictor in the predictor list, followed by the top spatial predictor) for all coding units with AMVP patterns (1205, e.g., regular AMVP, affine AMVP). Other predictors can also be added (1230) to the predictor list. After constructing the predictor list, motion information can be encoded or decoded based on the predictor list (1240). Note that the reverse is also possible, i.e., the AMVP pattern uses a top-left scan, and the merging pattern uses a left-top scan.

[0181] In another classification approach, when using a list of motion vector predictors, coding modes can be categorized into non-subblock modes and subblock modes. In subblock modes, motion compensation is performed at the subblock level when the coding mode is indicated at the coding unit level, and subblocks of the CU can have different motion vectors. In VVC, only affine candidates (from affine AMVP or subblock merging) and SbTMVP candidates are available. However, it can include other potential subblock tools as FRUC (frame rate up-conversion) and OBMC (overlapping block motion compensation).

[0182] In one implementation, we propose using a top-left scan (1310, e.g., the top spatial predictor is the first predictor in the predictor list, followed by the left spatial predictor) for all coding units with non-subblock patterns (1305, e.g., merge, MMVD, TPM, IBC, AMVP), and a left-top scan (1320, e.g., the left spatial predictor is the first predictor in the predictor list, followed by the top spatial predictor) for all coding units with subblock patterns (1305, affine AMVP, subblock merge). Other predictors can also be added (1330) to the predictor list. After constructing the predictor list, motion information can be encoded or decoded based on the predictor list (1340). Note that the reverse is also possible, i.e., top-left is used for subblock patterns, and left-top is used for non-subblock patterns.

[0183] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or purpose of a particular step and / or action may be modified or combined.

[0184] The various methods and other aspects described in this patent application can be used to modify modules of the video encoder 200 and decoder 300 (e.g., motion estimation and compensation modules (270, 275, 375)), such as Figure 2 and Figure 3 As shown. Furthermore, aspects of the invention are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, as well as any extensions of such standards and recommendations. Unless otherwise specified or technically excluded, the aspects described in this application may be used alone or in combination.

[0185] One embodiment provides a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the above embodiments. One or more embodiments of this embodiment also provide a computer-readable storage medium storing instructions for encoding or decoding video data according to the above methods. One or more embodiments of this embodiment also provide a computer-readable storage medium storing a bitstream generated according to the above methods. One or more embodiments of this embodiment also provide a method and apparatus for transmitting or receiving a bitstream generated according to the above methods.

[0186] Various specific implementations involve decoding. As used in this patent application, "decoding" can encompass, for example, all or part of the process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by the decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the phrase "decoding process" specifically refers to a subset of operations or broadly refers to a wider range of decoding processes will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0187] Various specific implementations are involved in encoding. In a manner similar to the discussion above regarding “decoding,” the term “encoding,” as used in this patent application, can encompass, for example, all or part of the process performed on an input video sequence to produce an encoded bitstream.

[0188] Note that the syntax elements used in this article, such as those indicating the RST kernel index and those indicating whether to use planar intra-prediction mode, are descriptive terms. Therefore, they do not preclude the use of other syntax element names.

[0189] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may also be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented in apparatus (e.g., processors), which generally refer to processing devices, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0190] The reference to "an implementation scheme" or "implementation scheme" or "a specific implementation" or "specific implementation," and other variations thereof, means that the specific features, structures, characteristics, etc., described in connection with the implementation scheme are included in at least one implementation scheme. Therefore, the appearance of the phrase "in an implementation scheme" or "in an implementation scheme" or "in a specific implementation" or "in a specific implementation," and any other variations appearing throughout this application, do not necessarily refer to the same implementation scheme.

[0191] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, or information retrieved from memory.

[0192] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0193] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information from memory, or one or more of these. Moreover, "receiving" typically involves one or more of the following during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0194] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as possible listed.

[0195] It will be apparent to those skilled in the art that embodiments may produce various signals formatted to carry, for example, storable or transmissible information. The information may include, for example, instructions for performing a method or data generated by one of the embodiments. For example, signals may be formatted to carry a bitstream of the embodiments. Such signals may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and using a modulated carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, signals can be transmitted via a variety of different wired or wireless links. Signals may be stored on a processor-readable medium.

Claims

1. A method for video encoding, the method comprising: Access blocks of images in a video sequence, wherein the blocks are to be encoded using an encoding mode; A first list of motion vector predictors for the sub-block coding pattern is generated based on a first plurality of spatial predictors associated with the sub-block coding pattern, wherein the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the left adjacent block. A second list of motion vector predictors for the non-sub-block coding pattern is generated based on a second plurality of spatial predictors associated with the non-sub-block coding pattern, wherein a first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the top adjacent block, and a second spatial predictor following the first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the left adjacent block; and Based on the first list of motion vector predictors or the second list of motion vector predictors, the motion vector associated with the block is encoded.

2. The method according to claim 1, wherein, The sub-block coding mode is either the Sub-block Temporal Motion Vector Prediction (SbTMVP) mode, the Frame Rate Upconversion (FRUC) coding mode, the Overlapping Block Motion Compensation (OBMC) coding mode, the Affine Advanced Motion Vector Prediction (AMVP) mode, or the Sub-block Merging mode.

3. The method according to claim 1, further comprising: A third list of motion vector predictors for the AMVP coding pattern is generated based on a third plurality of spatial predictors associated with the advanced motion vector prediction (AMVP) coding pattern. The first spatial predictor in the third plurality of spatial predictors in the third list of motion vector predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the third plurality of spatial predictors in the third list of motion vector predictors is obtained by scanning the left adjacent block. The motion vectors are further encoded based on the first list of motion vector predictors, the second list of motion vector predictors, or the third list of motion vector predictors.

4. The method according to claim 3, further comprising: A fourth list of motion vector predictors for the merged coding pattern is generated based on a fourth plurality of spatial predictors associated with the merged coding pattern. The first spatial predictor in the fourth list of spatial predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor is obtained by scanning the left adjacent block. The motion vector is further encoded based on the first list of motion vector predictors, the second list of motion vector predictors, the third list of motion vector predictors, or the fourth list of motion vector predictors.

5. The method according to claim 2, wherein, The non-sub-block coding mode is the MMVD (Merge-in-Motion) mode, the TPM (Triangle Partitioning) mode, or the IBC (Intra-Block Copying) mode.

6. A method for video decoding, the method comprising: Access blocks of images in a video sequence, wherein the blocks are to be decoded using an encoding mode; A first list of motion vector predictors for the sub-block coding pattern is generated based on a first plurality of spatial predictors associated with the sub-block coding pattern, wherein the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the left adjacent block. A second list of motion vector predictors for the non-sub-block coding pattern is generated based on a second plurality of spatial predictors associated with the non-sub-block coding pattern, wherein a first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the top adjacent block, and a second spatial predictor following the first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the left adjacent block; and Based on the first motion vector predictor list or the second motion vector predictor list, decode the motion vector associated with the block.

7. The method according to claim 6, wherein, The sub-block coding mode is either the Sub-block Temporal Motion Vector Prediction (SbTMVP) mode, the Frame Rate Upconversion (FRUC) coding mode, the Overlapping Block Motion Compensation (OBMC) coding mode, the Affine Advanced Motion Vector Prediction (AMVP) mode, or the Sub-block Merging mode.

8. The method according to claim 6, further comprising: A third list of motion vector predictors for the AMVP coding pattern is generated based on a third plurality of spatial predictors associated with the advanced motion vector prediction (AMVP) coding pattern. The first spatial predictor in the third plurality of spatial predictors in the third list is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the third plurality of spatial predictors in the third list is obtained by scanning the left adjacent block. The motion vectors are further decoded based on the first list of motion vector predictors, the second list of motion vector predictors, or the third list of motion vector predictors.

9. The method according to claim 8, further comprising: A fourth list of motion vector predictors for the merged coding pattern is generated based on a fourth plurality of spatial predictors associated with the merged coding pattern. The first spatial predictor in the fourth list of spatial predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor is obtained by scanning the left adjacent block. The motion vectors are further decoded based on the first list of motion vector predictors, the second list of motion vector predictors, the third list of motion vector predictors, or the fourth list of motion vector predictors.

10. The method according to claim 7, wherein, The non-sub-block coding mode is the MMVD (Merge-in-Motion) mode, the TPM (Triangle Partitioning) mode, or the IBC (Intra-Block Copying) mode.

11. An apparatus for video encoding, the apparatus comprising at least one memory and one or more processors, wherein, The one or more processors are configured to: Access blocks of images in a video sequence, wherein the blocks are to be encoded using an encoding mode; A first list of motion vector predictors for the sub-block coding pattern is generated based on a first plurality of spatial predictors associated with the sub-block coding pattern, wherein the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the left adjacent block. A second list of motion vector predictors for the non-sub-block coding pattern is generated based on a second plurality of spatial predictors associated with the non-sub-block coding pattern, wherein a first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the top adjacent block, and a second spatial predictor following the first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the left adjacent block; and Based on the first list of motion vector predictors or the second list of motion vector predictors, the motion vector associated with the block is encoded.

12. The apparatus according to claim 11, wherein, The sub-block coding mode is either the Sub-block Temporal Motion Vector Prediction (SbTMVP) mode, the Frame Rate Upconversion (FRUC) coding mode, the Overlapping Block Motion Compensation (OBMC) coding mode, the Affine Advanced Motion Vector Prediction (AMVP) mode, or the Sub-block Merging mode.

13. The apparatus according to claim 11, wherein, The one or more processors are further configured to: A third list of motion vector predictors for the AMVP coding pattern is generated based on a third plurality of spatial predictors associated with the advanced motion vector prediction (AMVP) coding pattern. The first spatial predictor in the third plurality of spatial predictors in the third list of motion vector predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the third plurality of spatial predictors in the third list of motion vector predictors is obtained by scanning the left adjacent block. The motion vectors are further encoded based on the first list of motion vector predictors, the second list of motion vector predictors, or the third list of motion vector predictors.

14. The apparatus according to claim 13, wherein, The one or more processors are further configured to: A fourth list of motion vector predictors for the merged coding pattern is generated based on a fourth plurality of spatial predictors associated with the merged coding pattern. The first spatial predictor in the fourth list of spatial predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor is obtained by scanning the left adjacent block. The motion vector is further encoded based on the first list of motion vector predictors, the second list of motion vector predictors, the third list of motion vector predictors, or the fourth list of motion vector predictors.

15. The apparatus according to claim 12, wherein, The non-sub-block coding mode is the MMVD (Merge-in-Motion) mode, the TPM (Triangle Partitioning) mode, or the IBC (Intra-Block Copying) mode.

16. An apparatus for video decoding, the apparatus comprising at least one memory and one or more processors, wherein, The one or more processors are configured to: Access blocks of images in a video sequence, wherein the blocks are to be decoded using an encoding mode; A first list of motion vector predictors for the sub-block coding pattern is generated based on a first plurality of spatial predictors associated with the sub-block coding pattern, wherein the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the first plurality of spatial predictors in the first list of motion vector predictors is obtained by scanning the left adjacent block. A second list of motion vector predictors for the non-sub-block coding pattern is generated based on a second plurality of spatial predictors associated with the non-sub-block coding pattern, wherein a first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the top adjacent block, and a second spatial predictor following the first spatial predictor among the second plurality of spatial predictors in the second list of motion vector predictors is obtained from scanning the left adjacent block; and Based on the first motion vector predictor list or the second motion vector predictor list, decode the motion vector associated with the block.

17. The apparatus according to claim 16, wherein, The sub-block coding mode is either the Sub-block Temporal Motion Vector Prediction (SbTMVP) mode, the Frame Rate Upconversion (FRUC) coding mode, the Overlapping Block Motion Compensation (OBMC) coding mode, the Affine Advanced Motion Vector Prediction (AMVP) mode, or the Sub-block Merging mode.

18. The apparatus according to claim 16, wherein, The one or more processors are further configured to: A third list of motion vector predictors for the AMVP coding pattern is generated based on a third plurality of spatial predictors associated with the advanced motion vector prediction (AMVP) coding pattern. The first spatial predictor in the third plurality of spatial predictors in the third list is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor in the third plurality of spatial predictors in the third list is obtained by scanning the left adjacent block. The motion vectors are further decoded based on the first list of motion vector predictors, the second list of motion vector predictors, or the third list of motion vector predictors.

19. The apparatus according to claim 18, wherein, The one or more processors are further configured to: A fourth list of motion vector predictors for the merged coding pattern is generated based on a fourth plurality of spatial predictors associated with the merged coding pattern. The first spatial predictor in the fourth list of spatial predictors is obtained by scanning the top adjacent block, and the second spatial predictor following the first spatial predictor is obtained by scanning the left adjacent block. The motion vectors are further decoded based on the first list of motion vector predictors, the second list of motion vector predictors, the third list of motion vector predictors, or the fourth list of motion vector predictors.

20. The apparatus according to claim 17, wherein, The non-sub-block coding mode is the MMVD (Merge-in-Motion) mode, the TPM (Triangle Partitioning) mode, or the IBC (Intra-Block Copying) mode.

Images