Method and device for candidate derivation for affine merge modes in video coding

Abstract

A method, an apparatus, and a non-transitory storage medium for video decoding and encoding are provided. In one decoding method, a decoder obtains a first candidate position and a second candidate position. The decoder obtains a third candidate position based on the first and second candidate positions, and obtains a virtual block based on the first, second, and third candidate positions. The decoder may obtain multiple CPMVs for the virtual block based on translation MVs at the first, second, and third candidate positions, and may project the multiple CPMVs for the virtual block onto a current block to obtain a translation MV based on a specific position in the current block, or may project multiple second CPMVs for the current block.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 296,467, entitled "Candidate Derivation for Affine Merge Mode in Video Coding," filed January 4, 2022, which is incorporated by reference in its entirety for all purposes. FIELD This disclosure relates to video encoding and compression, and more particularly, but not exclusively, to methods and apparatus for improving affine merge candidate derivation for affine motion prediction modes in a video encoding or decoding process. [Background technology]

[0002] Various video encoding techniques may be used to compress video data. Video encoding is performed according to one or more video encoding standards. For example, currently, some known video encoding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), jointly developed by ISO / IEC MPEG and ITU-T VECG. AOMediaVideo1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its predecessor standard VP9. Audio Video Coding (AVS), referring to digital audio and digital video compression standards, is another video compression standard series developed by the Chinese Audio and Video Coding Standard Workgroup. Most of the existing video coding standards are built on the well-known hybrid video coding framework, i.e., using block-based prediction methods (e.g., inter-prediction, intra-prediction) to reduce the redundancy present in a video image or sequence, and using transform coding to reduce the energy of the prediction error. An important goal of video coding techniques is to compress video data into a format that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0003] The first generation AVS standard includes the Chinese national standard "Information Technology, Advanced Audio Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+). It can provide about 50% bit rate saving at the same perceived quality compared with the MPEG-2 standard. The AVS1 standard video part was promulgated as the Chinese national standard in February 2006. The second generation AVS standard includes the Chinese national standard series "Information Technology, Efficient Multimedia Coding" (known as AVS2), which is mainly targeted in the transmission of extra HD TV programs. The coding efficiency of AVS2 is twice that of AVS+. In May 2016, AVS2 was issued as the Chinese national standard. Meanwhile, the AVS2 standard video part was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as one international standard for applications. The AVS3 standard is a new generation video encoding standard for UHD video applications that aims to exceed the coding efficiency of the latest international standard HEVC. In March 2019, the 68th AVS meeting finalized the AVS3-P2 baseline, which provides approximately 30% bitrate savings over the HEVC standard. Currently, there is one reference software, called the High Performance Model (HPM), which is maintained by the AVS group to certify the reference implementation of the AVS3 standard. Summary of the Invention [Problem to be solved by the invention]

[0004] This disclosure provides examples of techniques related to improving motion vector candidate derivation for a motion prediction mode in a video encoding or decoding process. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, a method of video decoding is provided. The method includes: a decoder may obtain a third candidate position for a third affine candidate based on a first candidate position for the first affine candidate and a second candidate position for the second affine candidate, the first affine candidate and the second affine candidate being from a plurality of non-adjacent blocks not adjacent to a current block, may obtain a virtual block based on the first candidate position, the second candidate position, and the third candidate position, may obtain a plurality of control point motion vectors (CPMVs) for the virtual block based on translational motion vectors (MVs) at the first candidate position, the second candidate position, and the third candidate position, may obtain a translational MV based on a specific position in the current block in response to determining that the current block is coded as a normal inter mode, or may project a plurality of CPMVs for the virtual block to a current block to obtain a plurality of second CPMVs for the current block in response to determining that the current block is coded as an affine mode. Further, the decoder may insert a translational MV into a normal merge list in response to determining that the current block is to be coded as a normal inter mode, or may insert multiple second CPMVs into an affine merge candidate list or an affine advanced motion vector prediction (AMVP) candidate list in response to determining that the current block is to be coded as an affine mode.

[0006] According to a second aspect of the present disclosure, a method of video decoding is provided. The method includes: a decoder may obtain one or more first parameters based on a first neighboring block from a plurality of inter-coded neighboring blocks of a current block, may obtain one or more second parameters based on a second neighboring block from a plurality of affine-coded neighboring blocks of the current block or from a plurality of historical affine-coded neighboring blocks, may use the one or more first parameters and the one or more second parameters to construct an affine model, and may project the affine model to a current block to obtain a translational MV based on a specific position in the current block in response to determining that the current block is coded as a normal inter mode, or obtain a plurality of CPMVs for the current block in response to determining that the current block is coded as an affine mode. Furthermore, the decoder may insert the translational MV into a normal merge list in response to determining that the current block is coded as a normal inter mode, or may insert a plurality of CPMVs into an affine merge candidate list or an affine AMVP candidate list in response to determining that the current block is coded as an affine mode.

[0007] According to a third aspect of the present disclosure, a method of video decoding is provided. The method includes: a decoder may obtain one or more motion vector (MV) candidates from one or more candidate lists according to a predetermined order, the one or more candidate lists including an affine AMVP candidate list, a normal merge candidate list, and an affine merge candidate list, the one or more MV candidates are from a plurality of neighboring blocks to a current block, and may obtain one or more CPMVs for the current block based on the one or more MV candidates.

[0008] According to a fourth aspect of the present disclosure, a method of video encoding is provided. The method includes: an encoder may determine a third candidate position for a third affine candidate based on a first candidate position for the first affine candidate and a second candidate position for the second affine candidate, the first affine candidate and the second affine candidate being from a plurality of non-adjacent blocks that are not adjacent to a current block, may determine a virtual block based on the first candidate position, the second candidate position, and the third candidate position, may determine a plurality of control point motion vectors (CPMVs) for the virtual block based on translational motion vectors (MVs) at the first candidate position, the second candidate position, and the third candidate position, may determine a translational MV based on a specific position in the current block in response to determining that the current block is coded as a normal inter mode, or may project a plurality of CPMVs for the virtual block onto a current block to determine a plurality of second CPMVs for the current block in response to determining that the current block is coded as an affine mode. Further, in response to determining that a translational MV is obtained, the encoder may insert the translational MV into a normal merge list, and in response to determining that multiple second CPMVs are obtained, the encoder may insert multiple second CPMVs into an affine merge candidate list or an affine AMVP candidate list.

[0009] According to a fifth aspect of the present disclosure, a method of video encoding is provided. The method includes: an encoder may determine one or more first parameters based on a first neighboring block from a plurality of inter-coded neighboring blocks of a current block, may determine one or more second parameters based on a second neighboring block from a plurality of affine-coded neighboring blocks of the current block or from a plurality of historical affine-coded neighboring blocks, may use the one or more first parameters and the one or more second parameters to construct an affine model, and in response to determining that the current block is coded as a normal inter mode, may determine a translational MV based on a specific position within the current block, or in response to determining that the current block is coded as an affine mode, may project the affine model to the current block to obtain a plurality of CPMVs for the current block. Further, the encoder may insert a translational MV into a normal merge list in response to determining that the current block is to be coded as a normal inter mode, and may insert multiple CPMVs into an affine merge candidate list or an affine advanced motion vector prediction (AMVP) candidate list in response to determining that the current block is to be coded as an affine mode.

[0010] According to a sixth aspect of the present disclosure, a method of video encoding is provided. The method includes: an encoder may determine one or more MV candidates from one or more candidate lists according to a predetermined order, the one or more candidate lists including an affine AMVP candidate list, a normal merge candidate list, and an affine merge candidate list, the one or more MV candidates are from a plurality of neighboring blocks to a current block, and may determine one or more CPMVs for the current block based on the one or more MV candidates.

[0011] According to a seventh aspect of the present disclosure, an apparatus for video decoding is provided. The apparatus includes one or more processors and a memory configured to store instructions executable by the one or more processors. Further, the one or more processors are configured to perform a method according to the first aspect, the second aspect, or the third aspect when the instructions are executed.

[0012] According to an eighth aspect of the present disclosure, there is provided an apparatus for video encoding. The apparatus includes one or more processors and a memory configured to store instructions executable by the one or more processors. Further, the one or more processors are configured to perform a method according to the fourth aspect, the fifth aspect, or the sixth aspect when the instructions are executed.

[0013] According to a ninth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium for storing computer-executable instructions which, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform a method according to the first aspect, the second aspect or the third aspect.

[0014] According to a twelfth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium for storing computer-executable instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform a method according to the fourth aspect, the fifth aspect or the sixth aspect to encode a current block into a bitstream and transmit the bitstream.

[0015] A more particular description of the embodiments of the present disclosure will be made by reference to specific embodiments illustrated in the accompanying drawings, in which the embodiments will be described and explained with additional specificity and detail, given that the drawings are only representative of some embodiments and therefore should not be considered limiting in scope. [Brief description of the drawings]

[0016] [Figure 1A] 1 is a block diagram illustrating a system for encoding and decoding video blocks in accordance with some embodiments of the present disclosure. [Figure 1B] FIG. 2 is a block diagram of an encoder according to some embodiments of the present disclosure. [Figure 1C] 2 is a block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, in accordance with some embodiments of the present disclosure. [Figure 1D] 2 is a block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, in accordance with some embodiments of the present disclosure. [Figure 1E] 2 is a block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, in accordance with some embodiments of the present disclosure. [Figure 1F] 2 is a block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes, in accordance with some embodiments of the present disclosure. [Diagram 2] FIG. 2 is a block diagram of a decoder in accordance with some embodiments of the present disclosure. [Figure 3A] 1 is a diagram illustrating block partitions within a multi-type tree structure in accordance with some embodiments of the present disclosure. [Figure 3B] 1 is a diagram illustrating block partitions within a multi-type tree structure in accordance with some embodiments of the present disclosure. [Figure 3C] 1 is a diagram illustrating block partitions within a multi-type tree structure in accordance with some embodiments of the present disclosure. [Figure 3D] 1 is a diagram illustrating block partitions within a multi-type tree structure in accordance with some embodiments of the present disclosure. [Figure 3E] 1 is a diagram illustrating block partitions within a multi-type tree structure in accordance with some embodiments of the present disclosure. [Figure 4A] FIG. 2 illustrates an example four-parameter affine model, in accordance with some embodiments of the present disclosure. [Figure 4B] FIG. 2 illustrates an example four-parameter affine model, in accordance with some embodiments of the present disclosure. [Figure 4C] 1 is a diagram illustrating the locations of spatial merge candidates, according to some embodiments of the present disclosure. [Figure 4D] 1 is a diagram illustrating candidate pairs considered for redundancy check of spatial merge candidates, according to some embodiments of the present disclosure. [Figure 4E] 4 is a diagram illustrating motion vector scaling for temporal merging candidates in accordance with some embodiments of the present disclosure. [Figure 4F] 1 is a diagram illustrating candidate positions for temporal merge candidates C0 and C1 according to some embodiments of the present disclosure. [Diagram 5] FIG. 2 illustrates an example six-parameter affine model, in accordance with some embodiments of the present disclosure. [Figure 6] 1 is a diagram illustrating neighboring blocks for inherited affine merging candidates, in accordance with some embodiments of the present disclosure. [Figure 7] 1 is a diagram illustrating neighboring blocks for constructing affine merge candidates, in accordance with some embodiments of the present disclosure. [Figure 8] 4 is a diagram illustrating non-adjacent blocks for inherited affine merging candidates, in accordance with some embodiments of the present disclosure. [Figure 9] 4A-4C are diagrams illustrating the derivation of constructed affine merge candidates using neighboring blocks, in accordance with some embodiments of the present disclosure. [Figure 10] FIG. 2 illustrates vertical scanning of non-adjacent blocks in accordance with some embodiments of the present disclosure. [Figure 11] 4A-4C are diagrams illustrating horizontal scanning of non-adjacent blocks in accordance with some embodiments of the present disclosure. [Figure 12]FIG. 2 illustrates combined vertical and horizontal scanning of non-adjacent blocks in accordance with some embodiments of the present disclosure. [Figure 13A] 1 is a diagram illustrating a neighboring block having the same size as a current block, in accordance with some embodiments of the present disclosure. [Figure 13B] 1 is a diagram illustrating a neighboring block having a different size than a current block, in accordance with some embodiments of the present disclosure. [Figure 14A] A figure illustrating an example in which the bottom-left or top-right block of the bottom-most or right-most block in the previous distance is used as the bottom-most or right-most block of the current distance, according to some embodiments of the present disclosure. [Figure 14B] A figure illustrating an example in which the left or top block of the bottom-most or right-most block in the previous distance is used as the bottom-most or right-most block of the current distance, according to some embodiments of the present disclosure. [Figure 15A] 11A-11C are diagrams illustrating scanning positions at the bottom-left and top-right positions used for non-adjacent blocks above and to the left, according to some embodiments of the present disclosure. [Figure 15B] FIG. 13 illustrates a scanning position at the bottom right position used for both the top and left non-adjacent blocks, according to some embodiments of the present disclosure. [Figure 15C] FIG. 13 illustrates a scanning position at the bottom left position used for both the top and left non-adjacent blocks, according to some embodiments of the present disclosure. [Figure 15D] FIG. 13 illustrates a scanning position at the right-top position used for both the top and left non-adjacent blocks according to some embodiments of the present disclosure. [Figure 16] FIG. 10 illustrates a simplified scanning process for deriving construction merge candidates, according to some embodiments of the present disclosure. [Figure 17A] FIG. 10 illustrates an example of spatial neighborhood for deriving inherited affine merge candidates, according to some embodiments of the present disclosure. [Figure 17B]FIG. 10 illustrates an example of spatial neighborhood deriving constructed affine merge candidates, according to some embodiments of the present disclosure. [Figure 18] 1 illustrates an example of an inheritance-based derivation method for deriving candidate affine constructions, according to some embodiments of the present disclosure. [Figure 19] 1 is a diagram illustrating templates and reference samples of templates in Reference List 0 and Reference List 1 according to some embodiments of the present disclosure. [Figure 20] 10A-10C are diagrams illustrating templates and reference samples of the templates for blocks with sub-block motion using motion information of the sub-blocks of a current block, according to some embodiments of the present disclosure. [Figure 21] FIG. 1 illustrates an exemplary computing environment with a connected user interface in accordance with some embodiments of the present disclosure. [Figure 22] 1 is a flowchart illustrating a method for video decoding according to some embodiments of the present disclosure. [Diagram 23] 23 is a flowchart illustrating a method for video encoding corresponding to the method for video decoding as shown in FIG. 22 according to some embodiments of the present disclosure. [Figure 24] 1 is a flowchart illustrating a method for video decoding according to some embodiments of the present disclosure. [Diagram 25] 25 is a flowchart illustrating a method for video encoding corresponding to the method for video decoding as shown in FIG. 24 according to some embodiments of the present disclosure. [Figure 26] 1 is a flowchart illustrating a method for video decoding according to some embodiments of the present disclosure. [Figure 27] 27 is a flowchart illustrating a method for video encoding corresponding to the method for video decoding as shown in FIG. 26 according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Reference will now be made in detail to certain implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one skilled in the art that various alternatives may be used. For example, it will be apparent to one skilled in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0018] The terms used in the disclosure are adopted only for the purpose of describing certain embodiments and are not intended to limit the disclosure. The singular forms "a / an", "said" and "the" in the disclosure and the appended claims are intended to include the plural forms unless otherwise clearly indicated throughout the disclosure. Also, the term "and / or" used in the disclosure will be understood to refer to and include one or any or all possible combinations of the associated items listed.

[0019] References throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that a particular feature, structure, or characteristic described is included in at least one embodiment or example. A feature, structure, element, or characteristic described in connection with one or some embodiments is applicable to other embodiments, unless expressly specified otherwise.

[0020] Throughout the disclosure, the terms "first," "second," "third," etc. are all used solely as nomenclature to refer to related elements, e.g., devices, components, compositions, steps, etc., without implying any spatial or chronological order, unless clearly specified otherwise. For example, a "first device" and a "second device" may refer to two separately formed devices, or two parts, components, or operating states of the same device, and may be arbitrarily named.

[0021] The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "sub-unit" may include memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. The components may be physically attached or located adjacent to each other, or may not be physically attached or located.

[0022] As used herein, the terms "if" or "when" may be understood to mean "upon" or "in response to," depending on the context. When these terms appear in a claim, they may not indicate that the associated limitation or feature is conditional or optional. For example, a method may include steps where: i) function or action X' is performed when or if condition X exists; and ii) function or action Y' is performed when or if condition Y exists. A method may be implemented with both the ability to perform function or action X' and the ability to perform function or action Y'. Thus, functions X' and Y' may both be performed at different times for multiple executions of the method.

[0023] A unit or module may be implemented purely by software, purely by hardware, or by a combination of hardware and software. In a purely software implementation, for example, a unit or module may include functionally related code blocks or software components that are directly or indirectly linked together to perform a particular function.

[0024] 1A is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel according to some implementations of the present disclosure. As shown in FIG. 1A, the system 10 includes a source device 12 that generates and encodes video data to be subsequently decoded by a destination device 14. The source device 12 and the destination device 14 may include any of a wide variety of electronic devices, including a desktop or laptop computer, a tablet computer, a smartphone, a set-top box, a digital television, a camera, a display device, a digital media player, a video gaming console, or a video streaming device. In some implementations, the source device 12 and the destination device 14 are equipped with wireless communication capabilities.

[0025] In some implementations, the destination device 14 may receive the encoded video data to be decoded via a link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one embodiment, the link 16 may include a communication medium to enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other equipment that may be useful to facilitate communication from the source device 12 to the destination device 14.

[0026] In some other implementations, the encoded video data may be transmitted from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by destination device 14 via input interface 28. Storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further embodiments, storage device 32 may correspond to a file server or another intermediate storage device that may hold the encoded video data generated by source device 12. Destination device 14 may access the stored video data from storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing encoded video data and transmitting the encoded video data to destination device 14. Exemplary file servers include web servers (e.g., for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. Destination device 14 may access the encoded video data through any standard data connection, including a wireless channel (e.g., a Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., a Digital Subscriber Line (DSL), cable modem, etc.), or a combination of both that is suitable for accessing the encoded video data stored in the file server. The transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0027] As shown in FIG. 1A, source device 12 includes video source 18, video encoder 20, and output interface 22. Video source 18 may include sources such as video supplemental devices, e.g., video cameras, video archives containing previously captured video, video feeding interfaces for receiving video from a video content provider, and / or computer graphics systems for generating computer graphics data as source video, or combinations of such sources. As one example, if video source 18 is a video camera of a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, implementations described in this application may be generally applicable to video encoding and may be applied to wireless and / or wired applications.

[0028] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data may also (or instead) be stored in storage device 32 for later access, decoding, and / or playback by destination device 14 or other devices. Output interface 22 may further include a modem and / or a transmitter.

[0029] Destination device 14 includes an input interface 28, a video decoder 30, and a display device. Input interface 28 may include a receiver and / or modem and may receive encoded video data over link 16. The encoded video data communicated to link 16 and provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included in the encoded video data transmitted over a communications medium, stored on a storage medium, or stored on a file server.

[0030] In some implementations, destination device 14 may include a display device 34, which can be an integrated display device as well as an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0031] Video encoder 20 and video decoder 30 may operate according to proprietary or industry standards, such as VVC, HEVC, MPEG-4, Part 10, AVC, or extensions of such standards. It should be understood that the present application is not limited to a particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally believed that video encoder 20 of source device 12 may be configured to encode video data according to those current or later standards. Similarly, it is also generally believed that video decoder 30 of destination device 14 may be configured to decode video data according to those current or later standards.

[0032] Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented partially in software, an electronic device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of an encoder / decoder (codec) combined with the respective device.

[0033] Like HEVC, VVC is built on a block-based hybrid video coding framework. FIG. 1B is a block diagram illustrating a block-based video encoder according to some implementations of the present disclosure. In the encoder 100, an input video signal is processed block by block, called a coding unit (CU). The encoder 100 may be a video encoder 20 as shown in FIG. 1A. In VTM-1.0, a CU can be up to 128×128 pixels. However, unlike HEVC, which partitions blocks only based on a quadtree, in VVC, one coding tree unit (CTU) is partitioned into CUs based on a quadtree / binary tree / ternary tree to fit variable local characteristics. In addition, the concept of multiple partition unit types in HEVC is removed, i.e., the separation of CU, prediction unit (PU), and transform unit (TU) no longer exists in VVC, and instead, each CU is always used as a basic unit for both prediction and transformation without further partitioning. In a multi-type tree structure, a CTU is first partitioned by a quadtree structure, and then each quadtree leaf node can be further partitioned by binary and ternary tree structures.

[0034] 3A-3E are schematic diagrams illustrating a multi-type tree division mode according to some implementations of the present disclosure. Each of FIGs. 3A-3E shows five division types, including a quadrant (FIG. 3A), a vertical bisection (FIG. 3B), a horizontal bisection (FIG. 3C), a vertical extended trisection (FIG. 3D), and a horizontal extended trisection (FIG. 3E).

[0035] For each given video block, spatial prediction and / or temporal prediction may be performed. Spatial prediction (or "intra prediction") uses pixels from samples of already coded neighboring blocks (called reference samples) in the same video picture / slice to predict the current video block. Spatial prediction reduces spatial redundancy inherent in the video signal. Temporal prediction (also called "inter prediction" or "motion compensated prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal references. Also, if multiple reference pictures are supported, one reference picture index is transmitted in addition, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from.

[0036] After spatial and / or temporal prediction, an intra / inter mode decision circuit 121 in the encoder 100 chooses the best prediction mode, for example, based on a rate-distortion optimization method. The block predictor 120 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using the transform circuit 102 and the quantization circuit 104. The resulting quantized residual coefficients are inverse quantized by the inverse quantization circuit 116 and inverse transformed by the inverse transform circuit 118 to form a reconstructed residual, which is then added back to the predictive block to form a reconstructed signal for the CU. Additionally, in-loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO), and / or an adaptive in-loop filter (ALF), may be applied to the reconstructed CU before it is placed in a reference picture store of a picture buffer 117 and used to code a subsequent video block. To form the output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit 106 to be further compressed and packed to form the bitstream.

[0037] For example, blocking filters are available in the current version of VVC, along with AVC and HEVC. In HEVC, an additional in-loop filter called SAO is defined to further improve coding efficiency. In the current version of the VVC standard, yet another in-loop filter called ALF is being actively investigated and has a good chance of being included in the final standard.

[0038] These in-loop filter operations are optional. Performing these operations helps to improve coding efficiency and visual quality. They may also be turned off as a decision made by the encoder 100 to save computational complexity.

[0039] It should be noted that intra prediction is normally based on unfiltered reconstructed pixels, and inter prediction is based on filtered reconstructed pixels if those filter options are turned on by the encoder 100.

[0040] FIG. 2 is a block diagram illustrating a block-based video decoder 200 that can be used in conjunction with many video coding standards. This decoder 200 is similar to the reconstruction-related section present in the encoder 100 of FIG. 1B. The block-based video decoder 200 can be the video decoder 30 as shown in FIG. 1A. In the decoder 200, an incoming video bitstream 201 is first decoded through entropy decoding 202 to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed through inverse quantization 204 and inverse transform 206 to obtain a reconstructed prediction residual. The block predictor mechanism, implemented in the intra / inter mode selection 212, is configured to perform either intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residual from the inverse transform 206 and the prediction output generated by the block predictor mechanism using a summer 214.

[0041] The reconstructed blocks may further pass through an in-loop filter 209 before being stored in a picture buffer 213, which serves as a reference picture store. The reconstructed video in the picture buffer 213 may be transmitted to drive a display device and may be used to predict later video blocks. In situations where the in-loop filter 209 is turned on, a filtering operation is performed on those reconstructed pixels to derive the final reconstructed video output 222.

[0042] In the current VVC and AVS3 standards, the motion information of the currently coded block is either copied from spatial or temporal neighboring blocks specified by the merge candidate index or obtained by explicit signaling of motion estimation. The focus of this disclosure is to improve the accuracy of motion vectors for affine merge mode by improving the derivation method of affine merge candidates. To facilitate the description of this disclosure, the existing affine merge mode design in the VVC standard is used as an example to promote the proposed idea. Although the existing affine mode design in the VVC standard is used as an example throughout this disclosure, it will be noted by those skilled in the art of modern video coding technology that the proposed technology can also be applied to different designs of affine motion prediction mode or other coding tools with the same or similar design spirit.

[0043] In a typical video coding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other instances, a frame may be monochrome and therefore include only one two-dimensional array of luma samples.

[0044] As shown in FIG. 1C, video encoder 20 (or, more specifically, a partitioning unit in a prediction processing unit of video encoder 20) generates a coded representation of a frame by first partitioning the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively in a raster scan order from left to right and top to bottom. Each CTU is a maximum logical coding unit, and the width and height of the CTU are signaled by video encoder 20 in a sequence parameter set, such that all CTUs in a video sequence have the same size, which is one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a particular size. As shown in FIG. 1D, each CTU may include one CTB of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe the nature of different types of units of coding blocks of pixels and how the video sequence may be reconstructed at video decoder 30, including inter or intra prediction, intra prediction modes, motion vectors, and other parameters. For monochrome pictures or pictures with three separate color planes, the CTU may contain syntax elements used to code a single coding tree block and samples of the coding tree block. A coding tree block may be an N×N block of samples.

[0045] To achieve better performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination thereof, on the coding tree blocks of the CTU to partition the CTU into smaller CUs. As depicted in FIG. 1E, 64×64 CTU 400 is first partitioned into four smaller CUs, each of which has a block size of 32×32. Among the four smaller CUs, CU 410 and CU 420 are each partitioned by block size into four 16×16 CUs. Two 16×16 CUs 430 and 440 are each further partitioned by block size into four 8×8 CUs. FIG. 1F depicts a quad tree data structure illustrating the final result of the partitioning process of CTU 400 as depicted in FIG. 1E, where each leaf node of the quad tree corresponds to one CU of a respective size ranging from 32×32 to 8×8. Similar to the CTU depicted in FIG. 1D, each CU may include a CB of luma samples, and two corresponding coding blocks of chroma samples for an identically sized frame, and syntax elements used to code the samples of the coding blocks. In monochrome pictures or pictures with &3 separate color planes, a CU may include a single coding block and syntax structures used to code the samples of the coding block. It should be noted that the quadtree partition depicted in FIG. 1E-1F is for illustrative purposes only, and one CTU may be divided into CUs based on quadtree / ternary tree / binary tree partitioning to fit variable local characteristics. In a multi-type tree structure, one CTU is partitioned by a quadtree structure, and each quadtree leaf CU may be further partitioned into binary tree and ternary tree structures. As shown in FIG. 3A-3E, there are five possible partition types of coding blocks with width W and height H, namely, quadtree partition, horizontal binary tree partition, vertical binary tree partition, horizontal ternary tree partition, and vertical ternary tree partition.

[0046] In some implementations, video encoder 20 may further partition the coding blocks of a CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same prediction, inter or intra, is applied. A PU of a CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements used to predict the PBs. In a monochrome picture or a picture with three separate color planes, a PU may include a single PB and syntax elements used to predict the PBs. Video encoder 20 may generate predictive luma, Cb and Cr blocks for the luma, Cb and Cr PBs for each PU of the CU.

[0047] Video encoder 20 may use intra prediction or inter prediction to generate predictive blocks for a PU. If video encoder 20 uses intra prediction to generate predictive blocks for a PU, video encoder 20 may generate predictive blocks for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate predictive blocks for a PU, video encoder 20 may generate predictive blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0048] After video encoder 20 generates predictive luma, Cb, and Cr blocks for one or more PUs of a CU, video encoder 20 may generate a luma residual block for the CU by subtracting the predictive luma block of the CU from its original luma coding block, such that each sample in the luma residual block of the CU indicates a difference between a luma sample in one of the predictive luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, such that each sample in the Cb residual block of the CU indicates a difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0049] Further, as illustrated in FIG. 1E, video encoder 20 may use quadtree partitioning to decompose the luma, Cb, and Cr residual blocks of a CU into one or more luma, Cb, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) block of samples to which an identical transform is applied. A TU of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some embodiments, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and syntax structures used to transform samples of the transform block.

[0050] Video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar quantities. Video encoder 20 may apply one or more transforms to a Cb transform block of the TU to generate a Cb coefficient block for the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0051] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 may quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients and provide further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream including a sequence of bits forming a representation of the coding frame and associated data, which is either stored in storage device 32 or transmitted to destination device 14.

[0052] After receiving the bitstream generated by video encoder 20, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally reciprocal to the encoding process performed by video encoder 20. For example, video decoder 30 may perform an inverse transform on coefficient blocks associated with the TUs of the current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs the coding blocks of the current CU by adding samples of predictive blocks for the PUs of the current CU to corresponding samples of transform blocks of the TUs of the current CU. After reconstructing the coding blocks for each CU of the frame, video decoder 30 may reconstruct the frame.

[0053] As mentioned above, video coding achieves video compression mainly using two modes: intra-frame prediction (or intra prediction) and inter-frame prediction (or inter prediction). It should be noted that IBC is considered as either intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because of the use of motion vectors to predict a current video block from a reference video block.

[0054] However, ever improving video data capture techniques and more refined video block sizes for preserving content within the video data also substantially increases the amount of data required to represent the motion vectors for the current frame. One way to overcome this challenge is to take advantage of the fact that a group of neighboring CUs in both spatial and temporal domains not only have similar video data for prediction purposes, but also the motion vectors between those neighboring CUs are similar. Thus, it is possible to use the motion information of spatially neighboring CUs and / or temporally co-located CUs as an approximation of the motion information (e.g., motion vector) of the current CU by utilizing their spatial and temporal correlations, also referred to as the "motion vector predictor (MVP)" of the current CU.

[0055] Instead of encoding the actual motion vector of the current CU determined by the motion estimation unit as described above in connection with FIG. 1B into the video bitstream, the motion vector predictor of the current CU is subtracted from the actual motion vector of the current CU to produce a motion vector difference (MVD) for the current CU. By doing so, the motion vectors determined by the motion estimation unit for each CU of a frame do not need to be encoded into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0056] Similar to the process of choosing a predictive block in a reference frame during inter-frame prediction of a code block, a set of rules is required to build a motion vector candidate list (also known as a "merge list") for a current CU using those potential candidate motion vectors associated with spatially neighboring CUs and / or temporally co-located CUs of the current CU, and then select one member from the motion vector candidate list as the motion vector predictor for the current CU. By doing so, there is no need to transmit the motion vector candidate list itself from video encoder 20 to video decoder 30, and the index of the selected motion vector predictor in the motion vector candidate list is sufficient for video encoder 20 and video decoder 30 to use the same motion vector predictor in the motion vector candidate list to encode and decode the current CU.

[0057] Affine Model In HEVC, only the translational motion model is applied for motion compensated prediction. In the real world, there exist many kinds of motions, e.g., zoom in / out, rotation, depth motion, and other irregular motions. In VVC and AVS3, affine motion compensated prediction is applied by signaling one flag for each inter-coded block to indicate whether the translational motion model or the affine motion model is applied for inter prediction. In the current VVC and AVS3 designs, two affine modes, including a 4-parameter affine mode and a 6-parameter affine mode, are supported for one affine-coded block.

[0058] The 4-parameter affine model has the following parameters: two parameters for translational motion in the horizontal and vertical directions respectively, one parameter for zoom motion, and one parameter for rotational motion for both directions. In this model, the horizontal zoom parameter is equal to the vertical zoom parameter, and the horizontal rotation parameter is equal to the vertical rotation parameter. To achieve a better match of the motion vectors and affine parameters, the affine parameters will be derived from two MVs (also called control point motion vectors (CPMVs)) located at the top left and top right corners of the current block. As shown in Figures 4A-4B, the affine motion field of a block is calculated by the two CPMVs (V 0 ,V 1 Based on the control point motion, the motion field of one affine coding block (v x ,v y )teeth,

number

[0059] The 6-parameter affine mode has the following parameters: two parameters for translation in each of the horizontal and vertical directions, two parameters for zoom and rotation in each of the horizontal directions, and another two parameters for zoom and rotation in each of the vertical directions. The 6-parameter affine motion model is coded by three CPMVs. As shown in Figure 5, the three control points of one 6-parameter affine block are located at the top-left, top-right, and bottom-left corners of the block. The motion at the top-left control point is related to the translation motion, the motion at the top-right control point is related to the rotation and zoom motion in the horizontal direction, and the motion at the bottom-left control point is related to the rotation and zoom motion in the vertical direction. Compared to the 4-parameter affine motion model, the rotation and zoom motion in the horizontal direction of the 6-parameters cannot be identical to those in the vertical direction. (V 0 ,V 1,V 2 ) are the MVs of the top-left, top-right, and bottom-left corners of the current block in FIG. 5. Then, the motion vectors (v x ,v y )teeth,

number

[0060] Affine Merge Mode In affine merge mode, the CPMV for the current block is not explicitly signaled, but is derived from neighboring blocks. In particular, in this mode, the motion information of spatial neighboring blocks is used to generate the CPMV for the current block. The affine merge mode candidate list has a limited size. For example, in the current VVC design, there may be a maximum of five candidates. The encoder may evaluate and choose the best candidate index based on a rate-distortion optimization algorithm. The chosen candidate index is then signaled to the decoder side. The affine merge candidate may be determined in three ways. In the first way, the affine merge candidate may be inherited from the neighboring affine coded block. In the second way, the affine merge candidate may be constructed from the translation MV from the neighboring block. In the third way, the zero MV is used as the affine merge candidate.

[0061] There may be at most two candidates for the inheritance method, which are taken from the neighboring block located at the bottom left of the current block (e.g., the scanning order is from A0 to A1 as shown in FIG. 6) and the neighboring block located at the top right of the current block (e.g., the scanning order is from B0 to B2 as shown in FIG. 6) if available.

[0062] For the construction method, the candidates are combinations of adjacent translational MVs, which can be generated by two steps.

[0063] Step 1: Obtain four translational MVs, including MV1, MV2, MV3, and MV4, from the available neighbors. MV1: MV from one of the three adjacent blocks close to the top left corner of the current block. As shown in Figure 7, the scanning order is B2, B3, and A2. MV2: MV from one of the two adjacent blocks close to the top right corner of the current block. As shown in FIG. 7, the scanning order is B1 and B0. MV3: MV from one of the two adjacent blocks close to the bottom left corner of the current block. As shown in Figure 7, the scanning order is A1 and A0. MV4: MV from the temporally co-located neighbor block near the bottom right corner of the current block. As shown in the figure, the neighbor block is T.

[0064] Step 2: Derive combinations based on the four translational MVs from step 1. Combination 1: MV1, MV2, MV3, Combination 2: MV1, MV2, MV4, Combination 3: MV1, MV3, MV4, Combination 4: MV2, MV3, MV4, Combination 5: MV1, MV2, Combination 6: MV1, MV3.

[0065] When the merge candidate list is not full after filling it with inheritance and construction candidates, zero MV is inserted at the end of the list.

[0066] Affine AMVP mode Affine Advanced Motion Vector Prediction (AMVP) mode can be applied for CUs with both width and height of 16 or more. An affine flag at the CU level is signaled in the bitstream to indicate whether affine AMVP mode is used, and then another flag is signaled to indicate whether it is 4-parameter affine or 6-parameter affine. In this mode, the CPMV of the current CU and the difference of their CPMV predictor (CPMVP) are signaled in the bitstream. The affine AVMP candidate list size is 2, and the affine AMVP candidate list is signaled by using the following four types of CPMV candidates in the following order: - inherited affine AMVP candidates extrapolated from the CPMVs of neighboring CUs, - Construct affine AMVP candidate CPMVP derived using translational MVs of neighboring CUs, - Translational MVs from adjacent CUs, - Temporal MV from co-located CUs, - Zero MV.

[0067] The checking order of inherited affine AMVP candidates is the same as that of inherited affine merge candidates. The only difference is that for AMVP candidates, only affine CUs with the same reference picture as the current block are considered. No pruning process is applied when inserting inherited affine motion predictors into the candidate list.

[0068] The constructed AMVP candidates are derived from the same spatial neighborhood as in the affine merge mode. The same checking order is used as is done for affine merge candidate construction. In addition, the reference picture indexes of the neighboring blocks are also checked. The first block in the checking order that is inter-coded and has the same reference picture as in the current CU is used. If the current CU is coded in 4-parameter affine mode and mv 0 and mv 1 If both are available, mv0 and mv 1 are added to the affine AMVP candidate list as one candidate. If the current CU is coded in 6-parameter affine mode and all three CPMVs are available, they are added to the affine AMVP candidate list as one candidate. Otherwise, the constructed AMVP candidate is set as not available.

[0069] If the number of candidates in the affine AMVP list is still less than 2 after the valid inherited affine AMVP candidates and construction AMVP candidates are inserted, mv 0 , mv 1 , and mv 2 are added in order. Finally, if it is still not full, zero MVs are used to fill the affine AMVP list.

[0070] Regular Inter-Merge Mode In some embodiments, a canonical inter-merge candidate list is constructed by including the following five types of candidates, in order: (1) Spatial MVP from spatially adjacent CUs, (2) Temporal MVP from co-located CUs; (3) History-based MVP from a first-in-first-out (FIFO) table, (4) Pairwise average MVP, and (5) Zero MV.

[0071] The size of the merge list is signaled in the Sequence Parameter Set header, and the maximum allowed size of the merge list is 6. For each CU code in merge mode, the index of the best merge candidate is coded using Truncated Unary Binary (TU). The first bin of the merge index is coded by context, and bypass coding is used for the other bins.

[0072] The derivation process for each category of merging candidates is provided above. In some embodiments, parallel derivation of merging candidate lists may be supported for all CUs within a certain size of an area.

[0073] Spatial candidate derivation The derivation of spatial merge candidates in VVC is identical to that in HEVC, except that the positions of the first two merge candidates are swapped. A maximum of four merge candidates are selected from among the candidates located at the positions represented in FIG. 4C. The order of derivation is B0, A0, B1, A1, and B2. When one or more CUs in positions B0, A0, B1, A1 are not available (e.g., because they belong to another slice or tile) or are intra-coded, only position B2 is considered. After the candidate in position A1 is added, the addition of the remaining candidates is subjected to a redundancy check, which ensures that candidates with identical motion information are eliminated from the list so that coding efficiency is improved. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only the pairs linked with the arrows in FIG. 4D are considered, and only the candidates are added to the list if the corresponding candidates used for the redundancy check do not have identical motion information. FIG. 4D illustrates candidate pairs considered for redundancy check of spatial merging candidates.

[0074] Temporal candidate derivation In this step, only one candidate is added to the list. In particular, in this derivation of the temporal merge candidate, the scaling motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list and reference index to be used for the derivation of the co-located CU are explicitly signaled in the slice header. The scaling motion vector for the temporal merge candidate is obtained as illustrated by the dotted line in FIG. 4E, and is scaled from the motion vector of the co-located CU using POC distances, tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero.

[0075] The position for the temporal candidate is selected between candidates C0 and C1, as depicted in Figure 4F. If the CU in position C0 is not available, is intra-coded, or is outside the current row of the CTU, then position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0076] History-Based Merge Candidate Deriving History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and temporal motion vector prediction (TMVP). In this method, the motion information of a previously coded block is stored in a table and used as the MVP for the current CU. The table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (emptied) when a new CTU row is encountered. Whenever there is a non-subblock inter-coded CU, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0077] The HMVP table size S may be set to 6, which indicates that up to 5 History-Based MVP (HMVP) candidates may be added to the table. When inserting a new motion candidate into the table, a constrained first-in-first-out (FIFO) rule is utilized, and a redundancy check is first applied to find if an identical HMVP exists in the table. If so, the identical HMVP is removed from the table, all subsequent HMVP candidates are moved forward, and the identical HMVP is inserted into the last entry of the table.

[0078] HMVP candidates are used in the merge candidate list construction process. The most recent few HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. Redundancy checks are applied to HMVP candidates for spatial or temporal merge candidates.

[0079] To reduce the number of operations for redundancy checking, the following simplifications are introduced: First, the last two entries in the table are checked for redundancy with respect to each of the A1 and B1 spatial candidates. Second, the process of building a merge candidate list from HMVP is completed when the total number of available merge candidates reaches a maximum of the allowed merge candidates minus one.

[0080] Pairwise average merging candidate derivation A pairwise average candidate is generated by averaging a predefined pair of candidates in an existing merge candidate list using the first two merge candidates. The first merge candidate may be defined as p0Cand and the second merge candidate may be defined as p1Cand, respectively. The average motion vector is calculated separately according to the availability of motion vectors in p0Cand and p1Cand for each reference list. If both motion vectors in one list are available, they are averaged even when those two motion vectors point to different reference pictures and that reference picture is set to the one in p0Cand; if only one motion vector is available, it uses the one directly and keeps this list invalid if no motion vector is available. Also, if the half-pel interpolation filter index of p0Cand and p1Cand is different, it is set to 0.

[0081] When the merge list is not full after the pairwise average merge candidates are added, zero MVPs are inserted at the end of the merge list until the maximum number of merge candidates is encountered.

[0082] Adaptive Reordering of Merge Candidates by Template Matching (ARMC) The reordering method, named as ARMC, is applied to the regular merge mode, the template matching (TM) merge mode, and the affine merge mode (excluding SbTMVP candidates, where SbTMVP stands for sub-block-based temporal motion vector prediction candidates). For the TM merge mode, the merge candidates are reordered before the refinement process.

[0083] After the merge candidate list is constructed, the merge candidates are divided into several subgroups. The subgroup size is set to 5. The merge candidates in each subgroup are reordered in ascending order by cost value based on template matching. For simplicity, the merge candidates in the first subgroup but not in the last subgroup are not reordered.

[0084] The template matching cost is measured by the sum of absolute differences (SAD) between the template's samples of the current block and their corresponding reference samples. The template contains a set of reconstructed samples that are neighbors to the current block. The template's reference samples are identified by the same motion information of the current block.

[0085] When a merge candidate utilizes bi-prediction, the reference samples of the merge candidate's template are also generated by bi-prediction as shown in FIG.

[0086] For a subblock-based merging candidate with subblock size equal to Wsub*Hsub, the top template contains several sub-templates with size of Wsub×1, and the left template contains several sub-templates with size of 1×Hsub, where Wsub is the width of the subblock and Hsub is the height of the subblock. As shown in Figure 20, the motion information of the subblocks in the first row and first column of the current block is used to derive the reference samples of each sub-template.

[0087] For current video standards VVC and AVS, only neighboring blocks are used to derive affine merge candidates for the current block, as shown in Figures 6 and 7 for inheritance and construction candidates, respectively. To increase the diversity of merge candidates and further exploit spatial correlation, it is straightforward to extend the coverage of neighboring blocks from neighboring regions to non-neighboring regions.

[0088] In the current video standards VVC and AVS, each affine inheritance candidate is derived from one neighboring block with affine motion information, while each affine construction candidate is derived from two or three neighboring blocks with translational motion information. To further exploit spatial correlation, new candidate derivation methods that combine affine and translational motion can be investigated.

[0089] The candidate derivation method proposed for the affine merge mode can be extended to other coding modes, such as the affine AMVP mode and the canonical merge mode.

[0090] In this disclosure, the candidate derivation process for the affine merge mode may be extended by using not only adjacent blocks but also non-adjacent blocks. The detailed methods may be summarized in the following aspects, including affine merge candidate pruning, non-adjacent based derivation process for affine inheritance merge candidates, non-adjacent based derivation process for affine construct merge candidates, inheritance based derivation method for affine construct merge candidates, HMVP based derivation method for affine construct merge candidates, and candidate derivation methods for affine AMVP mode and regular merge mode.

[0091] Affine Merge Candidate Pruning As the affine merge candidate list in typical video coding standards usually has a limited size, candidate pruning is a necessary process to remove redundant ones. This pruning process is necessary for both affine merge inheritance candidates and construction candidates. As explained in the introduction, the CPMV of the current block is not directly used for affine motion compensation. Instead, the CPMV needs to be transformed into a translational MV at the position of each sub-block in the current block. The transformation process is performed by adhering to a general affine model, as shown below.

number

[0092] For the 6-parameter affine model, three CPMVs are available, designated V0, V1, and V2. The six model parameters a, b, c, d, e, and f are then

number

[0093] For a 4-parameter affine model, when the top-left corner CPMV and top-right corner CPMV, called V0 and V1, are available, the six parameters a, b, c, d, e, and f are

number

[0094] For a 4-parameter affine model, when the top-left corner CPMV and bottom-left corner CPMV, called V0 and V2, are available, the six parameters a, b, c, d, e, and f are

number

[0095] In the above equations (4), (5), and (6), w and h represent the width and height of the current block, respectively.

[0096] When two merge candidate sets of CPMV are compared for redundancy checking, it is proposed to check the similarity of the six affine model parameters. Therefore, the candidate pruning process can be performed in two steps.

[0097] In step 1, two candidate sets of CPMVs are assumed, and corresponding affine model parameters for each candidate set are derived. More specifically, the two candidate sets of CPMVs are derived from two sets of affine model parameters, e.g., (a 1 ,b1 ,c 1 ,d 1 ,e 1 ,f 1 ) and (a 2 ,b 2 ,c 2 ,d 2 ,e 2 ,f 2 )

[0098] In step 2, a similarity check is performed between the two sets of affine model parameters based on one or more predefined thresholds. In one embodiment, (a 1 -a 2 ), (b 1 -b 2 ), (c 1 -c 2 ), (d 1 -d 2 ), (e 1 -e 2 ), and (f 1 -f 2 ) are below a positive threshold, such as a value of all 1, then the two candidates are considered to be similar and one of them may be pruned / removed and not placed in the merge candidate list.

[0099] In some embodiments, the divide or right shift operation in step 1 may be removed to simplify the computation in the CPMV pruning process.

[0100] In particular, the model parameters of c, d, e, and f can be calculated without dividing by the width w and height h of the current block. For example, taking the above equation (4) as an example, the approximate model parameters of c', d', e', and f' can be calculated as the following equation (7).

number

[0101] In the case where only two CPMVs are available, some of the model parameters are derived from other parts of the model parameters that depend on the width or height of the current block. In this case, the model parameters can be transformed to take into account the influence of the width and height. For example, in the case of equation (5), the approximate model parameters of c', d', e', and f' can be calculated based on the following equation (8). In the case of equation (6), the approximate model parameters of c', d', e', and f' can be calculated based on the following equation (9).

number

number

[0102] In step 2 above, a threshold is needed to evaluate the similarity between two candidate sets of CPMV. There may be multiple ways to define the threshold. In one embodiment, the threshold may be defined for each comparable parameter. Table 1 is an example of this embodiment showing the threshold defined for each comparable model parameter. In another embodiment, the threshold may be defined by considering the size of the currently coded block. Table 2 is an example of this embodiment showing the threshold defined by the size of the currently coded block.

[0103] [Table 1]

[0104] [Table 2]

[0105] In another embodiment, the thresholds may be defined by considering the weight or height of the current block. Tables 3 and 4 are examples of this embodiment. Table 3 shows the thresholds defined by the width of the current coding block, and Table 4 shows the thresholds defined by the height of the current coding block.

[0106] [Table 3]

[0107] [Table 4]

[0108] In another embodiment, the threshold may be defined as a group of fixed values. In another embodiment, the threshold may be defined by any combination of the above embodiments. In one example, the threshold may be defined by considering different parameters as well as the weight and height of the current block. Table 5 is one example of this embodiment showing the threshold defined by the height of the currently coded block. In any of the above proposed embodiments, the comparable parameter may represent any parameter defined in any of the formulas from (4) to (9), if necessary.

[0109] [Table 5]

[0110] The advantage of using the transferred affine model parameters for candidate redundancy check is that it results in a unified similarity check process for candidates with different affine model types, for example, one merge candidate may use a 6-parameter affine model with 3 CPMVs and another candidate may use a 4-parameter affine model with 2 CPMVs, it takes into account the different influence of each CPMV in the merge candidate when deriving the target MV in each sub-block, and it provides the similarity significance of the two affine merge candidates relative to the width and height of the current block.

[0111] Non-adjacent based resolution process for affine inheritance merge candidates For inheritance merge candidates, the non-adjacent based derivation process can be performed in three steps: Step 1 is for candidate scanning, Step 2 is for CPMV projection, and Step 3 is for candidate pruning.

[0112] In step 1, non-adjacent blocks are scanned and selected by the following method.

[0113] Scanning Area and Distance In some embodiments, the non-adjacent blocks may be scanned from the area to the left and the area above the current coding block, and the scanning distance may be defined as the number of coding blocks from the scanning position to the left or top side of the current coding block.

[0114] As shown in Figure 8, multiple lines of non-adjacent blocks may be scanned on either the left or top of the current coding block. The distances shown in Figure 8 represent the number of coding blocks from each candidate position to the left or top side of the current block. For example, an area with "distance 2 (D2)" on the left side of the current block indicates that the candidate adjacent blocks located within this area are two blocks away from the current block. Similar indications may be applied to other scanning regions with different distances.

[0115] In one or more embodiments, the non-adjacent blocks at each distance may have the same block size as the currently coded block, as shown in FIG. 13A. As shown in FIG. 13A, the non-adjacent blocks 1301 on the left and 1302 on the top have the same size as the currently coded block 1303. In some embodiments, the non-adjacent blocks at each distance may have a different block size than the currently coded block, as shown in FIG. 13B. The neighboring block 1304 is a neighboring block to the currently coded block 1303. As shown in FIG. 13B, the non-adjacent blocks 1305 on the left and 1306 on the top have the same size as the currently coded block 1307. The neighboring block 1308 is a neighboring block to the currently coded block 1307.

[0116] It should be noted that when the non-adjacent blocks at each distance have the same block size as the currently coded block, the value of the block size is adaptively changed according to the partition granularity in each different area in the image. When the non-adjacent blocks at each distance have a different block size than the currently coded block, the value of the block size may be predefined as a constant value, such as 4×4, 8×8, or 16×16. The 4×4 non-adjacent motion field shown in Figures 10 and 12 is an example in this case, and the motion field may be considered as a special case of a sub-block, but is not limited thereto.

[0117] Similarly, the non-adjacent coding blocks shown in Figure 11 may have different sizes as well. In one embodiment, the non-adjacent coding blocks may have a size as the current coding block that is adaptively changed. In another embodiment, the non-adjacent coding blocks may have a predefined size with a fixed value, such as 4x4, 8x8, or 16x16.

[0118] Based on the defined scanning distance, the total size of the scanning area on either the left or the top of the current encoding clock may be determined by a configurable distance value. In one or more embodiments, the maximum scanning distance on the left and top side may use the same value or different values. FIG. 13 shows an example where the maximum distance on both the left and top sides share the same value of 2. The maximum scanning distance value(s) may be determined by the encoder side and may be signaled in the bitstream. Alternatively, the maximum scanning distance value(s) may be predefined as a fixed value(s), such as a value of 2 or 4. When the maximum scanning distance is predefined as a value of 4, it indicates that the scanning process is completed when the candidate list is full, or all non-adjacent blocks with a maximum distance of 4 have been scanned, whichever comes first.

[0119] In one or more embodiments, within each scanning region at a particular distance, the start and end neighboring blocks may be position dependent.

[0120] In some embodiments, for the left scanning area, the starting neighboring block may be the adjacent lower left block of the starting neighboring block of the neighboring scanning area with a shorter distance. For example, as shown in FIG. 8, the starting neighboring block of the "distance 2" scanning area on the left side of the current block is the adjacent lower left neighboring block of the starting neighboring block of the "distance 1 (D1)" scanning area. In FIG. 8, D1, D2, D3 indicate distance 1, distance 2, and distance 3, respectively. The ending neighboring block may be the adjacent left block of the ending neighboring block of the upper scanning area with a shorter distance. For example, as shown in FIG. 8, the ending neighboring block of the "distance 2" scanning area on the left side of the current block is the adjacent left neighboring block of the ending neighboring block of the "distance 1" scanning area on the current block.

[0121] Similarly, for the upper scanning area, the starting neighboring block may be the neighboring right-top block of the starting neighboring block of the adjacent scanning area with the shorter distance, and the ending neighboring block may be the neighboring left-top block of the ending neighboring block of the adjacent scanning area with the shorter distance.

[0122] Scanning Order When neighboring blocks are scanned within a non-contiguous region, certain order or / and rules may be followed to determine the selection of the scanned neighboring blocks.

[0123] In some embodiments, the left area may be scanned first, followed by scanning the area above. As shown in Figure 8, three lines of the non-adjacent area on the left side (e.g., from distance 1 (D1) to distance 3 (D3)) may be scanned first, followed by scanning three lines of the non-adjacent area above the current block.

[0124] In some embodiments, the left area and the top area may be scanned alternately. For example, as shown in FIG. 8, the left scanning area with "distance 1" is scanned first, followed by scanning the top area with "distance 1".

[0125] For scanning areas located on the same side (e.g., left or top areas), the scanning order is from the area with the shortest distance to the area with the longest distance. This order can be flexibly combined with other embodiments of the scanning order. For example, the left and top areas can be scanned alternately, and the order for the areas on the same side is scheduled to be from the shortest distance to the longest distance.

[0126] Within each scanning region at a particular distance, a scanning order may be defined. In one embodiment, for the left scanning region, scanning may start from the lower neighboring block to the top neighboring block. For the top scanning region, scanning may start from the right block to the left block.

[0127] Scanning completed For inheritance merge candidates, the neighboring blocks coded by affine mode are defined as qualified candidates. In some embodiments, the scanning process may be performed iteratively. For example, scanning performed in a particular area at a particular distance may be stopped in an instance when the first X qualified candidates are identified, where X is a predefined positive value. For example, as shown in FIG. 8, scanning in the left scanning region with distance 1 may be stopped when the first one or more qualified candidates are identified. Then, the next iteration of the scanning process is started by targeting another scanning region regulated by a predefined scanning order / rule.

[0128] In one or more embodiments, X may be defined for each distance. For example, at each distance, X is set to 1, which means that the scanning is completed for each distance when the first qualified candidate is found and the scanning process is resumed from a different distance in the same area or from the same or different distance in a different area. It should be noted that the value of X may be set as the same value or different values ​​for different distances. The scanning process for an area is fully completed when the maximum number of qualified candidates is found from all allowable distances (e.g., bounded by the maximum distance) of the area.

[0129] In another embodiment, X can be defined for an area. For example, X is set to 3, which means that the scanning is completed for the whole area (e.g., the area to the left or above the current block) when the first three qualified candidates are found and the scanning process is resumed from the same or different distance of another area. It should be noted that the value of X can be set as the same value or different values ​​for different areas. When the maximum number of qualified candidates is found from all areas, the scanning process is completed globally.

[0130] The value of X can be defined for both distance and area. For example, for each area (e.g., the area to the left or above the current block), X is set to 3, and for each distance, X is set to 1. The value of X can be set as the same value or different values ​​for different areas and distances.

[0131] In some embodiments, the scanning process may be performed continuously, for example, scanning performed within a particular area at a particular distance may be stopped in instances when all covered neighboring blocks have been scanned and no more qualified candidates have been identified or a maximum allowed number of candidates has been reached.

[0132] During the candidate scanning process, each candidate non-adjacent block is determined and scanned by adhering to the scanning method proposed above. For easier implementation, each candidate non-adjacent block can be indicated and specified by a specific scanning position. Once a specific scanning area and distance are determined by adhering to the method proposed above, the scanning position can be determined accordingly based on the following method.

[0133] In one method, the bottom-left and top-right positions are used for the top and left non-adjacent blocks, as shown in FIG. 15A.

[0134] Alternatively, the bottom right position is used for both the top and left non-adjacent blocks, as shown in FIG. 15B.

[0135] Alternatively, the bottom left position is used for both the above and left non-adjacent blocks, as shown in FIG. 15C.

[0136] Alternatively, the right-top position is used for both the top and left non-adjacent blocks, as shown in FIG. 15D.

[0137] For easier illustration, in Figures 15A to 15D, each non-adjacent block is assumed to have the same block size as the current block. Without loss of generality, this illustration can be easily extended to non-adjacent blocks with different block sizes.

[0138] Furthermore, step 2 may utilize the same process of CPMV projection as used in the current AVS and VVC standards, where it is assumed that the current block shares the same affine model with the selected neighboring block, and then the coordinates of two or three corner pixels (e.g., if the current block uses a 4-parameter model, two coordinates (top left pixel / sample location and top right pixel / sample location) are used, and if the current block uses a 6-parameter model, three coordinates (top left pixel / sample location, top right pixel / sample location, and bottom left pixel / sample location) are used) are plugged into equation (1) or (2), depending on whether the neighboring block is coded with a 4-parameter or 6-parameter affine model, to generate two or three CPMVs.

[0139] In step 3, any qualified candidate identified in step 1 and converted in step 2 may undergo a similarity check against all existing candidates already in the merge candidate list. The details of the similarity check have already been explained in the "Affine Merge Candidate Pruning" section above. If the newly qualified candidate is found to be similar to any existing candidate in the candidate list, then this newly qualified candidate is removed / pruned.

[0140] Non-adjacent based resolution process for affine constructed merge candidates In the case of deriving inherited merge candidates, one neighboring block is identified at a time, and this single neighboring block needs to be coded in affine mode and may contain two or three CPMVs. In the case of deriving constructed merge candidates, two or three neighboring blocks are identified at a time, and each identified neighboring block does not need to be coded in affine mode, and only one translational MV is taken from this block.

[0141] FIG. 9 presents an example in which construct affine merge candidates can be derived by using non-adjacent blocks. In FIG. 9, A, B, and C are the geographical locations of three non-adjacent blocks. A virtual coding block is formed using the location of A as the top left corner, the location of B as the top right corner, and the location of C as the bottom left corner. Considering a virtual CU as an affine coding block, the MVs at the locations of A', B', and C' can be derived by adhering to equation (3), and the model parameters (a, b, c, d, e, f) can be calculated by the translation MVs at the locations of A, B, and C. Once derived, the MVs at the locations of A', B', and C' can be used as three CPMVs for the current block, and the existing process of generating construct affine merge candidates (the one used in AVS and VVC standards) can be used.

[0142] For constructed merge candidates, the non-adjacent based derivation process may be performed in five steps. The non-adjacent based derivation process may be performed in five steps in a device such as an encoder or a decoder. Step 1 is for candidate scanning; step 2 is for affine model determination; step 3 is for CPMV projection; step 4 is for candidate generation; and step 5 is for candidate pruning. In step 1, non-adjacent blocks may be scanned and selected by the following method:

[0143] Scanning Area and Distance In some embodiments, to maintain rectangular coding blocks, the scanning process is performed only for two non-adjacent blocks, and the third non-adjacent block may depend on the horizontal and vertical positions of the first and second non-adjacent blocks.

[0144] In some embodiments, as shown in Figure 9, the scanning process is performed only for the positions of B and C. The position of A can be uniquely determined by the horizontal position of C and the vertical position of B.

[0145] To form a valid hypothetical coding block, location A may need to be at least valid. The validity of location A may be defined as whether motion information at location A is available or not. In one embodiment, the coding block located at location A may need to be coded in inter mode so that motion information is available to form a hypothetical coding block.

[0146] In some embodiments, the scanning regions and distances may be defined according to a particular scanning direction.

[0147] In some embodiments, the scanning direction may be perpendicular to the side of the current block. One example is shown in Figure 10, where the scanning area is defined as one line of contiguous motion fields to the left or top of the current block. The scanning distance is defined as the number of motion fields from the scanning position to the side of the current block. It should be noted that the size of the motion field may depend on the maximum granularity of the applicable video coding standard. In the example shown in Figure 10, the size of the motion field is assumed to be aligned with the current VVC standard and set to 4x4.

[0148] In some embodiments, the scanning direction may be parallel to the sides of the current block. One example is shown in Figure 11, where the scanning region is defined as one line of contiguous coding blocks to the left or above the current block.

[0149] In some embodiments, the scanning direction can be a combination of vertical and horizontal scanning to the side of the current block. One example is shown in FIG. 12. As shown in FIG. 12, the scanning direction can also be a combination of parallel and diagonal. Scanning at position B starts from left to right, then in a diagonal direction to the block to the left and above. Scanning at position B repeats as shown in FIG. 12. Similarly, scanning at position C starts from the top to the bottom, then in a diagonal direction to the block to the left and above. Scanning at position C repeats as shown in FIG. 12.

[0150] Scanning Order In some embodiments, the scanning order may be defined as from a position with a smaller distance to a position with a larger distance to the current coding block, which may be applied in the case of vertical scanning.

[0151] In some embodiments, the scanning order may be defined as a fixed pattern. This fixed pattern scanning order may be used for candidate locations with similar distances. One example is the case of horizontal scanning. In one example, the scanning order may be defined as a top-bottom direction for the left scanning area and a left-to-right direction for the top scanning area, similar to the example shown in FIG. 11.

[0152] For the case of a combined scanning method, the scanning order can be a combination of fixed pattern and distance dependent, similar to the example shown in FIG.

[0153] Scanning completed For constructing merge candidates, the qualified candidates do not need to be affine encoded since only translational MVs are required.

[0154] Depending on the number of candidates required, the scanning process may be concluded when the first X qualified candidates have been identified, where X is a positive value.

[0155] As shown in FIG. 9, three corners named A, B, and C are needed to form a virtual coding block. For easier implementation, the scanning process in step 1 may be performed only to identify non-adjacent blocks located at corners B and C, and the coordinate of A may be accurately determined by taking the horizontal coordinate of C and the vertical coordinate of B. In this way, the formed virtual coding block is constrained to be rectangular. In the case where either the B or C point is not available, e.g., outside the boundary, or the motion information in the non-adjacent blocks corresponding to B or C is not available (e.g., the block is coded in intra mode or screen-content mode case), the horizontal or vertical coordinate of C may be defined as the horizontal or vertical coordinate, respectively, of the top-left point of the current block.

[0156] In another embodiment, when corner B and / or corner C are initially determined from the scanning process in step 1, the non-adjacent blocks located at corners B and / or C may be identified accordingly. Second, the location(s) of corners B and / or C may be reset to a pivot point within the corresponding non-adjacent block, such as the center of mass of each non-adjacent block. For example, the center of mass may be defined as the geometric center of each adjacent block.

[0157] As shown in Fig. 9, when the scanning process is performed for corners B and C, the process can be performed jointly or independently. In an embodiment of independent scanning, the scanning method proposed before can be applied separately for corners B and C. In an embodiment of joint scanning, there can be different methods as follows:

[0158] In one embodiment, pairwise scanning may be performed. In one example of pairwise scanning, the candidate positions for corners B and C advance simultaneously. For easier illustration and without loss of generality, FIG. 17B will be taken as an example. As shown in FIG. 17B, the scanning for corner B starts in a bottom-up direction from a first non-adjacent block located on the upper side of the current block. The scanning for corner C starts in a right-left direction from a first non-adjacent block located on the left side of the current block. Thus, in the example shown in FIG. 17B, the pairwise scanning may be defined as the candidate positions for both B and C advance by one unit of step size, where one unit of step size is defined as the height of the current coding block for corner B and the width of the current coding block for corner C.

[0159] In another embodiment, an alternative scanning may be performed. In one example of the alternative scanning, the candidate positions for corners B and C are advanced alternately. In one step, only the position of B or C may be advanced, and the position of C or B remains unchanged. In one example, the position of corner B may be increased incrementally from the first non-adjacent block to a distance in a maximum number of non-adjacent blocks, and the position of corner C remains in the first non-adjacent block. In the next round, the position of corner C moves to the second non-adjacent block, and the position of corner B is traversed again from the first to the maximum value. The rounds continue until all combinations are traversed.

[0160] For the purpose of unification, the methods of defining scanning regions and distances, scanning order, and scanning completion proposed for deriving inheritance merge candidates may be fully or partially reused for deriving construction merge candidates. In one or more embodiments, the same methods defined for inheritance merge candidate scanning, including but not limited to scanning regions and distances, scanning order, and scanning completion, may be fully reused for construction merge candidate scanning.

[0161] In some embodiments, the same method defined for inheritance merge candidate scanning can be partially reused for construction merge candidate scanning. Figure 16 shows an example of this case. In Figure 16, the block size of each non-adjacent block is the same as the current block, which is defined similarly as inheritance candidate scanning, but the overall process is a simplified version since the scanning at each distance is limited to only one block.

[0162] Figures 17A and 17B show another example of this case, where both the non-adjacent inheritance merge candidate and the non-adjacent construction merge candidate are defined with the same block size as the current coding block, and the scanning order, scanning region, and scanning completion condition may be defined differently.

[0163] In FIG. 17A, the maximum distance for the left non-adjacent neighborhood is 4 coding blocks and the maximum distance for the top non-adjacent neighborhood is 5 coding blocks. Also, at each distance, the scanning direction is bottom-up for the left side and right-left for the top side. In FIG. 17B, the maximum distance for the non-adjacent neighborhood is 4 for both the left side and the top side. In addition, scanning at a particular distance is not available because there is only one block at each distance. In FIG. 17A, if M qualified candidates are identified, the scanning operation within each distance may be completed. The value of M may be a predefined fixed value, such as a value of 1 or any other positive integer, or a signaled value determined by the encoder, or a configurable value at the encoder or decoder. In one embodiment, the value of M may be equal to the merge candidate list size.

[0164] 17A and 17B, when N qualified candidates are identified, scanning operations at different distances may be completed. The value of N may be a predefined fixed value, such as a value of 1 or any other positive integer, or a signaled value determined by the encoder, or a configurable value in the encoder or decoder. In one embodiment, the value of N may be the same as the merge candidate list size. In another embodiment, the value of N may be the same as the value of M.

[0165] In both Figures 17A and 17B, non-adjacent spatial neighbors having a closer distance to the current block may be prioritized, which indicates that a non-adjacent spatial neighbor with distance i is scanned or checked before a neighbor with distance i+1, where i may be a non-negative integer representing a particular distance.

[0166] At a particular distance, at most two non-adjacent spatial neighbors are used, which means that at most one neighbor from one side of the current block, e.g., left and top, is selected for inheritance or construction candidate derivation if available. As shown in Figure 17A, the checking order of the left and top neighbors is bottom-top and right-left, respectively. For Figure 17B, this rule can also be applied, with the difference being that at any particular distance, there can be only one option per side of the current block.

[0167] For a construction candidate, the positions of one left and top non-adjacent spatial neighbor are first determined independently, as shown in Fig. 17B. Then, the position of the left-top neighbor, which can surround a rectangular virtual block together with the left and top non-adjacent neighbors, can be determined accordingly. Then, as shown in Fig. 9, the motion information of the three non-adjacent neighbors is used to form CPMVs at the left-top (A), right-top (B), and left-bottom (C) of the virtual block, which are finally projected to the current CU to generate a corresponding construction candidate.

[0168] In step 2, the translational MVs at the positions of the selected candidates after step 1 are evaluated and a suitable affine model can be determined. For easier illustration and without loss of generality, Figure 9 is again used as an example.

[0169] Due to factors such as hardware constraints, implementation complexity, and different reference indexes, the scanning process may be completed before a sufficient number of candidates are identified, for example, motion information of the motion fields in one or more of the selected candidates after step 1 may not be available.

[0170] If motion information for all three candidates is available, the corresponding hypothetical coding block represents a 6-parameter affine model. If motion information for one of the three candidates is not available, the corresponding hypothetical coding block represents a 4-parameter affine model. If motion information for more than one of the three candidates is not available, the corresponding hypothetical coding block may be unusable to represent a valid affine model.

[0171] In some embodiments, if motion information is not available for the top left corner of the virtual coding block, e.g., corner A in FIG. 9, or if motion information is not available for both the top right corner, e.g., corner B in FIG. 9, and the bottom left corner, e.g., corner C in FIG. 9, the virtual block may be set as invalid and may be unusable for representing a valid model, and then steps 3 and 4 may be skipped during the current iteration.

[0172] In some embodiments, if either the top right corner, e.g., corner B in FIG. 9, or the bottom left corner, e.g., corner C in FIG. 9, is unavailable, but not both, then the virtual block may represent a valid 4-parameter affine model.

[0173] In step 3, if the hypothetical coding block is capable of representing a valid affine model, the same projection process used for the inheritance merge candidates may be used.

[0174] In one or more embodiments, the same projection process used for the inheritance merge candidates may be used: in this case, the 4-parameter model represented by the hypothetical coding block from step 2 is projected onto a 4-parameter model for the current block, and the 6-parameter model represented by the hypothetical coding block from step 2 is projected onto a 6-parameter model for the current block.

[0175] In some embodiments, the affine model represented by the virtual coding block from step 2 is always projected onto a 4-parameter model or a 6-parameter model for the current block.

[0176] According to equations (5) and (6), there are two types of four-parameter affine models: type A, where V 0 and V 1 The upper left corner CPMV and the upper right corner CPMV, referred to as V, are available. 0 and V 2 The advantage of this approach is that a top left corner CPMV and a bottom left corner CPMV, referred to as the top left corner CPMV and the bottom left corner CPMV, are available.

[0177] In one or more embodiments, the type of the projected 4-parameter affine model is the same type of the 4-parameter affine model represented by the virtual coding block, e.g., if the affine model represented by the virtual coding block from step 2 is a 4-parameter affine model of type A or B, then the projected affine model for the current block is also of type A or B, respectively.

[0178] In some embodiments, the 4-parameter affine model represented by the virtual coding block from step 2 is always projected to a 4-parameter model of the same type for the current block. For example, a 4-parameter affine model of type A or B represented by the virtual coding block is always projected to a 4-parameter affine model of type A.

[0179] In step 4, based on the projected CPMV after step 3, in one embodiment, the same candidate generation process used in the current VVC or AVS standard may be used. In another embodiment, the temporal motion vectors used in the candidate generation process used in the current VVC or AVS standard may not be used for the non-adjacent block based derivation method. When a temporal motion vector is not used, it indicates that the generated combination does not include any temporal motion vector.

[0180] In step 5, any newly generated candidate after step 4 may undergo a similarity check against all existing candidates already in the merge candidate list. The details of the similarity check have already been explained in the "Affine Merge Candidate Pruning" section. If the newly generated candidate is found to be similar to any existing candidate in the candidate list, the newly generated candidate is removed or pruned.

[0181] In some embodiments, a virtual coding block is formed by determining three corner points A, B, and C, and then the translation MVs of the 4×4 blocks located at the three corners are used to represent an affine model for the virtual coding block. Finally, the affine model of the virtual coding block is projected to the current coding block. This overall configuration can be used to derive a first type of affine candidate built from non-adjacent spatial neighbors (e.g., the sub-blocks located near the three corner points A, B, and C are non-adjacent spatial neighbors). In some embodiments, this method can be applied to affine modes such as affine merge mode and affine AMVP mode, and this method can also be applied to canonical modes such as canonical merge mode and canonical AMVP mode, because the projected affine model can be used to derive the translation MVs based on a specific position (e.g., center position) inside the prediction block or coding block.

[0182] An Inheritance-Based Derivation Method for Affine Construct Merge Candidates For each affine inheritance candidate, all motion information is inherited from one selected spatial neighboring block coded in affine mode. The inheritance information includes CPMV, reference index, prediction direction, affine model type, etc. On the other hand, for each affine construction candidate, all motion information is constructed from two or three selected spatial or temporal neighboring blocks, and the selected neighboring block is not coded in affine mode, and only translational motion information is needed from the selected neighboring block.

[0183] In this section, a new candidate derivation method is disclosed that combines features of inheritance and construction candidates.

[0184] In some embodiments, the combination of inheritance and construction may be achieved by separating the affine model parameters into different groups, with one group of affine parameters inherited from one neighboring block and another group of affine parameters inherited from the other neighboring block.

[0185] In one embodiment, the parameters of one affine model are constructed from two groups. As shown in formula (3), the affine model may include six parameters, including a, b, c, d, e, and f. The translational parameters {a, b} may represent one group, and the non-translational parameters {c, d, e, f} may represent another group. With this grouping method, the two groups of parameters may be inherited independently from two different neighboring blocks in a first step, and then concatenated / constructed to be a complete affine model in a second step. In this case, the group with non-translational parameters needs to be inherited from one affine-coded neighboring block, and the group with translational parameters may be from any inter-coded neighboring block, which may or may not be coded in affine mode. It will be noted that the affine-coded neighboring blocks may be selected from adjacent or non-adjacent affine neighboring blocks based on the scanning method previously proposed for the affine inheritance candidate, such as the method shown in Figure 17A, which is the scanning method / rule including scanning region and distance, scanning order, and scanning completion used in the section "Non-adjacent based derivation process for affine inheritance merge candidates", and the scanning method may be performed for both adjacent or non-adjacent blocks. Alternatively, the affine-coded neighboring blocks may not physically exist, but may be virtually constructed from regular inter-coded neighboring blocks, such as the method shown in Figure 17B, which is the scanning method / rule including scanning region and distance, scanning order, and scanning completion used in the section "Non-adjacent based derivation process for affine construction merge candidates".

[0186] In some embodiments, the neighboring blocks associated with each group may be determined in different ways. In one method, the neighboring blocks for the different groups of parameters may be all from the non-adjacent neighboring regions, and the scanning method may be designed similar to the previously proposed method for the non-adjacent based derivation process. In another method, the neighboring blocks for the different groups of parameters may be all from the adjacent neighboring regions, and the scanning method may be the same as the current VVC or AVS video standard. In another method, the neighboring blocks for the different groups of parameters may be partial from the adjacent regions and partial from the non-adjacent neighboring regions.

[0187] When neighboring blocks are scanned from non-adjacent neighborhood areas to build the current type of candidate, the scanning process may be performed differently from the non-adjacent based derivation process for affine inheritance candidates. In one or more embodiments, the scanning area, distance, and order may be defined similarly, but the scanning completion rule may be specified differently. For example, non-adjacent blocks may be scanned exclusively within a defined maximum distance in each area. In this case, all non-adjacent blocks within the distance may be scanned by adhering to the scanning order. In some embodiments, the scanning area may be different. For example, in addition to the left and top areas, the neighboring area and the non-adjacent area at the bottom right of the current coding block may be scanned to determine neighbors to generate translation or / and non-translation parameters. In addition, the neighbors scanned in the bottom right area may be used to find co-located temporal neighbors instead of spatial neighbors. One scanning criterion may be conditional based on whether the co-located temporal neighbor at the bottom right has already been used to generate an affine construction neighbor. If it has already been used, scanning is not performed, otherwise scanning is performed. Instead, if the temporal neighbor(s) at the same location on the bottom right are available, meaning they are already in use, scanning is performed, otherwise no scanning is performed.

[0188] When several groups of affine parameters are combined to build a new candidate, there may be several rules to be observed. The first one is the eligibility criteria. In one embodiment, the related neighboring block or block(s) per group may be checked whether they use the same reference picture for at least one direction or both directions. In another embodiment, the related neighboring block or block(s) per group may be checked whether they use the same precision / resolution for motion vectors.

[0189] When a criterion is checked, the first X relevant neighboring block(s) per group may be used. The value of X may be defined as the same or different values ​​for different groups of parameters. For example, the first one or two neighboring blocks containing non-translational affine parameters may be used, and the first three or four neighboring blocks containing translational affine parameters may be used.

[0190] The second is a construction formula. In one embodiment, the CPMV of a new candidate can be derived in the following formula: JPEG2025501213000015.jpg13132(x,y) is the position of a corner within the current coding block (e.g., (0,0) for the top-left corner CPMV, (width,0) for the top-right corner CPMV), {c,d,e,f} is one group of parameters from one neighboring block, and {a,b} is another group of parameters from another neighboring block.

[0191] In another embodiment, the CPMV of the new candidate may be derived in the following manner: JPEG2025501213000016.jpg14132(Δw,Δh) is the distance between the top left corner of the current coding block and the top left corner of one of the neighboring blocks associated with one group of parameters, such as the associated neighboring block of the group {a,b}. The definition of the other parameters in this formula is the same as the above embodiment. The parameters may be grouped in another way, where (a,b,c,d,e,f) is formed as one group and (Δw,Δh) is formed as another group. Also, the two groups of parameters are from two different neighboring blocks. Alternatively, the value of (Δw,Δh) may be predefined as a fixed value such as (0,0) or any constant value that does not depend on the distance between the neighboring block and the current block.

[0192] FIG. 18 shows an example of an inheritance-based derivation method for deriving an affine construction candidate. In FIG. 18, there are three steps for deriving an affine construction candidate. In step 1, according to a certain grouping strategy, the encoder or decoder may perform scanning of neighboring blocks and non-neighboring blocks for each group. In the case of FIG. 18, two groups are defined, neighboring 1 is coded in affine mode and provides non-translational affine parameters, and neighboring 2 provides translational affine parameters. Neighboring 1 may be obtained according to the process in the section "Non-neighbor-based derivation process for affine inheritance merge candidates" as shown in FIG. 15A-15D and 17A, and neighboring 1 may be an adjacent or non-neighboring block of the current block. Furthermore, neighboring 2 may be obtained according to the process as shown in FIG. 16 and 17B.

[0193] In some embodiments, Neighbor 1, coded in affine mode, may be scanned from adjacent or / and non-adjacent regions by adhering to the scanning method proposed above. In some embodiments, Neighbor 2, coded in affine or non-affine mode, may also be scanned from adjacent or non-adjacent regions. For example, Neighbor 2 may be from one of the scanned adjacent or non-adjacent regions if the motion information has not already been used to derive some affine merge or AMVP candidates, or from the bottom right position of the current block if a co-located TMVP candidate at this position is available and / or has already been used to derive some affine merge or MVP candidates. Alternatively, a small coordinate offset (e.g., +1 or +2 or 1 or 2 for the vertical or / and horizontal coordinates) may be applied when determining the position of Neighbor 2 to provide slightly more diversified motion information for building new candidates.

[0194] In step 2, with the parameters and positions determined in step 1, a specific affine model can be defined, which can derive different CPMVs according to the coordinates (x, y) of the CPMVs. For example, as shown in FIG. 18, the non-translational parameters {c, d, e, f} can be obtained based on the neighbor 1 obtained in step 1, and the translational parameters {a, b} can be obtained based on the neighbor 2 obtained in step 1. Furthermore, the distance parameters Δw, Δh can be calculated based on the position (x, y) of the current block. 1 ,y 1 ) and the adjacent 2 positions (x 2 ,y 2 ), the distance parameters Δw, Δh may indicate the horizontal and vertical distances between the current block and neighbor 1 or neighbor 2, respectively. For example, the distance parameters Δw, Δh may indicate the horizontal distance (x 1 -x 2 ), and the vertical distance (y 1 -y 2 In particular, Δw=x 1-x 2 , Δh=y 1 -y 2 It is.

[0195] In step 3, two or three CPMVs are derived for the currently coded block, and the currently coded block may be constructed to form new affine candidates.

[0196] In some embodiments, other prediction information may be further constructed. If a neighboring block is checked to have the same direction and / or reference picture, the prediction direction (e.g., bi-predictive or uni-predictive) and the index of the reference picture may be the same as the related neighboring block. Instead, the prediction information is determined by reusing the least overlapping information among the related neighboring blocks from different groups. For example, if only one direction reference index from one neighboring block is the same as the same direction reference index of the other neighboring block, the prediction direction of the new candidate is determined as uni-predictive, and the same reference index and direction are reused.

[0197] In some embodiments, an affine model may be constructed by combining model parameters from different inheritances. In one example, translational model parameters may be inherited from translational blocks (e.g., from adjacent spatially adjacent 4x4 blocks or / and non-adjacent spatially adjacent 4x4 blocks) and non-translational model parameters may be inherited from affine-coded blocks (e.g., from adjacent spatially adjacent affine-coded blocks or / and non-adjacent spatially adjacent affine-coded blocks). Alternatively, non-translational model parameters may be inherited from history-coded affine blocks instead of explicitly scanned non-adjacent spatially adjacent affine-coded blocks, and the history-coded affine blocks may be adjacent spatially adjacent or non-adjacent spatially adjacent. This overall configuration may be used to derive a second type of affine candidate constructed from non-adjacent spatial neighbors (e.g., non-translational model parameters may be inherited from non-adjacent spatial neighbors). In some embodiments, because a generative affine model can be used to derive a translational MV based on a specific position (e.g., a center position) within a prediction block or a coding block, this method may be applied to affine modes such as affine merge mode and affine AMVP mode, and this method may also be applied to canonical modes such as canonical merge mode and canonical AMVP mode.

[0198] An HMVP-Based Deriving Method for Affine Construct Merge Candidates In the case of adjacent neighbor-based derivation processing, already defined in the current video standards VVC and AVS and described in the above section and in FIG. 7, a fixed order of scanning for adjacent neighbors is performed to identify two or three adjacent blocks. In the case of non-adjacent based derivation processing, as proposed in the previous section and in FIG. 17B, two non-adjacent neighbors are identified during another fixed order of scanning. In other words, for both adjacent neighbor-based derivation method and non-adjacent neighbor-based derivation method, a certain depth of local scanning is necessary to identify the number of neighbors. This scanning process relies on local buffering around each current block and also incurs a certain amount of computational complexity.

[0199] On the other hand, as described in the introduction section, the HMVP merge mode is already adopted in current VVC and AVS, and the translational motion information from neighboring blocks is already stored in the history table. In this case, the scanning process can be replaced by searching the HMVP table.

[0200] Therefore, for the previously proposed non-neighbor-based derivation process and inheritance-based derivation process, instead of the scanning method as shown in Fig. 17B and Fig. 18, the translational motion information may be obtained from the HMVP table. However, to subsequently derive affine construction candidates, position information, width, height, and reference information are also required, which may be accessible when the current HMVP table may be modified. Therefore, it is proposed to extend the HMVP table to store additional information in addition to the motion information of each historical neighbor. In one embodiment, the additional information may include the position of affine or non-affine neighboring blocks, or affine motion information such as CPMV or equivalent regular motion derived from CPMV (for example, this regular motion may be from an internal sub-block of the affine-coded neighboring block), reference index, etc.

[0201] Candidate Derivation Methods for Affine AMVP and Regular Merge Modes As described in the above section, for affine AMVP mode, an affine candidate list is also needed to derive the CPMV predictor. As a result, all the above proposed derivation methods can be applied to the affine AMVP mode as well. The only difference is that when the above proposed derivation methods are applied in AMVP, the selected neighboring block should have the same reference picture index as the currently coded block.

[0202] For regular merge mode, the candidate list is also built only with translational candidate MVs, not CPMVs. In this case, all the above proposed derivation methods can still be applied by adding an additional derivation step, which is to derive a translational MV for the current block, which can be realized by selecting a specific rotation position (x,y) within the current block, and observing the same equation (3). In other words, to derive the CPMV of an affine block, the positions of the three corners of the block can be used as the rotation positions (x,y) in equation (3), and to derive the translational MV of a regular inter-coded block, the center position of the block can be used as the rotation position (x,y) in equation (3). Once the translational MV is derived for the current block, it can be inserted into the candidate list as another candidate.

[0203] When new candidates are derived based on the methods proposed above for affine AMVP and regular merge modes, the replacements of the new candidates may be reordered.

[0204] In one embodiment, a newly derived candidate may be inserted into the affine AMVP candidate list by adhering to the following order: (1) inherited from adjacent spatial neighbors; (2) constructed from adjacent spatial neighbors; (3) inherited from non-adjacent spatial neighbors; (4) constructed from non-adjacent spatial neighbors; (5) translational MVs from adjacent spatial neighbors; (6) temporal MVs from adjacent temporal neighbors, and (7) Zero MV.

[0205] In another embodiment, a newly derived candidate may be inserted into the affine AMVP candidate list by adhering to the following order: (1) inherited from adjacent spatial neighbors; (2) constructed from adjacent spatial neighbors; (3) inherited from non-adjacent spatial neighbors; (4) translational MVs from adjacent spatial neighbors; (5) constructed from non-adjacent spatial neighbors; (6) temporal MVs from adjacent temporal neighbors, and (7) Zero MV.

[0206] In another embodiment, a newly derived candidate may be inserted into the affine AMVP candidate list by adhering to the following order: (1) inherited from adjacent spatial neighbors; (2) constructed from adjacent spatial neighbors; (3) translational MVs from adjacent spatial neighbors; (4) inherited from non-adjacent spatial neighbors; (5) constructed from non-adjacent spatial neighbors; (6) temporal MVs from adjacent temporal neighbors, and (7) Zero MV.

[0207] In another embodiment, a newly derived candidate may be inserted into the affine AMVP candidate list by adhering to the following order: (1) inherited from adjacent spatial neighbors; (2) constructed from adjacent spatial neighbors; (3) translational MVs from adjacent spatial neighbors; (4) Temporal MVs from adjacent temporal neighbors, (5) inherited from non-adjacent spatial neighbors; (6) constructed from non-adjacent spatial neighbors, and (7) Zero MV.

[0208] In another embodiment, a newly derived candidate may be inserted into the affine AMVP candidate list by adhering to the following order: (1) inherited from adjacent spatial neighbors; (2) constructed from adjacent spatial neighbors; (3) translational MVs from adjacent spatial neighbors; (4) Temporal MVs from adjacent temporal neighbors, (5) inherited from non-adjacent spatial neighbors, and (6) Zero MV.

[0209] In another embodiment, the newly derived candidates are inserted into the affine AMVP candidate list by adhering to the following order: (1) inherited from adjacent spatial neighbors; (2) constructed from adjacent spatial neighbors; (3) translational MVs from adjacent spatial neighbors; (4) Translational MV from temporal neighbors, (5) inherited from non-adjacent spatial neighbors, and (6) Zero MV.

[0210] It will be noted that candidates constructed from non-adjacent spatial neighborhoods may be referred to as first type candidates or / and second type candidates constructed from non-adjacent spatial neighborhoods.

[0211] In another embodiment, the newly derived candidates may be inserted into the regular merge candidate list by adhering to the following order: (1) Spatial MVP from adjacent spatial neighbors, (2) Temporal MVP from adjacent co-located neighbors, (3) spatial MVP from non-adjacent spatial neighbors, (4) Inheritance MVP from non-adjacent spatially affine neighbors, (5) Constructing MVP from non-adjacent spatial neighbors (6) History-based MVP from FIFO tables, (7) Pairwise average MVP, and (8) Zero MV.

[0212] Reordering the affine merge candidate list In one embodiment, non-adjacent spatial merge candidates may be inserted into the affine merge candidate list by adhering to the following order: 1. Sub-block based temporal motion vector prediction (SbTMVP) candidate if available, 2. inherited from adjacent neighbors, 3. inherited from non-adjacent neighborhoods, 4. constructed from adjacent neighbors, 5. constructed from non-adjacent neighborhoods, 6. zero MV.

[0213] In another embodiment, non-adjacent spatial merge candidates may be inserted into the affine merge candidate list by adhering to the following order: 1. SbTMVP candidate if available, 2. inherited from adjacent neighbors, 3. constructed from adjacent neighbors, 4. inherited from non-adjacent neighbors, 5. constructed from non-adjacent neighbors, 6. zero MV.

[0214] In another embodiment, non-adjacent spatial merge candidates may be inserted into the e-affine merge candidate list by adhering to the following order: 1. SbTMVP candidate if available, 2. inherited from adjacent neighbors, 3. constructed from adjacent neighbors, 4. one set of zero MVs, inherited from non-adjacent neighbors, 6. constructed from non-adjacent neighbors, 7. remaining zero MVs if the list is still not full.

[0215] In another embodiment, non-adjacent spatial merge candidates may be inserted into the affine merge candidate list by adhering to the following order: 1. SbTMVP candidates if available, 2. inherited from adjacent neighbors, 3. inherited from non-adjacent neighbors with distance less than X, 4. constructed from adjacent neighbors, 5. constructed from non-adjacent neighbors, 6. constructed from inherited translational and non-translational neighbors, 7. zero MV if the list is still not full.

[0216] In another embodiment, non-adjacent spatial merge candidates may be inserted into the affine merge candidate list by adhering to the following order: 1. SbTMVP candidate if available, 2. inherited from adjacent neighbors, 3. inherited from non-adjacent neighbors, 4. first candidate constructed from adjacent neighbors, 5. first X candidates constructed from inherited translational and non-translated neighbors, 6. constructed from non-adjacent neighbors, 7. another Y candidates constructed from inherited translational and non-translated neighbors, 8. zero MV if the list is still not full.

[0217] In some embodiments, the values ​​of X and Y may be predefined fixed values, such as a value of 2, or signaling values ​​received by the decoder (sequence / slice / block / CTU level signaling parameters), or configurable values ​​at the encoder / decoder, or the number of available left and top neighbors of each individual coding block, or dynamically determined values ​​(e.g., X<=3, Y<=3) according to any combination of methods for determining the values ​​of X and Y. In one embodiment, the value of X may be the same as the value of Y. In another embodiment, the value of X may be different from the value of Y.

[0218] In another embodiment, non-adjacent spatial merge candidates may be inserted into the affine merge candidate list by adhering to the following order: 1. SbTMVP candidate if available, 2. inherited from adjacent neighbors, inherited from neighbors not adjacent by a distance less than X, 4. constructed from adjacent neighbors, 5. constructed from neighbors not adjacent by a distance less than Y, 6. inherited from neighbors not adjacent by a distance greater than X, 7. constructed from neighbors not adjacent by a distance greater than Y, 8. zero MV. In this embodiment, the values ​​X and Y may be predefined fixed values, such as a value of 2, or signaling values ​​determined by the encoder, or configurable values ​​at the encoder or decoder. In one embodiment, the value of X may be the same as the value of Y. In another embodiment, the value of N may be different from the value of M.

[0219] In some embodiments, when a new candidate is derived by using an inheritance-based derivation method that constructs a CPMV by combining affine motion and translational MVs, the placement of this new candidate may depend on the placement of other constructed candidates.

[0220] In one embodiment, for different construction candidates, the reordering of the affine merge candidate list may adhere to the following order: (1) constructed from adjacent spatial neighbors; (2) constructed from combining adjacent spatially affine neighbors and translational MVs, (3) constructed from non-adjacent spatial neighbors, and (4) It is constructed from combining non-adjacent spatially affine adjacent and translational MVs.

[0221] In another embodiment, for different construction candidates, the reordering of the affine merge candidate list may adhere to the following order: (1) constructed from adjacent spatial neighbors; (2) constructed from non-adjacent spatial neighbors; (3) constructed from combining adjacent spatially affine neighbors and translational MVs; and (4) It is constructed from combining non-adjacent spatially affine adjacent and translational MVs.

[0222] In another embodiment or embodiments, the reordering of the affine merge candidates may be partially or completely interleaved among candidates of different categories (e.g., interleaving may indicate that candidates from the same category may not be adjacently placed in the candidate list). In some embodiments, there may be seven categories of affine merge candidates arranged in the affine merge candidate list: (1) SbTMVP candidates, if available; (2) candidates for inheritance from adjacent neighbors; (3) inheritance candidates from non-adjacent neighbors, (4) Construction candidates from adjacent neighborhoods, (5) A second type of construction candidate from non-adjacent neighborhoods; (6) construction candidates of the first type from non-adjacent neighborhoods; and (7) Zero MV. The particular ordering discussed above may be applied to any candidate list, including affine AMVP candidate lists, regular merge candidate lists, and affine merge candidate lists.

[0223] The ordering of candidates for the above different categories may be designed in different ways, since candidates placed at later positions in the affine merge list may incur a higher signaling overhead when selected and signaled by the encoder.

[0224] In one or more embodiments, the order of the candidates may remain the same as the insertion order above. To reorder the subsequent candidates, an adaptive reordering method may be applied, which may be a template-based method (ARMC) or a non-template-based method such as a bilateral matching method.

[0225] In one or more embodiments, the order of candidates may be reordered in a particular pattern, which may be applied in any candidate list, including affine AMVP candidate lists, regular merge candidate lists, and affine merge candidate lists.

[0226] In some embodiments, the reordering pattern may depend on a number of available candidates for each category.

[0227] In one embodiment, the reordering pattern may be defined as follows: (1) SbTMVP, if a candidate is available; (2) candidates for inheritance from adjacent neighbors; (3) Construction candidates from adjacent neighborhoods, (4) the first X succession candidates from non-adjacent neighborhoods (e.g., X can be a pre-fixed number such as 1 or a signaled number); (5) the first Y building candidates from a second type of building candidates from a non-adjacent neighborhood (e.g., Y may be defined similarly to X); (6) the first Z building candidates from the first type of building candidates from the non-adjacent neighborhood (e.g., Z may be defined similarly to X); (7) remaining succession candidates from non-adjacent neighborhoods (e.g., X can be a pre-fixed number such as 1 or a signaled number); (8) remaining construction candidates from the second type of construction candidates from non-adjacent neighborhoods (e.g., Y may be defined similarly to X); (9) remaining construction candidates from the first type of construction candidates from non-adjacent neighborhoods (e.g., Z may be defined similarly to X); and (10) Zero MV.

[0228] In one or more embodiments, the reordering pattern may be an interleaved manner, which may merge different candidates from different categories. In one example, the interleaved pattern may be defined as follows: (1) SbTMVP, if a candidate is available; (2) candidates for inheritance from adjacent neighbors; (3) X1 construction candidates from adjacent neighborhoods; (4) the first Y1 construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (5) the first Z1 succession candidates from non-adjacent neighborhoods, (6) the first K1 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (7) the first X2 construction candidates from adjacent neighborhoods, (8) the first Y2 construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (9) the first Z2 succession candidates from non-adjacent neighborhoods, (10) the first K2 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (11)... (12) the first Xi construction candidates from adjacent neighborhoods; (13) the first Yi construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (14) the first Zi succession candidates from non-adjacent neighborhoods, (15) the first Ki construction candidates from the first type of construction candidates from the non-adjacent neighborhood; and (16) Zero MV. The values ​​of (Xi, Yi, Zi, Ki) can be pre-fixed numbers such as 1 or a signaled number. If the number of available candidates in one category is less than other categories, the position of the candidate for this category is skipped and the remaining available candidates of the other categories take over this position.

[0229] In one or more embodiments, the reordering pattern may be a combination version that takes into account both availability and interleaving methods. In one example, the combination pattern may be defined as follows: (1) SbTMVP, if a candidate is available; (2) candidates for inheritance from adjacent neighbors; (3) the first X1 construction candidates from adjacent neighborhoods; (4) all available construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (5) remaining construction candidates from adjacent neighborhoods; (6) the first Z1 succession candidates from non-adjacent neighborhoods, (7) the first K1 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (8)... (9) the first Zi succession candidates from non-adjacent neighborhoods, (10) the first Ki construction candidates from the first type of construction candidates from the non-adjacent neighborhood; and (11) Zero MV.

[0230] Improved reordering of affine candidate lists Based on the candidate derivation method proposed above, one or more candidates may be derived for an existing affine merge candidate list, or an affine AMVP candidate list, or a normal merge candidate list, and the size of the corresponding list may be adjusted statistically (e.g., configurable size) or adaptively (e.g., dynamically changed according to availability at the encoder and then signaled to the decoder). It will be noted that when one or more new candidates are derived for the normal merge candidate list, the new candidates are first derived as affine candidates, and then converted into translational motion vectors by using the rotation position (e.g., center sample or pixel position) in the coding block and associated affine model before being inserted into the normal merge candidate list.

[0231] In one or more embodiments, an adaptive reordering method such as ARMC may be applied to one or more of the above candidate lists after the candidate lists have been updated or built by adding some new candidates derived by the candidate derivation method proposed above.

[0232] In another embodiment, a temporal candidate list may be created first, and the temporal candidate list may have a size larger than the existing candidate list (e.g., affine merge candidate list, affine AMVP candidate list, normal merge candidate list). Once the temporal candidate list is built by adding newly derived candidates and statistically ordered by using the insertion method proposed above, an adaptive reordering method such as ARMC may be applied to reorder the temporal candidate list. After adaptive reordering, the first N candidates of the temporal candidate list are inserted into the existing candidate list, and the value of N may be a fixed value or a configurable value. In one embodiment, the value of N may be the same as the size of the existing candidate list, and the selected N candidates from the temporal candidate list are identified.

[0233] In the above application scenarios applying an adaptive reordering method such as ARMC, the following methods may be used to improve the performance or / and reduce the complexity of the applied reordering method.

[0234] In some embodiments, when a template matching cost is used to reorder different candidates, a cost function such as the sum of absolute differences (SAD) between the samples of the template of the current block and their corresponding reference samples may be used. The reference samples of the template may be identified by the same motion information of the current block. In the case where fractional motion information is used for the current block, an interpolation filtering process may be used to generate the predicted samples of the template. Since the generated predicted samples are not used for the final block reconstruction but rather to compare the motion accuracy between different candidates, the prediction accuracy of the template samples may be relaxed by using an interpolation filter with smaller taps. For example, in the case of adaptively reordering an affine merge candidate list, a 2-tap or 4-tap interpolation filter may be used to generate the predicted samples for the selected template of the current block. Or, the nearest integer sample (skipping the interpolation filtering process entirely) may be used as the predicted sample of the template. When the template matching method is used to adaptively reorder candidates in other candidate lists, such as a normal merge candidate list or an affine AMVP candidate list, an interpolation filter with smaller taps may be used as well.

[0235] In some embodiments, when template matching costs are used to reorder different candidates, a cost function such as the SAD between the samples of the template of the current block and their corresponding reference samples may be used. The corresponding reference samples may be identified at integer or fractional positions. When fractional positions are identified, a level of prediction accuracy may be achieved by performing an interpolation filter process. Due to limited prediction accuracy, the calculated matching costs for different candidates may include noise level differences. To reduce the effect of noise level cost differences, the calculated matching costs may be adjusted by removing a few least significant bits before the candidate sorting process.

[0236] In some embodiments, if not enough candidates are derived by using different derivation methods, the candidate lists may be padded with zero MVs at the end of each list. In this case, the candidate cost may be calculated only for the first zero MV, and the remaining zero MVs may be statistically assigned an arbitrarily large cost value, so that those repeated zero MVs are placed at the end of the corresponding candidate list.

[0237] In some embodiments, all zero MVs may be statistically assigned an arbitrarily large cost value, such that all zero MVs are placed at the end of the corresponding candidate list.

[0238] In some embodiments, the previous completion method may be applied to the reordering method to reduce the complexity at the decoder side.

[0239] In one or more embodiments, when the candidate list is constructed, different types of candidates may be derived and inserted into the list. If one candidate or one type of candidate does not participate in the reordering process but is selected and signaled to the decoder, the reordering process applied to other candidates may be completed early. In one example, in the case of applying ARMC to an affine merge candidate list, SbTMVP candidates may be excluded from the reordering process. In this case, if the signaled merge index value for an affine coded block indicates an SbTMVP candidate at the decoder side, the ARMC process may be skipped or completed early for this affine block.

[0240] In another embodiment, if one candidate or one type of candidate does not participate in the reordering process but is selected and signaled to the decoder, both the derivation process and the reordering process for this particular candidate or this particular type of candidate may be skipped. It should be noted that the skipped derivation process and the reordering process only apply to the particular candidate or the particular type of candidate, and the remaining candidates or the types of candidates are still executed, and the derivation process is skipped, which indicates that the related operations of deriving the particular candidate or the particular type of candidate are skipped, but the predefined list position of the particular candidate or the particular type of candidate (e.g., according to the predefined insertion order) may still be maintained, and the candidate contents such as motion information may just be invalid due to the skipped derivation process. Similarly, during the reordering process, the cost calculation of this particular candidate or the particular type of candidate may be skipped, and the list position of this particular candidate or the particular type of candidate may not be changed after reordering other candidates.

[0241] 21 illustrates a computing environment (or computing device) 2110 connected to a user interface 2160. The computing environment 2110 can be part of a data processing server. In some embodiments, the computing device 2110 can perform any of the various methods or processes (encoding / decoding methods or processes) described below according to various embodiments of the present disclosure. The computing environment 2110 can include a processor 2120, a memory 2140, and an I / O interface 2150.

[0242] The processor 2120 typically controls the overall operation of the computing environment 2110, such as operations associated with display, data acquisition, data communication, and image processing. The processor 2120 may include one or more processors for executing instructions to perform all or part of the steps in the methods described above. Moreover, the processor 2120 may include one or more modules that facilitate interaction between the processor 2120 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single chip machine, or a GPU, etc.

[0243] The memory 2140 is configured to store various types of data to support the operation of the computing environment 2110. The memory 2140 may include predefined software 2142. Examples of such data include instructions for any application or method operating on the computing environment 2110, video data sets, image data, etc. The memory 2140 may be implemented by using any type of volatile or non-volatile memory device, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk, or a combination thereof.

[0244] The I / O interface 2150 provides an interface between the processor 2120 and peripheral interface modules, such as a keyboard, a click wheel, and buttons. The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 2150 may be connected to encoders and decoders.

[0245] In some embodiments, a non-transitory computer readable storage medium is also provided that includes a number of programs, such as contained in memory 2140, executable by processor 2120 in computing environment 2110 to perform the methods described above. For example, the non-transitory computer readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0246] A non-transitory computer-readable storage medium has stored thereon a plurality of programs for execution by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, causing the computing device to perform the above-described method for motion prediction.

[0247] In some embodiments, the computing environment 2110 may be implemented with one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic circuits (PLDs), field programmable gate arrays (FPGAs), graphical processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0248] FIG. 22 is a flowchart illustrating a method for video decoding according to an embodiment of the present disclosure.

[0249] In step 2201, on the decoder side, the processor 2120 may obtain a third candidate position based on the first candidate position and the second candidate position.

[0250] In some embodiments, the first candidate location and the second candidate location may be from a number of non-adjacent neighboring locations, the number of non-adjacent neighboring locations being a number of blocks away from one side of the current block. For example, the first candidate location may be one or two blocks away from the left side of the current block, and the second candidate location may be one or two blocks away from the top side of the current block. The number of blocks is not limited to one or two.

[0251] In some embodiments, the processor 2120 may obtain a first candidate position according to a first scanning distance and a first scanning area, where the first scanning distance indicates a first distance between the first candidate position and a left side of the current block, and the first scanning area is located to the left of the current block. Furthermore, the processor 2120 may obtain a second candidate position according to a second scanning distance and a second scanning area, where the second scanning distance indicates a second distance between the second candidate position and a top side of the current block, and the second scanning area is located above the current block. For example, the first distance may include one or more blocks away from the left side of the current block, and the second distance may include one or more blocks away from the top side of the current block.

[0252] For example, the first candidate position may be obtained by adhering to a scanning process as shown in Figure 10, 11, or 12. In some embodiments, the scanning area and distance may be defined according to a specific scanning direction. In some embodiments, the scanning direction may be perpendicular to the side of the current block. One example is shown in Figure 10, where the scanning area is defined as one line of continuous motion fields to the left or above the current block. The scanning distance is defined as the number of motion fields from the scanning position (the position of the MV candidate) to the side of the current block. In some embodiments, the size of the motion field may depend on the maximum granularity of the applicable video coding standard. In the example shown in Figure 10, the size of the motion field is assumed to be aligned with the current VVC standard and set to 4x4.

[0253] In some embodiments, the scanning direction may be parallel to the sides of the current block. One example is shown in Figure 11, where the scanning area is defined as one line of contiguous coding blocks to the left or above the current block. For example, the first scanning area is one line of contiguous coding blocks to the left of the current block, and the second scanning area is one line of contiguous coding blocks above the current block.

[0254] In step 2202, the processor 2120 may obtain a virtual block based on the first candidate location, the second candidate location, and the third candidate location.

[0255] In some embodiments, the virtual block is a rectangular coding block, and the third candidate location is determined based on a vertical position of the first candidate location and a horizontal position of the second candidate location.

[0256] For example, as shown in Fig. 9, the first candidate location is the bottom left corner C, and the second candidate location is the top right corner B. Based on the locations of B and corner C used as the two end points of a rectangle, a virtual block (a block in the dotted line and all sides are parallel to the current block) can be formed, so that another end point A can be obtained. End point A is the third candidate location.

[0257] In step 2203, the processor 2120 may obtain multiple first control point motion vectors (CPMVs) for the virtual block based on the translation MVs at the first candidate position, the second candidate position, and the third candidate position in accordance with equation (2) above.

[0258] In step 2204, the processor 2120 may project multiple first CPMVs for the virtual block onto the current block to obtain a translational MV based on a particular position within the current block in response to determining that the current block is to be coded as normal inter mode, or multiple second CPMVs for the current block in response to determining that the current block is to be coded as affine mode.

[0259] For example, the CPMVs at corners A, B, and C are projected onto the current block with end points A', B', and C', respectively, as shown in Fig. 9. Furthermore, the CPMVs of the current block can be transformed into translational MVs at specific locations within the current block by adhering to the general affine model as shown in equations (3) or (4).

[0260] In some embodiments, the particular location may include a center location within the current block.

[0261] In some embodiments, as shown in step 2205, the processor 2120 may insert a translational MV into the normal merge list in response to determining that the current block is coded as a normal inter mode. For example, when the processor 2120 determines that a translational MV is obtained, it may insert the translational MV into the normal merge list. In some embodiments, the processor 2120 may further obtain and insert additional translational MVs into the normal merge list until the normal merge list is full.

[0262] In some embodiments, as shown in step 2206, the processor 2120 may obtain a plurality of second CPMVs from the affine merge candidate list or the affine AMVP candidate list in response to determining that the current block is coded as an affine mode. For example, when the processor 2120 determines the plurality of second CPMVs, it may insert the plurality of second CPMVs into the affine merge candidate list or the affine AMVP candidate list. In some embodiments, the processor 2120 may further obtain and insert additional plurality of second CPMVs into the affine merge candidate list or the affine AMVP candidate list until the affine merge candidate list or the affine AMVP candidate list is full.

[0263] FIG. 23 is a flow chart illustrating a method for video encoding that corresponds to the method for video decoding as shown in FIG.

[0264] In step 2301, on the encoder side, the processor 2120 may obtain a third candidate position based on the first candidate position and the second candidate position.

[0265] In some embodiments, the first candidate location and the second candidate location may be from a number of non-adjacent neighboring locations, the number of non-adjacent neighboring locations being a number of blocks away from one side of the current block. For example, the first candidate location may be one or two blocks away from the left side of the current block, and the second candidate location may be one or two blocks away from the top side of the current block. The number of blocks is not limited to one or two.

[0266] In some embodiments, the processor 2120 may obtain a first candidate position according to a first scanning distance and a first scanning area, where the first scanning distance indicates a first distance between the first candidate position and a left side of the current block, and the first scanning area is located to the left of the current block. Furthermore, the processor 2120 may obtain a second candidate position according to a second scanning distance and a second scanning area, where the second scanning distance indicates a second distance between the second candidate position and a top side of the current block, and the second scanning area is located above the current block. For example, the first distance may include one or more blocks away from the left side of the current block, and the second distance may include one or more blocks away from the top side of the current block.

[0267] For example, the first candidate position may be obtained by adhering to a scanning process as shown in Figure 10, 11, or 12. In some embodiments, the scanning area and distance may be defined according to a specific scanning direction. In some embodiments, the scanning direction may be perpendicular to the side of the current block. One example is shown in Figure 10, where the scanning area is defined as one line of continuous motion fields to the left or above the current block. The scanning distance is defined as the number of motion fields from the scanning position (the position of the MV candidate) to the side of the current block. In some embodiments, the size of the motion field may depend on the maximum granularity of the applicable video coding standard. In the example shown in Figure 10, the size of the motion field is assumed to be aligned with the current VVC standard and set to 4x4.

[0268] In some embodiments, the scanning direction may be parallel to the sides of the current block. One example is shown in Figure 11, where the scanning area is defined as one line of contiguous coding blocks to the left or above the current block. For example, the first scanning area is one line of contiguous coding blocks to the left of the current block, and the second scanning area is one line of contiguous coding blocks above the current block.

[0269] In step 2302, the processor 2120 may determine a virtual block based on the first candidate location, the second candidate location, and the third candidate location.

[0270] In some embodiments, the virtual block is a rectangular coding block, and the third candidate location is determined based on a vertical position of the first candidate location and a horizontal position of the second candidate location.

[0271] For example, as shown in Fig. 9, the first candidate location is the bottom left corner C, and the second candidate location is the top right corner B. Based on the locations of B and corner C used as the two end points of a rectangle, a virtual block (a block in the dotted line and all sides are parallel to the current block) can be formed, so that another end point A can be obtained. End point A is the third candidate location.

[0272] In step 2303, the processor 2120 may obtain multiple first control point motion vectors (CPMVs) for the virtual block based on the translation MVs at the first candidate position, the second candidate position, and the third candidate position in accordance with equation (2) above.

[0273] In step 2304, the processor 2120 may obtain a translational MV based on a particular position within the current block in response to determining that the current block is to be coded as a normal inter mode, or may project multiple first CPMVs for the virtual block onto the current block to obtain multiple second CPMVs for the current block in response to determining that the current block is to be coded as an affine mode.

[0274] For example, the CPMVs at corners A, B, and C are projected onto the current block with end points A', B', and C', respectively, as shown in Fig. 9. Furthermore, the CPMVs of the current block can be transformed into translational MVs at specific locations within the current block by adhering to the general affine model as shown in equations (3) or (4).

[0275] In some embodiments, the particular location may include a center location within the current block.

[0276] In some embodiments, in response to determining that a translational MV is obtained, the processor 2120 may insert the translational MV into the regular merge list, as shown in step 2305. The processor 2120 may further insert additional translational MVs into the regular merge list until the regular merge list is full.

[0277] In some embodiments, the processor 2120 may insert multiple second CPMVs into the affine merge candidate list or the affine AMVP candidate list in response to determining that multiple second CPMVs are obtained, as shown in step 2306. Further, the processor 2120 may insert additional multiple second CPMVs into the affine merge candidate list or the affine AMVP candidate list until the affine merge candidate list or the affine AMVP candidate list is full.

[0278] In some embodiments, the encoder side and the decoder side may each build a full-normal merge list, and the decoder side may receive a candidate index indicating a translational MV in the full-normal merge list sent by the encoder. In one or more embodiments, the decoder may select a translational MV from the full-normal merge list based on the candidate index.

[0279] FIG. 24 is a flowchart illustrating a method for video decoding according to an embodiment of the present disclosure.

[0280] In step 2401, on the decoder side, the processor 2120 may obtain one or more first parameters based on a first neighboring block from a plurality of inter-coded neighboring blocks of a current block.

[0281] In step 2402, the processor 2120 may obtain one or more second parameters based on a second neighboring block from the plurality of affine coded neighboring blocks of the current block or from the plurality of historical affine coded neighboring blocks.

[0282] In step 2403, the processor 2120 may construct one or more affine models using the one or more first parameters and the one or more second parameters.

[0283] In step 2404, the processor 2120 may project an affine model onto the current block to obtain a translational MV based on a particular position within the current block in response to determining that the current block is to be coded as a normal inter mode, or to obtain multiple CPMVs for the current block in response to determining that the current block is to be coded as an affine mode.

[0284] In some examples, the one or more first parameters may include a plurality of translational parameters associated with an affine model, and the one or more second parameters may include a plurality of non-translational parameters associated with the affine model.

[0285] In some embodiments, the processor 2120 may obtain multiple inter-coded neighboring blocks from multiple neighboring blocks and multiple non-adjacent blocks, where the multiple neighboring blocks are adjacent to the current block and the multiple non-adjacent blocks are each located a number of blocks away from one side of the current block.

[0286] In some embodiments, the processor 2120 may obtain a plurality of affine-coded neighboring blocks from a plurality of affine-coded neighboring blocks and a plurality of affine-coded non-adjacent blocks.

[0287] In some embodiments, the processor 2120 may obtain a plurality of history affine coded neighboring blocks from a plurality of neighboring blocks and a plurality of non-neighboring blocks.

[0288] In some embodiments, as shown in Figure 18, Neighbor 1 is one first neighboring block obtained by scanning a left scanning region of the current block according to the derivation process for the affine inheritance merge candidate, and one or more such first neighboring blocks may be obtained by exclusively scanning the left scanning region and / or the top scanning region is at different scanning distances. The different scanning distances may include Distance 1 (D1), Distance 2 (D2), Distance 3 (D3) as shown in Figure 8, where D1, D2, D3 respectively indicate that the scanned MV candidate is at Distance 1, Distance 2, or Distance 3 from one side of the current block along a direction perpendicular to the side.

[0289] Further, as shown in Fig. 18, Neighbor 2 is one second neighboring block obtained by scanning a left scanning region of the current block according to the affine construction merge candidate derivation process, and one or more such second neighboring blocks may be obtained by exclusively scanning the left scanning region and / or the top scanning region is at different scanning distances. The different scanning distances may include Distance 1 (D1), Distance 2 (D2), Distance 3 (D3) as shown in Fig. 8, where D1, D2, D3 respectively indicate that the scanned MV candidate is at Distance 1, Distance 2, or Distance 3 from one side of the current block along a direction perpendicular to the side.

[0290] In some embodiments, the processor 2120 may insert a translational MV into the normal merge list in response to determining that the current block is encoded as a normal inter mode, as shown in step 2405. Additionally, the processor 2120 may obtain and insert additional translational MVs into the normal merge list until the normal merge list is full.

[0291] In some embodiments, the processor 2120 may insert multiple CPMVs into the affine merge candidate list or the affine AMVP candidate list in response to determining that the current block is coded as affine mode, as indicated in step 2406. Additionally, the processor 2120 may obtain and insert additional multiple CPMVs into the affine merge candidate list or the affine AMVP candidate list until the affine merge candidate list or the affine AMVP candidate list is full.

[0292] FIG. 25 is a flow chart illustrating a method for video encoding that corresponds to the method for video decoding as shown in FIG.

[0293] In step 2501, on the encoder side, the processor 2120 may obtain one or more first parameters based on a first neighboring block from a plurality of inter-coded neighboring blocks of a current block.

[0294] In step 2502, the processor 2120 may obtain one or more second parameters based on a second neighboring block from the plurality of affine coded neighboring blocks of the current block or from the plurality of historical affine coded neighboring blocks.

[0295] In step 2503, the processor 2120 may construct one or more affine models using the one or more first parameters and the one or more second parameters.

[0296] In step 2504, the processor 2120 may project an affine model onto the current block to obtain a translational MV based on a particular position within the current block in response to determining that the current block is to be coded as a normal inter mode, or to obtain multiple CPMVs for the current block in response to determining that the current block is to be coded as an affine mode.

[0297] In some examples, the one or more first parameters may include a plurality of translational parameters associated with an affine model, and the one or more second parameters may include a plurality of non-translational parameters associated with the affine model.

[0298] In some embodiments, the processor 2120 may obtain multiple inter-coded neighboring blocks from multiple neighboring blocks and multiple non-adjacent blocks, where the multiple neighboring blocks are adjacent to the current block and the multiple non-adjacent blocks are each located a number of blocks away from one side of the current block.

[0299] In some embodiments, the processor 2120 may obtain a plurality of affine-coded neighboring blocks from a plurality of affine-coded neighboring blocks and a plurality of affine-coded non-adjacent blocks.

[0300] In some embodiments, the processor 2120 may obtain a plurality of history affine coded neighboring blocks from a plurality of neighboring blocks and a plurality of non-neighboring blocks.

[0301] In some embodiments, as shown in Figure 18, Neighbor 1 is one first neighboring block obtained by scanning a left scanning region of the current block according to the derivation process for the affine inheritance merge candidate, and one or more such first neighboring blocks may be obtained by exclusively scanning the left scanning region and / or the top scanning region is at different scanning distances. The different scanning distances may include Distance 1 (D1), Distance 2 (D2), Distance 3 (D3) as shown in Figure 8, where D1, D2, D3 respectively indicate that the scanned MV candidate is at Distance 1, Distance 2, or Distance 3 from one side of the current block along a direction perpendicular to the side.

[0302] Further, as shown in Fig. 18, Neighbor 2 is one second neighboring block obtained by scanning a left scanning region of the current block according to the affine construction merge candidate derivation process, and one or more such second neighboring blocks may be obtained by exclusively scanning the left scanning region and / or the top scanning region is at different scanning distances. The different scanning distances may include Distance 1 (D1), Distance 2 (D2), Distance 3 (D3) as shown in Fig. 8, where D1, D2, D3 respectively indicate that the scanned MV candidate is at Distance 1, Distance 2, or Distance 3 from one side of the current block along a direction perpendicular to the side.

[0303] In some embodiments, as shown in step 2505, in response to determining that a translational MV is obtained, the processor 2120 may insert the translational MV into the normal merge list. Additionally, the processor 2120 may obtain and insert an additional translational MV into the normal merge list in response to determining that the current block is to be coded as a normal inter mode until the normal merge list is full. In some embodiments, the processor 2120 selects a candidate from a full normal merge candidate list, i.e., a normal merge candidate list that is fully populated by a candidate, and signals the candidate to a corresponding decoder.

[0304] In some embodiments, as shown in step 2506, the processor 2120 may insert multiple CPMVs into the affine merge candidate list or the affine AMVP candidate list in response to determining that multiple CPMVs are obtained. Additionally, the processor 2120 may obtain and insert additional multiple CPMVs into the affine merge candidate list or the affine AMVP candidate list in response to determining that the current block is coded as an affine mode until the affine merge candidate list or the affine AMVP candidate list is full. In some embodiments, the processor 2120 may select a candidate from a full affine merge candidate list, i.e., an affine merge candidate list that is fully populated by the candidate, or a full affine AMVP candidate list, i.e., an affine AMVP candidate list that is fully populated by the candidate, and may signal the candidate to a corresponding decoder.

[0305] In some embodiments, the encoder and decoder may each construct a full-affine merge candidate list or a full-affine AMVP candidate list, and the decoder may receive a candidate index indicating a candidate in the full-affine merge list or the full-affine AMVP candidate list sent by the encoder. According to the received candidate index, the decoder may obtain the candidate (i.e., CPMV) indicated by the candidate index.

[0306] FIG. 26 is a flowchart illustrating a method for video decoding according to an embodiment of the present disclosure.

[0307] In step 2601, on the decoder side, the processor 2120 may obtain one or more MV candidates from one or more candidate lists according to a predetermined order, where the one or more candidate lists may include an affine AMVP candidate list, a normal merge candidate list, and an affine merge candidate list, and the one or more MV candidates are from multiple adjacent blocks to the current block.

[0308] In some embodiments, the one or more MV candidates may include MV candidates of multiple categories, and different MV candidates in the same category are not adjacently located in the one or more candidate lists.

[0309] In some embodiments, the one or more MV candidates may include MV candidates of multiple categories, and the predetermined order is obtained based on some MV candidates in each category.

[0310] In some examples, the multiple categories of MV candidates may include SbTMVP candidates, multiple inheritance candidates from adjacent blocks, multiple construction candidates from adjacent blocks, multiple inheritance candidates from non-adjacent blocks including a first number of inheritance candidates from the non-adjacent blocks and the remaining inheritance candidates from the non-adjacent blocks, multiple construction candidates from a first type of construction candidates from non-adjacent blocks including a third number of construction candidates from the first type of construction candidates and the remaining construction candidates from the first type of construction candidates, and multiple construction candidates from a second type of construction candidates from non-adjacent blocks including a second number of construction candidates from the second type of construction candidates and the remaining construction candidates from the second type of construction candidates.

[0311] For example, the overall process according to Fig. 22 or Fig. 23 may be used to derive a first type of affine candidate that is constructed from non-adjacent spatial neighbors (e.g., sub-blocks located near three corner points A, B, and C are non-adjacent spatial neighbors). Furthermore, the overall process according to Fig. 24 or Fig. 25 may be used to derive a second type of affine candidate that is constructed from non-adjacent spatial neighbors (e.g., non-translational model parameters may be inherited from non-adjacent spatial neighbors).

[0312] Moreover, the processor 2120 may further obtain from the candidate list a plurality of inheritance candidates from adjacent blocks after the SbTMVP candidate, may obtain from the candidate list a plurality of construction candidates from adjacent blocks after the plurality of inheritance candidates from the adjacent blocks, may obtain from the candidate list a first number of inheritance candidates from non-adjacent blocks after the plurality of construction candidates from the adjacent blocks, may obtain from the candidate list a second number of construction candidates from a second type of construction candidate after the first number of inheritance candidates from the non-adjacent blocks, may obtain from the candidate list a third number of construction candidates from the first type of construction candidate after the second number of construction candidates from the second type of construction candidate, may obtain from the candidate list a remaining inheritance candidate from a non-adjacent block after the third number of construction candidates from the first type of construction candidate, may obtain from the candidate list a remaining construction candidate from the second type of construction candidate after the remaining inheritance candidate from the non-adjacent block, or may obtain from the candidate list a remaining construction candidate from the first type of construction candidate after the remaining construction candidate from the second type of construction candidate.

[0313] For example, the predefined order for arranging the MV candidates may be defined as follows: (1) SbTMVP, if a candidate is available; (2) candidates for inheritance from adjacent neighbors; (3) Construction candidates from adjacent neighborhoods, (4) the first X succession candidates from non-adjacent neighborhoods (e.g., X can be a pre-fixed number such as 1 or a signaled number); (5) the first Y building candidates from a second type of building candidates from a non-adjacent neighborhood (e.g., Y may be defined similarly to X); (6) the first Z building candidates from the first type of building candidates from the non-adjacent neighborhood (e.g., Z may be defined similarly to X); (7) remaining succession candidates from non-adjacent neighborhoods (e.g., X can be a pre-fixed number such as 1 or a signaled number); (8) remaining construction candidates from the second type of construction candidates from non-adjacent neighborhoods (e.g., Y may be defined similarly to X); (9) remaining construction candidates from the first type of construction candidates from non-adjacent neighborhoods (e.g., Z may be defined similarly to X); and (10) Zero MV.

[0314] In the above example, the first number is X, the second number is Y, and the third number is Z.

[0315] In some embodiments, the first number, the second number, and the third number may be obtained in one of the following manners: predefining the first number, the second number, and the third number as fixed values, respectively, or receiving the first number, the second number, and the third number signaled at some level by the encoder.

[0316] In some embodiments, the processor 2120 may obtain MV candidates from the candidate list according to a predetermined order based on one or more numbers of MV candidates in each category, and the one or more numbers of MV candidates in each category are arranged in an interleaved manner.

[0317] In some embodiments, the multiple categories of MV candidates may include an SbTMVP candidate, a multiple inheritance candidate from adjacent blocks, a multiple construction candidate from adjacent blocks including Xi construction candidates from the adjacent blocks, a multiple inheritance candidate from non-adjacent blocks including Zi inheritance candidates from the non-adjacent blocks, a multiple construction candidate from a first type of construction candidate from non-adjacent blocks including Ki construction candidates from the first type of construction candidate, and a multiple construction candidate from a second type of construction candidate from non-adjacent blocks including Yi construction candidates from the second type of construction candidate.

[0318] Further, the multiple categories may include multiple sets of categories, each set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki construction candidates from a first type of construction candidates, and Yi construction candidates from a second type of construction candidates, where Xi, Yi, Zi, and Ki are each positive integers, and i is a positive integer greater than 2.

[0319] Moreover, the processor 2120 may further obtain MV candidates from a second / later set of categories from the candidate list, the second / later set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki-1 construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates after the first / previous set of categories including Xi-1 construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates.

[0320] In some embodiments, the processor 2120 may further obtain construction candidates from adjacent blocks in a set of categories including X1 construction candidates from adjacent blocks, Z1 inheritance candidates from non-adjacent blocks, K1 construction candidates from a first type of construction candidate, and Y1 construction candidates from a second type of construction candidate after one or more inheritance candidates from adjacent blocks, or may obtain one or more inheritance candidates from adjacent blocks after the SbTMVP candidate.

[0321] In some embodiments, the value of i may be 2, 3, or more. Although 3 is taken as an example, the value of i is not limited to 3. For example, the multiple categories of MV candidates may include an SbTMVP candidate, multiple inheritance candidates from adjacent blocks, multiple construction candidates from adjacent blocks including X1 construction candidates of the adjacent block, X2 construction candidates from the adjacent block, and X3 construction candidates from the adjacent block, multiple inheritance candidates from non-adjacent blocks including Z1 inheritance candidates from the non-adjacent block, Z2 inheritance candidates from the non-adjacent block, and Z3 inheritance candidates from the non-adjacent block, multiple construction candidates from a first type of construction candidate from non-adjacent blocks including K1 construction candidates from a first type of construction candidate, K2 construction candidates from the first type of construction candidate, and K3 construction candidates from the first type of construction candidate, and multiple construction candidates from a second type of construction candidate from non-adjacent blocks including Y1 construction candidate from a second type of construction candidate, a first Y2 construction candidate from the second type of construction candidate block, and Y3 construction candidate from the second type of construction candidate.

[0322] Further, the multiple categories may include a first set of categories including X1 construction candidates from adjacent blocks, Z1 inheritance candidates from non-adjacent blocks, K1 construction candidates from a first type of construction candidate, and Y1 construction candidates from a second type of construction candidate; a second set of categories including X2 construction candidates from adjacent blocks, Z2 inheritance candidates from non-adjacent blocks, K2 construction candidates from the first type of construction candidate, and Y2 construction candidates from the second type of construction candidate; and a third set of categories including X3 construction candidates from adjacent blocks, Z3 inheritance candidates from non-adjacent blocks, K3 construction candidates from the first type of construction candidate, and Y3 construction candidates from the second type of construction candidate.

[0323] Moreover, the processor 2120 may further obtain, from the candidate list, an MV candidate from a second set of categories that is disposed after the first set of categories, from the candidate list, an MV candidate from a third set of categories that is disposed after the second set of categories, obtain a construction candidate from an adjacent block from the first set of categories after one or more inheritance candidates from the adjacent block, or obtain one or more inheritance candidates from an adjacent block after the SbTMVP candidate.

[0324] In one embodiment, the interleaved scheme may be defined as follows: (1) SbTMVP candidates, if available; (2) candidates for inheritance from adjacent neighbors; (3) the first X1 construction candidates from adjacent neighborhoods; (4) the first Y1 construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (5) the first Z1 succession candidates from non-adjacent neighborhoods, (6) the first K1 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (7) the first X2 construction candidates from adjacent neighborhoods, (8) the first Y2 construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (9) the first Z2 succession candidates from non-adjacent neighborhoods, (10) the first K2 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (11)... (12) the first Xi construction candidates from adjacent neighborhoods; (13) the first Yi construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (14) the first Zi succession candidates from non-adjacent neighborhoods, (15) the first Ki construction candidates from the first type of construction candidates from the non-adjacent neighborhood; and (16) Zero MV.

[0325] In some embodiments, Xi, Yi, Zi, and Ki are obtained in one of the following ways: by predefining Xi, Yi, Zi, and Ki as fixed values, respectively, or by receiving Xi, Yi, Zi, and Ki signaled at any level by the encoder.

[0326] In some embodiments, in response to determining that the number of available MV candidates in one category is less than the other categories, the processor 2120 may obtain available MV candidates in the other categories to be placed in one or more positions of the one category. For example, if the number of available candidates in one category is less than the other categories, a candidate position for this category is skipped and the remaining available candidates of the other categories take over this position.

[0327] In some embodiments, the processor 2120 may retrieve MV candidates from the candidate list according to a predetermined order based on available MV candidates in one or more categories, with MV candidates in other categories being placed in the candidate list in an interleaved manner.

[0328] For example, a combined version of the above that takes into account both availability and interleaving methods may be defined as follows: (1) SbTMVP candidates, if available; (2) candidates for inheritance from adjacent neighbors; (3) the first X1 construction candidates from adjacent neighborhoods; (4) all available construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (5) remaining construction candidates from adjacent neighborhoods; (6) the first Z1 succession candidates from non-adjacent neighborhoods, (7) the first K1 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (8)... (9) the first Zi succession candidates from non-adjacent neighborhoods, (10) the first Ki construction candidates from the first type of construction candidates from the non-adjacent neighborhood; and (11) Zero MV.

[0329] At step 2602, the processor 2120 may obtain one or more CPMVs for the current block based on the one or more MV candidates.

[0330] FIG. 27 is a flow chart illustrating a method for video encoding that corresponds to the method for video decoding as shown in FIG.

[0331] In step 2701, the processor 2120 on the encoder side may obtain one or more MV candidates from one or more candidate lists according to a predetermined order, where the one or more candidate lists may include an affine AMVP candidate list, a normal merge candidate list, and an affine merge candidate list, and the one or more MV candidates are from multiple adjacent blocks to the current block.

[0332] In some embodiments, the one or more MV candidates may include MV candidates of multiple categories, and different MV candidates in the same category are not adjacently located in the one or more candidate lists.

[0333] In some embodiments, the one or more MV candidates may include multiple categories of MV candidates, and the predetermined order is obtained based on the number of MV candidates in each category.

[0334] In some examples, the multiple categories of MV candidates may include SbTMVP candidates, multiple inheritance candidates from adjacent blocks, multiple construction candidates from adjacent blocks, multiple inheritance candidates from non-adjacent blocks including a first number of inheritance candidates from the non-adjacent blocks and the remaining inheritance candidates from the non-adjacent blocks, multiple construction candidates from a first type of construction candidates from non-adjacent blocks including a third number of construction candidates from the first type of construction candidates and the remaining construction candidates from the first type of construction candidates, and multiple construction candidates from a second type of construction candidates from non-adjacent blocks including a second number of construction candidates from the second type of construction candidates and the remaining construction candidates from the second type of construction candidates.

[0335] Moreover, the processor 2120 may further obtain from the candidate list a plurality of inheritance candidates from adjacent blocks after the SbTMVP candidate, may obtain from the candidate list a plurality of construction candidates from adjacent blocks after the plurality of inheritance candidates from the adjacent blocks, may obtain from the candidate list a first number of inheritance candidates from non-adjacent blocks after the plurality of construction candidates from the adjacent blocks, may obtain from the candidate list a second number of construction candidates from a second type of construction candidate after the first number of inheritance candidates from the non-adjacent blocks, may obtain from the candidate list a third number of construction candidates from the first type of construction candidate after the second number of construction candidates from the second type of construction candidate, may obtain from the candidate list a remaining inheritance candidate from a non-adjacent block after the third number of construction candidates from the first type of construction candidate, may obtain from the candidate list a remaining construction candidate from the second type of construction candidate after the remaining inheritance candidate from the non-adjacent block, or may obtain from the candidate list a remaining construction candidate from the first type of construction candidate after the remaining construction candidate from the second type of construction candidate.

[0336] For example, the predefined order for arranging the MV candidates may be defined as follows: (1) SbTMVP candidates, if available; (2) candidates for inheritance from adjacent neighbors; (3) Construction candidates from adjacent neighborhoods, (4) the first X succession candidates from non-adjacent neighborhoods (e.g., X can be a pre-fixed number such as 1 or a signaled number); (5) the first Y building candidates from a second type of building candidates from a non-adjacent neighborhood (e.g., Y may be defined similarly to X); (6) the first Z building candidates from the first type of building candidates from the non-adjacent neighborhood (e.g., Z may be defined similarly to X); (7) remaining succession candidates from non-adjacent neighborhoods (e.g., X can be a pre-fixed number such as 1 or a signaled number); (8) remaining construction candidates from the second type of construction candidates from non-adjacent neighborhoods (e.g., Y may be defined similarly to X); (9) remaining construction candidates from the first type of construction candidates from non-adjacent neighborhoods (e.g., Z may be defined similarly to X); and (10) Zero MV.

[0337] In the above example, the first number is X, the second number is Y, and the third number is Z.

[0338] In some embodiments, the first number, the second number, and the third number may be obtained in one of the following manners: predefining the first number, the second number, and the third number as fixed values, respectively, or signaling the first number, the second number, and the third number signaled at a level.

[0339] In some embodiments, the processor 2120 may insert the MV candidates into the candidate list according to a predetermined order based on the one or more number of MV candidates in each category, where the one or more number of MV candidates in each category are arranged in an interleaved manner.

[0340] In some embodiments, the multiple categories of MV candidates may include an SbTMVP candidate, a multiple inheritance candidate from adjacent blocks, a multiple construction candidate from adjacent blocks including Xi construction candidates from the adjacent blocks, a multiple inheritance candidate from non-adjacent blocks including Zi inheritance candidates from the non-adjacent blocks, a multiple construction candidate from a first type of construction candidate from non-adjacent blocks including Ki construction candidates from the first type of construction candidate, and a multiple construction candidate from a second type of construction candidate from non-adjacent blocks including Yi construction candidates from the second type of construction candidate.

[0341] Further, the multiple categories may include multiple sets of categories, each set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki construction candidates from a first type of construction candidates, and Yi construction candidates from a second type of construction candidates, where Xi, Yi, Zi, and Ki are each positive integers, and i is a positive integer greater than 2.

[0342] Moreover, the processor 2120 may further insert into the candidate list MV candidates from a second / later set of categories, the second / later set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki-1 construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates after a first / previous set of categories including Xi-1 construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates.

[0343] In some embodiments, the processor 2120 may further insert into the candidate list one or more inheritance candidates from adjacent blocks in the set of categories, including X1 construction candidates of adjacent blocks, Z1 inheritance candidates from non-adjacent blocks, K1 construction candidates from the first type of construction candidates, and Y1 construction candidates from the second type of construction candidates, insert into the candidate list one or more inheritance candidates from adjacent blocks after the SbTMVP candidate, or insert into the candidate list one or more zero MV candidates after the MV candidate from the last set of categories in response to determining that the candidate list is not full.

[0344] In some embodiments, the value of i may be 2, 3, or more. Although 3 is taken as an example, the value of i is not limited to 3. For example, the multiple categories of MV candidates may include an SbTMVP candidate, multiple inheritance candidates from adjacent blocks, multiple construction candidates from adjacent blocks including X1 construction candidates of the adjacent block, X2 construction candidates from the adjacent block, and X3 construction candidates from the adjacent block, multiple inheritance candidates from non-adjacent blocks including Z1 inheritance candidates from the non-adjacent block, Z2 inheritance candidates from the non-adjacent block, and Z3 inheritance candidates from the non-adjacent block, multiple construction candidates from a first type of construction candidate from non-adjacent blocks including K1 construction candidates from a first type of construction candidate, K2 construction candidates from the first type of construction candidate, and K3 construction candidates from the first type of construction candidate, and multiple construction candidates from a second type of construction candidate from non-adjacent blocks including Y1 construction candidate from a second type of construction candidate, a first Y2 construction candidate from the second type of construction candidate block, and Y3 construction candidate from the second type of construction candidate.

[0345] Further, the multiple categories may include a first set of categories including X1 construction candidates from adjacent blocks, Z1 inheritance candidates from non-adjacent blocks, K1 construction candidates from a first type of construction candidate, and Y1 construction candidates from a second type of construction candidate; a second set of categories including X2 construction candidates from adjacent blocks, Z2 inheritance candidates from non-adjacent blocks, K2 construction candidates from the first type of construction candidate, and Y2 construction candidates from the second type of construction candidate; and a third set of categories including X3 construction candidates from adjacent blocks, Z3 inheritance candidates from non-adjacent blocks, K3 construction candidates from the first type of construction candidate, and Y3 construction candidates from the second type of construction candidate.

[0346] Moreover, processor 2120 may further insert into the candidate list an MV candidate from a second set of categories that is disposed after the first set of categories, may insert into the candidate list an MV candidate from a third set of categories that is disposed after the second set of categories, may insert into the candidate list a construction candidate from an adjacent block from the first set of categories after the one or more inheritance candidates from the adjacent block, or may insert into the candidate list one or more inheritance candidates from an adjacent block after the SbTMVP candidate.

[0347] In one embodiment, the interleaved scheme may be defined as follows: (1) SbTMVP candidates, if available; (2) candidates for inheritance from adjacent neighbors; (3) the first X1 construction candidates from adjacent neighborhoods; (4) the first Y1 construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (5) the first Z1 succession candidates from non-adjacent neighborhoods, (6) the first K1 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (7) the first X2 construction candidates from adjacent neighborhoods, (8) the first Y2 construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (9) the first Z2 succession candidates from non-adjacent neighborhoods, (10) the first K2 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (11)... (12) the first Xi construction candidates from adjacent neighborhoods; (13) the first Yi construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (14) the first Zi succession candidates from non-adjacent neighborhoods, (15) the first Ki construction candidates from the first type of construction candidates from the non-adjacent neighborhood; and (16) Zero MV.

[0348] In some embodiments, Xi, Yi, Zi, and Ki are obtained in one of the following ways: by predefining Xi, Yi, Zi, and Ki as fixed values, respectively, or by receiving Xi, Yi, Zi, and Ki signaled at any level by the encoder.

[0349] In some embodiments, in response to determining that the number of available MV candidates in one category is less than the other categories, the processor 2120 may obtain available MV candidates in the other categories to be placed in one or more positions of the one category. For example, if the number of available candidates in one category is less than the other categories, a candidate position for this category is skipped and the remaining available candidates of the other categories take over this position.

[0350] In some embodiments, the processor 2120 may insert MV candidates into the candidate list according to a predetermined order based on available MV candidates in one or more categories, with MV candidates in other categories being arranged in the candidate list in an interleaved manner.

[0351] For example, a combined version of the above that takes into account both availability and interleaving methods may be defined as follows: (1) SbTMVP candidates, if available; (2) candidates for inheritance from adjacent neighbors; (3) the first X1 construction candidates from adjacent neighborhoods; (4) all available construction candidates from the second type of construction candidates from non-adjacent neighborhoods; (5) remaining construction candidates from adjacent neighborhoods; (6) the first Z1 succession candidates from non-adjacent neighborhoods, (7) the first K1 construction candidates from the first type of construction candidates from non-adjacent neighborhoods; (8)... (9) the first Zi succession candidates from non-adjacent neighborhoods, (10) the first Ki construction candidates from the first type of construction candidates from the non-adjacent neighborhood; and (11) Zero MV.

[0352] At step 2702, the processor 2120 may obtain one or more CPMVs for the current block based on the one or more MV candidates.

[0353] In some embodiments, an apparatus for video decoding is provided, the apparatus including a processor 2120 and a memory 2140 configured to store instructions executable by the processor, the processor being configured to, upon execution of the instructions, perform any of the methods as illustrated in Figures 22, 24, and 26.

[0354] In some embodiments, an apparatus for video encoding is provided, the apparatus including a processor 2120 and a memory 2140 configured to store instructions executable by the processor, the instructions being configured to, when executed, perform any of the methods as illustrated in Figures 23, 25 and 27.

[0355] In some other embodiments, a non-transitory computer-readable storage medium having instructions stored thereon is provided. When the instructions are executed by the processor 2120, the instructions cause the processor to perform any of the methods illustrated in Figures 22-27. In one embodiment, a number of programs may be executed by the processor 2120 in the computing environment 2110 to receive (e.g., from the video encoder 20 in Figure 2) a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) and to perform the decoding method described above in accordance with the received bitstream or data stream. In another embodiment, the programs may be executed by the processor 2120 in the computing environment 2110 to perform the encoding method described above for encoding video information (e.g., video blocks representing video frames and / or one or more associated syntax elements, etc.) into a bitstream or data stream and may also be executed by the processor 2120 in the computing environment 2110 to transmit the bitstream or data stream (e.g., to the video decoder 30 in FIG. 3). Alternatively, the non-transitory computer-readable storage medium may store therein a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.) generated by an encoder (e.g., the video encoder 20 in FIG. 2) using the encoding method described above, for use by a decoder (e.g., the video decoder 30 in FIG. 3) in decoding video data. The non-transitory computer-readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, or an optical data storage device, etc.

[0356] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure, including such departures from the disclosure, which adhere to its general principles as come within known or customary practice in the art. It is intended that the specification and examples be considered as illustrative only.

[0357] It will be appreciated that the present disclosure is not limited to the exact embodiments described above and illustrated in the accompanying drawings, but various modifications and changes can be made without departing from the scope thereof.

Claims

1. 1. A method for video decoding, comprising: obtaining, by a decoder, a third candidate position based on the first candidate position and the second candidate position; obtaining, by the decoder, a virtual block based on the first candidate location, the second candidate location, and the third candidate location; obtaining, by the decoder, a plurality of first control point motion vectors (CPMVs) for the virtual block based on translational motion vectors (MVs) at the first candidate position, the second candidate position, and the third candidate position; projecting, by the decoder, the plurality of first CPMVs of the virtual block onto a current block to obtain a translational CPMV based on a particular position within the current block in response to determining that the current block is to be coded as a normal inter mode, or to obtain a plurality of second CPMVs for the current block in response to determining that the current block is to be coded as an affine mode; A method comprising:

2. in response to determining, by the decoder, that the current block is to be coded as a regular inter mode, inserting the translational MV into a regular merge list; responsive to determining, by the decoder, that the current block is to be coded as an affine mode, inserting the second plurality of CPMVs into an affine merge candidate list or an affine advanced motion vector prediction (AMVP) candidate list; The method of claim 1 further comprising:

3. acquiring, by the decoder, additional translational MVs and inserting them into the normal merge list until the normal merge list is full; obtaining, by the decoder, an additional number of second CPMVs and inserting them into the affine merge candidate list or the affine AMVP candidate list until the affine merge candidate list or the affine AMVP candidate list is full; The method of claim 2 , further comprising:

4. The method of claim 1 , wherein the particular location comprises a center location within the current block.

5. The method of claim 1 , wherein the virtual block is a rectangular coding block, and the third candidate location is determined based on a vertical position of the first candidate location and a horizontal position of the second candidate location.

6. obtaining, by the decoder, the first candidate position according to a first scanning distance and a first scanning area, where the first scanning distance indicates a first distance between the first candidate position and a left side of the current block, and the first scanning area is located to the left of the current block; obtaining, by the decoder, the second candidate position according to a second scanning distance and a second scanning area, where the second scanning distance indicates a second distance between the second candidate position and a top side of the current block, and the second scanning area is located above the current block; The method of claim 1 further comprising:

7. 1. A method of video decoding, comprising: obtaining, by a decoder, one or more first parameters based on a first neighboring block from a plurality of inter-coded neighboring blocks of the current block; obtaining, by the decoder, one or more second parameters based on a second neighboring block from a plurality of affine coded neighboring blocks of the current block or from a plurality of historical affine coded neighboring blocks; constructing, by the decoder, an affine model using the one or more first parameters and the one or more second parameters; projecting, by the decoder, the affine model onto a current block to obtain a translational MV based on a particular position within the current block in response to determining that the current block is to be coded as a regular inter mode, or to obtain a plurality of CPMVs for the current block in response to determining that the current block is to be coded as an affine mode; A method comprising:

8. in response to determining, by the decoder, that the current block is to be coded as a regular inter mode, inserting the translational MV into a regular merge list; responsive to determining, by the decoder, that the current block is to be coded as an affine mode, inserting the plurality of CPMVs into an affine merge candidate list or an affine advanced motion vector prediction (AMVP) candidate list; The method of claim 7 further comprising:

9. acquiring, by the decoder, additional translational MVs and inserting them into the normal merge list until the normal merge list is full; obtaining, by the decoder, additional CPMVs and inserting them into the affine merge candidate list or the affine AMVP candidate list until the affine merge candidate list or the affine AMVP candidate list is full; The method of claim 8 further comprising:

10. 8. The method of claim 7, wherein the one or more first parameters include a plurality of translational parameters associated with the affine model, and the one or more second parameters include a plurality of non-translational parameters associated with the affine model.

11. obtaining, by the decoder, the plurality of inter-coded neighboring blocks from a plurality of neighboring blocks and a plurality of non-adjacent blocks, the plurality of neighboring blocks being adjacent to the current block, and the plurality of non-adjacent blocks each being located a number of blocks away from one side of the current block; obtaining, by the decoder, the plurality of affine-coded adjacent blocks from the plurality of affine-coded adjacent blocks and the plurality of affine-coded non-adjacent blocks; obtaining, by the decoder, the plurality of history affine coded neighboring blocks from the plurality of neighboring blocks and the plurality of non-adjacent blocks; The method of claim 7 further comprising:

12. 1. A method for video decoding, comprising: Obtaining, by a decoder, one or more motion vector (MV) candidates from one or more candidate lists according to a predetermined order, the one or more candidate lists including an affine advanced motion vector prediction (AMVP) candidate list, a normal merge candidate list, and an affine merge candidate list, the one or more MV candidates being from a plurality of neighboring blocks to a current block; obtaining, by the decoder, one or more control point motion vectors (CPMVs) for the current block based on the one or more MV candidates; A method comprising:

13. the one or more MV candidates include MV candidates of a plurality of categories; Different MV candidates in the same category are not placed adjacently in said one or more candidate lists. The method of claim 12.

14. the one or more MV candidates include MV candidates of a plurality of categories; The predetermined order is obtained based on the number of MV candidates in each category. The method of claim 12.

15. The MV candidates for the plurality of categories are Multiple building candidates from adjacent blocks; a plurality of succession candidates from non-adjacent blocks, including a first number of succession candidates from the non-adjacent blocks and remaining succession candidates from the non-adjacent blocks; a plurality of construction candidates from the first type of construction candidates from non-adjacent blocks including a third number of construction candidates from the first type of construction candidates and remaining construction candidates from the first type of construction candidates; a plurality of construction candidates from the second type of construction candidates from non-adjacent blocks including a second number of construction candidates from the second type of construction candidates and remaining construction candidates from the second type of construction candidates; obtaining, by the decoder, from a candidate list, a first number of succession candidates from the non-adjacent blocks after the plurality of construction candidates from adjacent blocks; obtaining, by the decoder, from the candidate list, the second number of construction candidates from the second type of construction candidates after the first number of inheritance candidates from the non-adjacent blocks; obtaining, by the decoder, from the candidate list, the third number of construction candidates from the first type of construction candidates subsequent to the second number of construction candidates from the second type of construction candidates; obtaining, by the decoder, from the candidate list, remaining succession candidates from the non-adjacent blocks after the third number of construction candidates from the first type of construction candidates; obtaining, by the decoder, from the candidate list, the remaining construction candidates from the second type construction candidates after the remaining inheritance candidates from the non-adjacent blocks; or obtaining, by the decoder, from the candidate list, the remaining construction candidates from the first type of construction candidates after the remaining construction candidates from the second type of construction candidates; The method of claim 14 further comprising:

16. The MV candidates for the plurality of categories are Sub-block-based temporal motion vector prediction (SbTMVP) candidates; a plurality of succession candidates from adjacent blocks; The method comprises the steps of: obtaining, by the decoder, from the candidate list, a number of successor candidates from the neighboring blocks following the SbTMVP candidate; or obtaining, by the decoder, from the candidate list, a number of construction candidates from the neighboring blocks after a number of inheritance candidates from the neighboring blocks; The method of claim 15 further comprising one of:

17. The first number, the second number, and the third number are expressed in the following manner: predefining the first number, the second number, and the third number as fixed values, respectively; or receiving the first number, the second number, and the third number signaled at a level by an encoder; The method of claim 15, wherein the method is obtained in one of the following ways:

18. 15. The method of claim 14, further comprising: obtaining, by the decoder, MV candidates from the candidate list according to the predetermined order based on one or more numbers of MV candidates in each category, the one or more numbers of MV candidates in each category being arranged in an interleaved manner.

19. The MV candidates for the plurality of categories are A number of construction candidates from adjacent blocks, including Xi construction candidates from the adjacent blocks; a plurality of succession candidates from non-adjacent blocks, including Zi succession candidates from non-adjacent blocks; A number of construction candidates from a first type of construction candidate from a non-adjacent block including Ki construction candidates from the first type of construction candidate; A plurality of construction candidates from the second type of construction candidates from non-adjacent blocks, including Yi construction candidates from the second type of construction candidates; The plurality of categories includes a plurality of sets of categories, each set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates; Xi, Yi, Zi, and Ki are each a positive integer, i being a positive integer greater than 2; The method comprises the steps of: obtaining, by the decoder, from the candidate list, MV candidates from a second set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki-1 construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates, after a first set of categories including Xi-1 construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki-1 construction candidates from the first type of construction candidates, and Yi-1 construction candidates from the second type of construction candidates; The method of claim 18 , further comprising one of:

20. The MV candidates for the plurality of categories are Sub-block-based temporal motion vector prediction (SbTMVP) candidates; a plurality of succession candidates from adjacent blocks; The method comprises the steps of: obtaining, by the decoder, construction candidates from neighboring blocks in a set of categories including X construction candidates from neighboring blocks, Z construction candidates from non-neighboring blocks, K construction candidates from the first type of construction candidates, and Y construction candidates from the second type of construction candidates, after one or more inheritance candidates from neighboring blocks; or obtaining, by the decoder, the one or more successor candidates from adjacent blocks following the SbTMVP candidate; 20. The method of claim 19, further comprising one of:

21. Xi, Yi, Zi, and Ki are expressed by the following formula: predefining Xi, Yi, Zi, and Ki as fixed values, or receiving Xi, Yi, Zi, and Ki signaled by an encoder; 20. The method of claim 19, wherein the method is obtained in one of the following ways:

22. 20. The method of claim 19, further comprising, in response to determining that a number of available MV candidates in one category is less than another category, obtaining, by the decoder, available MV candidates in the other category located at one or more positions in the one category.

23. 15. The method of claim 14, further comprising: obtaining, by the decoder, MV candidates from a candidate list according to a predetermined order based on available MV candidates in one or more categories, and MV candidates in other categories are placed in the candidate list in an interleaved manner.

24. 1. A method for video encoding, comprising: obtaining, by the encoder, a third candidate position based on the first candidate position and the second candidate position; obtaining, by the encoder, a virtual block based on the first candidate location, the second candidate location, and the third candidate location; obtaining, by the encoder, a plurality of first control point motion vectors (CPMVs) for the virtual block based on translational motion vectors (MVs) at the first candidate position, the second candidate position, and the third candidate position; projecting the first plurality of CPMVs for the virtual block onto the current block in response to determining, by the encoder, that the current block is to be coded as a normal inter mode, to obtain a translational MV based on a particular position within the current block, or in response to determining, by the encoder, that the current block is to be coded as an affine mode, to obtain a second plurality of CPMVs for the current block; A method comprising:

25. In response to determining, by the encoder, that the translation MV is obtained, inserting the translation MV into a regular merge list; responsive to determining, by the encoder, that the second CPMVs are obtained, inserting the second CPMVs into an affine merge candidate list or an affine advanced motion vector prediction (AMVP) candidate list; 25. The method of claim 24, further comprising:

26. The method of claim 24 , wherein the particular location comprises a center location within the current block.

27. 25. The method of claim 24, wherein the virtual block is a rectangular coding block, and the third candidate location is determined based on a vertical position of the first candidate location and a horizontal position of the second candidate location.

28. Obtaining, by the encoder, the first candidate position according to a first scanning distance and a first scanning area, where the first scanning distance indicates a first distance between the first candidate position and a left side of the current block, and the first scanning area is located to the left of the current block; obtaining, by the encoder, the second affine candidate according to a second scanning distance and a second scanning region, where the second scanning distance indicates a second distance between the second candidate position and a top side of the current block, and the second scanning region is located above the current block; 27. The method of claim 26, further comprising:

29. 1. A method for video encoding, comprising: obtaining, by an encoder, one or more first parameters based on a first neighboring block from a plurality of inter-coded neighboring blocks of a current block; obtaining, by the encoder, one or more second parameters based on a second neighboring block from a plurality of affine-coded neighboring blocks of the current block or from a plurality of historical affine-coded neighboring blocks; constructing, by the encoder, an affine model using the one or more first parameters and the one or more second parameters; projecting, by the encoder, the affine model onto the current block to obtain a translational MV based on a particular position within the current block in response to determining that the current block is to be coded as a regular inter mode, or to obtain a number of CPMVs for the current block in response to determining that the current block is to be coded as an affine mode. A method comprising:

30. in response to determining, by the encoder, that the current block is to be coded as a regular inter mode, inserting the translational MV into a regular merge list; in response to determining, by the encoder, that the current block is to be coded as an affine mode, inserting the plurality of CPMVs into an affine merge candidate list or an affine advanced motion vector prediction (AMVP) candidate list; 30. The method of claim 29, further comprising:

31. 31. The method of claim 30, further comprising, in response to the encoder determining that the current block is to be coded as a regular inter mode, obtaining and inserting an additional translational MV into the regular merge candidate list until the regular merge candidate list is full.

32. 32. The method of claim 31, further comprising selecting, by the encoder, a candidate from a full regular merge candidate list and signaling the candidate to a decoder.

33. 31. The method of claim 30, further comprising, in response to the encoder determining that the current block is to be encoded in affine mode, obtaining and inserting additional CPMVs into the affine merge candidate list or the affine AMVP candidate list until the affine merge candidate list or the affine AMVP candidate list is full.

34. 34. The method of claim 33, further comprising selecting, by the encoder, a candidate from a full-affine merge candidate list or a full-affine AMVP candidate list and signaling the candidate to a decoder.

35. 30. The method of claim 29, wherein the one or more first parameters include a plurality of translational parameters associated with the affine model, and the one or more second parameters include a plurality of non-translational parameters associated with the affine model.

36. obtaining, by the encoder, the plurality of inter-coded neighboring blocks from a plurality of neighboring blocks and a plurality of non-adjacent blocks, the plurality of neighboring blocks being adjacent to the current block, and the plurality of non-adjacent blocks being each located a number of blocks away from one side of the current block; obtaining, by the encoder, the plurality of affine coded adjacent blocks from the plurality of affine coded adjacent blocks and the plurality of affine coded non-adjacent blocks; obtaining, by the encoder, the plurality of history affine encoded neighboring blocks from the plurality of neighboring blocks and the plurality of non-adjacent blocks; 30. The method of claim 29, further comprising:

37. 1. A method for video encoding, comprising: Obtaining, by an encoder, one or more motion vector (MV) candidates from one or more candidate lists according to a predetermined order, the one or more candidate lists including an affine advanced motion vector prediction (AMVP) candidate list, a normal merge candidate list, and an affine merge candidate list, the one or more MV candidates being from a plurality of neighboring blocks to a current block; obtaining, by the encoder, one or more control point motion vectors (CPMVs) for the current block based on the one or more MV candidates; A method comprising:

38. the one or more MV candidates include MV candidates of a plurality of categories; Different MV candidates in the same category are not placed adjacently in said one or more candidate lists.

38. The method of claim 37.

39. the one or more MV candidates include MV candidates of a plurality of categories; The predetermined order is obtained based on the number of MV candidates in each category.

38. The method of claim 37.

40. The MV candidates for the plurality of categories are Multiple building candidates from adjacent blocks; a plurality of succession candidates from non-adjacent blocks, including a first number of succession candidates from the non-adjacent blocks and remaining succession candidates from the non-adjacent blocks; a plurality of construction candidates from the first type of construction candidates from non-adjacent blocks including a third number of construction candidates from the first type of construction candidates and remaining construction candidates from the first type of construction candidates; a plurality of construction candidates from the second type of construction candidates from non-adjacent blocks including a second number of construction candidates from the second type of construction candidates and remaining construction candidates from the second type of construction candidates; The method comprises the steps of: inserting, by the encoder, into a candidate list a number of successor candidates from the neighboring blocks after the SbTMVP candidate; inserting, by the encoder, into the candidate list a plurality of construction candidates from the neighboring blocks after a plurality of inheritance candidates from the neighboring blocks; inserting, by the encoder, into the candidate list a first number of successor candidates from the non-adjacent block after a plurality of construction candidates from the adjacent blocks; inserting, by the encoder, into the candidate list the second number of construction candidates from the second type of construction candidates after the first number of inheritance candidates from the non-adjacent block; inserting, by the encoder, into the candidate list the third number of building candidates from the first type of building candidates after the second number of building candidates from the second type of building candidates; inserting, by the encoder, into the candidate list remaining successor candidates from the non-adjacent blocks after the third number of construction candidates from the first type of construction candidates; inserting, by the encoder, into the candidate list a remaining construction candidate from the second type construction candidate after a remaining inheritance candidate from the non-adjacent block; or inserting, by the encoder, into the candidate list the remaining construction candidates from the first type of construction candidates after the remaining construction candidates from the second type of construction candidates; 40. The method of claim 39, further comprising one of:

41. The MV candidates for the plurality of categories are Sub-block-based temporal motion vector prediction (SbTMVP) candidates; a plurality of succession candidates from adjacent blocks; The method comprises the steps of: inserting, by the encoder, into the candidate list a number of successor candidates from the neighboring blocks after the SbTMVP candidate; inserting, by the encoder, into the candidate list a number of construction candidates from the neighboring blocks after a number of inheritance candidates from the neighboring blocks; or responsive to determining, by the encoder, that the candidate list is not full, inserting into the candidate list one or more zero MV candidates after the remaining construction candidates from the first type of construction candidates; 41. The method of claim 40, further comprising one of:

42. The first number, the second number, and the third number are expressed in the following manner: predefining the first number, the second number, and the third number as fixed values, respectively; or signaling, by the encoder, the first number, the second number, and the third number at a level; 41. The method of claim 40, wherein the method is obtained in one of the following ways:

43. 40. The method of claim 39, further comprising inserting, by the encoder, into the candidate list MV candidates according to the predetermined order based on one or more numbers of MV candidates in each category, the one or more numbers of MV candidates in each category being arranged in an interleaved manner.

44. The MV candidates for the plurality of categories are A number of construction candidates from adjacent blocks, including Xi construction candidates from the adjacent blocks; a plurality of succession candidates from non-adjacent blocks, including Zi succession candidates from non-adjacent blocks; A number of construction candidates from a first type of construction candidate from a non-adjacent block including Ki construction candidates from the first type of construction candidate; A plurality of construction candidates from the second type of construction candidates from non-adjacent blocks, including Yi construction candidates from the second type of construction candidates; The plurality of categories includes a plurality of sets of categories, each set of categories including Xi construction candidates from adjacent blocks, Zi inheritance candidates from non-adjacent blocks, Ki construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates; Xi, Yi, Zi, and Ki are each a positive integer, i being a positive integer greater than 2; The method comprises the steps of: inserting, by the encoder, into the candidate list MV candidates from a second set of categories including Xi construction candidates from neighboring blocks, Zi inheritance candidates from non-neighboring blocks, Ki construction candidates from the first type of construction candidates, and Yi construction candidates from the second type of construction candidates after a first set of categories including Xi-1 construction candidates from neighboring blocks, Zi-1 inheritance candidates from non-neighboring blocks, Ki-1 construction candidates from the first type of construction candidates, and Yi-1 construction candidates from the second type of construction candidates; 44. The method of claim 43, further comprising one of:

45. The MV candidates for the plurality of categories are Sub-block-based temporal motion vector prediction (SbTMVP) candidates; a plurality of succession candidates from adjacent blocks; The method comprises the steps of: inserting, by the encoder, into the candidate list, construction candidates from neighboring blocks in a set of categories including X construction candidates from neighboring blocks, Z construction candidates from non-neighboring blocks, K construction candidates from the first type of construction candidates, and Y construction candidates from the second type of construction candidates, after one or more inheritance candidates from neighboring blocks; inserting, by the encoder, the one or more successor candidates from a neighboring block after the SbTMVP candidate; or responsive to determining, by the encoder, that the candidate list is not full, inserting into the candidate list one or more zero MV candidates after the MV candidates from a last set of categories; 44. The method of claim 43, further comprising one of:

46. Xi, Yi, Zi, and Ki are expressed by the following formula: predefining Xi, Yi, Zi, and Ki as fixed values, or signaling Xi, Yi, Zi, and Ki at a level by said encoder; 45. The method of claim 44, wherein the method is obtained in one of the following ways:

47. 45. The method of claim 44, further comprising, in response to determining that a number of available MV candidates in one category is less than other categories, inserting, by the encoder, into the candidate list available MV candidates in the other category located at one or more positions of the one category.

48. 40. The method of claim 39, further comprising inserting, by the encoder, into a candidate list MV candidates according to a predetermined order based on available MV candidates in one or more categories, wherein MV candidates in other categories are arranged in the candidate list in an interleaved manner.

49. 1. An apparatus for video decoding, comprising: one or more processors; a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors; The one or more processors are configured, when the instructions are executed, to perform the method of any one of claims 1 to 23. Device.

50. 1. An apparatus for video encoding, comprising: one or more processors; a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors; The one or more processors are configured, when the instructions are executed, to perform the method of any one of claims 24 to 48. Device.

51. 24. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform a method according to any one of claims 1 to 23 based on the bitstream.

52. 49. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform the method of any one of claims 24 to 48 to encode a current video block into a bitstream and transmit the bitstream.

Images