Method and apparatus for motion storage in geometric segmentation mode

Abstract

Methods, apparatuses, and non-transitory storage media for video decoding and encoding are provided. In a decoding method, a decoder obtains at least one of a first motion vector or a second motion vector, in which, in a geometric partition mode, a coding unit is geometrically partitioned into a geometrically partitioned first portion and a geometrically partitioned second portion for prediction, the first motion vector is from the first portion and the second motion vector is from the second portion. In addition, the decoder may determine a value of the stored motion vector type based on the mixing width, the motion index, and the partial index. Further, the decoder may store the motion vector in a corresponding motion field based on the stored value of the motion vector type and the first motion vector or the second motion vector.

Description

[0001] This application is filed based on and claims priority to U.S. Provisional Application No. 63 / 354,230, filed on June 21, 2022, entitled “Method and Apparatus for Motion Storage in Geometric Partitioning Mode,” the entire contents of which are incorporated by reference for all purposes. Technical Field

[0002] The present disclosure relates to video coding and compression. More specifically, the present invention relates to a method and apparatus for improving coding efficiency of geometric partitioning (GPM) mode. Background Art

[0003] Various video codec technologies can be used to compress video data. Video coding is performed according to one or more video codec standards. For example, today, some well-known video codec standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part2) and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part10), which are jointly developed by ISO / IEC MPEG and ITU-T VECG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its previous standard VP9. Audio Video Coding (AVS), a digital audio and digital video compression standard, is another set of video compression standards developed by the Chinese Audio and Video Coding Standard Working Group. Most existing video codec standards are based on the well-known hybrid video codec framework, that is, block-based prediction methods (e.g., inter-frame prediction, intra-frame prediction) are used to reduce the redundancy present in video images or sequences, and transform coding is used to compress the energy of prediction errors. An important goal of video codec technology is to compress video data into a form using a lower bit rate while avoiding or minimizing the degradation of video quality.

[0004] The first version of the VVC standard was completed in July 2020, providing approximately 50% bit rate savings or equivalent perceptual quality compared to the previous generation video codec standard HEVC. Although the VVC standard provides significant codec improvements over its predecessors, there is evidence that superior coding efficiency can be achieved through additional coding tools. Recently, the Joint Video Exploration Team (JVET), in collaboration with ITU-TVECG and ISO / IEC MPEG, began exploring advanced technologies that can achieve substantial enhancements in codec efficiency relative to VVC. In April 2021, a software code base called the Enhanced Compression Model (ECM) was established for future video codec exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, in which several existing modules (e.g., intra / inter prediction, transform, loop filter, etc.) are further extended and / or improved. In the future, any new coding tools beyond the VVC standard will need to be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC). Summary of the invention

[0005] The present disclosure provides examples of techniques related to improving the encoding and decoding efficiency of a geometric partitioning mode (GPM) in a video encoding or decoding process.

[0006] According to a first aspect of the present disclosure, a method for motion field storage is provided. In the method for motion field storage, a decoder may determine at least one of a first motion vector or a second motion vector, wherein, in a geometric partitioning mode, a coding unit is geometrically partitioned into a geometrically partitioned first part and a geometrically partitioned second part for prediction, and the first motion vector is from the first part and the second motion vector is from the second part. In addition, the decoder may determine a value of a stored motion vector type based on a motion index and a partial index. In addition, the decoder may store a motion vector in a corresponding motion field based on the value of the stored motion vector type, wherein the motion vector includes a first motion vector, a second motion vector, or a combined motion vector obtained based on the first motion vector and the second motion vector.

[0007] According to a second aspect of the present disclosure, a method for motion field storage is provided. In the method for motion field storage, an encoder may determine at least one of a first motion vector or a second motion vector, wherein, in a geometric partitioning mode, a coding unit is geometrically partitioned into a geometrically partitioned first part and a geometrically partitioned second part for prediction, and the first motion vector is from the first part and the second motion vector is from the second part. In addition, the encoder may determine a value of a stored motion vector type based on a motion index and a partial index. In addition, the encoder may store a motion vector in a corresponding motion field based on the value of the stored motion vector type, wherein the motion vector may include a first motion vector, a second motion vector, or a combined motion vector obtained based on the first motion vector and the second motion vector.

[0008] According to a third aspect of the present disclosure, there is provided an apparatus comprising one or more processors and a memory, wherein the memory is coupled to the one or more processors and configured to store instructions executable by the one or more processors. In addition, the one or more processors are configured to perform the method according to the first aspect when executing the instructions.

[0009] According to a fourth aspect of the present disclosure, there is provided an apparatus comprising one or more processors and a memory, wherein the memory is coupled to the one or more processors and is configured to store instructions executable by the one or more processors. In addition, the one or more processors are configured to execute the method according to the second aspect when executing the instructions.

[0010] According to a fifth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform the method according to the first aspect.

[0011] According to a sixth aspect of the present disclosure, a non-temporary computer-readable storage medium storing computer-executable instructions is provided, which, when executed by one or more computer processors, causes the one or more computer processors to perform a method according to the second aspect to encode a current block into a bit stream and send the bit stream.

[0012] According to a seventh aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing a bit stream to be decoded by the method according to the first aspect.

[0013] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, storing a bit stream generated by the method according to the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A more particular description of examples of the present disclosure will be presented by reference to specific examples shown in the accompanying drawings. Given that these drawings depict only some examples and are not therefore considered limiting in scope, the examples will be described and explained with additional specificity and detail through use of the accompanying drawings.

[0015] Figure 1A is a block diagram of a video encoder according to some examples of the present disclosure.

[0016] Figure 1B A block diagram illustrating an exemplary video encoder according to some examples of the present disclosure.

[0017] Figure 1C is a block diagram illustrating an exemplary video decoder according to some examples of the present disclosure.

[0018] Figure 2 is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of the present disclosure.

[0019] Figure 3 is a block diagram of a video decoder according to some examples of the present disclosure.

[0020] Figure 4 Allowable GPM partitioning according to some examples of the present disclosure is shown.

[0021] Figure 5 Selection of a unidirectional prediction motion vector from a motion vector in a merge candidate list of a GPM according to some examples of the present disclosure is shown.

[0022] Figure 6 Template matching algorithms according to some examples of the present disclosure are shown.

[0023] Figure 7 A selected set of pixels are shown on which gradient analysis is performed according to some examples of the present disclosure.

[0024] Figure 8 Convolution of a 3x3 Sobel gradient filter is shown according to some examples of the present disclosure.

[0025] Fig. 9 Prediction fusion by weighted averaging of two HoG modes and planes is shown according to some examples of the present disclosure.

[0026] Fig.10 Templates and their reference samples used in TIMD according to some examples of the present disclosure are shown.

[0027] Fig.11 A mix of reordered templates for GPM split mode is shown according to some examples of the present disclosure.

[0028] FIG. 12A to FIG. 12C Available IPM candidates for a GPM with inter and intra prediction according to some examples of the present disclosure are shown.

[0029] Fig.12D Disabling combinations of GPM with inter- and intra-prediction are shown according to some examples of the present disclosure.

[0030] Fig.13 Shown are the distances from any location within a block to a segmentation edge for a geometric segmentation mode according to some examples of the present disclosure.

[0031] Fig.14 Some examples according to the present disclosure are shown Fig.13 Example of quantization of distances in .

[0032] Fig.15 Soft blending regions on both sides of a segmentation boundary are shown according to some examples of the present disclosure.

[0033] Fig.16 GPM mixing in the current ECM 4.0 is shown according to some examples of the present disclosure.

[0034] Fig.17 is a block diagram illustrating a system for encoding and decoding video blocks according to some examples of the present disclosure.

[0035] Fig.18 is a diagram illustrating a computing environment coupled with a user interface according to some examples of the present disclosure.

[0036] Fig.19A is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of the present disclosure.

[0037] Fig.19B is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of the present disclosure.

[0038] Fig.19C is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of the present disclosure.

[0039] Fig.19D is a diagram illustrating block partitioning in a multi-type tree structure according to some examples of the present disclosure.

[0040] Fig. 20 is a flowchart illustrating a method for video decoding according to some examples of the present disclosure.

[0041] Fig.21 is a diagram showing some examples according to the present disclosure. Fig. 20 The method for video decoding shown in FIG. 1 corresponds to a flow chart of a method for video encoding. DETAILED DESCRIPTION

[0042] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, many non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0043] The terms used in this disclosure are adopted only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms "a / an", "said" and "the" in this disclosure and the appended claims are also intended to include the plural forms, unless otherwise clearly indicated throughout the present disclosure. It will also be understood that the term "and / or" used in this disclosure refers to and includes one or any or all possible combinations of the listed multiple related items.

[0044] References throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that the particular features, structures, or characteristics described are included in at least one embodiment or example. Unless explicitly stated otherwise, features, structures, elements, or characteristics described in conjunction with one or some embodiments are also applicable to other embodiments.

[0045] Throughout the disclosure, unless otherwise explicitly specified, the terms "first", "second", "third", etc. are used only as nomenclature for reference to related elements (e.g., devices, components, compositions, steps, etc.), without implying any spatial or temporal order. For example, "first device" and "second device" may refer to two separately formed devices, or two parts, components or operating states of the same device, and may be named arbitrarily.

[0046] The terms "module", "sub-module", "circuit", "sub-circuit", "circuitry", "sub-circuitry", "unit", or "sub-unit" may include a memory (shared, dedicated, or group) storing code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components connected directly or indirectly. These components may or may not be physically attached to or located adjacent to each other.

[0047] As used herein, the terms "if" or "when..." may be understood to mean "at..." or "in response to...", depending on the context. These terms, if they appear in a claim, may not indicate that the associated limitation or feature is conditional or optional. For example, a method may include the following steps: i) when condition X exists or if condition X exists, perform function or action X', and ii) when condition Y exists or if condition Y exists, perform function or action Y'. The method may be implemented with the ability to perform function or action X' and the ability to perform function or action Y'. Therefore, both functions X' and Y' may be performed at different times during multiple executions of the method.

[0048] A unit or module may be implemented purely by software, purely by hardware, or by a combination of hardware and software. In a pure software implementation, for example, a unit or module may include functionally related code blocks or software components linked together directly or indirectly to perform a specific function.

[0049] Similar to all previous video codec standards, ECM is built on a block-based hybrid video codec framework. Figure 1A A block diagram of a general block-based hybrid video coding system is given. The input video signal is processed block by block, called coding unit (CU). In ECM-1.0, a CU can be up to 128x128 pixels. However, like VVC, a coding tree unit (CTU) is split into CUs based on quadtree / binarytree / ternarytree to adapt to the changing local characteristics.

[0050] In the multi-type tree structure, a CTU is first split by a quadtree structure, and then each quadtree leaf node can be further split by a binary tree and a ternary tree structure.

[0051] Figure 2 is a schematic diagram showing a multi-type tree splitting mode according to some embodiments of the present disclosure. Figure 2 As shown in , there are five types of splits, quaternary split, horizontal binary split, vertical binary split, horizontal ternary split, and vertical ternary split.

[0052] exist Figure 2In the video processing, spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples of already encoded neighboring blocks in the same video picture / slice (which are called reference samples) to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also called "inter-frame prediction" or "motion compensated prediction") uses reconstructed pixels from already encoded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is usually signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference.

[0053] And, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture storage the temporal prediction signal comes from. After spatial and / or temporal prediction, the mode decision block in the encoder selects the best prediction mode, for example, based on a rate-distortion optimization method. The prediction block is then subtracted from the current video block; and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are inversely quantized and inversely transformed to form a reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Further loop filtering (such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF)) may be applied to the reconstructed CU before it is placed in the reference picture memory and used to encode future video blocks. In order to form an output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit to be further compressed and packaged to form a bitstream.

[0054] Figure 3 A block diagram of a block-based video decoder according to some examples of the present disclosure is provided. In the block-based video decoder 200, the video bitstream 201 is first entropy decoded at the entropy decoding unit 202. The coding mode and prediction information are sent to the spatial prediction unit (if intra-frame coding) or the temporal prediction unit (if inter-frame coding) to form a prediction block. The residual transform coefficients are sent to the inverse quantization unit 204 and the inverse transform unit 206 to reconstruct the residual block. The prediction block and the residual block are then added together. The reconstructed block may be further subjected to loop filtering 209 before it is stored in the reference picture buffer 213. The reconstructed video in the reference picture buffer 213 is then sent out for display and for predicting future video blocks.

[0055] The main focus of this disclosure is to further improve the encoding and decoding efficiency of GPM in ECM. In the following, some relevant encoding tools in ECM are briefly reviewed. After that, some defects in the existing design of current GPM are discussed. Finally, solutions to improve the existing GPM in ECM are proposed.

[0056] Fig.17 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Fig.17 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. Source device 12 and destination device 14 may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0057] In some embodiments, the target device 14 may receive the encoded video data to be decoded via the link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14. In one example, the link 16 may include a communication medium that enables the source device 12 to directly send the encoded video data to the target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and sent to the target device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, a switch, a base station, or any other device that may be beneficial to facilitate communication from the source device 12 to the target device 14.

[0058] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device 32 may correspond to a file server or another intermediate storage device that may hold the encoded video data generated by the source device 12. The target device 14 may access the stored video data from the storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing encoded video data and sending the encoded video data to the target device 14. Exemplary file servers include a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. Target device 14 may access the encoded video data through any standard data connection suitable for accessing encoded video data stored on a file server, including a wireless channel (e.g., a Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both wireless channels and wired connections. The transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both streaming and download transmissions.

[0059] like Fig.17 As shown in , source device 12 includes video source 18, video encoder 20 and output interface 22. Video source 18 may include sources such as or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera of a security monitoring system, source device 12 and target device 14 may form a camera phone or a video phone. However, the embodiments described in this application may be generally applicable to video encoding and decoding, and may be applied to wireless and / or wired applications.

[0060] Captured, pre-captured or computer-generated video may be encoded by the video encoder 20. The encoded video data may be sent directly to the target device 14 via the output interface 22 of the source device 12. The encoded video data may also (or alternatively) be stored on a storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or a transmitter. In some examples, the video encoder 20 may be a video encoding system as shown in FIG. 1.

[0061] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included in the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server. In some examples, video decoder 30 may be, for example, Figure 3 The video decoding system shown.

[0062] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device and an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0063] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any of these current standards or future standards.

[0064] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0065] In a typical video encoding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore include only one two-dimensional array of luma samples.

[0066] like Fig.19A As shown in FIG. 1 , the video encoder 20 (or more specifically, the segmentation unit 45) (as shown in FIG. Figure 1B ) generates an encoded representation of a frame by first partitioning the frame into a set of CTUs. A video frame may include an integer number of CTUs sequentially ordered from left to right and from top to bottom in a raster scan order. Each CTU is the largest logical coding unit and is encoded by the video encoder 20 (as shown in FIG. Figure 1B The width and height of the CTU are signaled in the sequence parameter set (as shown in FIG. 1 ), so that all CTUs in the video sequence have the same size of one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a specific size. Fig.19B As shown in , each CTU may include one CTB for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the properties of different types of units for coding pixel blocks and how they may be encoded in the video decoder 30 (e.g., Figure 1C The video sequence is reconstructed at (as shown), including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vector and other parameters. In a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements for encoding the samples of the coding tree block. The coding tree block may be an N×N sample block.

[0067] To achieve better performance, the video encoder 20 (eg Figure 1B As shown in FIG. 1 , a tree partitioning, such as a binary tree partitioning, a ternary tree partitioning, a quadtree partitioning, or a combination thereof, may be recursively performed on the coding tree block of the CTU, and the CTU may be divided into smaller CUs. Fig.19C As depicted in , a 64×64 CTU 400 is first divided into four smaller CUs, each CU having a block size of 32×32. Among the four smaller CUs, CU 410 and CU 420 are respectively divided into four CUs with a block size of 16×16. Two 16×16 CUs 430 and CU 440 are further divided into four CUs with a block size of 8×8. Fig.19D Depicted is a diagram showing Fig.19C The quadtree data structure of the final result of the partitioning process of the CTU 400 depicted in FIG. 4 corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Fig.19B Each CU may include a CB of luma samples and two corresponding coding blocks of chroma samples of the same size frame, and syntax elements for encoding the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures for encoding the samples of the coding block. It should be noted that Fig.19C and Fig.19D The quadtree partitioning depicted in FIG is for illustrative purposes only, and one CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In a multi-type tree structure, one CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to a binary and ternary tree structure. Figure 2 As shown, there are five possible partitioning types for a coding block with width W and height H, namely, quadruple partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning.

[0068] In some embodiments, the video encoder 20 may further partition the coding block of the CU into one or more (M×N) PBs. A PB is a rectangular (square or non-square) sample block to which the same prediction (inter or intra) is applied. The PU of a CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements for predicting the PBs. In a monochrome picture or a picture with three separate color planes, a PU may include a single PB and a syntax structure for predicting the PB. The video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for the luma PB, Cb PB, and Cr PB of each PU of the CU.

[0069] Video encoder 20 may use intra prediction or inter prediction to generate a prediction block for a PU. If video encoder 20 uses intra prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0070] After the video encoder 20 generates the predicted luma block, the predicted Cb block, and the predicted Cr block for one or more PUs of a CU, the video encoder 20 may generate a luma residual block for the CU by subtracting the predicted luma block of the CU from the original luma coding block of the CU, so that each sample in the luma residual block of the CU indicates the difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, respectively, so that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU may indicate the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0071] In addition, if Fig.19C As shown in , the video encoder 20 can use quadtree partitioning to decompose the luma residual block, Cb residual block and Cr residual block of the CU into one or more luma transform blocks, Cb transform blocks and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. The TU of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements for transforming the transform block samples. Therefore, each TU of a CU may be associated with a luma transform block, a Cb transform block and a Cr transform block. In some examples, the luma transform block associated with the TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and a syntax structure for transforming the samples of the transform block.

[0072] The video encoder 20 may apply one or more transforms to the luma transform block of the TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficient may be a scalar. The video encoder 20 may apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 may apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0073] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, the video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, the video encoder 20 may output a bitstream including a sequence of bits that form a representation of an encoded frame and associated data, and the bitstream is stored in the storage device 32 or sent to the target device 14.

[0074] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. The video decoder 30 can reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally inverse to the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coding block of the current CU by adding the samples of the prediction block for the PU of the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coding block for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0075] As mentioned above, video codec mainly uses two modes, namely, intra-frame prediction (or intra-frame prediction) and inter-frame prediction (or inter-frame prediction) to achieve video compression. It should be noted that IBC can be regarded as intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because motion vectors are used to predict the current video block based on the reference video block.

[0076] But with the ever-improving video data capture technology and finer video block sizes for retaining details in the video data, the amount of data required to represent the motion vector for the current frame has also increased significantly. One way to overcome this challenge benefits from the fact that not only a set of neighboring CUs in both the spatial and temporal domains have similar video data for prediction purposes, but also the motion vectors between these neighboring CUs are similar. Therefore, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vector) of the current CU (which is also called the "motion vector predictor" (MVP) of the current CU) by exploring their spatial and temporal correlations.

[0077] Instead of combining as above Figure 1B The actual motion vector of the current CU determined by the motion estimation unit 42 is encoded into the video bitstream, and the motion vector prediction value of the current CU is subtracted from the actual motion vector of the current CU to generate a motion vector difference (MVD) for the current CU. By doing so, the motion vector determined by the motion estimation unit 42 for each CU of the frame does not need to be encoded into the video bitstream, and the amount of data used to represent the motion information in the video bitstream can be significantly reduced.

[0078] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a coding block, both the video encoder 20 and the video decoder 30 need to adopt a set of rules for constructing a motion vector candidate list (also called a "merge list") for the current CU using those potential candidate motion vectors associated with the spatial neighboring CUs and / or temporal co-located CUs of the current CU, and then selecting a member from the motion vector candidate list as the motion vector prediction value for the current CU. By doing so, the motion vector candidate list itself does not need to be sent from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for the video encoder 20 and the video decoder 30 to use the same motion vector prediction value within the motion vector candidate list to encode and decode the current CU.

[0079] Geometric Partitioning Mode (GPM)

[0080] In VVC, geometric partitioning mode is supported for inter prediction. Geometric partitioning mode is signaled by a CU level flag as a special merge mode. In the current GPM design, for each possible CU size with width and height not less than 8 and not greater than 64 (excluding 8×64 and 64×8), a total of 64 partitions are supported by GPM mode.

[0081] When this mode is used, the CU Figure 4The geometric positioning straight line shown is split into two parts. The position of the split line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU is inter-predicted using its own motion; only unidirectional prediction is allowed for each partition, that is, each part has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that only two motion compensated predictions are required for each CU, the same as conventional bidirectional prediction. If a geometric partitioning mode is used for the current CU, a geometric partitioning index indicating the partitioning mode (angle and offset) of the geometric partitioning, as well as two merge indices (one merge index for each partition) are further signaled. The number of maximum GPM candidate sizes is explicitly signaled at the sequence level.

[0082] One-way prediction candidate list construction

[0083] To derive a unidirectional prediction motion vector for geometric partitioning, first derive a unidirectional prediction candidate list directly from the regular merge candidate list generation process. Denote n as the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector of the nth merge candidate (where X equals the parity of n) is used as the nth unidirectional prediction motion vector for geometric partitioning mode. These motion vectors are Figure 5 In the case where the corresponding LX motion vector of the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the unidirectional prediction motion vector of the geometric partition mode instead.

[0084] Blending along geometric partition edges

[0085] After obtaining each geometric partition using its own motion, blending is applied to the two unidirectional prediction signals to derive samples around the geometric partition edges. The blending weight for each position of the CU is derived based on the distance from each individual sample position to the corresponding partition edge.

[0086] like Fig.13 As shown, the distance (or displacement) d(x_c, y_c) from any position within the block to the segmentation edge (or segmentation boundary) is mathematically defined by the Hessian norm form:

[0087]

[0088] Among them, x c and c Indicates the position relative to the center of the block; denotes the angle parameter, and ρ denotes the offset parameter of the segmentation boundary.

[0089] In order to realize displacement calculation in practice, the angle parameter and the offset parameter are quantized into integers.

[0090] d(x c ,y c )=((x c +ρ x，j )cosLut[i])>>3+((y c +ρ y，j )cosLut[(i+8)%32])>>3 (2)

[0091] Among them, ρ x，j and ρ y，j is a quantized offset that depends on the width and height of the block; and cosLut[i] represents the quantized cosine lookup table for angle parameter index i.

[0092] Finally, the continuous sample point positions (x c ,y c ) is quantized to integer positions (m,n), and the displacement d(m,n) is given by:

[0093] d(m,n)=((m+ρ x.j )<<1-w+1)cosLut[i]+((n+ρ y，j )<<1-h+1)cosLut[(i+8)%32]. (3)

[0094] Fig.14 Some examples according to the present disclosure are shown Fig.13 Quantification of the distance in .

[0095] Quantized d(m,n) and d(x c ,y c ) is given by the following formula

[0096]

[0097] Fig.15 Soft blending regions on both sides of a segmentation boundary according to some examples of the present disclosure are shown. Fig.15 As shown, a soft mix region with a mix width of θ luminance samples is defined on both sides of the segmentation boundary. Outside the soft mix region, only weighting values ​​0 or 8 can be selected. Within the soft mix region, the weighting value ω(x c ,y c ):

[0098]

[0099] The ramp function is also quantized to integer positions to obtain weighted values ​​ω(m,n), which are used in the mixing process of GPM, i.e.,

[0100] ω(m,n)=Clip3(0,8,(d(m,n)+32+4)>>3) (6)

[0101] Two mixing matrices (W0 and W1) are generated using these values ​​in different locations.

[0102] The predicted value of GPM is then given by

[0103]

[0104] Motion field storage for geometric segmentation mode

[0105] Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combination Mv of Mv1 and Mv2 are stored in the motion field of the geometric partition mode coded CU.

[0106] The type of motion vector stored for each individual position in the motion field is determined as:

[0107] sType = abs(motionIdx) < 32? 2

[0108] (motionIdx≤0?(1-partIdx):partIdx) (8)

[0109] Wherein, motionIdx is equal to d(4x+2, 4y+2), which is recalculated from equation (1). partIdx depends on the angle index i.

[0110] If sType is equal to 0 or 1, store Mv1 or Mv2 in the corresponding motion field, otherwise if sType is equal to 2, store the combined Mv from Mv1 and Mv2. Generate the combined Mv using the following process:

[0111] (1) If Mv1 and Mv2 are from different reference picture lists (one from L0 and the other from L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector.

[0112] (2) Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored.

[0113] GPM signaling design

[0114] According to the current GPM design, the use of GPM is indicated by signaling a flag at the CU level. The flag is signaled only when the current CU is encoded and decoded by merge mode or skip mode. Specifically, when the flag is equal to one, it indicates that the current CU is predicted by GPM. Otherwise (the flag is equal to zero), the CU is encoded and decoded by another merge mode (such as normal merge mode, merge mode with motion vector difference, combined inter and intra prediction, etc.). When GPM is enabled for the current CU, a syntax element (i.e., merge_gpm_partition_idx) is further signaled to indicate the applied geometric partitioning mode (which specifies the direction and offset of the straight line from the center of the CU to split the CU into two partitions, such as Figure 4 ). Thereafter, two syntax elements merge_gpm_idx0 and merge_gpm_idx1 are signaled to indicate the indexes of the unidirectional prediction merge candidates for the first GPM partition and the second GPM partition. More specifically, those two syntax elements are used to determine the unidirectional MVs of the two GPM partitions from the unidirectional prediction merge list. According to the current GPM design, in order to make the two unidirectional MVs more different, the two indexes cannot be the same. Based on such prior knowledge, the unidirectional prediction merge index of the first GPM partition is first signaled and used as a prediction value to reduce the signaling overhead of the unidirectional prediction merge index of the second GPM partition. In detail, if the second unidirectional prediction merge index is less than the first unidirectional prediction merge index, its original value is directly signaled. Otherwise (the second unidirectional prediction merge index is greater than the first unidirectional prediction merge index), its value is subtracted by 1 before being signaled to the bitstream. On the decoder side, the first unidirectional prediction merge index is first decoded. Then, for decoding of the second unidirectional prediction merge index, if the resolved value is less than the first unidirectional prediction merge index, the second unidirectional prediction merge index is set equal to the resolved value; otherwise (the resolved value is equal to or greater than the first unidirectional prediction merge index), the second unidirectional prediction merge index is set equal to the resolved value plus one. Table 1 shows the existing syntax elements for the GPM mode in the current VVC specification.

[0115] Table 1 Existing GPM syntax elements in the merged data syntax table of the VVC specification

[0116]

[0117] On the other hand, in the current GPM design, truncated unary codes are used for binarization of two unidirectional prediction merge indexes (i.e., merge_gpm_idx0 and merge_gpm_idx1). In addition, because the two unidirectional prediction merge indexes cannot be the same, different maximum values ​​are used to truncate the codewords of the two unidirectional prediction merge indexes, and the maximum values ​​are set to be equal to MaxGpmMergeCand-1 and MaxGpmMergeCand-2 for merge_gpm_idx0 and merge_gpm_idx1, respectively. MaxGPMMergeCand is the number of candidates in the unidirectional prediction merge list. On the decoder side, when the received value of merge_gpm_idx1 is equal to or greater than merge_gpm_idx0, its value will be increased by 1, taking into account that the values ​​of merge_gpm_idx0 and merge_gpm_idx1 cannot be the same. When GPM / mode is applied, two different binarization methods are applied to translate the syntax merge_gpm_partition_idx into a binary bit string. Specifically, the syntax elements are binarized by fixed-length codes and truncated binary codes in VVC.

[0118] Motion signaling for regular inter-mode

[0119] Similar to the HEVC standard, in addition to the merge / skip mode, both VVC and AVS3 allow an inter-CU to explicitly specify its motion information in the bitstream. In general, the motion information signaling in both VVC and AVS3 is maintained the same as that in the HEVC standard. Specifically, an inter-prediction syntax (i.e., inter_pred_idc) is first signaled to indicate whether the prediction signal is from list L0, L1, or both. For each reference list used, the corresponding reference picture is identified by signaling a reference picture index ref_idx_lx (x=0, 1) for the corresponding reference list, and the corresponding MV is represented by an MVP index mvp_lx_flag (x=0, 1) used to select the MV prediction value (MVP), followed by its motion vector difference (MVD) between the target MV and the selected MVP. In addition, in the VVC standard, a control flag mvd_11zero_flag is signaled at the slice level. When mvd_11zero_flag is equal to 0, L1 MVD is signaled in the bitstream; otherwise (when mvd_11zero_flag flag is equal to 1), L1 MVD is not signaled and its value is always inferred to be zero at the encoder and decoder.

[0120] With CU-level weighted bidirectional prediction

[0121] In previous standards before VVC and AVS3, when weighted prediction (WP) is not applied, a bidirectional prediction signal is generated by averaging unidirectional prediction signals obtained from two reference pictures. In VVC, a tool coding (i.e., bidirectional prediction with CU-level weights (BCW)) is introduced to improve the efficiency of bidirectional prediction. Specifically, instead of simple averaging, the bidirectional prediction in BCW is extended by allowing a weighted average of two prediction signals, as described:

[0122] P'(i,j)=((8-w)·P0(i,j)+w·P1(i,j)+4)>>3 (9)

[0123] In VVC, when the current picture is a low-latency picture (where all reference pictures have smaller picture order count (POC) values ​​than the current picture), the weight of a BCW coding block is allowed to be selected from a set of predefined weight values ​​w∈{-2, 3, 4, 5, 10}, and a weight of 4 represents the traditional bidirectional prediction case where two unidirectional prediction signals are equally weighted. For low latency, only 3 weights w∈{3, 4, 5} are allowed. In general, although there are some design similarities between WP and BCW, the two coding tools are aimed at solving illumination change problems at different granularities. However, because the interaction between WP and BCW may potentially complicate the VVC design, it is not allowed to enable both tools at the same time. Specifically, when WP is enabled for a slice, the BCW weights of all bidirectionally predicted CUs in the slice are not signaled and are inferred to be 4 (i.e., equal weights are applied).

[0124] Template Matching

[0125] Template matching (TM) is a decoder-side MV derivation method to refine the motion information of the current CU by finding the best match between a template consisting of the neighboring reconstructed samples above and to the left of the current CU and a reference block (i.e., the same size as the template) in a reference picture. Figure 6 As shown in , an MV is searched around the initial motion of the current CU within a search range of [-8, +8] pixels. The best match can be defined as the MV that achieves the lowest matching cost (e.g., sum of absolute differences (SAD), sum of absolute transform differences (SATD), etc.) between the current template and the reference template. There are two different ways to apply TM mode to inter-frame coding.

[0126] In AMVP mode, MVP candidates are determined based on template matching differences to find the MVP candidate that achieves the minimum difference between the current block template and the reference block template, and then TM is performed only for this specific MVP candidate for MV refinement. TM refines the MVP candidate starting from full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode) within the [-8, +8] pixel search range using an iterative diamond search. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode), followed sequentially by half-pixel and quarter-pixel depending on the AMVR mode specified in the following table. This search process ensures that the MVP candidate still maintains the same MV accuracy as indicated by the AMVR mode after the TM process.

[0127] Table 2 Search patterns for AMVR mode and merge mode

[0128]

[0129] In merge mode, a similar search method is applied to the merge candidates indicated by the merge index. As shown in the table above, depending on whether the optional interpolation filter is used based on the merge motion information (which is used when AMVR is in half-pixel mode), the TM can perform all the way down to 1 / 8 pixel MVD precision or skip those beyond half-pixel MVD precision.

[0130] Decoder-side intra mode derivation (DIMD)

[0131] DIMD is an intra-coding tool in which the luma intra prediction mode (IPM) is not sent via the bitstream. Instead, the IPM is derived in the same way at the encoder and decoder using previously coded / decoded pixels. The DIMD method performs texture gradient processing to derive the 2 best modes. These two modes and the planar mode are then applied to the block and their predictions are weighted averaged. The selection of DIMD is signaled in the bitstream for intra-coded blocks using a flag. At the decoder, if the DIMD flag is true, the intra prediction mode is derived during the reconstruction process using the same previously coded neighboring pixels. If not, the intra prediction mode is parsed from the bitstream as in the classic intra coding mode.

[0132] In order to derive the intra prediction mode for a block, a set of neighboring pixels must first be selected and the gradient analysis will be performed on this set of neighboring pixels. For normative purposes, these pixels should be in the decoded / reconstructed pool of pixels. Figure 7As shown, the template is selected to be T pixels to the left and T pixels above the current block. Next, a gradient analysis is performed on the pixels of the template. This allows the determination of the main angular direction of the template, which is assumed (and this is the core premise of the proposed method) to have a high probability of being the same as the main angular direction of the current block. A simple 3x3 Sobel gradient filter is used and is defined by the following matrix that will be convolved with the template:

[0133] and

[0134] For each pixel of the template, each of these two matrices is point-wise multiplied with a 3x3 window centered on the current pixel and consisting of its 8 direct neighbors, and the results are summed. Thus, two values ​​Gx (from the multiplication with Mx) and Gy (from the multiplication with My) corresponding to the gradient at the current pixel are obtained in the horizontal and vertical directions, respectively.

[0135] Figure 8 801 is a pixel that is not available for gradient analysis. Pixel 803 (including pixel 801) is a pixel for which gradient analysis can be performed. Pixel 802 is a pixel for which gradient analysis is not possible due to the lack of some neighbors. Pixel 804 is an available (reconstructed) pixel outside the considered template used in the gradient analysis of pixel 803. In the case where pixel 804 is not available (for example, because the block is too close to the boundary of the picture), the gradient analysis of all pixels 803 using this pixel 804 is not performed. For each pixel 803, the intensity (G) and orientation (O) of the gradient are calculated using Gx and Gy as follows:

[0136] G=|G x |+|G y | and

[0137] The orientation of the gradient is then converted to an intra-frame angular prediction mode for indexing a histogram (initialized to zero first). The histogram value in the intra-frame angular mode is increased by G. Once all pixels 803 in the template have been processed, the histogram will contain the cumulative value of the gradient strength for each intra-frame angular mode. The IPM corresponding to the two highest histogram bars is selected for the current block. If the maximum value in the histogram is 0 (indicating that gradient analysis cannot be performed, or the area constituting the template is flat), DC mode is selected as the intra-frame prediction mode for the current block.

[0138] The two IPMs corresponding to the two highest Histogram of Oriented Gradients (HoG) bins are combined with the plane mode. Prediction fusion is applied as a weighted average of the above three predictions. For this purpose, the weight of the plane is fixed to 21 / 64 (~1 / 3). Then, the remaining weight of 43 / 64 (~2 / 3) is shared between the two HoG IPMs, proportional to the amplitude of their HoG bins. Fig. 9 The process is visualized.

[0139] The derived intra modes are included into the master list of intra most probable modes (MPMs), so the DIMD process is performed before building the MPM list. The master derived intra modes of a DIMD block are stored with the block and are used for MPM list construction for neighboring blocks.

[0140] Template-based Intra Mode Derivation (TIMD)

[0141] For each intra mode in MPM, calculate Fig.10 The sum of absolute transform differences (SATD) between the predicted samples and the reconstructed samples of the template area shown in , and the intra mode of the first two modes with the smallest SATD cost is selected, then fused using weights, and this weighted intra prediction is used to encode the current CU.

[0142] The costs of the two selection modes are compared with the threshold. In the test, a cost factor of 2 is applied as follows:

[0143] costMode2<2*costMode1.

[0144] If this condition is true, then fusion is applied, otherwise only mode 1 is used.

[0145] The weight of a mode is calculated based on its SATD cost as follows:

[0146] weight1=costMode2 / (costMode1+costMode2)

[0147] weight2=1-weight1.

[0148] Geometric segmentation mode with template matching (TM)

[0149] In ECM, template matching (TM) is applied above the geometric partitioning mode. When GPM is enabled for the current CU, two sets of unidirectional motion information of GPM are derived for each part of GPM from the GPM merge candidate list, respectively. The GPM merge candidate list is constructed as follows.

[0150] First, interleaved list-0 MV candidates and list-1 MV candidates are derived directly from the regular merge candidate list, where list-0 MV candidates have higher priority than list-1 MV candidates. A pruning method with an adaptive threshold based on the current CU size is applied to remove redundant MV candidates.

[0151] Secondly, interleaved list-1 MV candidates and list-0 MV candidates are further derived directly from the regular merge candidate list, where list-1 MV candidates have higher priority than list-0 MV candidates.The same pruning method with adaptive threshold is applied to remove redundant MV candidates.

[0152] Third, fill in zero MV candidates until the GPM candidate list is full.

[0153] A CU-level flag is further signaled to indicate that TM is enabled for GPM, that is, the two motion vectors of the GPM are further refined using template matching. During this template matching, one of the following template types may be used for refinement, i.e., above (A), left (L), above plus left (A+L) of the current block. In the current ECM3.1 (adopted from JVET W0065), the template type is selected based on the angle parameter of the GPM, as shown in Table 3 below.

[0154] Table 3 Template type selection

[0155]

[0156]

[0157] The motion is then refined by minimizing the difference between the current template and the template in the reference picture using the same search pattern with the merge mode of the half-pixel interpolation filter disabled.

[0158] Template matching based reordering for GPM splitting mode

[0159] In VVC and ECM-3.1, there are 64 GPM split modes, and a fixed-length binary code is used to signal the use of the split mode for each GPM coding unit (CU). This coding approach may mean that all GPM split modes are treated as equally probable events, so a fixed-length code can be used for signaling accordingly.

[0160] The template matching based GPM split pattern reordering method was first proposed in JVET document JVET-Y0135. The template matching cost of each GPM split pattern is calculated, and the split patterns are reordered based on the cost on both the encoder and decoder sides. Only the best N candidates are available, where N is less than or equal to 64. Golomb-Rice codes are used instead of fixed-length binary codes to signal the GPM pattern index.

[0161] The reordering method of the GPM split mode is a two-step process after generating the corresponding reference templates of the two GPM splits in the coding unit, as follows:

[0162] ● Mix the reference templates of the two GPM partitions using the corresponding weights of the splitting patterns (i.e., obtain 64 mixed reference templates), and calculate the corresponding TM costs of these mixed reference templates;

[0163] ● Re-sort the TM costs in ascending order and mark the best N candidates as available split patterns.

[0164] The edges on the template extend from the edges of the current CU, such as Fig.11 The corresponding weights used in the template blending process are calculated similarly to the GPM weight derivation process. The only difference is as follows:

[0165] The sample point positions on the template (relative to the original positions of the CU) are used to derive weights;

[0166] • The weights are mapped to 0 and 8 before use, depending on which is closer, and thus the edges on the template are cleanly cut for computational simplicity in the blending process of the templates.

[0167] GPM with inter-frame and intra-frame prediction

[0168] In previous GPM designs, inter prediction with two unidirectional predictions was used to generate the final prediction. An approach combining inter and intra prediction for GPM was introduced by JVET-X0166 and JVET-Y0065.

[0169] In GPM with inter and intra prediction, the final prediction samples are generated by weighting the inter prediction samples and intra prediction samples for each GPM separated region. Each part contains a flag to indicate whether inter or intra prediction is used. The samples for inter prediction are derived through the same scheme as the current GPM, while the samples for intra prediction are derived through the intra prediction mode (IPM) candidate list and the index signaled from the encoder. The IPM candidate list size is predefined to be 3. The available IPM candidates are parallel angle modes (such as Fig. 12A parallel mode as shown in ), perpendicular angle mode relative to the GPM block boundary (as shown in Fig. 12B vertical mode as shown in ), and Fig. 12C In addition, as shown in Fig.12DThe GPM with intra and intra prediction shown is restricted in this method to reduce the signaling overhead of IPM and avoid the size increase of intra prediction circuit on hardware decoder. In addition, direct motion vector and IPM storage on GPM mixed region is introduced to further improve the codec performance.

[0170] In addition, the IPM list within the GPM frame can be further improved by DIMD, TIMD and angle mode from neighboring blocks. More specifically, the parallel mode is used in the first position of the IPM list, and then the IPM candidates from the neighbors' TIMD, DIMD and angle mode are used, and pruning between candidates is performed.

[0171] For neighbor mode derivation, there are up to five positions for available neighbor blocks, but they are restricted by the angles of the GPM block boundaries as shown in Table 4 below.

[0172] Table 4 Location of neighboring blocks subject to angle constraints of GPM

[0173] GPM angle 0 2 3 4 5 8 11 12 13 14 1st Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A Split Angle 16 18 19 20 21 24 27 28 29 30 1st Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A

[0174] The motion is then refined by minimizing the difference between the current template and the template in the reference picture using the same search pattern of merge mode with the half-pixel interpolation filter disabled.

[0175] GPM Adaptive Mixing

[0176] The GPM adaptive hybrid method has been disclosed in JVET-Z0127.

[0177] During the development of VVC, Geometric Partitioning Mode (GPM) was adopted. During the development of ECM, GPM was further enhanced by, for example, GPM+TM, GPM+MMVD, and inter+intra GPM. However, the mixing strength of GPM has not been improved since the original GPM design. That is, Fig.16 As shown, for all different contents, the blending strength, blending area width or blending width θ is fixed.

[0178] The weighting values ​​in the blending mask can be given by a ramp function:

[0179]

[0180] With a fixed θ=2 pixels in the current ECM (VVC) design, the ramp function can be quantized as

[0181] ω m,n =Clip3(0,8,(d(m,n)+32+4)>>3) (13)

[0182] It is claimed that this design may not always be optimal because a fixed blending region width cannot always provide the best blending quality for various types of video content. For example, screen video content often contains strong textures and sharp edges, which requires a narrow blending region to preserve edge information. For camera-captured content, blending is often required; however, the blending region width depends on multiple factors, such as the actual boundaries of the moving object and the motion uniqueness of the two segmentations.

[0183] To solve the above problems, an adaptive blending scheme is proposed for GPM, which dynamically adjusts the width of the blending region around the GPM partition boundary. Specifically, the width of the blending region (i.e., θ) is allowed to be selected from a set of predefined values ​​(e.g., {0, 1, 2, 4, 8}). The optimal blending region width is determined at the encoder for each GPM CU and is signaled to the decoder based on a syntax element merge_gpm_blending_width_idx. In this scheme, all predefined blending strength values ​​are shiftable, and all limiting and shifting operations in the existing GPM blending process can be maintained without any changes.

[0184] In addition, the range of weights is increased from [0,8] to [0,32] to accommodate the increased width of the GPM mixing region. Specifically, the weights are calculated as

[0185]

[0186] Deblocking filter

[0187] The deblocking filtering process is similar to that in HEVC. In VVC, the deblocking filtering process is applied to CU boundaries, transform sub-block boundaries, and prediction sub-block boundaries. The prediction sub-block boundaries include the prediction unit boundaries introduced by the SbTMVP and affine modes, and the transform sub-block boundaries include the transform unit boundaries introduced by the SBT and ISP modes, as well as the transforms caused by the implicit splitting of large CUs. As done in HEVC, the processing order of the deblocking filter is defined as horizontal filtering for the vertical edges of the entire picture first, followed by vertical filtering for the horizontal edges. This particular order enables any of multiple horizontal filtering or vertical filtering processes to be applied in parallel threads, or can still be implemented on a CTB-by-CTB basis with only a small processing delay. Compared to HEVC deblocking, the following modifications are introduced.

[0188] - The filter strength of the deblocking filter depends on the average brightness level of the reconstructed samples.

[0189] - Deblocking tC table is extended and adapted to 10-bit video.

[0190] - 4×4 grid deblocking for luma.

[0191] -Stronger deblocking filter for luma.

[0192] -Stronger deblocking filter for chroma.

[0193] - Deblocking filter for sub-block boundaries.

[0194] - Deblocking decisions adapted to smaller motion differences.

[0195] In VVC and ECM, motion information is stored for temporal motion vector prediction. In GPM of VVC, bidirectional motion is stored in a mixed region, which can benefit temporal motion vector prediction. However, in current ECM, when GPM with inter- and intra-frame prediction is enabled, only unidirectional motion is stored, which may not be efficient for codec performance.

[0196] A method and apparatus for further improving the motion storage of GPM in ECM is provided. In VVC, the motion field of a CU with GPM consists of three parts: Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combination Mv of Mv1 and Mv2. However, in the current ECM, when GPM with intra-frame and inter-frame prediction is enabled, only Mv1 and Mv2 are stored in the motion field of the CU with GPM.

[0197] Motion storage for GPM with inter and intra prediction

[0198] In one or more embodiments, it is proposed to store Mv1, Mv2, and the combination Mv1 and Mv2 in the motion field of a CU with GPM inter-frame and inter-frame prediction such as VVC. The stored motion vector type of each individual position in the motion field is determined as:

[0199] sType = abs(motionIdx) < 32? 2

[0200] (motionIdx≤0?(1-partIdx):partIdx) (15)

[0201] where motionIdx is equal to d(4x+2, 4y+2), which is recalculated from equation (1). partIdx depends on the angle index i.

[0202] If sType is equal to 0 or 1, then Mv1 or Mv2 is stored in the corresponding motion field, otherwise if sType is equal to 2, then the combined Mv from Mv1 and Mv2 is stored. The combined Mv is generated using the following process:

[0203] - If Mv1 and Mv2 come from different reference picture lists (one from L0 and the other from L1),

[0204] Mv1 and Mv2 are then simply combined to form a bidirectional prediction motion vector.

[0205] - Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored.

[0206] Motion storage for GPM with intra and inter prediction

[0207] In the current ECM, if a CU is encoded with a GPM with intra and inter prediction, no motion information is stored in the motion field of the intra partition, and the unidirectional prediction motion is stored in the motion field of the inter partition. In some embodiments, the motion storage of a GPM with intra and inter is modified as follows:

[0208] - If sType is equal to 2, the motion information of the inter-frame segmentation is stored.

[0209] - If segmentation 1 is intra-frame segmentation, segmentation 2 is inter-frame segmentation

[0210] - If sType is equal to 0, no motion information is stored.

[0211] - If sType is equal to 1, the motion information of the inter-frame segmentation is stored.

[0212] --If segmentation 1 is inter-frame segmentation, segmentation 2 is intra-frame segmentation

[0213] - If sType is equal to 0, the motion information of the inter-frame segmentation is stored.

[0214] - If sType is equal to 1, no motion information is stored.

[0215] GPM's Sports Memory with Adaptive Hybrid

[0216] In JVET-Z0127, a GPM with adaptive mixing is proposed. When a GPM with adaptive mixing is enabled, the mixing area may vary for different mixing widths θ. In some embodiments, for a GPM with adaptive mixing, the calculation of sType is modified as follows:

[0217] sType=abs(motionIdx)<16*θ? 2

[0218] (motionIdx≤0?(1-partIdx):partIdx)(16)

[0219] Fig.18A computing environment 1610 is shown coupled to a user interface 1650. The computing environment 1610 may be part of a data processing server. The computing environment 1610 includes a processor 1620, a memory 1630, and an input / output (I / O) interface 1640.

[0220] The processor 1620 generally controls the overall operation of the computing environment 1610, such as operations associated with display, data acquisition, data communication, and image processing. The processor 1620 may include one or more processors for executing instructions to perform all or some of the steps in the above method. In addition, the processor 1620 may include one or more modules that facilitate the interaction between the processor 1620 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a graphics processing unit (GPU), etc.

[0221] The memory 1630 is configured to store various types of data to support the operation of the computing environment 1610. The memory 1630 may include predetermined software 1632. Examples of such data include instructions for any application or method operating on the computing environment 1610, video data sets, image data, etc. The memory 1630 may be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0222] I / O interface 1640 provides an interface between processor 1620 and peripheral interface modules (e.g., keyboard, click wheel, buttons, etc.). Buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. I / O interface 1640 may be coupled to an encoder and a decoder.

[0223] In one or more embodiments, a non-transitory computer-readable storage medium including a plurality of programs in, for example, a memory 1630 is also provided, wherein the plurality of programs can be executed by a processor 1620 in a computing environment 1610 to perform the above method. Alternatively, the non-transitory computer-readable storage medium may store a plurality of programs generated by an encoder (for example, Fig.17 The video encoder 20 in FIG. 1 generates a video signal for a decoder (eg, Fig.17 A bitstream or data stream including encoded video information (e.g., video information including one or more syntax elements) used by the video decoder 30 in the video decoder 30 when decoding video data. The non-transitory computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0224] In one or more embodiments, a computing device is also provided, comprising: one or more processors (e.g., processor 1620); and a non-transitory computer-readable storage medium or memory 1630 in which multiple programs that can be executed by the one or more processors are stored, wherein the one or more processors are configured to perform the above-mentioned method when executing the multiple programs.

[0225] In one or more embodiments, a computer program product including a plurality of programs in, for example, memory 1630 is also provided, which may be executed by processor 1620 in computing environment 1610 to perform the above method. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0226] In one or more embodiments, the computing environment 1610 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.

[0227] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0228] Unless otherwise specifically stated, the order of steps of the method according to the present disclosure is intended to be illustrative only, and the steps of the method according to the present disclosure are not limited to the order specifically described above, but can be changed according to actual conditions. In addition, at least one of the steps of the method according to the present disclosure can be adjusted, combined or deleted according to actual needs.

[0229] The examples are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it will be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

[0230] Fig. 20 is a flow chart illustrating a method for video decoding according to an example of the present disclosure. The example method may be implemented by a decoder.

[0231] In step 2001, the processor 1620 at the decoder side may obtain at least one of a first motion vector or a second motion vector, wherein in the geometric partitioning mode, the coding unit is geometrically partitioned into a geometrically partitioned first part and a geometrically partitioned second part for prediction, the first motion vector is from the first part and the second motion vector is from the second part. For example, Figure 4 As shown in , the CU is split into two parts by a geometric positioning line.

[0232] In step 2002, the processor 1620 at the decoder side may determine a value of a stored motion vector type based on a motion index and a partial index.

[0233] In step 2003, the processor 1620 on the decoder side may store a motion vector in a corresponding motion field based on the value of the stored motion vector type, wherein the motion vector may include a first motion vector, a second motion vector, or a combined motion vector obtained based on the first motion vector and the second motion vector.

[0234] In some examples, processor 1620 may also determine a value of the stored motion vector type based on the blend width, the motion index, and the partial index. The stored motion vector type may be, but is not limited to, sType in equation (15) or (16). The blend width may be, for example, Fig.15 or Fig.16 The blend width or blend area width θ is shown. The motion index may be, but is not limited to, motionIdx in equation (15) or (16), and the part index may be, but is not limited to, partIdx in equation (15) or (16).

[0235] In some examples, the value of the blend width depends on the blend width index, and the partial index depends on the angle index. In some examples, the angle index indicates an index of an angle parameter of a segmentation mode of the geometric segmentation.

[0236] In some examples, the first portion of the geometric partition and the second portion of the geometric partition are both predicted using inter mode.

[0237] In another example, when the value of the stored motion vector type is 0 or 1, the decoder stores the first motion vector or the second motion vector in the corresponding motion field; and when the value of the stored motion vector type is 2, the decoder obtains a combined motion vector based on the first motion vector and the second motion vector, and stores the combined motion vector in the corresponding motion field.

[0238] In one example, when the first motion vector and the second motion vector come from different reference pictures in different reference picture lists, the decoder determines that the combined motion vector is a combination of the first motion vector and the second motion vector, wherein the combined motion vector is a bidirectionally predicted motion vector; and when the first motion vector and the second motion vector come from a reference picture in the same reference picture list, the decoder determines the combined motion vector to be the second motion vector, wherein the combined motion vector is a unidirectionally predicted motion vector.

[0239] In one example, the geometric partitioning modes are intra and inter prediction.

[0240] In another example, the decoder stores the first motion vector or the second motion vector as a uni-directionally predicted motion vector in the corresponding motion field.

[0241] In one example, when the value of the stored motion vector type is 2, the decoder stores the inter-frame segmented motion vector in the corresponding motion field.

[0242] In one example, the first part of the geometric segmentation is an intra segmentation and the second part of the geometric segmentation is an inter segmentation.

[0243] In another example, when the value of the stored motion vector type is 0, the decoder does not store the motion vector in the corresponding motion field.

[0244] In some examples, when the value of the stored motion vector type is 1, the decoder stores the inter-segmented motion vector in the corresponding motion field, wherein the inter-segmented motion vector is the second motion vector.

[0245] In one example, the first part of the geometric segmentation is an inter segmentation and the second part of the geometric segmentation is an intra segmentation.

[0246] In another example, when the value of the stored motion vector type is 0, the decoder stores the inter-frame segmented motion vector in the corresponding motion field, wherein the inter-frame segmented motion vector is the first motion vector.

[0247] In another example, when the value of the stored motion vector type is 1, the decoder does not store the motion vector in the corresponding motion field.

[0248] Fig.21 is shown as Fig. 20 The method for video decoding shown corresponds to a flow chart of a method for video encoding. The example method may be implemented by an encoder.

[0249] In step 2101, the processor 1620 at the encoder side may obtain at least one of a first motion vector or a second motion vector, wherein in the geometric partitioning mode, the coding unit is geometrically partitioned into a geometrically partitioned first part and a geometrically partitioned second part for prediction, the first motion vector is from the first part and the second motion vector is from the second part. Figure 4 As shown in , the CU is split into two parts by a geometric positioning line.

[0250] In step 2102, the processor 1620 at the encoder side may determine a value of a stored motion vector type based on the motion index and the partial index.

[0251] In step 2103, the processor 1620 on the encoder side may store a motion vector in a corresponding motion field based on the value of the stored motion vector type, wherein the motion vector may include a first motion vector, a second motion vector, or a combined motion vector obtained based on the first motion vector and the second motion vector.

[0252] In some examples, processor 1620 may also determine a value of the stored motion vector type based on the blend width, the motion index, and the partial index. The stored motion vector type may be, but is not limited to, sType in equation (15) or (16). The blend width may be, for example, Fig.15 or Fig.16 The blend width or blend area width θ is shown. The motion index may be, but is not limited to, motionIdx in equation (15) or (16), and the part index may be, but is not limited to, partIdx in equation (15) or (16).

[0253] In some examples, the value of the blend width depends on the blend width index, and the partial index depends on the angle index. In some examples, the angle index indicates an index of an angle parameter of a segmentation mode of the geometric segmentation.

[0254] In some examples, the first portion of the geometric partition and the second portion of the geometric partition are both predicted using inter mode.

[0255] In another example, when the value of the stored motion vector type is 0 or 1, the encoder stores the first motion vector or the second motion vector in the corresponding motion field; and when the value of the stored motion vector type is 2, the encoder obtains a combined motion vector based on the first motion vector and the second motion vector, and stores the combined motion vector in the corresponding motion field.

[0256] In one example, when the first motion vector and the second motion vector come from different reference pictures in different reference picture lists, the encoder determines that the combined motion vector is a combination of the first motion vector and the second motion vector, wherein the combined motion vector is a bidirectionally predicted motion vector; and when the first motion vector and the second motion vector come from reference pictures in the same reference picture list, the encoder determines that the combined motion vector is the second motion vector, wherein the combined motion vector is a unidirectionally predicted motion vector.

[0257] In one example, the geometric partitioning modes are intra and inter prediction.

[0258] In another example, the encoder stores the first motion vector or the second motion vector as a uni-directionally predicted motion vector in the corresponding motion field.

[0259] In one example, when the value of the stored motion vector type is 2, the encoder stores the inter-frame segmented motion vector in the corresponding motion field.

[0260] In one illustration, the first portion of the geometric segmentation is an intra-frame segmentation and the second portion of the geometric segmentation is an inter-frame segmentation.

[0261] In another example, when the value of the stored motion vector type is 0, the encoder does not store the motion vector in the corresponding motion field.

[0262] In some examples, when the value of the stored motion vector type is 1, the encoder stores the inter-segmented motion vector in the corresponding motion field, wherein the inter-segmented motion vector is the second motion vector.

[0263] In one example, the first part of the geometric segmentation is an inter segmentation and the second part of the geometric segmentation is an intra segmentation.

[0264] In another example, when the value of the stored motion vector type is 0, the encoder stores the inter-frame segmented motion vector in the corresponding motion field, wherein the inter-frame segmented motion vector is the first motion vector.

[0265] In another example, when the value of the stored motion vector type is 1, the encoder does not store the motion vector in the corresponding motion field.

[0266] The above method can be implemented using a device including one or more circuits, including an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components. The device can use circuits combined with other hardware or software components to perform the above method. Each of the above modules, sub-modules, units or sub-units can be implemented at least in part using one or more circuits.

[0267] In view of the description and practice of the present disclosure disclosed herein, other examples of the present disclosure will be apparent to those skilled in the art. This application is intended to cover any variation, use or adjustment of the present disclosure that follows the general principles of the present disclosure, and includes deviations from the present disclosure that fall within known or customary practices in the art. It is intended that the description and examples be considered exemplary only.

[0268] It should be understood that the present disclosure is not limited to the exact examples described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for motion field storage, comprising: Obtaining, by a decoder, at least one of a first motion vector or a second motion vector, wherein, in a geometric partitioning mode, the coding unit is geometrically partitioned into a geometrically partitioned first portion and a geometrically partitioned second portion for prediction, and the first motion vector is from the first portion and the second motion vector is from the second portion; determining, by the decoder, a value of a stored motion vector type based on the motion index and the partial index; and The decoder stores a motion vector in a corresponding motion field based on the stored value of the motion vector type, wherein the motion vector includes a first motion vector, a second motion vector, or a combined motion vector obtained based on the first motion vector and the second motion vector.

2. The method according to claim 1, wherein: Determining, by the decoder, a value of a stored motion vector type based on the motion index and the partial index, comprises: A value of a stored motion vector type is determined by the decoder based on a mix width, the motion index, and the partial index, wherein the value of the mix width depends on the mix width index, and wherein the partial index depends on the angle index.

3. The method according to claim 2, wherein: Both the first part of the geometric partition and the second part of the geometric partition are predicted using the inter mode.

4. The method according to claim 3, wherein: Storing, by the decoder, the motion vector in a corresponding motion field based on a stored value of the motion vector type, comprises: In response to determining that the value of the stored motion vector type is 0 or 1, storing, by the decoder, the first motion vector or the second motion vector in the corresponding motion field; and In response to determining that the value of the stored motion vector type is 2, a combined motion vector is obtained by the decoder based on the first motion vector and the second motion vector, and the combined motion vector is stored by the decoder in the corresponding motion field.

5. The method according to claim 4, further comprising: In response to determining that the first motion vector and the second motion vector are from different reference pictures in different reference picture lists, determining, by the decoder, that the combined motion vector is a combination of the first motion vector and the second motion vector, wherein the combined motion vector is a bi-predictive motion vector; and In response to determining that the first motion vector and the second motion vector are from a same reference picture in a reference picture list, determining, by the decoder, that a combined motion vector is a second motion vector, wherein the combined motion vector is a uni-directionally predicted motion vector.

6. The method according to claim 2, wherein: The geometric partitioning modes are intra and inter prediction.

7. The method according to claim 6, further comprising: The first motion vector or the second motion vector is stored by the decoder as a unidirectional prediction motion vector in a corresponding motion field.

8. The method according to claim 6, further comprising: In response to determining that the value of the stored motion vector type is 2, the inter-frame segmented motion vector is stored by the decoder in the corresponding motion field.

9. The method according to claim 6, wherein: The first part of the geometric segmentation is intra segmentation and the second part of the geometric segmentation is inter segmentation.

10. The method according to claim 9, further comprising: In response to determining that the value of the stored motion vector type is 0, no motion vector is stored by the decoder in the corresponding motion field.

11. The method according to claim 9, further comprising: In response to determining that the value of the stored motion vector type is 1, the decoder stores the inter-frame segmented motion vector in the corresponding motion field, wherein the inter-frame segmented motion vector is a second motion vector.

12. The method according to claim 6, wherein: The first part of the geometric segmentation is inter-frame segmentation and the second part of the geometric segmentation is intra-frame segmentation.

13. The method according to claim 12, further comprising: In response to determining that the value of the stored motion vector type is 0, the decoder stores the inter-frame segmented motion vector in the corresponding motion field, wherein the inter-frame segmented motion vector is a first motion vector.

14. The method according to claim 12, further comprising: In response to determining that the value of the stored motion vector type is 1, no motion vector is stored by the decoder in the corresponding motion field.

15. A method for motion field storage, comprising: Obtaining, by an encoder, at least one of a first motion vector or a second motion vector, wherein, in a geometric partitioning mode, the coding unit is geometrically partitioned into a geometrically partitioned first portion and a geometrically partitioned second portion for prediction, and the first motion vector is from the first portion and the second motion vector is from the second portion; determining, by the encoder, a value of a stored motion vector type based on the motion index and the partial index; and The decoder stores a motion vector in a corresponding motion field based on the stored value of the motion vector type, wherein the motion vector includes a first motion vector, a second motion vector, or a combined motion vector obtained based on the first motion vector and the second motion vector.

16. The method according to claim 15, wherein: Determining, by the encoder, a value of a stored motion vector type based on the motion index and the partial index, comprises: A value of a stored motion vector type is determined by the encoder based on a blend width, the motion index, and the partial index, wherein the value of the blend width depends on the blend width index, and wherein the partial index depends on the angle index.

17. The method according to claim 16, wherein: Both the first part of the geometric partition and the second part of the geometric partition are predicted using the inter mode.

18. The method according to claim 17, wherein: Storing, by the encoder, the motion vector in a corresponding motion field based on a stored value of the motion vector type, comprises: In response to determining that the value of the stored motion vector type is 0 or 1, storing, by the encoder, the first motion vector or the second motion vector in the corresponding motion field; and In response to determining that the value of the stored motion vector type is 2, a combined motion vector is obtained by the decoder based on the first motion vector and the second motion vector, and the combined motion vector is stored by the encoder in the corresponding motion field.

19. The method according to claim 18, further comprising: In response to determining that the first motion vector and the second motion vector are from different reference pictures in different reference picture lists, determining, by the encoder, that the combined motion vector is a combination of the first motion vector and the second motion vector, wherein the combined motion vector is a bi-predictive motion vector; and In response to determining that the first motion vector and the second motion vector are from a same reference picture in a reference picture list, determining, by the encoder, that the combined motion vector is a second motion vector, wherein the combined motion vector is a uni-directionally predicted motion vector.

20. The method according to claim 16, wherein: The geometric partitioning modes are intra and inter prediction.

21. The method according to claim 20, further comprising: The first motion vector or the second motion vector is stored by the encoder as a unidirectional prediction motion vector in a corresponding motion field.

22. The method according to claim 20, further comprising: In response to determining that the value of the stored motion vector type is 2, the inter-frame segmented motion vector is stored by the encoder in the corresponding motion field.

23. The method according to claim 20, wherein: The first part of the geometric segmentation is intra segmentation and the second part of the geometric segmentation is inter segmentation.

24. The method according to claim 23, further comprising: In response to determining that the value of the stored motion vector type is 0, no motion vector is stored by the encoder in the corresponding motion field.

25. The method of claim 23, further comprising: In response to determining that the value of the stored motion vector type is 1, the encoder stores the inter-frame segmented motion vector in the corresponding motion field, wherein the inter-frame segmented motion vector is a second motion vector.

26. The method of claim 20, wherein: The first part of the geometric segmentation is inter-frame segmentation and the second part of the geometric segmentation is intra-frame segmentation.

27. The method according to claim 26, further comprising: In response to determining that the value of the stored motion vector type is 0, the encoder stores the inter-segmented motion vector in the corresponding motion field, wherein the inter-segmented motion vector is a first motion vector.

28. The method of claim 26, further comprising: In response to determining that the value of the stored motion vector type is 1, no motion vector is stored by the encoder in the corresponding motion field.

29. An apparatus comprising: one or more processors; as well as a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, Wherein, when executing the instructions, the one or more processors are configured to perform the method according to any one of claims 1 to 28.

30. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform the method of any one of claims 1-14 based on the bitstream.

31. A non-transitory computer-readable storage medium for storing computer-executable instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform the method according to any one of claims 15-28 to encode the current block into a bitstream, and transmit the bitstream.

32. A non-transitory computer-readable storage medium for storing a bit stream to be decoded by the method of any one of claims 1-14.

33. A non-transitory computer-readable storage medium for storing a bitstream generated by the method of any one of claims 15-28.

Images