Method and apparatus for rounding, truncating and storing motion vectors for inter-frame prediction

Abstract

A method for video encoding and decoding is provided, comprising: setting a motion vector (MV) rounding process, the MV rounding process being configured to perform MV rounding operations; applying the MV rounding process to at least a first MV used in a first prediction tool, the first prediction tool being selected from a first group of tools including: pair-wise merge candidates, a triangular prediction mode, and an affine mode; and applying the MV rounding process to at least a second MV used in a second prediction tool, the second prediction tool being selected from a second group of tools including: temporal motion vector prediction (TMVP), advanced temporal motion vector prediction (ATMVP), and a merge mode with motion vector difference (MMVD).

Description

[0001] This application is a divisional application of an application with a filing date of February 19, 2020, an application number of 202080014012.3 and an invention name of “Method and device for rounding, truncation and storage of motion vectors for inter-frame prediction”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Application No. 62 / 808,276, filed on February 20, 2019, entitled “Methods and Apparatus on Motion Vector Rounding and Storage for Inter Prediction,” and U.S. Provisional Application No. 62 / 816,025, filed on March 8, 2019, entitled “Methods and Apparatuses on Motion Vector Rounding and Storage for Inter Prediction,” both of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0004] The present application relates generally to video coding and compression, and in particular, but not limited to, methods and apparatus for motion vector rounding, truncation and storage for video coding. Background Art

[0005] Digital video is supported by various electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. These electronic devices transmit, receive, encode, decode and / or store digital video data by implementing video compression / decompression. Digital video devices implement video codec technologies, such as those described in standards defined by Versatile Video Codec (VVC), Joint Exploration Test Model (JEM), MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Codec (AVC), ITU-TH.265 / High Efficiency Video Codec (HEVC), and extensions of such standards.

[0006] Video codecs typically utilize prediction methods (e.g., inter-frame prediction, intra-frame prediction) that exploit the redundancy present in video images or sequences. An important goal of video codec technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality. As evolving video services become available, coding techniques with better codec efficiency are needed.

[0007] Video compression typically includes performing spatial domain (intra-frame) prediction and / or temporal domain (inter-frame) prediction to reduce or remove redundancy inherent in video data. In block-based video codecs, the input video signal is processed block by block. For each block (also called coding unit (CU)), spatial domain prediction and / or temporal domain prediction can be performed. Each CU can be encoded and decoded using intra-frame, inter-frame or IBC mode. Video blocks in an intra-frame codec (I) slice of a video frame are encoded using spatial domain prediction relative to reference samples in adjacent blocks within the same video frame. Video blocks in an inter-frame codec (P or B) slice of a video frame can use spatial domain prediction relative to reference samples in adjacent blocks within the same video frame or temporal prediction relative to reference samples in other previous and / or future reference video frames.

[0008] A prediction block for the current video block to be encoded is generated based on a spatial or temporal prediction of a previously encoded reference block (e.g., a neighboring block). The process of finding a reference block can be accomplished by a block matching algorithm. The residual data representing the pixel differences between the current block to be encoded and the prediction block is called a residual block or prediction error. The inter-frame encoding and decoding block is encoded based on the motion vector pointing to the reference block in the reference frame that forms the prediction block and the residual block. The process of determining the motion vector is generally called motion estimation. The intra-frame encoding and decoding block is encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, such as the frequency domain, to produce residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which are then entropy encoded into the video bitstream to achieve even further compression.

[0009] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or is directly transmitted to the electronic device by wire or wireless. The electronic device then performs video decompression (which is the reverse process of the above-mentioned video compression) by, for example, parsing the encoded video bitstream to obtain semantic elements from the bitstream and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the semantic elements obtained from the bitstream, and presents the reconstructed digital video data on a display of the electronic device.

[0010] As digital video quality increases from HD to 4Kx2K or even 8Kx4K, the amount of video data to be encoded / decoded increases exponentially. How to encode / decode video data more efficiently while maintaining the image quality of the decoded video data is an ongoing challenge.

[0011] At the Joint Video Experts Group (JVET) meeting, JVET defined the first draft of the Universal Video Codec (VVC) and the VVC Test Model 1 (VTM l) coding method. It was decided to include a quadtree with nested multi-type trees using binary and ternary partitioning codec block structures as the initial new codec feature of VVC. Since then, the reference software VTM for implementing the coding method and the VVC decoding process draft has been developed during the JVET meeting. Summary of the invention

[0012] In general, this disclosure describes examples of techniques related to motion vector rounding and truncation for video encoding and decoding.

[0013] According to a first aspect of the present disclosure, a method for video encoding and decoding is provided, including: setting a motion vector (MV) rounding process, the MV rounding process being configured to perform MV rounding operations; applying the MV rounding process to at least a first MV used in a first prediction tool, the first prediction tool being selected from a group consisting of: pair-wise merge candidates, a triangular prediction mode, and an affine mode; and applying the MV rounding process to at least a second MV used in a second prediction tool, the second prediction tool being selected from a group consisting of: temporal motion vector prediction (TMVP) and advanced temporal motion vector prediction (ATMVP) and a merge mode with motion vector difference (MMVD).

[0014] According to a second aspect of the present disclosure, a method for video encoding and decoding is provided, including: setting a motion vector (MV) clipping process, the MV clipping process being configured to perform a clipping operation; applying the MV clipping process to at least a first MV used in a first prediction tool, the first prediction tool being selected from a group consisting of: an advanced motion vector prediction (AMVP) mode, an advanced temporal motion vector prediction (ATMVP) mode, a triangle mode, a merge mode with motion vector difference (MMVD) mode, a decoder side motion vector derivation (DMVR) mode, and an affine inheritance mode; and applying the MV clipping process to at least a second MV used in a second prediction tool, the second prediction tool being selected from a group consisting of: an explicit normal inter mode and an explicit affine mode.

[0015] According to a third aspect of the present disclosure, a device for video encoding and decoding is provided, comprising: a processor; and a memory configured to store instructions executable by the processor; wherein the processor is configured, when executing the instructions, to: set a motion vector (MV) rounding process, the MV rounding process being configured to perform MV rounding operations; apply the MV rounding process to at least a first MV used in a first prediction tool, the first prediction tool being selected from a group consisting of: pair-wise merge candidates, a triangular prediction mode, and an affine mode; and apply the MV rounding process to at least a second MV used in a second prediction tool, the second prediction tool being selected from a group consisting of: temporal motion vector prediction (TMVP) and advanced temporal motion vector prediction (ATMVP) and a merge mode with motion vector difference (MMVD).

[0016] According to a fourth aspect of the present disclosure, a device for video encoding and decoding is provided, comprising: a processor; and a memory configured to store instructions executable by the processor; wherein the processor is configured, when executing the instructions, to: set a motion vector (MV) clipping process, the MV clipping process being configured to perform a clipping operation; apply the MV clipping process to at least a first MV used in a first prediction tool, the first prediction tool being selected from a group consisting of: an advanced motion vector prediction (AMVP) mode, an advanced temporal motion vector prediction (ATMVP) mode, a triangle mode, a mode with a merge mode of motion vector difference (MMVD), a decoder-side motion vector derivation (DMVR) mode, and an affine inheritance mode; and apply the MV clipping process to at least a second MV used in a second prediction tool, the second prediction tool being selected from a group consisting of: an explicit normal inter mode and an explicit affine mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] A more specific description of examples of the present disclosure will be presented by reference to specific examples illustrated in the accompanying drawings. Given that these drawings depict only some examples and are therefore not to be considered limiting of scope, these examples will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0018] Figure 1 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0019] Figure 2 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0020] Figure 3 is a schematic diagram illustrating block partitioning in a multi-type tree structure according to some embodiments of the present disclosure.

[0021] Figure 4is a schematic diagram illustrating locations of spatial domain merging candidates in a VVC according to some implementations of the present disclosure.

[0022] Figure 5 is a schematic diagram illustrating an MV scaling operation for temporal motion vector prediction (TMVP) according to some implementations of the present disclosure.

[0023] Figure 6 is a schematic diagram illustrating a 4-parameter affine model according to some embodiments of the present disclosure.

[0024] Figure 7 is a schematic diagram illustrating a 6-parameter affine model according to some embodiments of the present disclosure.

[0025] Figure 8 is a schematic diagram illustrating an affine CPMV storage method according to some embodiments of the present disclosure.

[0026] Fig. 9 is a schematic diagram illustrating triangular prediction partitioning according to some implementations of the present disclosure.

[0027] Fig.10 is a schematic diagram illustrating locations of candidate blocks for generating a unidirectional prediction MV list for a triangular prediction mode according to some embodiments of the present disclosure.

[0028] Fig.11 is a schematic diagram illustrating a weighted average of two unidirectional prediction samples along a partition edge of a triangular prediction mode according to some embodiments of the present disclosure.

[0029] Fig.12 is a schematic diagram illustrating a mapping function for absolute value-based MV rounding according to some embodiments of the present disclosure.

[0030] Fig.13 is a schematic diagram illustrating a mapping function for TMVP / ATMVP based MV rounding according to some embodiments of the present disclosure.

[0031] Fig.14 is a schematic diagram illustrating a mapping function for conventional MV rounding according to some embodiments of the present disclosure.

[0032] Fig.15 is a schematic diagram illustrating an example of CPMV and MV context update procedures for actual VVC implementations according to some embodiments of the present disclosure.

[0033] Fig.16 is a diagram illustrating an example of mantissa-exponent representation of time-domain MV storage according to some implementations of the present disclosure.

[0034] Fig.17is a block diagram illustrating an exemplary apparatus for video encoding and decoding according to some embodiments of the present disclosure.

[0035] Fig.18 is a flow chart illustrating an exemplary process of motion vector rounding for video encoding and decoding according to some embodiments of the present disclosure.

[0036] Fig.19 is a flowchart illustrating an exemplary process of motion vector extraction for video encoding and decoding according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0037] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on various types of electronic devices having digital video capabilities.

[0038] References throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that the particular feature, structure, or characteristic being described is included in at least one embodiment or example. Unless explicitly stated otherwise, features, structures, elements, or characteristics described in conjunction with one or some embodiments are also applicable to other embodiments.

[0039] Throughout the disclosure, the terms "first", "second", "third", etc. are used as special terms and are used only to refer to relevant elements, such as devices, parts, components, steps, etc., and do not imply any spatial or temporal order unless otherwise explicitly stated. For example, "first device" and "second device" may refer to two separately formed devices, two parts, components or operating states of the same device, and may be named arbitrarily.

[0040] As used herein, the terms “if” or “when” may be understood to mean “depending on” or “in response to.” These terms, if present in a claim, may not indicate that the associated limitation or feature is conditional or optional.

[0041] The terms "module", "sub-module", "circuit", "sub-circuit", "circuitry", "sub-circuitry", "unit" or "sub-unit" may include a memory (shared, dedicated or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components connected directly or indirectly. These components may or may not be physically attached to each other or located adjacent to each other.

[0042] A unit or module may be implemented purely by software, purely by hardware, or by a combination of hardware and software. For example, in a pure software implementation, a unit or module may include functionally related code blocks or software components that are linked together directly or indirectly to perform a specific function.

[0043] Figure 1 A block diagram illustrating an exemplary block-based hybrid video encoder 100 is shown, which can be used in conjunction with many video codec standards that use block-based processing. In the encoder 100, a video frame is segmented into multiple video blocks for processing. For each given video block, a prediction is formed based on an inter-frame prediction method or an intra-frame prediction method. In inter-frame prediction, one or more prediction values ​​are formed through motion estimation and motion compensation based on pixels from a previously reconstructed frame. In intra-frame prediction, prediction values ​​are formed based on reconstructed pixels in the current frame. Through mode decision, the best prediction value can be selected to predict the current block.

[0044] The prediction residual representing the difference between the current video block and its predicted value is sent to the transform circuitry 102. The transform coefficients are then sent from the transform circuitry 102 to the quantization circuitry 104 for entropy reduction. The quantized coefficients are then fed to the entropy encoding and decoding circuitry 106 to generate a compressed video bitstream. Figure 1 As shown in , prediction-related information 110 (such as video block segmentation information, motion vectors, reference picture indexes, and intra-prediction modes) from the inter-prediction circuit system and / or the intra-prediction circuit system 112 is also fed through the entropy encoding and decoding circuit system 106 and saved as a compressed video bitstream 114.

[0045] In encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by inverse quantization circuitry 116 and inverse transform circuitry 118. The reconstructed prediction residual is combined with block prediction value 120 to generate unfiltered reconstructed pixels of the current video block.

[0046] Spatial domain prediction (or "intra-frame prediction") uses pixel points of samples from already coded neighboring blocks (called reference samples) in the same video frame as the current video block to predict the current video block.

[0047] Temporal prediction (also called "inter prediction" or "motion compensated prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in video signals. The temporal prediction signal for a given codec unit (CU) or codec block is typically signaled by one or more motion vectors (MVs) that indicate the amount and direction of motion between the current CU and its temporal reference picture. Furthermore, if multiple reference pictures are supported, an additional reference picture index is sent to identify which reference picture in the reference picture storage the temporal prediction signal comes from.

[0048] After performing spatial and / or temporal prediction, the intra / inter mode decision circuitry 121 in the encoder 100 selects the best prediction mode, for example based on a rate distortion optimization method. The block prediction value 120 is then subtracted from the current video block; and the resulting prediction residual is decorrelated using the transform circuitry 102 and the quantization circuitry 104. The resulting quantized residual coefficients are inverse quantized by the inverse quantization circuitry 116 and inverse transformed by the inverse transform circuitry 118 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Further loop filtering 115 (such as a deblocking filter, sample adaptive offset (SAO) and / or adaptive loop filter (ALF)) may be applied to the reconstructed CU before it is placed into the reference picture storage of the picture buffer 117 and used to encode and decode future video blocks. To form the output video bitstream 114, the codec mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy codec circuitry 106 to be further compressed and packed to form a bitstream.

[0049] For example, deblocking filters can be used in the current versions of AVC, HEVC, and VVC. In HEVC, an additional loop filter called SAO (Sample Adaptive Offset) is defined to further improve the coding efficiency. In the current version of the VVC standard, another loop filter called ALF (Adaptive Loop Filter) is being actively studied and it is likely to be included in the final standard.

[0050] These loop filter operations are optional. Performing these operations helps improve codec efficiency and visual quality. They can also be turned off as a decision presented by the encoder 100 to save computational complexity.

[0051] It should be noted that intra prediction is usually based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels if these filter options are turned on by the encoder 100.

[0052] Figure 2 is a block diagram illustrating an exemplary block-based video decoder 200 that can be used in conjunction with many video codec standards. The decoder 200 is similar to the one residing in Figure 1 The decoder 200 first decodes the incoming video bitstream 201 by entropy decoding 202 to derive the quantization coefficient levels and prediction related information. The quantization coefficient levels are then processed by inverse quantization 204 and inverse transform 206 to obtain reconstructed prediction residuals. The block prediction value mechanism implemented in the intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixel points is obtained by adding the reconstructed prediction residuals from the inverse transform 206 and the prediction output generated by the block prediction value mechanism using an adder 214.

[0053] The reconstructed block may also pass through a loop filter 209 before it is stored in a picture buffer 213, which is used as a reference picture store. The reconstructed video in the picture buffer 213 may be sent to drive a display device, as well as used to predict future video blocks. With the loop filter 209 turned on, a filtering operation is performed on these reconstructed pixels to derive a final reconstructed video output 222.

[0054] Figure 3 is a schematic diagram illustrating block partitioning in a multi-type tree structure in VVC. Like HEVC, VVC is built on a block-based hybrid video codec framework. However, unlike HEVC, which partitions blocks only based on a quadtree (i.e., quadtree), in VVC, a codec tree unit (CTU) is divided into codec units (CU) to accommodate various local characteristics based on quadtrees, binary trees, or ternary trees. In addition, the concept of multiple partition unit types in HEVC is removed in VVC, that is, there is no distinction between CUs, prediction units (PUs), and transform units (TUs) in VVC; on the contrary, each CU is always used as a basic unit for prediction and transformation without further partitioning. In the multi-type tree structure, a CTU is first partitioned by a quadtree structure. Then, each quadtree leaf node can be further partitioned into CUs by a binary tree or ternary tree structure. As Figure 3 As shown in , currently five types of partitioning are used in VVC: quadruple partitioning 301 , horizontal binary partitioning 302 , vertical binary partitioning 303 , horizontal ternary partitioning 304 , and vertical ternary partitioning 305 .

[0055] Regular merge mode.

[0056] As with HEVC, merge mode is supported in VVC, where the motion information of a codec block is not signaled but derived from a set of spatial and / or temporal merge candidates based on a contention-based scheme; and accordingly, only the index of the selected merge candidate needs to be signaled from the encoder to the decoder to reconstruct the motion information.

[0057] To build a list of merge candidates, spatial domain motion vector candidates are first checked and added to the list. Figure 4 The diagram shows the locations of spatial domain merge candidates. Five spatial domain merge candidates are checked and added in the order A1 → B1 → B0 → A0 → B2. If a block at one of the spatial locations is intra-coded or outside the boundaries of the current slice, patch and / or picture, it is considered unavailable.

[0058] After all valid spatial domain candidates are inserted into the merge candidate list, temporal candidates are generated from the motion information of the co-located block in the co-located reference picture by using the temporal motion vector prediction (TMVP) technique. A scaled motion vector is derived based on the motion information of the co-located block in the co-located reference picture signaled in the patch group or slice header. Figure 5 The motion vector scaling operation for temporal motion vector prediction (TMVP) is shown. The scaled motion vector of the temporal merge candidate is obtained by scaling the motion vector of the co-located block col_PU using the picture order count (POC) distances tb and td, as Figure 5 As shown by the dotted line in , where tb is defined as the POC difference between the reference picture curr_ref of the current picture and the current picture curr_pic, and td is defined as the POC difference between the reference picture of the co-located picture col_ref and the co-located picture col_pic.

[0059] When both spatial and temporal motion vectors are inserted into the merge candidate list and the total number of existing merge candidates in the list is less than the maximum size of the merge list (which is signaled in the patch group header), the history-based merge candidates are added. The so-called history-based merge candidates include those motion vectors from previously coded CUs, which are kept in a separate motion vector list and are managed based on specific rules such as the first-in-first-out (FIFO) rule.

[0060] After inserting the history-based candidates, if the merge candidate list is not full, further pairwise average motion vector candidates are added to the list. As the name suggests, this type of candidate is constructed by averaging the candidates already in the current list. More specifically, based on a specific order or rule, two candidates in the merge candidate list are taken at a time and the average motion vector of these two candidates is appended to the current list. After inserting the pairwise average motion vectors, if the merge candidate list is still not full, zero motion vectors are added to make the list full.

[0061] Affine mode.

[0062] In HEVC, only the translational motion model is applied to motion compensated prediction. However, in the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, affine motion compensated prediction is used by signaling a flag for each inter-frame codec block to indicate whether the translational motion model or the affine motion model is applied to inter-frame prediction. For the current VVC design, an affine codec block can support two affine modes, including a 4-parameter affine model and a 6-parameter affine model. Figure 6 and 7 The 4-parameter affine model and the 6-parameter affine model are illustrated respectively.

[0063] The 4-parameter affine model has the following parameters: two parameters for translation motion in the horizontal and vertical directions, one parameter for zoom motion in both the horizontal and vertical directions, and one parameter for rotation motion in both the horizontal and vertical directions, where the horizontal zoom parameter is equal to the vertical zoom parameter, and the horizontal rotation parameter is equal to the vertical rotation parameter. In order to better adapt the motion vector and affine parameters, in VVC, these affine parameters are converted into two MVs (also called control point motion vectors (CPMVs)) located at the upper left and upper right corners of the current block. Figure 6 As shown in , the affine motion field vector of the block is described by two control points MV(v0,v1). Based on the motion of the control points, the motion field vector (v x , v y ) is described by the following equation:

[0064]

[0065] The 6-parameter affine mode has the following parameters: two parameters for horizontal and vertical translation, one parameter for horizontal zoom, one parameter for horizontal rotation, one parameter for vertical zoom, and one parameter for vertical rotation. The 6-parameter affine motion model is encoded and decoded with three MVs, which can also be called three CPMVs. Figure 7 As shown in , the three control points of a 6-parameter affine block are located at the upper left, upper right, and lower left corners of the block. The motion of the upper left control point is related to the translation motion; the motion of the upper right control point is related to the horizontal rotation and zoom motion; and the motion of the lower left control point is related to the vertical rotation and zoom motion. Compared with the 4-parameter affine motion model, the horizontal rotation and zoom motion of the 6-parameter affine motion model may be different from those in the vertical direction. Assume that (v0, v1, v2) is Figure 7 The MVs of the upper left corner, upper right corner, and lower left corner of the current block in the current block, then each sub-block (v x ,v y ) is derived using the three MVs at the control point by the following equation:

[0066]

[0067] In VVC, the CPMV of an affine codec block is stored in a separate buffer. The stored CPMV is only used to generate affine CPMV prediction values ​​for affine merge mode (i.e., inheriting the affine CPMV from the affine CPMV of the adjacent affine block) and affine explicit mode (i.e., signaling the affine CPMV according to a prediction-based scheme). The sub-block MV derived from the CPMV is used for motion compensation, MV prediction of translation MV, and deblocking.

[0068] Figure 8 The diagram illustrates the affine CPMV storage method in VVC. In order to avoid increasing the picture row buffer size for additional storage of CPMV, the affine motion data inheritance from the codec block of the above CTU is treated differently from the affine motion data inheritance from the adjacent CU in the same CTU. Specifically, for the current CU, if the spatial domain neighbor for affine motion data inheritance is in the above CTU row, the sub-block MV in the row buffer instead of the CPMV is used for AMVP export of the current CU. In this way, the CPMV is only stored in the local buffer (i.e., the affine block within a CTU) instead of in the row buffer. If the candidate CU is 6-parameter affine coded and decoded, the affine model is downgraded to a 4-parameter model. Figure 8 As shown in , along the top CTU boundary, the bottom left and bottom right sub-block motion vectors of the block are used for affine inheritance of the CU in the bottom CTU.

[0069] Triangular prediction model.

[0070] In VVC, triangular prediction mode is introduced for motion compensated prediction. Fig. 9 The figure shows the triangular prediction partition in VVC. Fig. 9As shown in , CU 902, 904 is divided into two triangular prediction units PU1 and PU2 in the diagonal direction or the anti-diagonal direction (i.e., divided from the upper left corner to the lower right corner or from the upper right corner to the lower left corner). Each triangular prediction unit in the CU is inter-predicted using its own unidirectional prediction motion vector and reference frame index, which are derived from the unidirectional prediction candidate list. Along the diagonal boundary between the two triangular prediction units, each 4x4 sub-block is predicted using two unidirectional prediction signals. After predicting the triangular prediction unit, an adaptive weighting process is performed on the diagonal edges. Then, the transformation and quantization process is applied to the entire CU. It should be noted that this mode is only applicable to the skip and merge modes in the current VVC. Although in Fig. 9 In FIG. 4 , the CU is shown as a square block, but the triangular prediction mode can also be applied to non-square (ie rectangular) shape CUs.

[0071] The uni-directional prediction MV candidate list may include one or more candidates, and each candidate may be a motion vector. Fig.10 The positions of candidate blocks for generating a unidirectional prediction MV list for triangular prediction mode are illustrated. In some examples, the unidirectional prediction motion vector candidate list may include two to five unidirectional prediction motion vector candidates. In some other examples, other numbers are possible. The unidirectional prediction motion vector candidate list is derived from seven neighboring blocks, including five spatial domain neighboring blocks (1 to 5) and two temporal co-located blocks (6 to 7), such as Fig.10 As shown in . The motion vectors of seven adjacent blocks are collected into a first merge list. Then, a unidirectional prediction candidate list is formed based on the first merge list motion vectors according to a specific order. According to the order, the unidirectional prediction motion vector from the first merge list is first put into the unidirectional prediction motion vector candidate list, followed by the reference picture list 0 or L0 motion vector of the bidirectional prediction motion vector, followed by the reference picture list 1 or L1 motion vector of the bidirectional prediction motion vector, and then the average motion vector of the L0 and L1 motion vectors of the bidirectional prediction motion vector. At this point, if the number of candidates is still less than the target number (5 in the current VVC), a zero motion vector is added to the list to meet the target number.

[0072] The corresponding prediction signal of each triangular partition is derived based on its unidirectional prediction MV. In addition, in order to mitigate the blocking artifacts along the diagonal or anti-diagonal edges between two partitions, a weighting process is applied to the two unidirectional prediction signals of the samples along the partition edges to derive the final prediction for the CU. Fig.11An example of a weighting process is shown where the values ​​{7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luma samples 1102 and chroma samples 1104, respectively.

[0073] Merge mode with Motion Vector Difference (MMVD).

[0074] In addition to the regular merge mode, in which the implicitly derived motion information is directly used for the generation of prediction samples for the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. The MMVD flag is signaled after the skip flag and merge flag are sent to specify whether the MMVD mode is used for the CU.

[0075] In MMVD, after the merge basis candidate is selected, it is further refined by signaled motion vector difference (MVD) information. Further information includes a merge candidate identifier, a distance index specifying the magnitude of the motion, and a direction index indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected as the MV basis (or starting point). The merge candidate identifier is signaled to specify which one to use.

[0076] The distance index specifies the motion magnitude information and indicates a predefined offset from the start point. As shown in Table 1, the offset is added to the horizontal component or the vertical component of the start MV.

[0077] Table 1 Relationship between distance index and predefined offset

[0078] Distance Index 0 1 2 3 4 5 6 7 Offset (in luma samples) 1 / 4 1 / 2 1 2 4 8 16 32 .

[0079] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions as shown in Table 2. The meaning of the MVD sign (such as + or -) may be different depending on the information of the starting MV. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV where both MVs point to the same side of the current picture (i.e., the POCs of both reference pictures are greater than the POC of the current picture, or are less than the POC of the current picture), the sign in Table 2 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional prediction MV and the two MVs point to different sides of the current picture (i.e., the POC of one reference picture is greater than the POC of the current picture, and the POC of the other reference picture is less than the POC of the current picture), the sign in Table 2 specifies the sign of the MV offset added to the list 0 MV component of the starting MV, and the sign of the list 1 MV has the opposite value.

[0080] Table 2 Symbols of MV offsets specified by direction index

[0081] Direction Index 00 01 10 11 X-axis + – N / A N / A Y-axis N / A N / A + – .

[0082] Decoder side motion vector derivation (DMVR).

[0083] Decoder-side motion vector derivation (DMVR) is a bidirectional prediction technique for merged blocks with two initially signaled MVs that can be further refined by using bidirectional matching prediction. Bidirectional matching is used to derive the motion information of the current CU by finding the best match between two blocks along the motion trajectory of the current CU in two different reference pictures. The cost function used in the matching process is the row subsample sum of absolute differences (SAD). After the matching process is completed, the refined MV is used for motion compensation in the prediction stage, boundary strength calculation in the deblocking filter, and temporal motion vector prediction for subsequent pictures. Under the assumption of continuous motion trajectory, the motion vectors MV0 and MV1 pointing to the two reference blocks should be proportional to the temporal distance between the current picture and the two reference pictures, i.e., TD0 and TD1. As a special case, when the current picture is located between the two reference pictures in the temporal domain and the temporal distances TD0 and TD1 from the current picture to the two reference pictures are the same, the bidirectional matching becomes a mirror-based bidirectional MV.

[0084] MV rounding operations need to be performed at multiple stages of motion compensated prediction in VVC:

[0085] 1) The stage of deriving the MV prediction value for TMVP and Advanced Temporal Motion Vector Prediction (ATMVP), where the MV of the co-located temporal block is based on the POC distance between the reference picture of the current picture and the current picture (i.e. Figure 5 tb in the same picture) and the POC distance between the reference picture of the same picture and the same picture (i.e. Figure 5 td) ratio is scaled;

[0086] 2) for the stage of generating pairwise merging candidates by averaging the MVs of two spatial / temporal MV candidates;

[0087] 3) For an affine codec block, the motion field vector (field) of the corresponding sub-block is derived from the corresponding CPMV according to equation (1) for a 4-parameter affine model and equation (2) for a 6-parameter affine model; and

[0088] 4) For triangular prediction mode, the additional unidirectional prediction MV candidates are added to the final MV candidate list by averaging the L0 and L1 MVs of the bidirectional prediction merge candidates.

[0089] On the other hand, for the current VVC specification, different MV rounding methods can be applied to the above inter-frame prediction methods. Specifically, assuming that A is the MV value before rounding, and Shift is the bitwise right shift applied to the MV rounding, then for different inter-frame prediction methods, the value of the rounded MV (i.e., L) is calculated in the following way,

[0090] For affine mode, merge candidate generation and triangular prediction mode pairwise:

[0091] L|=(A>=0)? (A+Offset)>>Shift:-((-A+Offset)>>Shift (3);

[0092] Or, for TMVP and ATMVP:

[0093] L|=(A+Offset+(A<0?1:0))>>Shift (4);

[0094] where Offset is a rounding offset set equal to (Shift>0)?1<<(Shift-1):0; and L is the value of MV after rounding.

[0095] For the current VVC, for pair-wise merge candidates, triangular prediction mode, and affine mode, an absolute value-based MV rounding method can be applied, as indicated in equation (3). However, this MV rounding method is relatively complex because it involves multiple calculation steps. Specifically, the encoding device may first calculate the absolute value of the input, then perform absolute value rounding, and finally multiply the rounded absolute value by the sign of the input. On the other hand, for TMVP and ATMVP, another rounding method can be applied, in which the offset value applied before the bitwise right shift depends on the sign of the input value. However, this non-uniform design means that different MV rounding methods need to be implemented for codec hardware implementations.

[0096] In order to reduce the complexity of hardware implementation, the following method can also be proposed to unify the MV rounding operation for inter-frame prediction in VVC.

[0097] In one example, the absolute value-based MV rounding method based on equation (3) is used for various MV rounding operations. That is, the same MV rounding process based on equation (3) is applied to MVs used in different prediction tools, which include at least pair-wise merge candidates, triangular prediction mode, affine mode, TMVP, ATMVP, and MMVD.

[0098] In another example, the rounding method based on equation (4) is used for various MV rounding operations used in inter-frame prediction. For the current VVC, the rounding method used for TMVP and ATMP is also applied to the rounding operations in the pair-wise merge candidate, triangular prediction mode, and affine mode. That is, the same MV rounding process based on equation (4) is applied to MVs used in different prediction tools, which include at least pair-wise merge candidates, triangular prediction mode, affine mode, TMVP, ATMVP, and MMVD.

[0099] In another example, a conventional rounding method is used to round various MVs in inter-frame prediction. In this example, the rounded MV value is directly obtained by offsetting the input value by the offset and then shifting it bit by bit, that is, L is calculated as follows

[0100] L=(A+Offset)>>Shift (5).

[0101] For the current VCC specification, the above rounding method (i.e., conventional MV rounding based on equation (5)) can also be the rounding method for sample rounding operations (such as motion compensated sample interpolation, adaptive loop filter (ALF), bidirectional optical flow (BDOF), generalized bidirectional prediction (GBi), etc.). Therefore, in this example where the conventional rounding method is applied to pair-wise merge candidates, triangular prediction mode, affine mode, TMVP, ATMVP, and MMVD, all sample and MV rounding operations in VVC are completely unified.

[0102] Fig.13 The mapping function for MV rounding based on absolute value is shown; Fig.14 A mapping function for MV rounding based on TMVP / ATMVP is shown; and Fig.15 The mapping functions for conventional MV rounding are shown. Based on the comparison of these mapping functions, the differences between the rounded values ​​calculated by the three methods are small. Specifically, the differences only exist when the input values ​​are equal to -0.5, -1.5, -2.5, ... Therefore, it can be expected that the impact of using different MV rounding methods on codec performance is relatively negligible.

[0103] In the current VVC, MVs are represented in 18 bits, i.e., the dynamic range of valid MVs is in [-131072, 131071]. For MMVD mode, the applied MV is generated by adding a signaled offset to a selected merge candidate specified by the MMVD base index signaled in the bitstream. Similarly, for DMVR mode, the MV of each bidirectionally predicted merged block is enhanced by symmetrically applying a motion refinement to the L0 and L1 MVs to minimize the bidirectional matching cost between the two prediction signals. However, even if the bit depth of the selected merge candidate MV is within 18 bits, due to the MV offset (explicitly signaled for MMVD mode and implicitly derived for DMVR mode), the final MV applied for each block encoded and decoded by MMVD or DMVR mode may potentially exceed the dynamic range of the valid MV, resulting in arithmetic overflow of the MV value and causing ambiguity when the final MV value is derived for different implementation platforms. In addition, a similar MV overflow problem also exists in the affine inheritance mode, in which the CPMV of a current affine codec block is derived from the CPMV of its adjacent affine codec block or the sub-block MV of its adjacent codec block based on a 4-parameter affine model or a 6-parameter affine model. In either case, due to the application of linear operations (e.g., addition, subtraction, multiplication), the derived CPMV may exceed the range of values ​​that can be represented by the effective bit depth (i.e., 18 bits).

[0104] To avoid MV value overflow, several methods can be used. In the example, a truncation operation is added, which is performed on the derived MVs of MMVD mode and DMVR mode and the derived CPMVs of affine inheritance mode to truncate them to within the effective bit depth (ie 18 bits).

[0105] In another example, a bitstream consistency is used to ensure that all derived MVs for MMVD mode and DMVR mode and derived CPMVs for affine inheritance mode are within the 18-bit dynamic range. That is, when the corresponding derived MV or CPMV exceeds [-2 17 ,2 17 -1] range, the encoder does not signal to the decoder to enable MMVD mode, DMVR mode, or affine inheritance mode for the current block.

[0106] For the current VVC, two different MV truncation methods can be applied to various inter-frame prediction modes. Specifically, for the MV truncation operations involved in AMVP, ATMVP, triangular prediction mode, and affine mode, when the MV value exceeds the range represented by the effective bit depth (i.e., 18 bits), a conventional truncation method is applied to truncate the MV value to the range boundary value, i.e., greater than 2 17 The MV value of -1 is truncated to 217 -1, and less than -2 17 The MV value is truncated to -2 17 The general intercept method is described by the following equation:

[0107]

[0108] where L is the value of MV before rounding; and L clip is the value of the truncated MV.

[0109] On the other hand, for the MV clipping operation involved in the explicit normal inter mode and the explicit affine mode, a periodic clipping method is applied, which is periodically repeated to keep the MV value in the range [-2 17 ,2 17 -1], as indicated by the following equation:

[0110]

[0111] where L is the MV value before rounding; M is the intermediate value during the MV truncation operation; and L clip is the value of the MV after truncation.

[0112] In addition, in order to reduce complexity while achieving a unified design, the MV interception method applied to different inter-frame modes in VVC can be unified. In one example, the conventional interception method indicated in equation (6) is used for all MV interception operations. That is, the same interception process based on equation (6) is applied to the MV used in the prediction tool including the following: AMVP mode, ATMVP mode, triangle mode, MMVD mode, DMVR mode, affine inheritance mode, explicit normal inter-frame mode and explicit affine mode. The interception process can also be applied to the derived CPMV in MMVD mode, DMVR mode and affine inheritance mode.

[0113] In another example, the periodic interception method shown in equation (7) is used for all MV interception operations. That is, the same interception process based on equation (7) is applied to MVs used in prediction tools including the following: AMVP mode, ATMVP mode, triangle mode, MMVD mode, DMVR mode, affine inheritance mode, explicit normal inter-frame mode and explicit affine mode. The interception process can also be applied to the derived CPMVs in MMVD mode, DMVR mode and affine inheritance mode.

[0114] Motion vector storage.

[0115] For spatial domain MV prediction in VVC, a hybrid motion field vector of affine CPMV (at 8x8 granularity) and sub-block MV (at 4x4 granularity) is used for affine motion inheritance and merging / skipping and advanced motion vector prediction (AMVP). In addition, in addition to history-based motion vector prediction, in the VVC standard, the CPMV and / or MV of a codec block can only be predicted based on the CPMV and / or MV of its temporary neighboring blocks. This means that the CPMV and / or MV of non-adjacent blocks of the codec block can be set after the CPMV and / or MV of the current block are fully reconstructed. Therefore, in actual hardware implementations, there is no need to store CPMV and MV for the entire CTU. Instead, the decoder only needs to store the CPMV and MV of the top row and left column as context. Fig.15 The CPMV and MV context update processes are shown to illustrate such concepts. Fig.15 In FIG. 1 , the CPMV scenario is shown in the light-colored dot region 1502 (assuming the minimum affine block size is 8×8), and the conventional MV scenario is shown in the dark-colored dot region 1504 (assuming the minimum inter-prediction block size is 4×4). Fig.15 As shown in , after the CPMV or MV of the current coding block is decoded, the corresponding left and upper contexts are updated, and the two areas 1502 and 1504 are further shifted downward and rightward until they span all coding blocks within a CTU.

[0116] In the example, assuming that the CTU size is 128x128 and MV is stored for each 4-sample segment, there are 32 segments for the top row, 32 segments for the left column, and 32 segments for storing the upper left context. In addition, the MV range in VVC is, for example, 18 bits. Therefore, local MV storage may require (32+32+32)x(18x 2(hor / ver)x 2(L0 / L1))=6912 bits=864 bytes. Similarly, assuming that CPMV is stored for each 8-sample segment, there are 16 segments for the top row, 16 segments for the left column, and 16 segments that need to be stored as CPMV contexts. Therefore, local CPMV storage may require (16+16+16)x(18x 3(CPMV)x 2(hor / ver)x 2(L0 / L1))=10368 bits=1296 bytes. In addition, considering that MV prediction and CPMV prediction may occur across CTU boundaries, an additional line buffer storage is required for each 4-sample segment of the picture width, i.e., picWidth / 4x(18x 2x2). Therefore, when using 4K video, the CPMV and MV storage usage for spatial domain CPMV and MV prediction in VVC is summarized in Table 3 below.

[0117] Table 3 Memory size required for storing spatial domain CPMV and MV in VVC

[0118] storage Local MV storage Local CPMV storage MV Line Buffer sum size 864 bytes 1296 bytes 8640 bytes 10800 bytes .

[0119] For the current VVC design, temporal MVs are compressed based on a sub-sampling grid of 8x8 luma samples to reduce the memory storage requirements for storing temporal MVs used for temporal motion prediction, such as TMVP and Advanced Temporal Motion Vector Prediction (ATMVP). Unlike spatial MVs stored in 18 bits, in order to further reduce MV storage requirements, temporal MVs are converted to mantissa-exponential format before being stored in an external buffer. The purpose of representing 18-bit temporal MVs in mantissa-exponential format is to more coarsely quantize large MV values ​​while maintaining the accuracy of small MV values. Fig.16 As shown in , for the current VVC specification, the mantissa-exponent representation used to store the time-domain MV is configured as a 7-bit signed mantissa (1 bit for the sign) and a 4-bit unsigned exponent.

[0120] Therefore, for the current VVC, different representation methods can be applied to store spatial domain MV and time domain MV. Specifically, spatial domain MV and CPMV are stored as 18-bit signed values, while time domain MV is stored using 7-bit signed mantissa and 4-bit unsigned exponent (i.e., 11 bits in total). That is, compared with spatial domain MV and CPMV, time domain MV has higher storage efficiency. However, in practice, the implementation cost of storing spatial domain MV and CPMV is more expensive than that of time domain MV, because spatial domain MV and CPMV need to be stored using on-chip memory, while time domain MV uses external memory to store.

[0121] For current VVC, a hybrid motion field vector of affine CPMV and sub-block MV can be used for affine motion inheritance and merging / skipping as well as advanced motion vector prediction (AMVP). In addition, since both MV and CPMV are stored with 18-bit precision, a large amount of on-chip memory (about 10800 bytes, as shown in Table 3) is required to store the MV and CPMV of neighboring blocks for spatial domain MV and CPMV prediction. In order to reduce this memory requirement, the following method is proposed to fully or partially convert the spatial domain MV and CPMV into the mantissa-exponent representation format before storage.

[0122] In the example, all spatial domain motion information is converted into a mantissa-exponent format before being stored in the on-chip memory. The converted motion information may include not only the MV and CPMV of the spatial domain neighboring blocks, but also the MV for the history-based merge candidates (i.e., history-based motion vector prediction (HMVP) MV). In addition, different combinations of mantissa and exponent parameters may be applied, which may provide various trade-offs between storage usage and MV accuracy. In an exemplary embodiment, the mantissa is set to 8 bits and the exponent is set to 4 bits, and Table 4 summarizes the rough on-chip memory usage when this mantissa-exponent setting is applied. Compared with Table 3, the use of mantissa-exponent representation for spatial domain MV and CPMV and for HMVP MV can efficiently reduce the total memory size of on-chip MV / CPMV storage by approximately 33%. In another example, the mantissa-exponent representation is applied to spatial domain MV and CPMV, but not to history-based MV candidates, i.e., the history-based MV candidates in the lookup table (LUT) are still stored in the original 18 bits.

[0123] Table 4 Memory size required to store spatial domain CPMV and MV when applying MV representation based on mantissa (8)-exponent (4)

[0124] storage Local MV storage Local MV storage MV Line Buffer sum size 576 bytes 864 bytes 5760 bytes 7200 bytes .

[0125] Due to the precision loss caused by MV quantization, the conversion of all spatial domain MVs and CPMVs and history-based MV candidates to the mantissa-exponent format may cause some obvious codec performance degradation. At the same time, the frequent conversion of MV and CPMV values ​​between the original MV domain and the mantissa-exponent MV domain may introduce additional codec complexity on both the encoder and decoder sides. In addition, it can be seen from Table 3 that compared with local MVs and CPMVs, MVs stored in the line buffer consume much more on-chip memory for storage, i.e., 80% of the total memory size. In order to achieve a better trade-off between MV and CPMV prediction efficiency and on-chip memory reduction, in another example, the mantissa-exponent representation is only applied to MVs stored in the line buffer (i.e., adjacent MVs from the top CTU), while all MVs and CPMVs stored in the local buffer (i.e., adjacent MVs and CPMVs within the same CTU) are still stored in the original 18 bits. Since MV quantization is only applied to MVs in the line buffer, in this example, the corresponding codec performance impact can be effectively reduced compared to the previous example in which all spatial domain MVs and CPMVs and history-based MV candidates are converted. At the same time, since there is no need to convert the MV or CPMV in the local buffer, and only the adjacent MV and CPMV from the top CTU row are converted to the mantissa-exponent domain (for MV storage) or converted back from the mantissa-exponent domain (for MV retrieval), the complexity overhead of converting MV and CPMV is reduced.

[0126] Table 5 shows the corresponding on-chip memory usage when the line buffer MV is stored in the 11-bit mantissa (7)-exponent (4) representation format. In this way, the total memory size of the on-chip MV / CPMV storage is reduced by about 32%, which is comparable to the 33% reduction achieved in the example shown in Table 3.

[0127] Table 5 Memory size required to store line buffer CPMV and MV using mantissa (7)-exponent (4) format

[0128] storage Local MV storage Local CPMV storage MV Line Buffer sum size 864 bytes 1296 bytes 5280 bytes 7440 bytes .

[0129] In another example, all spatial domain MVs and CPMVs are compressed based on the mantissa-exponent representation. In order to achieve a better trade-off between MV / CPMV prediction efficiency and storage requirements, different mantissa-exponent parameters are used to compress different spatial domain MVs. For example, different parameters can be used to compress MVs and CPMVs stored in the local buffer and MVs stored in the line buffer. Since the memory size used to store MVs in the line buffer is much more important than the memory size used to store MVs and CPMVs in the local buffer, the MVs in the line buffer may be further compressed than the MVs and / or CPMVs stored in the local buffer. In an exemplary embodiment, a 12-bit mantissa (8)-exponent (4) representation format for compressing MVs and CPMVs in local storage and an 11-bit mantissa (7)-exponent (4) representation format for compressing MVs in the line buffer can be used. In addition, the compressed MVs in the local buffer can include or exclude MVs for HMVP. That is, the history-based MV candidates in the LUT can be stored in a compressed format or in the original 18 bits.

[0130] In the above example, the storage requirements of the spatial domain MV and CPMV are reduced, thereby reducing the codec implementation cost. Using mantissa-exponent representation to represent the spatial domain MV and CPMV and the time domain MV in storage further reduces the hardware implementation complexity.

[0131] Fig.17 1700 is a block diagram illustrating an apparatus for video encoding and decoding according to some embodiments of the present disclosure. Apparatus 1700 may be a terminal such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

[0132] like Fig.17 As shown in , device 1700 may include one or more of the following components: a processing component 1702, a memory 1704, a power supply component 1706, a multimedia component 1708, an audio component 1710, an input / output (I / O) interface 1712, a sensor component 1714 and a communication component 1716.

[0133] The processing component 1702 generally controls the overall operation of the device 1700, such as operations related to display, phone calls, data communications, camera operations, and recording operations. The processing component 1702 may include one or more processors 1720 for executing instructions to complete all or part of the steps of the above method. In addition, the processing component 1702 may include one or more modules to facilitate interaction between the processing component 1702 and other components. For example, the processing component 1702 may include a multimedia module for facilitating interaction between the multimedia component 1708 and the processing component 1702.

[0134] The memory 1704 is configured to store different types of data to support the operation of the device 1700. Examples of such data include instructions for any application or method running on the device 1700, contact data, phone book data, messages, pictures, videos, etc. The memory 1704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the memory 1704 can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0135] The power supply component 1706 supplies power to the various components of the device 1700. The power supply component 1706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1700.

[0136] Multimedia component 1708 includes a screen, which provides an output interface between device 1700 and a user. In some examples, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen that receives input signals from a user. The touch panel may include one or more touch sensors for sensing touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or sliding action, but may also detect the duration and pressure associated with a touch or sliding operation. In some examples, multimedia component 1708 may include a front camera and / or a rear camera. When device 1700 is in an operating mode (such as a shooting mode or a video mode), the front camera and / or the rear camera may receive external multimedia data.

[0137] The audio component 1710 is configured to output and / or input audio signals. For example, the audio component 1710 includes a microphone (MIC). When the device 1700 is in an operating mode (such as a call mode, a recording mode, and a speech recognition mode), the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1704 or sent via the communication component 1716. In some examples, the audio component 1710 also includes a speaker for outputting an audio signal.

[0138] I / O interface 1712 provides an interface between processing unit 1702 and peripheral interface modules. The above peripheral interface modules may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to home buttons, volume buttons, power buttons, and lock buttons.

[0139] Sensor component 1714 includes one or more sensors for providing status assessment for device 1700 in different aspects. For example, sensor component 1714 can detect the on / off state of device 1700 and the relative position of components. For example, the components are the display and keyboard of device 1700. Sensor component 1714 can also detect position changes of device 1700 or components of device 1700, the presence or absence of user contact on device 1700, the orientation or acceleration / deceleration of device 1700, and temperature changes of device 1700. Sensor component 1714 can include a proximity sensor, which is configured to detect the presence of nearby objects without any physical contact. Sensor component 1714 can also include an optical sensor, such as a CMOS or CCD image sensor used in imaging applications. In some examples, sensor component 1714 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0140] Communication component 1716 is configured to promote wired or wireless communication between device 1700 and other equipment. Device 1700 can access wireless network based on communication standards such as WiFi, 4G or its combination. In an example, communication component 1716 receives broadcast signal or broadcast related information from external broadcast management system via broadcast channel. In an example, communication component 1716 may also include a near field communication (NFC) module for promoting short-range communication. For example, NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0141] In an example, apparatus 1700 may be implemented by one or more of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components to perform the above method.

[0142] The non-transitory computer-readable storage medium may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a flash memory, a hybrid drive or a solid-state hybrid drive (SSHD), a read-only memory (ROM), a compact disk read-only memory (CD-ROM), a tape, a floppy disk, etc.

[0143] Fig.18 is a flow chart illustrating an exemplary process of motion vector rounding for video coding.

[0144] In step 1802, processor 1720 sets a motion vector (MV) rounding process, the MV rounding process being configured to perform MV rounding operations.

[0145] The MV rounding process may be provided, for example, by an apparatus, a circuit system or a computer program code block.

[0146] In step 1804, processor 1720 applies an MV rounding process to at least a first MV used in a first prediction tool selected from a first group of tools including: pairwise merging candidates, triangular prediction mode, and affine mode.

[0147] In step 1806, the processor 1720 applies the MV rounding process to at least a second MV used in a second prediction tool, which is selected from a second group of tools including: temporal motion vector prediction (TMVP), advanced temporal motion vector prediction (ATMVP), and merge mode with motion vector difference (MMVD).

[0148] Fig.19 is a flow chart illustrating an exemplary process of motion vector extraction for video coding.

[0149] In step 1902, the processor 1720 sets a motion vector (MV) clipping process, which is configured to perform a clipping operation.

[0150] In step 1904, the processor 1720 applies the MV capture process to at least a first MV used in a first prediction tool, which is selected from a first group of tools including: advanced motion vector prediction (AMVP) mode, advanced temporal motion vector prediction (ATMVP) mode, triangular mode, merge mode with motion vector difference (MMVD) mode, decoder side motion vector derivation (DMVR) mode and affine inheritance mode.

[0151] In step 1906, the processor 1720 applies the MV clipping process to at least a second MV used in a second prediction tool selected from a second group of tools including: an explicit normal inter mode and an explicit affine mode.

[0152] In some examples, a device for video encoding and decoding is provided. The device includes a processor 1720; and a memory 1704, which is configured to store instructions executable by the processor; wherein the processor is configured to execute when executing the instructions Fig.18 or Fig.19 The method shown in .

[0153] In some other examples, a non-transitory computer-readable storage medium 1704 is provided having instructions stored therein. When the instructions are executed by the processor 1720, the instructions cause the processor to perform Fig.18 or Fig.19The method shown in .

[0154] The description of the present disclosure is presented for purposes of illustration and is not intended to be exhaustive or limiting of the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.

[0155] These examples are selected and described in order to explain the principles of the present disclosure and enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications adapted to the specific use contemplated. Therefore, it should be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.

Claims

1. A method for video decoding, comprising: determining an inter-frame prediction process; as well as In response to determining that the inter prediction process is one of the derivation processes for: performing a motion vector (MV) clipping operation on at least one MV in the inter prediction process according to a first MV clipping mode; Advanced Motion Vector Prediction (AMVP) mode, Advanced Temporal Motion Vector Prediction (ATMVP) mode, Merge Mode with Motion Vector Difference (MMVD) mode, Decoder Side Motion Vector Derivation (DMVR) mode, Affine Inheritance mode, or Explicit Affine mode, The upper limit of the first MV capture mode is 2 17 -1, and the lower limit of the first MV capture mode is -2 17 .

2. The method of claim 1, wherein the MV truncation operation is defined by the following equation: in L is the value of MV before rounding; and L clip is the value of the MV after truncation.

3. The method of claim 1, further comprising: The MV truncation process is applied to all MVs and CPMVs derived in the merge with motion vector difference (MMVD) mode, the decoder side motion vector derivation (DMVR) mode and the affine inheritance mode.

4. A device for video decoding, comprising: processor; and a memory configured to store instructions executable by the processor; Wherein, when executing the instruction, the processor is configured to: determining an inter-frame prediction process; as well as In response to determining that the inter prediction process is one of the derivation processes for: performing a motion vector (MV) clipping operation on at least one MV in the inter prediction process according to a first MV clipping mode; Advanced Motion Vector Prediction (AMVP) mode, Advanced Temporal Motion Vector Prediction (ATMVP) mode, Merge Mode with Motion Vector Difference (MMVD) mode, Decoder Side Motion Vector Derivation (DMVR) mode, Affine Inheritance mode, or Explicit Affine mode, The upper limit of the first MV capture mode is 2 17 -1, and the lower limit of the first MV capture mode is -2 17 .

5. The apparatus of claim 4, wherein the first MV capture mode is defined by the following equation: in L is the value of MV before rounding; and L clip is the value of the MV after truncation.

6. In the apparatus of claim 4, the processor is further configured to apply the MV truncation process to all MVs and CPMVs derived in a merge mode with motion vector difference (MMVD) mode, a decoder side motion vector derivation (DMVR) mode, and an affine inheritance mode.

7. A non-transitory computer-readable storage medium storing therein instructions and a bit stream to be decoded by the method for video decoding as claimed in any one of claims 1 to 3, wherein when the instructions are executed by a processor, the instructions cause the processor to perform the method as claimed in any one of claims 1 to 3.

8. A computer program product, comprising instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 3.

9. A non-transitory computer-readable storage medium storing a bit stream to be decoded by a processor through the method for video decoding according to any one of claims 1 to 3.

10. A method for storing a bit stream, comprising: A bit stream to be decoded by the method for video decoding according to any one of claims 1 to 3 is stored on a non-transitory computer-readable storage medium.

Images