Video encoding and decoding method and device

Abstract

A method and apparatus for video encoding and decoding, wherein a video decoder receives pixel data of a block to be decoded as a current block in a current frame of a video. When the current block is encoded and decoded using inter-frame mode and bidirectional prediction, the video decoder receives a first motion vector difference value; and determines a second motion vector difference value based on a first flag or a second flag. When the first flag is false, the video decoder uses the second flag to determine whether to set the second motion vector difference value to the inverse of the first motion vector difference value. When the first flag is true, the video decoder sets the second motion vector difference value to zero. Based on the first motion vector difference value and the second motion vector difference value, the video decoder reconstructs the current block by performing inter-frame prediction using motion information.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 851,140, filed May 22, 2019, the contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to video coding and decoding. In particular, the present invention relates to signaling motion vector differences for inter-frame prediction. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] High-Efficiency Video Coding (HEVC) is a new-generation international video codec standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is a codec architecture based on hybrid block-based motion compensation similar to DCT transform. The basic unit of compensation (called a coding unit (CU)) is a 2Nx2N square block, and each CU can be recursively divided into four smaller CUs until a predetermined minimum size is reached. Each CU includes one or more prediction units (PUs).

[0006] To improve the encoding and decoding efficiency of the hybrid codec architecture, HEVC provides two types of prediction modes for each PU: intra prediction and inter prediction. For intra prediction mode, spatially adjacent reconstructed pixels can be used to generate directional predictions, with up to 35 directions in HEVC. For inter prediction mode, temporally reconstructed reference frames can be used to generate motion-compensated predictions. There are three different modes: Skip, Merge, and Inter Advanced Motion Vector Prediction (AMVP) mode.

[0007] When a PU is coded in inter-AMVP mode, motion-compensated prediction is performed using transmitted motion vector differences (MVDs). MVDs are used with motion vector predictors (MVPs) to derive motion vectors (MVs). To determine the MVP in inter-AMVP mode, the advanced motion vector prediction (AMVP) architecture is used to select a MV predictor from a set of AMVP candidates consisting of two spatial MVPs and one temporal MVP. Therefore, in AMVP mode, the MVP index and corresponding MVDs are encoded and transmitted. Furthermore, the inter-prediction direction (indicating the prediction direction between bidirectional and unidirectional prediction, i.e., List0 (L0) and List1 (L1)) and the accompanying reference frame index for each list are also encoded and transmitted. Summary of the Invention

[0008] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious technologies described herein. Selective, but not all, embodiments are further described in the following detailed description. Therefore, the following summary is not intended to determine the essential characteristics of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0009] Some embodiments of the present invention provide a method for signaling motion vector differences. A video decoder receives pixel data of a block to be decoded as a current block in a current frame of a video. When the current block is encoded and decoded using inter-frame mode and bidirectional prediction, the video decoder receives a first motion vector difference; and derives a second motion vector difference based on a first flag or a second flag. When the first flag is false, the video decoder uses the second flag to determine whether to set the second motion vector difference to the inverse of the first motion vector difference. When the first flag is true, the video decoder sets the second motion vector difference to zero. Based on the first motion vector difference and the second motion vector difference, the video decoder reconstructs the current block by performing inter-frame prediction using motion information.

[0010] According to the video coding and decoding method and apparatus provided by the present invention, operations related to signaling of motion vector differences related to motion compensation prediction during inter-frame prediction are standardized. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The following drawings are provided to provide a further understanding of the present invention and are incorporated into and constitute a part of this specification. These drawings illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention. To clearly illustrate the concepts of the present invention, some components may be shown out of scale compared to the dimensions in actual embodiments, and these drawings are not necessarily drawn to scale.

[0012] Figure 1 Conceptually depicted is inter prediction of the current block when the SMVD mode flag is true.

[0013] Figure 2 Conceptually depicted is inter prediction when mvd_l1_zero_flag is true.

[0014] Figure 3 An example video encoder is shown.

[0015] Figure 4 Portions of a video encoder that can implement both the SMVD mode and the zero L1 MVD mode are shown.

[0016] Figure 5 A process is conceptually depicted to reconcile the SMVD mode with the zero L1 MVD mode.

[0017] Figure 6 An example video decoder is shown.

[0018] Figure 7 Portions of a video decoder that can implement both SMVD mode and zero L1 MVD mode are shown.

[0019] Figure 8 A process is conceptually depicted to reconcile the SMVD mode with the zero L1 MVD mode.

[0020] Figure 9 An electronic system is conceptually illustrated in which some embodiments of the present invention may be implemented. DETAILED DESCRIPTION

[0021] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced.

[0022] In the following detailed description, numerous specific details are provided by way of example to provide a thorough understanding of the relevant teachings. Any variations, derivations, and / or extensions of the teachings described herein are within the scope of protection of the present invention. To avoid unnecessarily obscuring aspects of the teachings of the present invention, known methods, procedures, components, and / or circuits in one or more exemplary embodiments disclosed herein are sometimes described at a relatively high level without further elaboration.

[0023] I. Symmetric Motion Vector Difference (SMVD) Mode

[0024] In addition to the MVD signaling used for the standard unidirectional and bidirectional prediction modes, an MVD signaling mode for bidirectional prediction (SMVD mode, also called symmetric MVD mode) is implemented. In symmetric MVD mode, motion information (including the reference picture indices for both List-0 and List-1 and the MVD for List-1) is not signaled but instead derived. Symmetric motion vector difference (SMVD) mode is an inter-prediction mode in which motion information is symmetrically determined or inferred. In particular, in some embodiments, the motion vector difference in the L1 direction (denoted as MVD1 or MVDL1) is determined based on the motion vector difference in the L0 direction (denoted as MVD0 or MVDL0). In some embodiments, the L1 reference picture and the L0 reference picture are equidistant from the current picture in time or POC (which may be determined based on the picture order count (POC) or any other factor related to the temporal domain or POC). In some embodiments, in SMVD mode, MVD1 is inferred to be equal to –MVD0. In some embodiments, when the SMVD mode flag is true, BDOF is not applied to prediction to reduce complexity.

[0025] In some embodiments, a reference pair (ref_idx_l0 and ref_idx_l1) is not signaled but inferred (ref_idx_l0 is the reference index pointing to reference picture list-0, and ref_idx_l1 is the reference index pointing to reference picture list-1). In some embodiments, MVD0 is signaled, but MVD1 is not signaled but inferred.

[0026] In some embodiments, at the CU level, if the consistency condition for SMVD mode is met, an SMVD mode flag (which indicates whether the symmetric mode (or SMVD mode) is used) is explicitly signaled. When the symmetric mode flag is true, only mvp_l0_flag (indicating the MVP index of list-0), mvp_l1_flag (indicating the MVP index of list-1), and MVD0 are explicitly signaled. The reference indexes for list-0 and list-1, denoted as (RefIdxSymL0, RefIdxSymL1), respectively, are set to the pair of reference pictures. MVD1 is set equal to (–MVD0). The MV calculation according to SMVD mode is according to the following formula:

[0027]

[0028] L0MV=(mvx0,mvy0), L0 predictor=(mvpx0,mvpy0), L0MVD or MVD0=(mvdx0,mvdy0). L1MV=(mvx1,mvy1), L1 predictor=(mvpx1,mvpy1). L1MVD or MVD1=(mvdx1,mvdy1), but because it is SMVD mode, it is inferred as (–mvdx0,–mvdy0) or –MVD0.

[0029] II. L1 motion vector difference derivation

[0030] In some embodiments, when the current CU or PU is inter-coded and bidirectional prediction is enabled (e.g., when the current CU or PU is in a B-frame or B-slice), a number of different settings can be used to implicitly derive MVD1. One of these settings (which can be used to implicitly derive MVD1) is SMVD mode. When the enable condition for SMVD mode (represented by condition A) is met and the SMVD mode flag is true, MVD1 is set equal to -MVD0 rather than being explicitly signaled. In some embodiments, the SMVD mode flag can be signaled at the CU / PU, CTU, slice, tile, tilegroup, SPS, PPS, or picture level.

[0031] Another setting that can be used to implicitly derive MVD1 is zero MVD1 mode, which can be enabled via the mvd_l1_zero_flag. When the enable condition for mvd_l1_zero_flag (represented by condition B) is met and mvd_l1_zero_flag is true, MVD1 is set to zero rather than being explicitly signaled. In some embodiments, mvd_l1_zero_flag can be signaled at the CU / PU, CTU, slice, tile, tile group, SPS, PPS, or picture level.

[0032] In some embodiments, the picture level refers to a picture header (PH). The PH syntax structure contains information that is common to all slices of the coded picture associated with the PH syntax structure.

[0033] Figure 1 Conceptually depicted is inter prediction of the current block when the SMVD mode flag is true. Figure 2Inter-frame prediction when mvd_l1_zero_flag is true is conceptually illustrated. As shown in FIG. 1-2 , a video codec encodes or decodes a pixel block 100 (e.g., a CU or PU) as a current block in a current picture 110 . L0 reference picture 120 and L1 reference picture 130 are equidistant from the current picture in time or POC. When the SMVD mode flag is true, MVD1 is set to –MVD0. When mvd_l1_zero_flag is true, MVD1 is set to zero.

[0034] Some embodiments of the present invention provide methods to handle (or reconcile) the interaction between the SMVD mode flag and the mvd_l1_zero_flag, since both flags are related to (affect) the derivation of MVD1. In some embodiments, the condition that the current block (or current CU / PU) is inter-coded and bi-directional prediction is allowed is a prerequisite for enabling the SMVD mode flag or mvd_l1_zero_flag. When this prerequisite is met for the current block, a video encoder can use the SMVD mode flag and / or mvd_l1_zero_flag to determine the derivation of MVD1. In some embodiments, when condition A is not met, the SMVD mode flag is inferred to be false. In some embodiments, when condition B is not met, mvd_l1_zero_flag is inferred to be false. In some of these embodiments, condition A and condition B are different. The following are examples of condition A and condition B:

[0035] In some embodiments, condition A (i.e., the enabling condition for SMVD mode) includes (or at least includes) that the CU is bi-predictive and / or that a reference picture that satisfies certain conditions exists. In some embodiments, BiDirPredFlag can be used to indicate whether a reference picture that satisfies certain conditions exists. BiDirPredFlag is derived at the slice level as follows: If mvd_l1_zero_flag is 1, BiDirPredFlag is set to 0. Otherwise, if the closest reference picture in list-0 and the closest reference picture in list-1 form a pair of forward and backward reference pictures or a pair of backward and forward reference pictures, and both list-0 and list-1 reference pictures are short-term reference pictures, BiDirPredFlag is set to 1. Otherwise, BiDirPredFlag is set to 0. In some embodiments, condition A includes (or at least includes) a selected reference picture in list-0 (denoted by RefIdxSymL0) and a selected reference picture in list-1 (denoted by RefIdxSymL1) forming a forward and backward reference picture pair or a backward and forward reference picture pair. In some embodiments, the selected picture in list-0 is the reference picture in list-0 that is closest (in time or POC) to the current picture. In some embodiments, the selected picture in list-1 is the reference picture in list-1 that is closest (in time or POC) to the current picture. In some embodiments, the selected picture in list-0 is signaled at the CU / PU, CTU, slice, tile, tile group, SPS, PPS, or picture level. In some embodiments, the selected picture in list-1 is signaled at the CU / PU, CTU, slice, tile, tile group, SPS, PPS, or picture level. In some embodiments, the selected picture in list-0 is the first reference picture in list-0. In some embodiments, the selected picture in list-1 is the first reference picture in list-1. In embodiments, any combination of the above may be applied.

[0036] In some embodiments, condition A includes (or at least includes) the closest reference picture in list-0 (denoted by RefIdxSymL0) and the closest reference picture in list-1 (denoted by RefIdxSymL1) forming a forward-reverse pair of reference pictures or a backward-reverse pair of reference pictures. In some embodiments, condition A includes (or at least includes) the first reference picture in list-0 and the first reference picture in list-1 are symmetrical. In other words, the first reference picture in list-0 and the first reference picture in list-1 form a forward-reverse pair of reference pictures (or a backward-reverse pair of reference pictures), and the distance between the current picture and the first reference picture in list-0 is the same as the distance between the current picture and the first reference picture in list-1. In some embodiments, when condition A is met, RefIdxSymL0 and RefIdxSymL1 are inferred to be zero.

[0037] In some embodiments, condition B includes (or at least includes) a pair of forward and backward reference pictures formed by any reference picture in list-0 and any reference picture in list-1. (This is referred to as the generalized P-and-B picture (or GPB) condition).

[0038] In some embodiments, if condition A (the condition for enabling the SMVD mode flag) is met, condition B is not met. In some embodiments, condition A includes (or at least includes) mvd_l1_zero_flag being false. In some embodiments, when the condition for enabling the SMVD mode flag is met, the condition for mvd_l1_zero_flag is automatically considered not met, and mvd_l1_zero_flag is set to false.

[0039] In some embodiments, if condition B (the enabling condition for mvd_l1_zero_flag) is met, condition A is not met. In some embodiments, condition B includes (or at least includes) the SMVD mode flag being false. In other words, when the condition for mvd_l1_zero_flag is met, the SMVD mode flag condition is automatically considered to be unmet, and the SMVD mode flag is set to false.

[0040] In some embodiments, when both the mvd_l1_zero_flag and the SMVD mode flag are true, MVD1 is set to zero (instead of -MVD0). In some embodiments, when both the mvd_l1_zero_flag and the SMVD mode flag are true, MVD1 is set to -MVD0 (instead of zero). In some embodiments, condition B is the opposite of condition A. In some embodiments, condition A is the opposite of condition B.

[0041] Any of the methods proposed above can be implemented in an encoder and / or a decoder. For example, any of the methods proposed can be implemented in an inter-frame encoding / decoding module, a motion compensation module, or an inter-frame candidate derivation module of an encoder. Alternatively, any of the methods proposed can be implemented as a circuit coupled to an inter-frame encoding / decoding module and / or a motion compensation module of an encoder, or a merging / inter-frame candidate derivation module of a decoder.

[0042] III. Exemplary Video Encoders

[0043] Figure 3 An exemplary video encoder 300 is shown. As shown, the video encoder 300 receives an input video signal from a video source 305 and encodes the signal into a bitstream 395. The video encoder 300 has several components or modules for encoding the signal from the video source 305, including at least some components selected from a transform module 310, a quantization module 311, an inverse quantization module 314, an inverse transform module 315, an intra-frame estimation module 320, an intra-frame prediction module 325, a motion compensation module 330, a motion estimation module 335, a loop filter 345, a reconstructed frame buffer 350, an MV buffer 365, an MV prediction module 375, and an entropy encoder 390. The motion compensation module 330 and the motion estimation module 335 are part of the inter-frame prediction module 340.

[0044] In some embodiments, modules 310-390 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device or electronic apparatus. In some embodiments, modules 310-390 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic apparatus. Although modules 310-390 are shown as separate modules, some modules may be combined into a single module.

[0045] A video source 305 provides a raw video signal representing uncompressed pixel data for each video frame. A subtractor 308 calculates the difference between the raw video pixel data from the video source 305 and the predicted pixel data 313 from the motion compensation module 330 or the intra-prediction module 325. A transform module 310 converts this difference (or residual pixel data or residual signal 309) into transform coefficients (e.g., by performing a discrete cosine transform, or DCT). A quantization module 311 quantizes the transform coefficients into quantized data (or quantized coefficients) 312, which are encoded into a bitstream 395 by an entropy encoder 390.

[0046] The inverse quantization module 314 inversely quantizes the quantized data (or quantized coefficients) 312 to obtain transform coefficients, and the inverse transform module 315 performs an inverse transform on the transform coefficients to generate a reconstructed residual 319. The reconstructed residual 319 is added to the predicted pixel data 313 to generate reconstructed pixel data 317. In some embodiments, the reconstructed pixel data 317 is temporarily stored in a line buffer (not shown) for use in intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 345 and stored in the reconstructed picture buffer 350. In some embodiments, the reconstructed picture buffer 350 is a memory external to the video encoder 300. In some embodiments, the reconstructed picture buffer 350 is a memory within the video encoder 300.

[0047] The intra-picture estimation module 320 performs intra-prediction based on the reconstructed pixel data 317 to generate intra-prediction data. The intra-prediction data is provided to the entropy encoder 390 to be encoded into the bitstream 395. The intra-prediction data is also used by the intra-prediction module 325 to generate predicted pixel data 313.

[0048] The motion estimation module 335 performs inter-prediction by providing MVs to reference pixel data of existing decoded video frames stored in the reconstructed picture buffer 350. These MVs are provided to the motion compensation module 330 to generate predicted pixel data.

[0049] Instead of encoding the complete actual MV into the bitstream, the video encoder 300 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 395.

[0050] The MV prediction module 375 generates a predicted MV based on a reference MV generated when encoding the current video frame, that is, a motion-compensated MV used to perform motion compensation. The MV prediction module 375 retrieves the reference MV from the current video frame from the MV buffer 365. The video encoder 300 stores the MV generated for the current video frame in the MV buffer 365 as a reference MV used to generate the predicted MV.

[0051] The MV prediction module 375 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference between the predicted MV and the motion compensated MV (MC MV) of the current video frame (residual motion data) is encoded into the bitstream 395 by the entropy encoder 390.

[0052] The entropy encoder 390 encodes various parameters and data into a bitstream 395 using entropy coding techniques such as Context-based Adaptive Binary Arithmetic Coding (CABAC) or Huffman encoding. The entropy encoder 390 encodes various header elements, flags, and quantized transform coefficients 312 along with residual motion data as syntax elements into the bitstream 395. The bitstream 395 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0053] The loop filter 345 performs a filtering or smoothing operation on the reconstructed pixel data 317 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering operation performed includes Sample Adaptive Offset (SAO). In some embodiments, the filtering operation includes an Adaptive Loop Filter (ALF).

[0054] Figure 4 Portions of a video encoder that can implement both the SMVD mode and the zero L1 MVD mode are shown. As previously described, the inter-prediction module 340, and in particular the motion estimation module 335, outputs a motion vector that is actually used for motion compensation (MC MV). In some embodiments, when the enabling conditions for the SMVD mode are met and the SMVD flag indicates the use of SMVD mode, the motion estimation begins a symmetric MVD search with an initial MV evaluation. This initial MV is evaluated from a set of initial MV candidates, including: MVs obtained from a uni-prediction search, MVs obtained from a bi-prediction search, and MVs from the AMVP list. The MV with the lowest rate-distortion cost is selected as the initial MV for the symmetric MVD motion search.

[0055] The value of the MC MV is determined based on whether the inter-prediction module 340 uses SMVD mode (smvd_l1_zero_flag = true), zero L1 MVD mode (mvd_l1_zero_flag = true), or some other mode to perform inter-prediction. If the current block is in a B-slice or B-frame where bidirectional prediction is enabled and / or the current block is encoded and decoded using bidirectional prediction, the inter-prediction module 340 performs motion estimation and motion compensation based on an MVD selection module 400, which provides MVD information associated with the SMVD mode flag and / or mvd_l1_zero_flag. If the conditions for enabling the SMVD mode flag are met, the SMVD mode flag may be signaled in the bitstream; otherwise, the SMVD mode flag may be derived without signaling. If the enabling conditions for mvd_l1_zero_flag are met, mvd_l1_zero_flag may be signaled in the bitstream; otherwise, mvd_l1_zero_flag may be derived without signaling. If SMVD mode is used, MC MV is set based on MVD1 = –MVD0. If zero L1 MVD mode is used, MC MV is set based on MVD1 = 0. If either the SMVD mode flag and / or mvd_l1_zero_flag is set to true, the entropy encoder 390 may not signal motion vector differences or residual motion data for L1 into the bitstream 395.

[0056] Figure 5 A process 500 is conceptually illustrated for reconciling an SMVD mode with a zero L1 MVD mode. In some embodiments, one or more processing units (e.g., processors) on a computing device implementing the encoder 300 executes the process 500 by executing instructions stored on a computer-readable medium. In some embodiments, an electronic device implementing the encoder 300 executes the process 500.

[0057] The encoder receives (at block 510) raw pixel data for a pixel block to be encoded in a bitstream as a current block in a current frame of a video. The encoder determines (at block 515) whether the current block is coded using inter-frame mode and bi-directional prediction. If so, the process proceeds to 520. If the current block is not coded using inter-frame mode and bi-directional prediction, the video encoder encodes the current block using another codec mode.

[0058] The encoder receives (at block 520) or calculates a first motion vector difference. The encoder determines (at block 525) whether a first flag (mvd_l1_zero_flag) is true or false. In some embodiments, the first flag (mvd_l1_zero_flag) is signaled at a picture level. In some embodiments, the mvd_l1_zero_flag can be signaled at the CU / PU, CTU, slice, tile, tile group, SPS, or PPS level. If the first flag is true, the encoder proceeds to block 530. If the first flag is false, the process proceeds to block 535. In some embodiments, the condition for enabling SMVD mode (or Condition A) includes at least mvd_l1_zero_flag being false.

[0059] At block 530, the encoder sets a second motion vector difference (i.e., MVD1) to zero because the first flag (mvd_l1_zero_flag) indicates that zero L1 MVD mode is used. In some embodiments, the first and second motion vector differences are associated with first and second reference pictures. In some embodiments, the first reference picture is selected from a first reference picture list and the second reference picture is selected from a second reference picture list. In some embodiments, the first and second reference pictures form a pair of forward and backward reference pictures. This is referred to as a generalized P and B picture (or GPB) condition. The process then proceeds to block 570.

[0060] At block 535, the encoder determines whether a second flag (SMVD mode flag) is true or false. In some embodiments, the second flag (SMVD mode flag) is signaled at the codec unit (CU) level for the current block. In some embodiments, the SMVD mode flag can be signaled at the CTU, slice, tile, tile group, SPS, PPS, or picture level. If the second flag is true, the process proceeds to 540. If the second flag is false, the process proceeds to 550.

[0061] At block 540, the encoder sets the second motion vector difference (i.e., MVD1) to the inverse of the first motion vector difference (i.e., −MVD0). In some embodiments, the first and second motion vector differences are associated with first and second reference pictures that are symmetrical or at the same distance from the current picture. In some embodiments, the encoder may select the first reference picture from a first reference picture list and the second reference picture from a second reference picture list. In some embodiments, the first and second reference pictures form a pair of backward and forward reference pictures, or a pair of forward and backward reference pictures. In some embodiments, the first reference picture is the picture in the first reference picture list that is closest in time or POC to the current picture, and the second reference picture is the picture in the second reference picture list that is closest in time or POC to the current picture. Processing then proceeds to block 570.

[0062] At block 550, the encoder sets the second motion vector difference (MVD1) based on a motion vector used for motion compensation and a predicted motion vector. The process then signals (at block 560) the second motion vector difference (MVD1) in the bitstream. In some embodiments, when either the first flag or the second flag is true, the second motion vector difference (MVD1) is not signaled in the bitstream.

[0063] Based on the first motion vector difference (MVD0) and the second motion vector difference (MVD1), the process encodes (at block 570) the current block into the bitstream by performing inter-prediction (particularly bi-prediction) using the motion information. In some embodiments, the SMVD mode flag and the mvd_l1_zero_flag are considered by the video encoder for inter-prediction only when bi-prediction is allowed for the current block and / or only when the current block is a bi-slice or a bi-picture.

[0064] IV. Exemplary Video Decoders

[0065] Figure 6 An exemplary video decoder 600 is shown. As shown, video decoder 600 is an image-decoding or video-decoding circuit that receives a bitstream 695 and decodes the contents of the bitstream into pixel data of a video frame for display. Video decoder 600 has several components or modules for decoding bitstream 695, including components selected from an inverse quantization module 605, an inverse transform module 610, an intra-frame prediction module 625, a motion compensation module 630, a loop filter 645, a decoded frame buffer 650, an MV buffer 665, an MV prediction module 675, and a parser 690. Motion compensation module 630 is part of inter-frame prediction module 640.

[0066] In some embodiments, modules 610-690 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 610-690 are modules of hardware circuits implemented by one or more integrated circuits of an electronic device. Although modules 610-690 are shown as separate modules, some modules may be combined into a single module.

[0067] The parser 690 (or entropy decoder) receives the bitstream 695 and performs preliminary parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 612. The parser 690 parses the various syntax elements using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0068] The inverse quantization module 605 inverse quantizes the quantized data (or quantized coefficients) 612 to obtain transform coefficients, and the inverse transform module 610 performs an inverse transform operation on the transform coefficients 616 to generate a reconstructed residual signal 619. The reconstructed residual signal 619 is added to the predicted pixel data 613 from the intra-prediction module 625 or the motion compensation module 630 to generate decoded pixel data 617. The decoded pixel data is filtered by the loop filter 645 and stored in the decoded picture buffer 650. In some embodiments, the decoded picture buffer 650 is a memory external to the video decoder 600. In some embodiments, the decoded picture buffer 650 is a memory within the video decoder 600.

[0069] The intra-prediction module 625 receives intra-prediction data from the bitstream 695 and, based thereon, generates predicted pixel data 613 from decoded pixel data 617 stored in the decoded picture buffer 650. In some embodiments, the decoded pixel data 617 is also stored in a line buffer (not shown) used for intra-picture prediction and spatial MV prediction.

[0070] In some embodiments, the contents of the decoded frame buffer 650 are used for display. The display device 655 directly retrieves the contents of the decoded frame buffer 650 for display, or retrieves the contents of the decoded frame buffer into a display buffer. In some embodiments, the display device receives pixel values from the decoded frame buffer 650 via a pixel transfer.

[0071] The motion compensation module 630 generates predicted pixel data 613 from the decoded pixel data 617 stored in the decoded picture buffer 650 according to motion compensated MVs (MC MVs). These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 695 to the predicted MVs received from the MV prediction module 675.

[0072] The MV prediction module 675 generates a predicted MV based on a reference MV generated when decoding the current video frame, that is, a motion-compensated MV used to perform motion compensation. The MV prediction module 675 retrieves the reference MV of the current video frame from the MV buffer 665. The video decoder 600 stores the motion-compensated MV generated for decoding the current video frame in the MV buffer 665 as a reference MV for generating the predicted MV.

[0073] The loop filter 645 performs a filtering or smoothing operation on the decoded pixel data 617 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering operation performed includes Sample Adaptive Offset (SAO). In some embodiments, the filtering operation includes an Adaptive Loop Filter (ALF).

[0074] Figure 7 Portions of a video decoder 600 that can implement SMVD mode and zero L1 MVD mode are shown. As shown, an entropy decoder 690 can parse the SMVD mode flag and / or mvd_l1_zero_flag from a bitstream 695. The entropy decoder can parse the SMVD mode flag if the conditions that enable the SMVD mode flag are met; otherwise, the SMVD mode flag can be derived without signaling. The entropy decoder can parse the mvd_l1_zero_flag if the conditions that enable the mvd_l1_zero_flag are met; otherwise, the mvd_l1_zero_flag can be derived without signaling. The entropy decoder 690 can also parse syntax elements from the bitstream that signal whether the current block is in a B-slice or B-frame and / or whether the current block is bi-predicted or not derived from syntax to determine whether the current block is in a B-slice or B-frame.

[0075] If the current block is in a B-slice or B-frame, bi-prediction is enabled, and / or the current block is coded using bi-prediction, an MVD selection module 700 determines whether to use SMVD mode or zero L1 MVD mode based on the SMVD mode flag and / or mvd_l1_zero_flag.

[0076] When the MVD selection module 700 determines that the SMVD mode is used, –MVD0 is used as the L1 motion vector difference (MVD1) when constructing motion vectors for motion compensation (MCMV). When the MVD selection module 700 determines that the zero L1 MVD mode is used, zero is used as the L1 motion vector difference when constructing motion vectors for motion compensation (MC MV). In other cases, the residual motion data parsed from the bitstream by the entropy decoder 690 is used to construct motion vectors for motion compensation. MVD0 and MVD1 are then used to calculate the motion vector for motion compensation (MC MV) by adding the predicted MV.

[0077] Figure 8 A process for reconciling the SMVD mode with the zero L1 MVD mode is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) on a computing device implementing decoder 600 may perform process 800 by executing instructions stored on a computer-readable medium. In some embodiments, an electronic device implementing decoder 600 may perform process 800.

[0078] The decoder receives (at block 810) data from a bitstream to decode a pixel block as a current block in a current frame of a video. The decoder determines (at block 815) whether the current block is coded using inter-mode and bi-directional prediction. If so, the process proceeds to block 820. If the current block is not coded using inter-mode and bi-directional prediction, the video decoder reconstructs the current block based on another codec mode.

[0079] The decoder receives (at block 820) a first motion vector difference (MVD0) from a bitstream. The decoder determines (at block 825) whether a first flag (mvd_l1_zero_flag) is true or false. In some embodiments, the first flag (mvd_l1_zero_flag) is signaled at a picture level. In some embodiments, the mvd_l1_zero_flag can be signaled at the CU / PU, CTU, slice, tile, tile group, SPS, or PPS level. If the first flag is true, the decoder proceeds to block 830. If the first flag is false, the process proceeds to block 835. In some embodiments, the condition for enabling SMVD mode (or Condition A) includes at least mvd_l1_zero_flag being false.

[0080] At block 830, the decoder sets a second motion vector difference value (i.e., MVD1) to zero because the first flag (mvd_l1_zero_flag) indicates that zero L1 MVD mode is used. In some embodiments, the first and second motion vector differences are associated with first and second reference pictures. In some embodiments, the first reference picture is selected from a first reference picture list and the second reference picture is selected from a second reference picture list. In some embodiments, the first and second reference pictures form a pair of forward and backward reference pictures. This is referred to as the generalized P and B picture (or GPB) condition. The process then proceeds to block 870.

[0081] At block 835, the decoder determines whether a second flag (SMVD mode flag) is true or false. In some embodiments, the second flag (SMVD mode flag) is signaled at the codec unit (CU) level for the current block. In some embodiments, the SMVD mode flag can be signaled at the CTU, slice, tile, tile group, SPS, PPS, or picture level. If the second flag is true, the process proceeds to 840. If the second flag is false, the process proceeds to 860.

[0082] At block 840, the decoder sets the second motion vector difference (i.e., MVD1) to the inverse of the first motion vector difference (i.e., -MVD0). In some embodiments, the first and second motion vector differences are associated with first and second reference pictures that are symmetrical or at the same distance from the current picture. In some embodiments, the decoder may select the first reference picture from a first reference picture list and the second reference picture from a second reference picture list. In some embodiments, the first and second reference pictures form a pair of backward and forward reference pictures, or a pair of forward and backward reference pictures. In some embodiments, the first reference picture is the picture in the first reference picture list that is closest in time or POC to the current picture, and the second reference picture is the picture in the second reference picture list that is closest in time or POC to the current picture. Processing then proceeds to block 870.

[0083] The process then receives (at block 860) the second motion vector difference (MVD1) in the bitstream. In some embodiments, when either the first flag or the second flag is true, the second motion vector difference (MVD1) is not signaled in the bitstream.

[0084] Based on the first motion vector difference and the second motion vector difference, the decoder reconstructs (at block 870) the current block by performing inter prediction (particularly bi-prediction) using the motion information. In some embodiments, the SMVD mode flag and the mvd_l1_zero_flag are considered by the video decoder for inter prediction only when bi-prediction is allowed for the current block and / or only when the current block is a bi-slice or a bi-picture (a prerequisite).

[0085] IX. Exemplary Electronic Systems

[0086] Many of the above features and applications can be implemented as software processing, which is specified as a set of instructions recorded on a computer readable storage medium (also referred to as a computer readable storage medium). When these instructions are executed by one or more computing units or processing units (for example, one or more processors, processor cores or other processing units), the instructions cause the processing unit to perform the actions represented by the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc. The computer readable medium does not include carrier waves and electrical signals connected wirelessly or wired.

[0087] In this specification, the term "software" is meant to include firmware in a read-only memory or an application stored in a magnetic storage device that can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program, while retaining different software inventions. In some embodiments, multiple software inventions can be implemented as independent programs. Finally, any combination of independent programs that together implement the software inventions described herein is within the scope of the present invention. In some embodiments, when installed to perform operations on one or more electronic systems, the software program defines one or more specific machine implementations that execute and implement the operations of the software program.

[0088] Figure 9An electronic system 900 is conceptually illustrated in which some embodiments of the present invention may be implemented. Electronic system 900 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or another type of electronic device. This electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. Electronic system 900 includes a bus 905, a processing unit 910, a graphics processing unit (GPU) 915, system memory 920, a network 925, read-only memory (ROM) 930, permanent storage 935, input devices 940, and output devices 945.

[0089] Bus 905 collectively represents all system buses, peripheral buses, and chipset buses that communicatively couple a wide range of internal devices within electronic system 900. For example, bus 905 communicatively couples to processing unit 910 via graphics processing unit 915, read-only memory 930, system memory 920, and persistent storage 935.

[0090] From these various memory units, the processing unit 910 retrieves instructions to be executed and data to be processed in order to perform the processes of the present invention. In various embodiments, the processing unit may be a single processor or a multi-core processor. Certain instructions are transmitted to and executed by the image processing unit 915. The image processing unit 915 may offload various computations or supplement the image processing provided by the processing unit 910.

[0091] Read-only memory 930 stores static data and instructions needed by processing unit 910 or other modules of the electronic system. Persistent storage 935, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when electronic system 900 is turned off. Some embodiments of the present invention use a mass storage device (e.g., a magnetic disk or optical disk and its corresponding disk drive) as persistent storage 935.

[0092] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as the permanent storage device. Like the permanent storage device 935, the system memory 920 is a read-write memory device. However, unlike the storage device 935, the system memory 920 is a volatile read-write memory, such as random access memory. The system memory 920 stores instructions and data that some processors need at runtime. In some embodiments, processing according to the present invention is stored in the system memory 920, the permanent storage device 935 and / or the read-only memory 930. For example, various memory units include instructions for processing multimedia clips according to some embodiments. From these various memory units, the processing unit 910 retrieves the executed instructions and processed data in order to perform the processing of certain embodiments.

[0093] The bus 905 is also connected to an input device 940 and an output device 945. The input device 940 enables a user to communicate information and select instructions to the electronic system. The input device 940 includes an alphanumeric keyboard and pointing device (also known as a "cursor control device"), a camera (such as a webcam), a microphone or similar device for receiving voice commands, etc. The output device 945 displays images generated by the electronic system or data output in other ways. The output device 945 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices such as a touch screen that serves as both an input device and an output device.

[0094] Finally, if Figure 9 As shown, bus 905 also couples electronic system 900 to a network 925 via a network adapter (not shown). In this manner, the computer can be part of a computer network (e.g., a local area network (LAN), a wide area network (WAN), or an intranet) or a network of networks (e.g., the Internet). Any or all components of electronic system 900 may be used in conjunction with the present invention.

[0095] Some embodiments include electronic components, such as a microprocessor, a storage device, and a memory that stores computer program instructions on a machine-readable medium or computer-readable medium (alternatively referred to as a computer-readable storage medium, a machine-readable medium, or a machine-readable storage medium). Some examples of computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-Rs), rewritable compact discs (CD-RWs), read-only digital versatile discs (e.g., DVD-ROMs, dual-layer DVD-ROMs), various recordable / rewritable DVDs (e.g., DVD RAMs, DVD-RWs, DVD+RWs, etc.), flash memory (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid-state hard drives, read-only and recordable Computer-readable media include optical disks, ultra-high-density optical disks, and any other optical or magnetic media, as well as floppy disks. Computer-readable media can store a computer program that is executed by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that produced by a compiler, and files containing high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0096] While the above discussion primarily refers to microprocessors or multi-core processors executing software, many of the aforementioned functions and applications are performed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. Additionally, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0097] As used in the specification and any claims of the present invention, the terms "computer," "server," "processor," and "memory" refer to electronic devices or other technical equipment. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display device refer to displaying on an electronic device. As used in the specification and any claims of the present invention, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are entirely limited to tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0098] While the present invention has been described with reference to numerous specific details, it will be appreciated by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit of the invention. Figure 5 and Figure 8 ) conceptually illustrates a process. The specific operations of these processes may not be performed in the exact order shown and described. These specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process may be implemented using several sub-processes or as part of a larger macro-process. Therefore, it will be understood by those skilled in the art that the present invention is not limited by the foregoing illustrative details, but is defined by the claims.

[0099] Additional Notes

[0100] The subject matter described in the present invention sometimes represents different components, which are contained in or connected to other different components. It will be understood that the described structures are merely examples and can actually be implemented by many other structures to achieve the same function. Conceptually, any arrangement of components that achieve the same function is actually "associated" so as to achieve the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to achieve a specific function are considered to be "interrelated" to achieve the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to each other to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve a specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting components, and / or wirelessly interactable and / or wirelessly interacting components, and / or logically interacting and / or logically interactable components.

[0101] Furthermore, with respect to the use of substantially any plural and / or singular terms, those skilled in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For sake of clarity, the present invention expressly contemplates the different singular / plural permutations.

[0102] Furthermore, it will be understood by those skilled in the art that, generally, the terms used in the present invention, and particularly in the claims, and the subject matter of the claims, are generally used as “open” terms, e.g., “including” should be interpreted as “including but not limited to,” “having” should be interpreted as “at least having,” “including” should be interpreted as “but not limited to,” etc. It will be further understood by those skilled in the art that if a specific number of claim contents are intended to be introduced, this will be explicitly indicated in the claim, and will not be displayed in the absence of such contents. For example, to aid understanding, the following claims may include the phrases “at least one” and “one or more” to introduce the claim contents. However, the use of these phrases should not be construed as implying the use of the indefinite article “a” or “an” to introduce the claim contents, thereby limiting any particular claim. Even when the same claim includes the introductory phrases “one or more” or “at least one,” the indefinite article, such as “a” or “an,” should be interpreted to mean at least one or more, and the same applies to the use of the explicit description used to introduce the claim. Furthermore, even if it is explicitly stated that Where an introductory statement explicitly refers to a specific number, one skilled in the art will recognize that such reference should be interpreted as indicating the referenced number. For example, "two references" without further modification means at least two references, or two or more references. Furthermore, where a statement similar to "at least one of A, B, and C" is used, such statement is generally intended to be understood by one skilled in the art. For example, "a system includes at least one of A, B, and C" would include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc. One skilled in the art will further understand that any separated word and / or phrase, whether in the specification, claims, or drawings, representing two or more alternative terms should be understood to include the possibility of one, one, or both of these terms. For example, "A or B" should be understood to include the possibility of "A," "B," or "A and B."

[0103] As can be seen from the foregoing, various embodiments have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the scope of the patent application represents the true scope and spirit.

Claims

1. A video decoding method, comprising: Receiving coded data associated with a current codec unit of a current frame of a video; Based on the coded data, determining whether the current codec unit is coded according to bidirectional prediction; as well as When it is determined that the current encoding and decoding unit performs encoding and decoding according to the bidirectional prediction: Determining a first motion vector difference in a first prediction direction of the bidirectional prediction; After determining the first motion vector difference, determining a second motion vector difference in a second direction of the bidirectional prediction includes: determining whether a first flag indicates that the second motion vector difference is zero; If it is determined that the first flag indicates that the second motion vector difference is zero, setting the second motion vector difference to a zero vector; If it is determined that the first flag does not indicate that the second motion vector difference is zero, when the second flag has a first value, setting the second motion vector difference to an inverse value of the first motion vector difference, and when the second flag has a second value, receiving the second motion vector difference from the encoded data; and The current codec unit is reconstructed according to the bidirectional prediction performed based on the first motion vector difference and the second motion vector difference.

2. The video decoding method according to claim 1, wherein: The first flag is signaled at a screen level.

3. The video decoding method according to claim 1, wherein: The second flag is signaled at the codec unit level of the current codec unit.

4. The video decoding method according to claim 1, wherein: When the first flag or the second flag is true, the second motion vector difference is not signaled.

5. The video decoding method according to claim 1, wherein: The advanced motion vector prediction mode is used for the current codec unit.

6. The video decoding method according to claim 1, wherein: The first motion vector difference and the second motion vector difference are associated with a first reference picture and a second reference picture that are symmetrical to the current picture or have the same distance from the current picture.

7. The video decoding method according to claim 6, wherein: The first reference picture is selected from a first reference picture list and the second reference picture is selected from a second reference picture list, wherein the first and second reference pictures form a pair of backward and forward reference pictures.

8. The video decoding method according to claim 7, wherein: The first reference picture is a picture in the first reference picture list that is temporally closest to the current picture, and the second reference picture is a picture in the second reference picture list that is temporally closest to the current picture.

9. The video decoding method according to claim 1, wherein: When the first flag is true, the second motion vector difference is set to zero.

10. The video decoding method according to claim 9, wherein: The first motion vector difference and the second motion vector difference are associated with a first reference frame and a second reference frame, wherein the first reference frame is selected from a first reference frame list and the second reference frame is selected from a second reference frame list, wherein the first reference frame and the second reference frame form a pair of forward and backward reference frames.

11. A video encoding method, comprising: Receiving pixel data of a current codec unit in a current picture; Determining whether the current codec unit is encoded according to bidirectional prediction; as well as When it is determined that the current encoding and decoding unit performs encoding and decoding according to the bidirectional prediction: Determining a first motion vector difference in a first prediction direction of the bidirectional prediction; After determining the first motion vector difference, determining a second motion vector difference in a second direction of the bidirectional prediction includes: determining whether a first flag indicates that the second motion vector difference is zero; If it is determined that the first flag indicates that the second motion vector difference is zero, setting the second motion vector difference to a zero vector; If it is determined that the first flag does not indicate that the second motion vector difference is zero, setting the second motion vector difference to an inverse value of the first motion vector difference when the second flag has a first value, and encoding the second motion vector difference when the second flag has a second value; and The current codec unit is encoded according to the bidirectional prediction performed based on the first motion vector difference and the second motion vector difference.

12. An electronic device for video decoding, comprising: A video decoder is configured to perform operations including: Receiving coded data associated with a current codec unit of a current frame of a video; Based on the coded data, determining whether the current codec unit is coded according to bidirectional prediction; and When it is determined that the current encoding and decoding unit performs encoding and decoding according to the bidirectional prediction: Determining a first motion vector difference in a first prediction direction of the bidirectional prediction; After determining the first motion vector difference, determining a second motion vector difference in a second direction of the bidirectional prediction includes: determining whether a first flag indicates that the second motion vector difference is zero; If it is determined that the first flag indicates that the second motion vector difference is zero, setting the second motion vector difference to a zero vector; If it is determined that the first flag does not indicate that the second motion vector difference is zero, when the second flag has a first value, setting the second motion vector difference to an inverse value of the first motion vector difference, and when the second flag has a second value, receiving the second motion vector difference from the encoded data; and The current codec unit is reconstructed according to the bidirectional prediction performed based on the first motion vector difference and the second motion vector difference.

13. A video encoding and decoding method, comprising: receiving data to be encoded or decoded as a current codec unit in a current frame of a video; Determining whether the current encoding and decoding unit performs encoding according to bidirectional prediction based on the received data; as well as When it is determined that the current encoding and decoding unit performs encoding and decoding according to the bidirectional prediction: Determining a first motion vector difference in a first prediction direction of the bidirectional prediction; After determining the first motion vector difference, determining a second motion vector difference in a second direction of the bidirectional prediction includes: determining whether a first flag indicates that the second motion vector difference is zero; If it is determined that the first flag indicates that the second motion vector difference is zero, setting the second motion vector difference to a zero vector; If it is determined that the first flag does not indicate that the second motion vector difference is zero, when the second flag has a first value, setting the second motion vector difference to an inverse value of the first motion vector difference, and when the second flag has a second value, encoding the second motion vector difference or receiving the second motion vector difference from encoded data associated with the current codec unit; and The current codec unit is encoded or decoded according to the bidirectional prediction performed based on the first motion vector difference and the second motion vector difference.

14. A non-volatile computer-readable storage medium storing data and instructions, which, when executed by a processor of an electronic device for video encoding and decoding, causes the device to perform the operation described in any one of the methods of claims 1-11 and 13.

Images