Video decoding method and apparatus

Abstract

The application provides a video decoding method and device, relates to the field of video coding technology, and can solve the problem of low video quality after coding in the prior art. The method comprises the following steps: parsing a code stream, wherein the code stream comprises a prediction mode identification bit and a merge motion vector difference (MMVD) identification bit, the prediction mode identification bit is used for indicating a prediction mode of a to-be-decoded image block, the prediction mode comprises at least one of a triangular prediction unit mode and a joint inter-intra prediction mode, and the MMVD identification bit is used for indicating an acquisition mode of motion information of the to-be-decoded image block; and then the to-be-decoded image block is decoded according to the prediction mode identification bit and the MMVD identification bit. The method is applied in a video coding process.

Description

[0001] This application is a divisional application. The original application has the application number 201811574721.8 and the original application date is December 21, 2018. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of video encoding and decoding technology, and in particular to a video decoding method and apparatus. Background Technology

[0003] With the development of internet technology, the popularity of various internet services continues to rise. Among them, users' demands for video service quality are also constantly increasing. Generally speaking, video quality is inseparable from the video encoding and decoding method. A better video encoding and decoding method can ensure that a lower bitrate is used to transmit the same quality video image, that is, less data can be transmitted.

[0004] In current video encoding and decoding schemes, at the encoding end, a video frame can be divided into several image blocks by selecting a partitioning mode. A prediction mode is then selected to predict the values ​​of these image blocks, which are sent to the decoding end. The decoding end then reconstructs the image block based on its predicted values.

[0005] Image blocks can be encoded and decoded by combining different partitioning and prediction modes. However, encoding and decoding image blocks using this combination method results in low-quality image blocks. Summary of the Invention

[0006] This application provides a video decoding method and apparatus that results in high-quality decoded images.

[0007] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, embodiments of this application provide a video decoding method, which is applied to a video decoding device or a chip of the video decoding device. The method includes: parsing a bitstream, the bitstream including a prediction mode identifier bit and a combined motion vector difference (MMVD) identifier bit, the prediction mode identifier bit being used to indicate the prediction mode of the image block to be decoded, the prediction mode including at least one of a triangular prediction unit mode and a joint inter-frame-intra-frame prediction mode, the MMVD identifier bit being used to indicate the method of obtaining motion information of the image block to be decoded, and then decoding the image block to be decoded according to the prediction mode identifier bit and the MMVD identifier bit.

[0009] The decoding method provided in this application involves a video decoder parsing the bitstream and decoding the image block to be decoded based on the prediction mode identifier and MMVD identifier bits contained in the bitstream. The combination of the prediction mode identifier and the MMVD identifier bits indicates whether MMVD is used for motion compensation of the image block to be decoded under the current prediction mode. Compared to existing technologies that decode image blocks by combining mode segmentation and prediction modes, the decoding method in this application can utilize MMVD to improve the accuracy of the prediction value of the current image block, resulting in a higher quality decoded image.

[0010] In one possible design, the prediction mode is a triangular prediction unit mode. The image block to be decoded consists of a first prediction unit and a second prediction unit. The MMVD identifier includes an MMVD indicator bit, a step size identifier, and a direction identifier. The MMVD indicator bit is used to indicate whether the motion information acquisition method of the image block to be decoded is MMVD.

[0011] Optionally, the image block to be decoded is decoded based on the prediction mode identifier and the MMVD identifier, specifically implemented as follows:

[0012] When the MMVD indicator bit is 1 and the prediction mode flag bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, it means that motion compensation of the image block to be decoded needs to be performed using MMVD. Optionally, MMVD can be used to perform motion compensation on the first prediction unit to obtain its MV. Specifically: obtain the predicted motion vector MVP of the first prediction unit, and determine the motion vector MV of the first prediction unit based on its MVP, step size flag, and direction flag. Optionally, MMVD can be used to perform motion compensation on the second prediction unit: obtain the MVP of the second prediction unit, and determine its MV based on its MVP, step size flag, and direction flag. In other words, MMVD can be used to perform motion compensation on the first prediction unit, and other motion compensation methods can be used to perform motion compensation on the second prediction unit. Alternatively, MMVD can be used to perform motion compensation on the second prediction unit, and other motion compensation methods can be used to perform motion compensation on the first prediction unit. Furthermore, MMVD can be used to perform motion compensation on both the first and second prediction units.

[0013] Then, the image block to be decoded is decoded based on the MV of the first prediction unit and the MV of the second prediction unit.

[0014] When the MMVD indicator bit is 0 and the prediction mode flag bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, it means that motion compensation for the image block to be decoded is not used using MMVD. In this case, the first prediction unit and the second prediction unit are decoded directly according to the existing technology process. Specifically, the prediction value of the first prediction unit and the prediction value of the second prediction unit are obtained, and the image block to be decoded is decoded based on the prediction values ​​of the first prediction unit and the second prediction unit.

[0015] In one possible design, the prediction mode is a joint inter-frame-intra-frame prediction mode.

[0016] Optionally, the image block to be decoded is decoded based on the prediction mode identifier and the MMVD identifier, specifically implemented as follows:

[0017] When the MMVD indicator bit is 1 and the prediction mode flag indicates that the prediction mode of the image block to be decoded is joint inter-frame-intra-frame prediction mode, it means that motion compensation of the image block to be decoded needs to be performed using MMVD. In this case, the image block to be decoded is decoded according to the following steps: Obtain the MVP of the image block to be decoded, and determine the MV of the image block to be decoded based on the MVP, stride flag, and direction flag. Determine the inter-frame prediction prediction value of the image block to be decoded based on the MV, and perform intra-frame prediction on the image block to be decoded to obtain the intra-frame prediction prediction value of the image block to be decoded. Decode the image block to be decoded based on the inter-frame prediction prediction value and the intra-frame prediction prediction value of the image block to be decoded.

[0018] When the MMVD indicator bit is 0 and the prediction mode flag bit indicates that the prediction mode of the image block to be decoded is the joint inter-frame-intra-frame prediction mode, it means that motion compensation for the image block to be decoded is not performed using MMVD. In this case, the image block to be decoded is directly decoded according to the existing technology process. Specifically, inter-frame prediction is performed on the image block to be decoded to obtain the predicted value of the inter-frame prediction of the image block to be decoded, and intra-frame prediction is performed on the image block to be decoded to obtain the predicted value of the intra-frame prediction of the image block to be decoded. Then, the image block to be decoded is decoded based on the predicted values ​​of the inter-frame prediction and the intra-frame prediction of the image block to be decoded.

[0019] In one possible design, the MMVD identifier bit also includes the MMVD initial position identifier.

[0020] When the prediction mode of the image block to be decoded is the triangular prediction unit mode:

[0021] Optionally, the predicted motion vector (MVP) of the first prediction unit is obtained. Specifically, the bitstream is parsed, the initial position identifier of the MMVD of the first prediction unit in the bitstream is obtained, and the MVP of the first prediction unit is obtained based on the initial position identifier of the MMVD of the first prediction unit.

[0022] Similarly, to obtain the MVP of the second prediction unit, the specific implementation is as follows: parse the bitstream, obtain the initial position identifier of the MMVD of the second prediction unit in the bitstream, and obtain the MVP of the second prediction unit based on the initial position identifier of the MMVD of the second prediction unit.

[0023] Optionally, the predicted motion vector MVP of the first prediction unit can be obtained by parsing the bitstream, obtaining the MVP information of the first prediction unit in the bitstream, and obtaining the MVP of the first prediction unit based on the MVP information of the first prediction unit.

[0024] The MVP of the second prediction unit is obtained by: obtaining the MVP information of the second prediction unit in the bitstream, and obtaining the MVP of the second prediction unit based on the MVP information of the second prediction unit.

[0025] When the prediction mode of the image block to be decoded is the joint inter-frame-intra-frame prediction mode:

[0026] Optionally, the MVP of the image block to be decoded can be obtained by parsing the bitstream, obtaining the initial MMVD position identifier of the image block to be decoded in the bitstream, and obtaining the MVP of the image block to be decoded based on the initial MMVD position identifier of the image block to be decoded.

[0027] Optionally, obtaining the MVP of the image block to be decoded can be implemented as follows: parsing the bitstream to obtain the MVP information of the image block to be decoded in the bitstream, and obtaining the MVP of the image block to be decoded based on the MVP information of the image block to be decoded.

[0028] Secondly, this application provides a video decoding device, which may be a video decoder or a chip within a video decoder. The device includes a parsing unit and a decoding unit.

[0029] The parsing unit is used to parse the bitstream, which includes a prediction mode identifier and a combined motion vector difference (MMVD) identifier. The prediction mode identifier is used to indicate the prediction mode of the image block to be decoded. The prediction mode includes at least one of the triangular prediction unit mode and the joint inter-frame-intra-frame prediction mode. The MMVD identifier is used to indicate the method of obtaining the motion information of the image block to be decoded.

[0030] The decoding unit is used to decode the image block to be decoded based on the prediction mode identifier bit and the MMVD identifier bit.

[0031] In one possible design, the MMVD identifier includes an MMVD indicator bit, a step size identifier, and a direction identifier. The MMVD indicator bit is used to indicate whether the motion information acquisition method of the image block to be decoded is MMVD.

[0032] In one possible design, the prediction mode is a triangular prediction unit mode, where the image block to be decoded consists of a first prediction unit and a second prediction unit.

[0033] Optionally, the decoding unit is configured to decode the image block to be decoded based on the prediction mode identifier bit and the MMVD identifier bit, including: when the MMVD indicator bit is 1 and the prediction mode identifier bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, obtaining the predicted motion vector MVP of the first prediction unit, and / or obtaining the MVP of the second prediction unit; determining the motion vector MV of the first prediction unit based on the MVP, step size identifier and direction identifier of the first prediction unit, and / or determining the MV of the second prediction unit based on the MVP, step size identifier and direction identifier of the second prediction unit; and decoding the image block to be decoded based on the MV of the first prediction unit and the MV of the second prediction unit.

[0034] Optionally, when the MMVD indicator bit is 0 and the prediction mode identifier bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, the decoding unit is used to decode the image block to be decoded according to the prediction mode identifier bit and the MMVD identifier bit, including: obtaining the prediction value of the first prediction unit and obtaining the prediction value of the second prediction unit; and decoding the image block to be decoded according to the prediction value of the first prediction unit and the prediction value of the second prediction unit.

[0035] In one possible design, the prediction mode is a joint inter-frame-intra-frame prediction mode.

[0036] Optionally, the decoding unit is configured to decode the image block to be decoded based on the prediction mode identifier bit and the MMVD identifier bit, including: obtaining the MVP of the image block to be decoded when the MMVD indicator bit is 1 and the prediction mode identifier bit indicates that the prediction mode of the image block to be decoded is a joint inter-frame-intra-frame prediction mode; determining the MV of the image block to be decoded based on the MVP, step size identifier, and direction identifier of the image block to be decoded; determining the predicted value of the inter-frame prediction of the image block to be decoded based on the MV of the image block to be decoded, and performing intra-frame prediction on the image block to be decoded to obtain the predicted value of the intra-frame prediction of the image block to be decoded; and decoding the image block to be decoded based on the predicted value of the inter-frame prediction and the predicted value of the intra-frame prediction of the image block to be decoded.

[0037] Optionally, when the MMVD indicator bit is 0 and the prediction mode identifier bit indicates that the prediction mode of the image block to be decoded is the joint inter-frame-intra-frame prediction mode, the decoding unit, used to decode the image block to be decoded according to the prediction mode identifier bit and the MMVD identifier bit, further includes: performing inter-frame prediction on the image block to be decoded to obtain the prediction value of the inter-frame prediction of the image block to be decoded, and performing intra-frame prediction on the image block to be decoded to obtain the prediction value of the intra-frame prediction of the image block to be decoded; and decoding the image block to be decoded according to the prediction value of the inter-frame prediction of the image block to be decoded and the prediction value of the intra-frame prediction of the image block to be decoded.

[0038] In one possible design, the MMVD identifier bit also includes the MMVD initial position identifier.

[0039] When the prediction mode of the image block to be decoded is a triangular prediction unit:

[0040] Optionally, obtaining the predicted motion vector MVP of the first prediction unit and / or obtaining the MVP of the second prediction unit includes: parsing the bitstream, obtaining the initial MMVD position identifier of the first prediction unit in the bitstream, and / or obtaining the initial MMVD position identifier of the second prediction unit in the bitstream; obtaining the MVP of the first prediction unit based on the initial MMVD position identifier of the first prediction unit, and / or obtaining the MVP of the second prediction unit based on the initial MMVD position identifier of the second prediction unit.

[0041] Optionally, obtaining the predicted motion vector MVP of the first prediction unit and / or obtaining the MVP of the second prediction unit includes: parsing the bitstream, obtaining the MVP information of the first prediction unit in the bitstream, and / or obtaining the MVP information of the second prediction unit in the bitstream; obtaining the MVP of the first prediction unit based on the MVP information of the first prediction unit, and / or obtaining the MVP of the second prediction unit based on the MVP information of the second prediction unit.

[0042] When the prediction mode of the image block to be decoded is the joint inter-frame-intra-frame prediction mode:

[0043] Optionally, obtaining the MVP of the image block to be decoded includes: parsing the bitstream to obtain the initial MMVD position identifier of the image block to be decoded in the bitstream; and obtaining the MVP of the image block to be decoded based on the initial MMVD position identifier of the image block to be decoded.

[0044] Optionally, obtaining the MVP of the image block to be decoded includes: parsing the bitstream to obtain the MVP information of the image block to be decoded in the bitstream; and obtaining the MVP of the image block to be decoded based on the MVP information of the image block to be decoded.

[0045] Thirdly, embodiments of this application provide a video decoding apparatus that has the function of implementing the video decoding method of any one of the first aspects described above. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0046] Fourthly, a video decoding apparatus is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, and when the video decoding apparatus is running, the processor executes the computer execution instructions stored in the memory to cause the video decoding apparatus to perform a video decoding method as described in any of the first aspects above.

[0047] Fifthly, a video decoding apparatus is provided, comprising: a processor; the processor is configured to be coupled to a memory, and after reading instructions from the memory, to execute a video decoding method as described in any of the first aspects above.

[0048] In a sixth aspect, a computer-readable storage medium is provided, which stores instructions that, when executed on a computer, enable the computer to perform the video decoding method of any of the first aspects described above.

[0049] In a seventh aspect, a computer program product containing instructions is provided, which, when run on a computer, enables the computer to execute the video decoding method of any one of the first aspects described above.

[0050] Eighthly, a circuit system is provided, the circuit system including processing circuitry configured to perform a video decoding method as described in any of the first aspects above.

[0051] Ninth aspect, a chip is provided, the chip including a processor and a memory coupled together, the memory storing program instructions, and when the program instructions stored in the memory are executed by the processor, the video decoding method of any one of the first aspects is implemented.

[0052] In a tenth aspect, a communication system is provided, which includes the video decoding device and the video encoding device described in the first aspect of the above aspects.

[0053] The technical effects of any of the design methods in aspects two through ten can be found in the technical effects of different design methods in aspect one, and will not be repeated here. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of image block division in the triangular prediction unit mode;

[0055] Figure 2 This is a schematic diagram of the vertical prediction pattern;

[0056] Figure 3 This is a system architecture diagram of the encoding / decoding system provided in the embodiments of this application;

[0057] Figure 4 This is a schematic diagram of the structure of a video encoder provided in an embodiment of this application;

[0058] Figure 5 This is a schematic diagram of the structure of the video decoder provided in the embodiments of this application;

[0059] Figure 6 Flowchart of the video decoding method provided in the embodiments of this application Figure 1 ;

[0060] Figure 7 Flowchart of the video decoding method provided in the embodiments of this application Figure 2 ;

[0061] Figure 8 Flowchart of the video decoding method provided in the embodiments of this application Figure 3 ;

[0062] Figure 9 This is a schematic diagram illustrating the calculation of predicted values ​​for image patches in the triangular prediction unit mode.

[0063] Figure 10 A schematic diagram of the two reference frames selected for the current frame;

[0064] Figure 11 This is a schematic diagram of the MMVD process;

[0065] Figure 12 This is a schematic diagram of the decoding device provided in the embodiments of this application;

[0066] Figure 13 This is a schematic diagram of the decoding device provided in the embodiments of this application;

[0067] Figure 14 This is a schematic diagram of the decoding device provided in an embodiment of this application. Detailed Implementation

[0068] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or different treatments of the same object, rather than to describe a specific order of objects. Furthermore, the terms "comprising" and "having," and any variations thereof, mentioned in the description of this application, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as preferred or advantageous over other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0069] First, the technical terms involved in the embodiments of this application will be introduced:

[0070] Video sequence: A complete image in a video is usually called a "frame", and a video composed of many frames in chronological order is called a video sequence.

[0071] Video coding technology addresses the issue of redundant information in video sequences, including spatial, temporal, visual, entropy, structural, knowledge, and importance redundancy. To minimize this redundancy and reduce the amount of data representing the video, video coding technology was developed to reduce storage space and bandwidth consumption. Video coding is also known as video compression.

[0072] Within the internationally accepted scope, video compression coding standards include, for example, Advanced Video Coding (AVC) Part 10 of the MPEG-2 and MPEG-4 standards developed by the Motion Picture Experts Group (MPEG), and H.263, H.264, and H.265 (also known as High Efficiency Video Coding standard) developed by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T).

[0073] It should be noted that in encoding algorithms based on a hybrid encoding architecture, the above compression encoding methods can be used in combination.

[0074] Partitioning Mode: The basic processing unit in video compression coding is the image block, which is obtained by dividing a frame / image at the encoding end. Partitioning mode refers to the partitioning method used when dividing a frame / image into multiple image blocks. Taking HEVC as an example, HEVC defines CTU, Coding Unit (CU), Prediction Unit (PU), and Transform Unit (TU). CTU, CU, PU, ​​and TU can all be used as the resulting image blocks after partitioning.

[0075] Prediction Mode: Due to the temporal and spatial redundancy of video sequences, these redundancy characteristics can be used to predict image patches. For example, within the same frame, the correlation between adjacent image patches allows the value of the current image patch to be inferred (predicted) based on neighboring patches. In different frames, for an image patch at a certain coordinate position, the value of the patch at the previous moment is correlated with its value at the next moment; therefore, the value of the patch at the next moment can be predicted based on the value of the patch at the previous moment. The method used to predict the current image patch is called the prediction mode.

[0076] It should be noted that prediction modes may differ across different video coding standards. For example, prediction modes include intra-frame prediction and inter-frame prediction. Intra-frame prediction includes vertical prediction mode, horizontal prediction mode, bottom-left prediction mode, bottom-right prediction mode, right vertical prediction mode, etc. Inter-frame prediction modes include motion estimation, etc.

[0077] Taking motion estimation in inter-frame prediction as an example, the purpose of motion estimation is to search for the most similar image patch to the current image patch among multiple reference frames. This most similar image patch is called the prediction patch. Similarity means that the texture features, chroma, brightness, etc., of the image patches are relatively close. When using unidirectional prediction, the reference frame can be the frame before or after the current frame. When using bidirectional prediction, the reference frames can be both the frame before and after the current frame. The video decoding method in this application does not limit the number of reference frames or the temporal order of the reference frames and the current frame.

[0078] Taking the vertical prediction mode in intra-frame prediction as an example, in vertical prediction mode, neighboring image patches can be used to predict the value of the current image patch. See also Figure 2 The current image block is the CU_8*8 (i.e., an 8*8 CU) indicated by the arrow, and adjacent image blocks can be... Figure 2The pixel prediction values ​​of CU_8*8 can be obtained from the pixel values ​​of these 8 CU_1*1, numbered 1 to 8. For example, the average value of these 8 CU_1*1 pixels can be used as the pixel prediction value of CU_8*8.

[0079] Motion information: When predicting an image patch, it is necessary to obtain the motion information of the image patch. For example, the motion information can be the motion vector (MV) of the image patch.

[0080] The following introduces two prediction modes: the triangular prediction unit mode and the combined intermerge / intra prediction mode.

[0081] Triangle Prediction Unit (Triangle PU) Mode: When performing intra-frame or inter-frame prediction, the current image patch is divided into two triangle prediction units (Triangle PUs). This division mode is called the triangle prediction unit mode.

[0082] For example, see Figure 1 The current image patch is divided into two triangular prediction units, PU1 and PU2. Optionally, such as... Figure 1 In the middle (a), triangular prediction units can be divided using a left-top / right-bottom division method, or as follows: Figure 1 In (b), the triangular prediction units are divided using a left-lower / right-upper division method.

[0083] On the encoding side (video encoder), for each triangulation prediction unit, the motion vector and reference frame of that triangulation prediction unit are selected from its prediction candidate list, and the prediction value of that triangulation prediction unit is determined based on its motion vector. Then, the prediction values ​​of the image region excluding the two triangulation prediction units (which may also be referred to as the hypotenuse region of the triangle in this paper) in the current image block are calculated to obtain the prediction value of the entire current image block. Afterwards, the entire current image block undergoes transformation and quantization processes to obtain a bitstream containing the image block data, and this bitstream is sent to the decoding side (video decoder). Correspondingly, on the decoding side, the video decoder can decode the two prediction units contained in the current image block separately to reconstruct the current image block.

[0084] Among them, see Figure 1 When calculating the predicted value of the hypotenuse region of a triangle, the following two sets of weighted parameters can be used:

[0085] It should be noted that when the reference frames of the two prediction units are different or the difference between their motion vectors is greater than 16 pixels, the second set of weighting parameters is used to weight the two prediction values; otherwise, the first set of weighting parameters is used to weight the two prediction values.

[0086] The first set of weighted parameters: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8}, {7 / 8, 4 / 8, 1 / 8}.

[0087] Here, {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} is used for the weighted prediction of the luminance components of PU1 and PU2. {7 / 8, 4 / 8, 1 / 8} is used for the weighted prediction of the chrominance components of PU1 and PU2.

[0088] The second set of weighted parameters: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8}.

[0089] Here, {7 / 8,6 / 8,5 / 8,4 / 8,3 / 8,2 / 8,1 / 8} is used for the predicted values ​​of the weighted luminance components of PU1 and PU2, and {6 / 8,4 / 8,2 / 8} is used for the predicted values ​​of the weighted chrominance components of PU1 and PU2.

[0090] For example, taking the prediction of the luminance component as an example, see [link to relevant documentation]. Figure 9 The predicted brightness value of the upper right triangular prediction unit (the upper right part filled with black) is P1, and the predicted brightness value of the lower left triangular prediction unit (the lower left part filled with black) is P2. Assume that the first set of weighted parameters is used to calculate the predicted value of the hypotenuse region of the triangle. Specifically, the hypotenuse region of the triangle includes pixel blocks labeled 7, 6, 4, 2, and 1. Correspondingly, the predicted value of the hypotenuse region is determined by the predicted values ​​of the pixel blocks labeled 7, 6, 4, 2, and 1. The predicted value of the pixel block labeled 7 is... The predicted value of the pixel block labeled 6 is The predicted value of pixel block labeled 4 is The predicted value of the pixel block labeled 2 is The predicted value of the pixel block labeled 1 is It is easy to understand that after obtaining the predicted value of each pixel block in the hypotenuse region of the triangle, the predicted value of the hypotenuse region of the triangle can be obtained from the predicted value of each pixel block.

[0091] It should be noted that, in the embodiments of this application, prediction can refer to unidirectional prediction or bidirectional prediction. The embodiments of this application do not limit the direction of prediction.

[0092] Motion Vector (MV): For the current image patch in the current frame, it is compared with a subset of image patches in the reference frame, and the best matching image patch among the subset of image patches is found. The displacement between the best matching image patch and the current image patch is called the motion vector. The reference frame can be a video frame preceding the current frame, the current frame, or a video frame following the current frame; it can be one frame or multiple frames.

[0093] Motion Vector Prediction (MVP): Due to the continuity of object motion, motion vectors are correlated in the time domain (also referred to as the temporal domain in this paper). That is, the motion vector of an image block in the previous frame that has the same coordinate position as the current image block can be used as the predicted value of the motion vector of the current image block; this predicted value is also called the predicted motion vector. Furthermore, motion vectors are also correlated in the spatial domain (also referred to as the spatial domain in this paper, referring to the position coordinates of an image block in a frame of video). Thus, the motion vectors of neighboring image blocks can be used as the predicted motion vector of the current image block.

[0094] Prediction Candidate List: In one example, the prediction candidate list includes 5 candidate predicted motion vectors (MVPs). These 5 MVPs are derived from 7 neighboring image patches. Image patches 1 to 5 are the spatially adjacent image patches of the current image patch in the same frame, and image patches 6 and 7 are image patches in other video frames that have the same coordinate positions as the current image patch. The MVs of 5 image patches can be selected from these 7 image patches, and these 5 selected image patch MVs can be added to the prediction candidate list as the 5 candidate MVPs for the current image patch. It is understood that if the current image patch has fewer than 5 neighboring image patches, a zero motion vector can be added to the prediction candidate list.

[0095] Joint inter-frame-intra-frame prediction mode: In this prediction mode, two prediction modes are used to predict the current image block. The first prediction value of the current image block is obtained by using the intra-frame prediction mode, and the second prediction value of the current image block is obtained by using the inter-frame prediction mode. The current image block is then reconstructed on the decoding side based on the first and second prediction values.

[0096] For the luma component, the intra-prediction modes in the joint inter-frame-intra-frame prediction mode include mean (DC) mode, planar (PLANAR) mode, horizontal prediction mode, and vertical prediction mode. For example, the prediction candidate list length for intra-prediction can be 3 or 4 based on the block size. When the width of an image block is greater than twice its height, the intra-prediction modes for that block include DC mode, PLANAR mode, and vertical prediction mode. When the height of an image block is greater than twice its width, the intra-prediction modes for that block include DC mode, PLANAR mode, and horizontal prediction mode.

[0097] Similarly, for the chroma component, the intra-frame prediction mode of the image patch can also be determined based on factors such as the size of the image patch. This will not be elaborated upon further here.

[0098] It should be noted that the triangular prediction unit mode may also be called the triangular mode or other names in some scenarios. The joint inter-frame-intra-frame prediction mode may also be called the merge mode, the joint inter-frame-intra-frame prediction mode, or the hybrid mode, or other names. It is understood that the mode name does not constitute a limitation on the mode itself.

[0099] Merge with motion vector difference (MMVD): After selecting one or more MVPs from the prediction candidate list, an offset can be added to the MVPs to determine the final MV in order to make the motion estimation results more accurate. This technique of adding an offset to the MVPs is called MMVD.

[0100] The superimposed offset is called MVD. The final MV = MVP + MVD. The superimposed offset, i.e., MVD, can be represented by the initial position of MMVD, the step size, and the direction of motion.

[0101] The MMVD initial position indicates the initial position of the MV, i.e., the position of the MVP. The position of the MVP can be indicated by the MMVD initial position flag. The MMVD initial position flag can be the Base candidate IDX flag. See Table 1 for an example definition of the Base candidate IDX flag:

[0102] Table 1

[0103]

[0104] As shown in Table 1 above, the Base candidate IDX flag can be 2 bits long. If the Base candidate IDX flag of the current image block is 00 (corresponding to decimal 0), it indicates that the encoder selects the first MVP in the prediction candidate list as the MVP of the current image block, and correspondingly, the decoder also selects the first MVP as the MVP of the current image block. When the Base candidate IDX flag is 10 (corresponding to decimal 2), it indicates that the encoder selects the third MVP in the prediction candidate list as the MVP of the current image block, and so on.

[0105] It should be noted that Table 1 is only an exemplary definition of the Base candidate IDX flag. It is understood that the definition of the Base candidate IDX flag may change in other scenarios, such as when the number of MVPs in the prediction candidate list changes. This application does not limit the specific definition of the Base candidate IDX flag.

[0106] The step size represents the distance offset of the MV relative to the MVP. This distance offset can be represented by pixel values; specifically, the step size can be indicated by the distance index (Distance IDX). See Table 2 for an example definition of the Distance IDX:

[0107] Table 2

[0108]

[0109] As shown in Table 2 above, for example, the length of the Distance IDX identifier can be 3 bits. Distance IDX identifier 000 (corresponding to decimal 0) indicates that the distance offset of MV relative to the initial MVP is 1 / 4 pixel, Distance IDX identifier 001 (corresponding to decimal 0) indicates that the distance offset of MV relative to MVP is 1 / 2 pixel, and so on.

[0110] Optionally, the direction of motion can be indicated by a Direction Index (IDX) identifier. It represents the direction of the motion vector difference (MVD) of the MVP based on the initial position. See Table 3 for an example definition of the Direction IDX identifier:

[0111] Table 3

[0112] Direction IDX 00 01 10 11 (x-axis) + – N / A N / A (y-axis) N / A N / A + –

[0113] As shown in Table 3 above, the length of the Direction IDX flag can be 2. Direction IDX flag 00 indicates that a positive MVD is superimposed on the MVP along the positive x-axis (horizontal direction), and Direction IDX flag 01 indicates that a negative MVD is superimposed on the MVP along the negative x-axis (horizontal direction). The terms "positive" and "negative" can be defined according to the application scenario of the video encoding / decoding system and are not limited here.

[0114] Thus, in one example, when the Base candidate IDX flag is 00, the Distance IDX flag is 001, and the Direction IDX flag is 01, it means that based on the first MVP in the predicted candidate list, an MVD offset by 1 / 2 pixels is superimposed on the first MVP along the negative x-axis.

[0115] See Figure 10 This shows two reference frames for the current frame: frame L0 and frame L1. Frame L0 is the frame preceding the current frame, and frame L1 is the frame following the current frame. For a detailed schematic diagram of the MMVD process, please refer to [link to MMVD diagram]. Figure 11 See also Figure 11 In (a), in reference frame L0, the central white dot represents the pixel position pointed to by the MVP. The four black dots adjacent to this white dot can represent the central white dot along the negative x-axis (e.g., ...). Figure 11 The position is obtained by offsetting the white dot below, along the positive x-axis (e.g., above), the negative y-axis (left), and the positive y-axis (right) by 1 / 4 pixel. Specifically, for the black dot above the central white dot, the MVD generated from the central white dot to that black dot has a step size of 1 / 4 pixel and an upward direction; for the black dot below the central white dot, the corresponding step size is 1 / 4 pixel and the direction is downward, and so on. Similarly, the four white dots in the box can represent the central white dot offset along the negative x-axis (e.g., below), the positive x-axis (e.g., above), the negative y-axis (left), and the positive y-axis (right). Figure 11 The position is obtained by offsetting by 1 / 2 pixels from the bottom, positive x-axis (e.g., top), negative y-axis (left), and positive y-axis (right), and so on. Similarly, in Figure 11 In (b), the step size and direction of each MVD generated from the central white dot to each pixel can be referenced. Figure 11 The description of (a) will not be repeated here.

[0116] The decoding method provided in this application is applicable to video encoding and decoding systems. Figure 3 The structure of a video encoding / decoding system is shown.

[0117] like Figure 3As shown, the video encoding / decoding system includes a source device 30 and a destination device 31. The source device 30 generates encoded video data; it may also be referred to as a video encoding device or video encoding apparatus. The destination device 31 decodes the encoded video data generated by the source device 30; it may also be referred to as a video decoding device or video decoding apparatus. The source device 30 and / or the destination device 31 may include at least one processor and a memory coupled to the at least one processor. The memory may include, but is not limited to, read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures; this application does not specifically limit this.

[0118] The source device 30 and the destination device 31 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0119] The destination device 31 may receive encoded video data from the source device 30 via link 32. Link 32 may include one or more media and / or devices capable of moving the encoded video data from the source device 30 to the destination device 31. In one example, link 32 may include one or more communication media enabling the source device 30 to transmit encoded video data directly to the destination device 31 in real time. In this example, the source device 30 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the destination device 31. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum, one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices enabling communication from the source device 30 to the destination device 31.

[0120] In another example, encoded video data can be output from output interface 303 to storage device 33. Similarly, encoded video data can be accessed from storage device 33 via input interface 313. Storage device 33 may contain various local access data storage media, such as Blu-ray discs, high-density digital video discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, or other suitable digital storage media for storing encoded video data.

[0121] In another example, storage device 33 may correspond to a file server or another intermediate storage device that stores encoded video data generated by source device 30. In this example, destination device 31 may obtain the video data stored in storage device 33 via streaming or downloading. The file server may be any type of server capable of storing and transmitting encoded video data to destination device 31. For example, the file server may include a World Wide Web (Web) server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk drive.

[0122] The target device 31 can access the encoded video data via any standard data connection (e.g., an Internet connection). Examples of data connection types include wireless channels, wired connections (e.g., cable modems, etc.), or combinations thereof, suitable for accessing encoded video data stored on a file server. The encoded video data can be transmitted from the file server via streaming, downloading, or a combination of both.

[0123] The decoding method of this application is not limited to wireless application scenarios. For example, the decoding method of this application can be applied to video encoding and decoding that supports a variety of multimedia applications, including: over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding of video data stored on data storage media, decoding of video data stored on data storage media, or other applications. In some instances, the video encoding and decoding system 1 can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0124] It should be noted that, Figure 3The video encoding / decoding system shown is merely an example and is not intended to limit the scope of the video encoding / decoding system in this application. The encoding / decoding method provided in this application can also be applied to scenarios where there is no data communication between the encoding and decoding devices. In other instances, the video data to be encoded or the encoded video data can be retrieved from local storage or streamed over a network. The video encoding device can encode the video data to be encoded and store the encoded video data in storage; the video decoding device can also retrieve the encoded video data from storage and decode it.

[0125] exist Figure 3 In this embodiment, source device 30 includes a video source 301, a video encoder 302, and an output interface 303. In some instances, the output interface 303 may include a modulator / demodulator (modem) and / or a transmitter. The video source 301 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video input interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources of video data.

[0126] The video encoder 302 encodes video data from the video source 301. In some instances, the source device 30 transmits the encoded video data directly to the destination device 31 via the output interface 303. In other instances, the encoded video data may be stored on the storage device 33 for later access by the destination device 31 for decoding and / or playback.

[0127] exist Figure 3 In this example, the destination device 31 includes a display device 311, a video decoder 312, and an input interface 313. In some examples, the input interface 313 includes a receiver and / or a modem. The input interface 313 can receive encoded video data via link 32 and / or from storage device 33. The display device 311 may be integrated with the destination device 31 or may be external to the destination device 31. Generally, the display device 311 displays the decoded video data. The display device 311 may include various display devices, such as liquid crystal displays, plasma displays, organic light-emitting diode displays, or other types of display devices.

[0128] Optionally, the video encoder 302 and the video decoder 312 may each be integrated with the audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other hardware and software to process the encoding of both audio and video in a common data stream or separate data streams.

[0129] The video encoder 302 and video decoder 312 may include at least one microprocessor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the encoding and decoding method provided in this application is implemented in software, the instructions for the software can be stored in a suitable non-volatile computer-readable storage medium, and at least one processor can be used to execute the instructions to implement this application.

[0130] The video encoder 302 and video decoder 312 in this application can operate according to video compression standards (such as HEVC) or other industry standards, and this application does not make any specific limitations on them.

[0131] Figure 4 This is a schematic block diagram of a video encoder 302 in an embodiment of this application. The video encoder 302 can perform intra-frame coding and inter-frame coding of video blocks within a video stripe. Intra-frame coding relies on spatial prediction to reduce or remove spatial redundancy of video within a given video frame or image. Inter-frame coding relies on temporal prediction to reduce or remove temporal redundancy of video within neighboring frames or images of a video sequence. The intra-frame coding mode (I-mode) can refer to any of several spatially based compression modes, such as one-way prediction (P-mode) or two-way prediction (B-mode). The inter-frame coding mode can refer to any of several temporally based compression modes.

[0132] exist Figure 4 In a feasible implementation, the video encoder 302 includes a segmentation unit 81, a transform unit 82, a quantization unit 83, a prediction unit 84, an entropy coding unit 85, a reference image memory 88, and a summer 802. The prediction unit 84 includes a motion estimation unit 841, a motion compensation unit 842, and an intra-frame prediction unit 843. For video block reconstruction, the video encoder 302 also includes an inverse quantization unit 86, an inverse transform unit 87, and a summer 801. In addition, the video encoder 302 may also include a deblocking filter (…). Figure 4 (Not shown in the image), this deblocking filter is used to filter block boundaries to remove block artifacts from the reconstructed video. When necessary, the deblocking filter will typically filter the output of the summer 801. In addition to the deblocking filter, the video encoder 302 may also include additional loop filters (in-loop or post-loop).

[0133] like Figure 4As shown, video encoder 302 receives video data, and segmentation unit 81 segments the data into video blocks. This segmentation may also include segmentation into slices, image blocks, or other larger units, and (for example) video block segmentation based on a quadtree structure of the Largest Coding Unit (LCU) and CUs. Exemplarily, video encoder 302 encodes components of video blocks within a video slice to be encoded. Generally, a slice may be divided into multiple video blocks (and may be divided into a set of video blocks called image blocks).

[0134] The prediction unit 84 can select one of several possible coding modes for the current video block based on the coding quality and cost calculation results (e.g., rate distortion cost (RD cost)). This could be one of several intra-frame coding modes or one of several coding-decoding modes. The prediction unit 84 can provide the obtained intra-frame coded or inter-frame coded block to the summer 802 to generate residual block data, and provide the obtained intra-frame decoded or inter-frame decoded block to the summer 801 to reconstruct the coded block for use as a reference image.

[0135] Motion estimation unit 841 and motion compensation unit 841 within prediction unit 84 perform inter-frame prediction decoding of the current video block relative to one or more prediction blocks in one or more reference images to provide temporal compression. Motion estimation unit 841 can be configured to determine the inter-frame prediction mode of a video stripe based on a predetermined mode of the video sequence. The predetermined mode can designate the video stripe in the sequence as a P-strip, B-strip, or a generalized P and B picture (GPB) stripe. Motion estimation unit 841 and motion compensation unit 842 can be highly integrated, but are described separately for conceptual purposes. Motion estimation is performed by motion estimation unit 841 to estimate the motion vector of the video block. For example, the motion vector can indicate the displacement of the PU of the video block within the current video frame or image relative to the prediction block within the reference image.

[0136] The predicted block is a block of PU that closely matches the video block to be decoded, obtained based on pixel differences. Pixel differences can be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some feasible implementations, the video encoder 302 can calculate the values ​​of sub-integer pixel positions of the reference image stored in the reference image memory 88. For example, the video encoder 302 can interpolate the values ​​of quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference image. Therefore, the motion estimation unit 841 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional pixel accuracy.

[0137] The motion estimation unit 841 calculates the motion vector of the PU in the video block of the inter-frame decoded strip by comparing the position of the PU with the position of the predicted block of the reference image. Reference images can be selected from either a first reference image list (list 0) or a second reference image list (list 1), each of which identifies one or more reference images stored in the reference image memory 88. The motion estimation unit 841 sends the calculated motion vector to the entropy coding unit 85 and the motion compensation unit 842.

[0138] Motion compensation performed by motion compensation unit 842 may involve extracting or generating prediction blocks based on motion vectors determined by motion estimation, possibly performing interpolation down to subpixel accuracy. After receiving the motion vector of the PU for the current video block, motion compensation unit 842 can locate the prediction block pointed to by the motion vector in one of the reference image lists. Video encoder 302 forms a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block, thereby forming a pixel difference. The pixel difference forms the residual data of the block and may include both luminance and chrominance difference components. Summer 801 represents one or more components performing this subtraction operation. Motion compensation unit 842 may also generate syntax elements associated with video blocks and video stripes for video decoder 312 to use for decoding video blocks of video stripes.

[0139] If the PU is located in the B strip, the image containing the PU can be associated with two reference image lists referred to as "List 0" and "List 1". In some feasible implementations, the image containing the B strip can be associated with a combination of lists that are a combination of List 0 and List 1.

[0140] Furthermore, if the PU is located in strip B, the motion estimation unit 841 can perform unidirectional or bidirectional prediction for the PU. In some feasible implementations, bidirectional prediction is a prediction based on images from reference image lists 0 and 1, respectively; in other feasible implementations, bidirectional prediction is a prediction based on reconstructed future frames and reconstructed past frames in the current frame's display order, respectively. When the motion estimation unit 841 performs unidirectional prediction for the PU, it can search for a reference block for the PU in the reference images of list 0 or list 1. The motion estimation unit 841 can then generate a reference index indicating the reference image containing the reference block in list 0 or list 1, and a motion vector indicating the spatial displacement between the PU and the reference block. The motion estimation unit 841 can output the reference index and motion vector as motion information for the PU. The motion compensation unit 842 can generate a predicted image block for the PU based on the reference block indicated by the PU's motion information.

[0141] When the motion estimation unit 841 performs bidirectional prediction for the PU, it can search for a reference block for the PU in the reference image in list 0, and can also search for another reference block for PU prediction in the reference image in list 1. The motion estimation unit 841 can generate a reference index indicating the reference images containing the reference blocks in lists 0 and 1, and a motion vector indicating the spatial displacement between the reference block and the PU. The motion estimation unit 841 can output the reference index and motion vector of the PU as motion information of the PU. The motion compensation unit 842 can generate a predicted image block of the PU based on the reference block indicated by the motion information of the PU.

[0142] In some feasible implementations, the motion estimation unit 841 does not output a complete set of motion information for the PU to the entropy coding unit 85. Instead, the motion estimation unit 841 may refer to the motion information of another PU to inform the motion information of a particular PU. For example, the motion estimation unit 841 may determine that the motion information of a PU is sufficiently similar to the motion information of its neighboring PUs. In this implementation, the motion estimation unit 841 may indicate an indication value in the syntax structure associated with the PU, which indicates to the video decoder 312 that the PU has the same motion information as its neighboring PUs or has motion information that can be derived from its neighboring PUs. In another implementation, the motion estimation unit 841 may identify candidate predicted motion vectors and motion vector differences (MVDs) associated with neighboring PUs in the syntax structure associated with the PU. The MVD indicates the difference between the motion vector of the PU and the indicated candidate predicted motion vectors associated with neighboring PUs. The video decoder 312 may use the indicated candidate predicted motion vectors and the MVD to determine the motion vector of the PU.

[0143] As described above, prediction unit 84 can generate a list of candidate predicted motion vectors for each PU of the CU. One or more of the candidate predicted motion vector lists may include one or more original candidate predicted motion vectors and one or more additional candidate predicted motion vectors derived from the original candidate predicted motion vectors.

[0144] The intra-prediction unit 843 within prediction unit 84 can perform intra-prediction decoding relative to one or more adjacent blocks in the same image or stripe as the current block to be decoded, to provide spatial compression. Therefore, as an alternative to inter-frame prediction (as described above) performed by motion estimation unit 841 and motion compensation unit 842, intra-prediction unit 843 can intra-predict the current block. Specifically, intra-prediction unit 843 can determine the intra-prediction mode to be used to encode the current block. In some feasible implementations, intra-prediction unit 843 can, for example, use various intra-prediction modes to encode the current block during individual encoding traversals, and intra-prediction unit 843 can select the appropriate intra-prediction mode to use from a test mode.

[0145] After prediction unit 84 generates a prediction block for the current video block via inter-frame prediction or intra-frame prediction, video encoder 302 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and applied to transform unit 82. Transform unit 82 transforms the residual video data into residual transform coefficients using, for example, a Discrete Cosine Transform (DCT) or a conceptually similar transform (e.g., Discrete Sine Transform (DST)). Transform unit 82 can transform the residual video data from the pixel domain to the transform domain (e.g., the frequency domain).

[0146] Transform unit 82 can send the resulting transform coefficients to quantization unit 83. Quantization unit 83 quantizes the transform coefficients to further reduce the code rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some feasible implementations, quantization unit 83 can then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, entropy coding unit 85 can perform the scan.

[0147] After quantization, entropy coding unit 85 can entropy-encode the quantized transform coefficients. For example, entropy coding unit 85 can perform Context-Adaptive Variable-Length Coding (CAVLC), Context-based Adaptive Binary Arithmetic Coding (CABAC), Syntax-based Context-Adaptive Binary Arithmetic Decoding (SBAC), Probabilistic Interval Segmentation Entropy (PIPE) decoding, or another entropy coding method or technique. Entropy coding unit 85 can also entropy-encode the motion vectors and other syntax elements of the currently decoded video strip. After entropy coding by entropy coding unit 85, the encoded bitstream can be transmitted to video decoder 312 or archived for later transmission or retrieval by video decoder 312.

[0148] Inverse quantization unit 86 and inverse transform unit 87 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for later use as a reference block in a reference image. Motion compensation unit 842 calculates the reference block by adding the residual block to a predicted block of one of the reference images in the reference image list. Motion compensation unit 842 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for motion estimation. Summer 801 adds the reconstructed residual block to the motion-compensated predicted block generated by motion compensation unit 842 to generate the reference block for storage in reference image memory 88. The reference block can be used by motion estimation unit 841 and motion compensation unit 842 as a reference block for inter-frame prediction of blocks in subsequent video frames or images.

[0149] It should be understood that other structural variations of the video encoder 302 can be used to encode video streams. For example, for certain image blocks or image frames, the video encoder 302 can directly quantize the residual signal without processing by the transform unit 82, and correspondingly, without processing by the inverse transform unit 87; or, for certain image blocks or image frames, the video encoder 302 does not generate residual data, and correspondingly, without processing by the transform unit 82, quantization unit 83, inverse quantization unit 86, and inverse transform unit 87; or, the video encoder 302 can directly store the reconstructed video block as a reference block without filter processing; or, the quantization unit 83 and inverse quantization unit 86 in the video encoder 302 can be combined together.

[0150] Figure 5 This is a schematic diagram of the structure of the video decoder 512 in an embodiment of this application. Figure 5As shown, the video decoder 512 includes an entropy decoding unit 50, a prediction unit 51, an inverse quantization unit 54, an inverse transform unit 53, a summer 55, and a reference image memory 52. ​​The prediction unit 51 includes a motion compensation unit 511 and an intra-frame prediction unit 512. In some feasible embodiments, the video decoder 512 can perform operations related to the image source... Figure 4 The video encoder 302 describes an exemplary, inverse decoding process of the encoding process.

[0151] During the decoding process, video decoder 512 receives from video encoder 302 an encoded video stream representing video blocks of encoded video strips and associated syntax elements. Entropy decoding unit 50 of video decoder 512 entropy decodes the stream to generate quantized coefficients, motion vectors, and other syntax elements. Entropy decoding unit 50 then forwards the motion vectors and other syntax elements to prediction unit 51. Video decoder 512 may receive syntax elements at the video strip level and / or the video block level.

[0152] When a video strip is decoded into an intra-decoded (I) strip, the intra-prediction unit 512 of the prediction unit 51 can generate prediction data for the video blocks of the current video strip based on the intra-prediction mode transmitted by the signal and the data from the previously decoded blocks from the current frame or image.

[0153] When a video image is decoded into inter-frame decoded (e.g., B, P, or GPB) stripes, the motion compensation unit 511 of the prediction unit 51 generates predictive blocks of video blocks for the current video image based on motion vectors and other syntax elements received from the entropy decoding unit 50. The predictive blocks may be generated from one of the reference images in a reference image list. The video decoder 512 may construct a reference image list (list 0 and list 1) based on the reference images stored in the reference image memory 52 using a default construction technique.

[0154] The motion compensation unit 511 determines the prediction information of video blocks in the current video strip by parsing motion vectors and other syntax elements, and uses the prediction information to generate predictive blocks for properly decoded current video blocks. For example, the motion compensation unit 511 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) of the video blocks used to decode the video strip, the inter-frame prediction strip type (e.g., B strip, P strip, or GPB strip), the construction information of one or more of the reference image list of the strip, the motion vector of each inter-frame coded video block of the strip, the inter-frame prediction state of each inter-frame decoded video block of the strip, and other information used to decode video blocks in the current video strip.

[0155] The motion compensation unit 511 can also perform interpolation based on an interpolation filter. The motion compensation unit 511 can use an interpolation filter, such as that used by the video encoder 302 during the encoding of a video block, to calculate the interpolated values ​​of sub-integer pixels of the reference block. In this application, the motion compensation unit 511 can determine the interpolation filter used by the video encoder 302 from the received syntax elements and use the interpolation filter to generate a predictive block.

[0156] If the PU is coded using inter-frame prediction, the motion compensation unit 511 can generate a list of candidate predicted motion vectors for the PU. The bitstream may include data identifying the position of selected candidate predicted motion vectors within the candidate predicted motion vector list of the PU. After generating the list of candidate predicted motion vectors for the PU, the motion compensation unit 511 can generate predictive image blocks for the PU based on one or more reference blocks indicated by the motion information of the PU. The reference blocks for the PU may be in a different time frame than the PU. The motion compensation unit 511 can determine the motion information of the PU based on selected motion information from the candidate predicted motion vector list of the PU.

[0157] The dequantization unit 54 dequantizes (e.g., dequantizes) the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 50. The dequantization process may involve determining the degree of quantization using quantization parameters calculated by the video encoder 302 for each video block in the video strip, and similarly determining the degree of dequantization to be applied. The inverse transform unit 53 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to produce a residual block in the pixel domain.

[0158] After the motion compensation unit 511 generates a predictive block for the current video block based on motion vectors and other syntax elements, the video decoder 512 forms a decoded video block by summing the residual block from the inverse transform unit 53 with the corresponding predictive block generated by the motion compensation unit 511. The summer 55 represents one or more components performing this summation operation. If necessary, a deblocking filter can also be applied to filter the decoded block to remove block artifacts. Other loop filters (in or after the decoding loop) can also be used to smooth pixel transitions or otherwise improve video quality. The decoded video block in a given frame or image is then stored in a reference image memory 52, which stores a reference image for subsequent motion compensation.

[0159] It should be understood that other structural variations of the video decoder 512 can be used for bitstream decoding. For example, for certain image blocks or frames, the entropy decoding unit 50 of the video decoder 512 may not decode the quantized coefficients, and therefore does not require processing by the inverse quantization unit 54 and the inverse transform unit 53. For example, the inverse quantization unit 54 and the inverse transform unit 53 in the video decoder 512 may be combined.

[0160] The aforementioned video encoder 302 and video decoder 512 can also be implemented in another way, for example, using a general-purpose digital processor system, such as... Figure 12 The encoding / decoding device 10 shown can be a part of the video encoder 302 or a part of the video decoder 312.

[0161] The encoding / decoding device 10 can be applied to either the encoding or decoding side. The encoding / decoding device 10 includes a processor 1001 and a memory 1002. The processor 1001 and the memory 1002 are connected (e.g., interconnected via a bus 1004). Optionally, the encoding / decoding device 10 may further include a transceiver 1003, which is connected to the processor 1001 and the memory 1002 for receiving / transmitting data.

[0162] The memory 1002 can be random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM). The memory 1002 is used to store relevant program code and video data.

[0163] Processor 1001 can be one or more central processing units (CPUs). If processor 1001 is a CPU, the CPU can be a single-core CPU or a multi-core CPU.

[0164] The processor 1001 is used to read the program code stored in the memory 1002 and execute it. Figures 6 to 8 The operation of any corresponding implementation scheme and its various feasible implementation methods.

[0165] The following, in conjunction with the above Figure 3 The video encoding and decoding system shown Figure 4 The video encoder 302 shown and Figure 5The video decoder 312 shown herein provides a detailed description of the encoding and decoding method provided in this application.

[0166] See Figure 6 This application provides a video decoding method, which includes the following steps:

[0167] S601, the video decoder obtains a bitstream containing image data.

[0168] Optionally, the bitstream obtained by the video decoder, which includes image data, includes syntax elements such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and slice header or slice segment header.

[0169] S602, The video decoder parses the bitstream to determine the prediction mode of the image block to be decoded and whether to use MMVD to perform motion compensation on the image block to be decoded.

[0170] As one possible implementation, the syntax elements of the bitstream include a prediction mode flag and an MMVD flag. The prediction mode flag indicates the prediction mode of the image block to be decoded. A detailed description of the prediction mode can be found above and will not be repeated here.

[0171] When the flag bit for a prediction mode of an image block to be decoded is 1, it indicates that the image block uses that prediction mode. For example, if the syntax elements of the bitstream include a flag bit for the triangular prediction unit mode, and this flag bit is 1, it means that on the encoding side, the video encoder uses the triangular prediction unit mode to encode the image block to be decoded; correspondingly, on the decoding side, the video decoder must use the triangular prediction unit mode to decode the image block to be decoded. The MMVD flag bit includes an MMVD indicator bit, which is used to indicate whether the image block to be decoded uses MMVD for motion compensation. When the MMVD indicator bit of the image block to be decoded is 1, it indicates that on the encoding side, the video encoder uses MMVD for motion compensation of the image block to be decoded; correspondingly, on the decoding side, the video decoder must also use MMVD for motion compensation of the image block to be decoded.

[0172] S603, the video decoder decodes the image block to be decoded based on the prediction mode identifier and the MMVD identifier.

[0173] The prediction mode involved in the embodiments of this application can be either the triangular prediction unit mode or the combined inter-merge / intra prediction mode. In the triangular prediction unit mode, the image block to be decoded is composed of, for example,... Figure 1 The first prediction unit (e.g., PU1) and the second prediction unit (e.g., PU2) shown are used. The video encoder performs motion estimation on the first prediction unit and the second prediction unit respectively. That is, the first prediction unit and the second prediction unit can have different motion vectors and different prediction values.

[0174] The MMVD flags also include the MMVD initial position flag, step size flag, and direction flag, which are used to indicate the offset of the motion vector relative to the predicted motion vector.

[0175] In one implementation, after receiving the bitstream, the video decoder parses the MMVD indicator bit and prediction mode flag bit contained in the bitstream. If it finds that the MMVD indicator bit of the image block to be decoded is 1, and the prediction mode flag bit of the image block to be decoded is the flag bit for the triangular prediction unit mode (TMU mode), then the prediction mode of the image block to be decoded is TMU mode, and motion compensation for the image block to be decoded needs to be performed using MMVD. In this case, see [link to relevant documentation]. Figure 7 S603 can be implemented through the following steps:

[0176] S6031a, The video decoder obtains the MVP of the first prediction unit, and / or obtains the MVP of the second prediction unit.

[0177] In the embodiments of this application, the video decoder may use either of the following two methods to obtain the MVP.

[0178] Method 1: The video decoder receives an encoded bitstream, which may include information about the MVP selected from the prediction candidate list. For example, on the encoding side, the prediction candidate list for the current image block (e.g., the current image block could be the first prediction unit) contains five candidate MVPs. The video encoder selects MVP1 as the MVP of the current image block and sends the selected MVP information to the video decoder, such as sending the index of MVP1. Thus, the video decoder can parse the bitstream and determine the MVP (e.g., MVP1) of the current image block based on the selected MVP information obtained after parsing the bitstream.

[0179] Method 2: The video decoder receives the encoded bitstream, which may include an MMVD initial position flag. The video decoder can parse the bitstream and determine the MVP of the current image block based on the MMVD initial position flag obtained from the parsed bitstream. The MMVD initial position flag can be used to indicate the selected MVP. The initial position flag can be the Base candidate IDX flag mentioned above.

[0180] S6032a, the video decoder determines the MV of the first prediction unit based on the MVP, step size identifier, and direction identifier of the first prediction unit, and / or determines the MV of the second prediction unit based on the MVP, step size identifier, and direction identifier of the second prediction unit.

[0181] For example, referring to Tables 1 to 3 above, taking the determination of the MV of the first prediction unit as an example, the video decoder parses the bitstream to obtain the step size identifier 000 and the direction identifier 00 of the first prediction unit. In S6031a, after parsing the bitstream, the initial position identifier of the first prediction unit is obtained as 00 (corresponding to the first MVP (2,2) in the prediction candidate list). Thus, the video decoder can determine that the final MV of the first prediction unit is (2+1 / 4, 2). Similarly, the process of determining the MV of the second prediction unit can be found in the relevant description of determining the MV of the first prediction unit.

[0182] It should be noted that, in this embodiment, when the video decoder determines that the prediction mode of the current image block to be decoded is the triangular prediction unit mode, and motion compensation of the image block to be decoded needs to be performed using MMVD, the above S6031a to S6032a can be performed only on the first prediction unit contained in the image block to be decoded. That is, the video decoder only uses MMVD to perform motion compensation on the first prediction unit, that is, after obtaining the MVP of the first prediction unit, the MV of the first prediction unit is determined according to the MVP, the step size flag, and the direction flag. In this case, motion compensation of the second prediction unit is not performed using MMVD, that is, the MVP of the second prediction unit can be directly used as the MV of the second prediction unit.

[0183] Of course, the video decoder may also perform S6031a to S6032a as described above only on the second prediction unit contained in the image block to be decoded. In another implementation, the video decoder may also perform S6031a to S6032a as described above on both the first prediction unit and the second prediction unit. That is, "and / or" in S6031a to S6032a as described above means that the corresponding steps are performed only on the first prediction unit, or only on the second prediction unit, or on both the first and second prediction units.

[0184] S6033a, the video decoder decodes the image block to be decoded based on the motion vector of the first prediction unit and the motion vector of the second prediction unit.

[0185] As one possible implementation, the video decoder uses the first prediction unit (e.g., Figure 1The MV of PU1 (shown) determines the prediction block corresponding to the first prediction unit in the reference frame (e.g., the frame before or after the current frame), and determines the prediction value of the first prediction unit obtained from the prediction block. The prediction block of the second prediction unit is determined based on the MV of the second prediction unit, and the prediction value of the second prediction unit obtained from the prediction block is also determined. Further, the video decoder determines the prediction value of the entire image block to be decoded based on the prediction values ​​of the first and second prediction units.

[0186] Here, the reference frame refers to other frames used to predict the first prediction unit in the current frame. The prediction block can be the image block in the reference frame that is most similar to the first prediction unit. Detailed descriptions of the reference frame and prediction block can be found above and will not be repeated here.

[0187] Then, the video decoder calculates the sum of the residual quantization value and the predicted value of the image block to be decoded based on the quantization value of the residual of the image block to be decoded transmitted by the video encoder, thereby decoding the image block to be decoded and reconstructing it.

[0188] In one implementation, after receiving the bitstream, the video decoder parses the MMVD indicator bit and prediction mode flag bit contained in the bitstream. If it finds that the MMVD indicator bit of the image block to be decoded is 0, and the prediction mode flag bit of the image block to be decoded is the flag bit of the triangular prediction unit mode, and the triangular prediction unit mode flag bit is 1, it indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, and no motion compensation is performed on the image block to be decoded using MMVD. In this case, S603 can be implemented as follows: the video decoder predicts the first prediction unit and obtains the predicted value of the first prediction unit, and predicts the second prediction unit and obtains the predicted value of the second prediction unit. Then, the video decoder decodes the image block to be decoded based on the predicted values ​​of the first and second prediction units to recover and reconstruct the image block to be decoded.

[0189] The method by which the video decoder obtains the prediction values ​​of the first prediction unit and the second prediction unit, and the method by which it decodes and reconstructs the image block to be decoded based on the prediction values ​​of the first prediction unit and the second prediction unit, can be found in the prior art process, and will not be repeated here.

[0190] In another implementation, if the MMVD indicator bit of the image block to be decoded is 1, and the prediction mode flag bit of the image block to be decoded is the flag bit of the joint inter-frame-intra prediction mode, and the flag bit of the joint inter-frame-intra prediction mode is 1, it indicates that the prediction mode of the image block to be decoded is the joint inter-frame-intra prediction mode, and motion compensation for the image block to be decoded needs to be performed using MMVD. In this case, see [link to relevant documentation]. Figure 8 S603 can be implemented through the following steps:

[0191] Among them, S6031b to S6033b are the inter-frame prediction processes for the image blocks to be decoded.

[0192] S6031b, the video decoder obtains the MVP of the image block to be decoded.

[0193] In the embodiments of this application, the video decoder may use any of the following methods to obtain the MVP of the image block to be decoded.

[0194] Method 1: Parse the bitstream encoded by the video encoder to obtain the MVP information of the image block to be decoded. This MVP information is used to indicate the MVP selected from the prediction candidate list. For example, if the video encoder selects MVP1 as the MVP of the image block to be decoded from the prediction candidate list, the video encoder sends this MVP information to the video decoder so that the video decoder can determine the MVP of the current image block to be decoded.

[0195] Method 2: Parse the bitstream encoded by the video encoder to obtain the initial MMVD position flag bit contained in the bitstream. This initial MMVD position flag bit is used to indicate the MVP selected from the prediction candidate list for the current image block to be decoded. For example, referring to Table 1 above, after the video decoder parses the bitstream, if the initial MMVD position flag bit in the bitstream is 11, then the video decoder determines that the MVP of the image block to be decoded is the 4th MVP in the prediction candidate list.

[0196] S6032b: The video decoder obtains the step size identifier and direction identifier of the image block to be decoded, and determines the MV of the image block to be decoded based on the MVP, step size identifier, and direction identifier of the image block to be decoded.

[0197] As one possible implementation, the video decoder parses the bitstream encoded by the video encoder to obtain the stride and orientation flags of the image blocks to be decoded contained in the bitstream. Then, the video decoder adds an offset to the MVP based on the MVP, stride, and orientation flags of the image block to be decoded, thus obtaining the final MV of the image block to be decoded. The detailed process of determining the MV of the image block using the MVP, stride, and orientation flags can be found in the description above and will not be repeated here.

[0198] S6033b: The video decoder determines the inter-frame prediction value of the image block to be decoded based on the MV of the image block to be decoded.

[0199] Specifically, the video decoder determines the encoded prediction block in the reference frame based on the MV of the image block to be decoded, and predicts the current image block to be decoded based on the prediction block to obtain the prediction value of the current image block to be decoded.

[0200] S6034b: The video decoder performs intra-frame prediction on the image block to be decoded to obtain the predicted value of the intra-frame prediction of the image block to be decoded.

[0201] Specifically, the video decoder uses a certain intra-frame prediction mode to predict the image block to be decoded. The predicted values ​​of the image block to be decoded include the predicted values ​​of the luminance component and the predicted values ​​of the chrominance component. The intra-frame prediction modes include, but are not limited to, vertical prediction mode, horizontal prediction mode, DC mode, and PLANA mode.

[0202] When predicting the luminance component of an image block to be decoded, the video decoder can determine the intra-prediction mode to use based on the size of the image block. Similarly, when predicting the chrominance component of an image block to be decoded, the video decoder can also determine the intra-prediction mode to use based on the size of the image block. The correspondence between image block size and intra-prediction mode can be found in the description above, and will not be repeated here.

[0203] S6035b: Decode the image block to be decoded based on the inter-frame prediction prediction value and the intra-frame prediction prediction value of the image block to be decoded.

[0204] Specifically, the video decoder weights the predicted values ​​of intra-frame prediction and inter-frame prediction to obtain the predicted value of the entire image block to be decoded. Based on the residual quantization value of the image block to be decoded sent by the video encoder, the decoder calculates the sum of the residual quantization value and the predicted value to decode and reconstruct the image block to be decoded.

[0205] As one possible implementation, if the MMVD indicator bit of the image block to be decoded is 0, and the prediction mode flag bit of the image block to be decoded is the flag bit of the joint inter-frame-intra-frame prediction mode, and the flag bit of the joint inter-frame-intra-frame prediction mode is 1, it indicates that the prediction mode of the image block to be decoded is the joint inter-frame-intra-frame prediction mode, and motion compensation for the image block to be decoded is not performed using MMVD. In this case, S603 can be specifically implemented as follows: the video decoder performs inter-frame prediction on the image block to be decoded to obtain the predicted value of the inter-frame prediction of the image block to be decoded, and performs intra-frame prediction on the image block to be decoded to obtain the predicted value of the intra-frame prediction of the image block to be decoded, and decodes and reconstructs the image block to be decoded based on the predicted value of the inter-frame prediction and the predicted value of the intra-frame prediction of the image block to be decoded.

[0206] The methods by which the video decoder obtains the inter-frame prediction and intra-frame prediction values ​​of the image block to be decoded, and the methods by which it decodes and reconstructs the image block to be decoded based on the inter-frame prediction and intra-frame prediction values, can be found in the existing technology process and will not be repeated here.

[0207] The decoding method provided in this application involves a video decoder parsing the bitstream and decoding the image block to be decoded based on the prediction mode identifier and MMVD identifier bits contained in the bitstream. The combination of the prediction mode identifier and the MMVD identifier bits indicates whether MMVD is used for motion compensation of the image block to be decoded under the current prediction mode. Compared to existing technologies that decode image blocks by combining mode segmentation and prediction modes, the decoding method in this application can utilize MMVD to improve the accuracy of the prediction value of the current image block, resulting in a higher quality decoded image.

[0208] Corresponding to the decoding side, embodiments of this application provide an encoding method applied in an encoding device, for example, in a device with video encoding capabilities. The encoding method includes: a video encoder determining the prediction mode of a current image block, determining whether to use MMVD for motion compensation of the current image block, and predicting the current image block according to the corresponding prediction mode. Further, the video encoder encodes the current image block to obtain a bitstream containing a prediction mode identifier bit and an MMVD identifier bit, and sends the bitstream to the video decoder. The prediction mode identifier bit is used to indicate the prediction mode of the image block to be decoded. This prediction mode can be a triangular prediction unit mode or a joint inter-frame-intra-frame prediction mode. When the identifier bit of the triangular prediction unit mode of the image block is 1, it indicates that the prediction mode used by the image block is the triangular prediction unit mode. When the identifier bit of the joint inter-frame-intra-frame prediction mode of the image block is 1, it indicates that the prediction mode used by the image block is the joint inter-frame-intra-frame prediction mode. The MMVD identifier bit is used to indicate whether the motion information acquisition method of the image block is MMVD, and to indicate the offset of the MVD relative to the MVP in the MMVD. Specifically, the MMVD flags include an MMVD indicator bit, which, when set to 1, indicates that the motion information of the image patch is acquired using MMVD. The MMVD flags also include an MMVD initial position flag, a step size flag, and a direction flag. These flags characterize the offset of the MVD relative to the MVP. For a detailed description of the prediction mode flags and the MMVD flags, please refer to the above text.

[0209] It should be noted that the embodiments of this application mainly introduce the encoding and decoding method when the prediction mode is the triangular prediction unit mode or the joint inter-frame-intra-frame prediction mode. It is easy to understand that the encoding and decoding method of the embodiments of this application can also be applied to other prediction modes or partitioning modes. In other prediction modes or partitioning modes, a block of images to be decoded may be divided into different prediction units, or a block of images to be decoded may use multiple prediction modes. The embodiments of this application do not limit the prediction mode to which the encoding and decoding method is applicable.

[0210] This application provides a decoding device, which can be a video decoder. Specifically, the decoding device is used to perform the steps executed by the video decoder in the above decoding method. The decoding device provided in this application may include modules corresponding to the respective steps.

[0211] This application embodiment can divide the decoding device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. The module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0212] When dividing each function into modules according to its corresponding function. Figure 13 A schematic diagram of a possible structure of the decoding device involved in the above embodiments is shown. For example... Figure 13 As shown, the decoding device 13 includes a parsing unit 130 and a decoding unit 131.

[0213] The parsing unit 130 is used to support the decoding device in performing S602 and other processes in the above embodiments, and / or other processes in the technology described herein.

[0214] The decoding unit 131 is used to support the decoding device in performing S603 and other processes in the above embodiments, and / or other processes in the technology described herein.

[0215] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0216] Of course, the decoding device provided in this application embodiment includes, but is not limited to, the above-described modules. For example, the decoding device may also include a storage unit 132.

[0217] Storage unit 132 can be used to store the program code and data of the decoding device.

[0218] When using integrated units, the structural schematic diagram of the decoding device provided in the embodiments of this application is as follows: Figure 14 As shown. In Figure 14 In this design, the decoding device 14 includes a processing module 140 and a communication module 141. The processing module 140 controls and manages the operation of the decoding device, for example, executing the steps performed by the parsing unit 130 and the decoding unit 131, and / or performing other processes described herein. The communication module 141 supports interaction between the decoding device and other devices, such as receiving bitstreams from a video encoder. Figure 14As shown, the decoding device may further include a storage module 142, which is used to store the program code and data of the decoding device, such as the contents stored in the storage unit 132.

[0219] The processing module 140 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication module 141 may be a transceiver, RF circuitry, or a communication interface, etc. The storage module 142 may be a memory.

[0220] All relevant content in each scenario involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0221] Both the decoding device 13 and the decoding device 14 described above can perform the above-described functions. Figure 6 or Figure 7 or Figure 8 The decoding method shown can be implemented using a video decoding device or other device with video encoding and decoding capabilities. Specifically, decoding device 13 and decoding device 14 can be video decoding devices or other devices with video encoding and decoding capabilities.

[0222] This application also provides a decoding device, which includes one or more processors, a memory, and a communication interface. The memory and the communication interface are coupled to one or more processors; the memory is used to store computer program code, which includes instructions. When one or more processors execute the instructions, the decoding device executes the decoding method of the embodiments of this application.

[0223] The decoding device here can be a video display device, a smartphone, a laptop, or other device that can process or play video.

[0224] This application also provides a video decoder, including a non-volatile storage medium and a central processing unit (CPU). The non-volatile storage medium stores an executable program, and the CPU is connected to the non-volatile storage medium and executes the executable program to implement the decoding method of the embodiments of this application.

[0225] This application also provides a decoder, which includes the decoding devices (decoding device 13 and decoding device 14) in the embodiments of this application and a reconstruction module, wherein the reconstruction module is used to determine the reconstructed pixel value of the current image block based on the predicted image pixels obtained by the decoding device.

[0226] Another embodiment of this application provides a computer-readable storage medium including one or more program codes, the one or more programs including instructions, wherein when a processor in a decoding device executes the program code, the decoding device performs, for example... Figure 6 or Figure 7 or Figure 8 The decoding method shown.

[0227] In another embodiment of this application, a computer program product is also provided, comprising computer-executable instructions stored in a computer-readable storage medium. At least one processor of a decoding device can read the computer-executable instructions from the computer-readable storage medium, and the at least one processor executes the computer-executable instructions to cause a terminal to perform execution. Figure 6 or Figure 7 or Figure 8 The steps of the video decoder in the decoding method shown.

[0228] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, it can appear, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated.

[0229] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. This available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0230] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0231] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0232] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0233] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0234] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of this application embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0235] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video decoding method, characterized in that, include: The parsed bitstream includes a merged motion vector difference (MMVD) flag, which indicates the method of acquiring motion information of the image block to be decoded; the MMVD flag includes an MMVD indicator, which indicates whether the method of acquiring motion information of the image block to be decoded is MMVD; the MMVD flag also includes a step size flag and a direction flag. The bitstream also includes a prediction mode identifier bit, which is used to indicate the prediction mode of the image block to be decoded, and the prediction mode includes a triangular prediction unit mode. When the MMVD indicator bit is 1, and the prediction mode flag bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, The image block to be decoded is composed of a first prediction unit and a second prediction unit; Obtain the predicted motion vector MVP of the first prediction unit, and / or obtain the MVP of the second prediction unit; The motion vector MV of the first prediction unit is determined based on the MVP, step size identifier, and direction identifier of the first prediction unit, and / or the MV of the second prediction unit is determined based on the MVP, step size identifier, and direction identifier of the second prediction unit. The image block to be decoded is decoded based on the MV of the first prediction unit and the MV of the second prediction unit; When the MMVD indicator bit is 0, and the prediction mode identifier bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, The image block to be decoded is composed of a first prediction unit and a second prediction unit; Obtain the MV and reference frame of the first prediction unit, and the MV and reference frame of the second prediction unit; The prediction block corresponding to the first prediction unit in the reference frame is determined based on the MV of the first prediction unit, and the prediction value of the first prediction unit is determined by the prediction block. The prediction block corresponding to the second prediction unit in the reference frame is determined based on the MV of the second prediction unit, and the prediction value of the second prediction unit predicted by the prediction block is determined. The predicted values ​​of the first prediction unit and the second prediction unit are weighted using a preset weighting parameter group to obtain the predicted value of the image block to be decoded. The image block to be decoded is decoded based on the predicted value of the image block to be decoded.

2. The method according to claim 1, characterized in that, The MMVD indicator bit being 1 indicates that the motion information acquisition method for the image block to be decoded is MMVD.

3. The method according to claim 1 or 2, characterized in that, The MMVD identifier also includes an MMVD initial position identifier, and the method further includes: obtaining the MVP of the image block to be decoded based on the MMVD initial position identifier of the image block to be decoded.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The bitstream is parsed to obtain the MVP information of the image block to be decoded; The MVP of the image block to be decoded is obtained based on the MVP information of the image block to be decoded.

5. The method according to claim 1, characterized in that, The prediction mode also includes a joint inter-frame-intra-frame prediction mode.

6. A video decoding device, characterized in that, include: The memory is used to store computer-executable instructions; A processor, the processor being configured to execute computer-executable instructions to perform the following steps: The parsed bitstream includes a merged motion vector difference (MMVD) flag, which indicates the method of acquiring motion information of the image block to be decoded; the MMVD flag includes an MMVD indicator, which indicates whether the method of acquiring motion information of the image block to be decoded is MMVD; the MMVD flag also includes a step size flag and a direction flag. The bitstream also includes a prediction mode identifier bit, which is used to indicate the prediction mode of the image block to be decoded, and the prediction mode includes a triangular prediction unit mode. When the MMVD indicator bit is 1, and the prediction mode flag bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, The image block to be decoded is composed of a first prediction unit and a second prediction unit; Obtain the predicted motion vector MVP of the first prediction unit, and / or obtain the MVP of the second prediction unit; The motion vector MV of the first prediction unit is determined based on the MVP, step size identifier, and direction identifier of the first prediction unit, and / or the MV of the second prediction unit is determined based on the MVP, step size identifier, and direction identifier of the second prediction unit. The image block to be decoded is decoded based on the MV of the first prediction unit and the MV of the second prediction unit; When the MMVD indicator bit is 0, and the prediction mode identifier bit indicates that the prediction mode of the image block to be decoded is the triangular prediction unit mode, The image block to be decoded is composed of a first prediction unit and a second prediction unit; Obtain the MV and reference frame of the first prediction unit, and the MV and reference frame of the second prediction unit; The prediction block corresponding to the first prediction unit in the reference frame is determined based on the MV of the first prediction unit, and the prediction value of the first prediction unit is determined by the prediction block. The prediction block corresponding to the second prediction unit in the reference frame is determined based on the MV of the second prediction unit, and the prediction value of the second prediction unit predicted by the prediction block is determined. The predicted values ​​of the first prediction unit and the second prediction unit are weighted using a preset weighting parameter group to obtain the predicted value of the image block to be decoded. The image block to be decoded is decoded based on the predicted value of the image block to be decoded.

7. The video decoding apparatus according to claim 6, characterized in that, The MMVD indicator bit being 1 indicates that the motion information acquisition method for the image block to be decoded is MMVD.

8. The video decoding apparatus according to claim 6 or 7, characterized in that, The MMVD identifier also includes an MMVD initial position identifier, and the step further includes: obtaining the MVP of the image block to be decoded based on the MMVD initial position identifier of the image block to be decoded.

9. The video decoding apparatus according to claim 6 or 7, characterized in that, The steps also include: The bitstream is parsed to obtain the MVP information of the image block to be decoded; The MVP of the image block to be decoded is obtained based on the MVP information of the image block to be decoded.

10. The video decoding apparatus according to claim 6, characterized in that, The prediction mode also includes a joint inter-frame-intra-frame prediction mode.

11. A video encoding method, characterized in that, include: The motion vector MV of the image block to be encoded is determined based on the predicted motion vector MVP, step size identifier, and direction identifier of the image block to be encoded. The bitstream includes a merged motion vector difference (MMVD) flag, which indicates the method of acquiring motion information of the image block to be encoded; the MMVD flag includes an MMVD indicator, which indicates whether the method of acquiring motion information of the image block to be encoded is MMVD; the MMVD flag also includes a step size flag and a direction flag; the bitstream also includes a prediction mode flag, which indicates the prediction mode of the image block to be encoded, and the prediction mode includes a triangular prediction unit mode; When the MMVD indicator bit is 1, and the prediction mode identifier bit indicates that the prediction mode of the image block to be encoded is the triangular prediction unit mode, The image block to be encoded is composed of a first prediction unit and a second prediction unit; Identify the MVP and motion vector difference (MVD) associated with the neighboring prediction units of the first prediction unit, and identify the MVP and MVD associated with the neighboring prediction units of the second prediction unit; The motion vector MV of the first prediction unit is determined based on the MVP and MVD associated with the neighboring prediction units of the first prediction unit, and the MV of the second prediction unit is determined based on the MVP and MVD associated with the neighboring prediction units of the second prediction unit. Obtain the reference frame of the first prediction unit and the reference frame of the second prediction unit; The prediction block corresponding to the first prediction unit in the reference frame is determined based on the MV of the first prediction unit, and the prediction value of the first prediction unit is determined by the prediction block. The prediction block corresponding to the second prediction unit in the reference frame is determined based on the MV of the second prediction unit, and the prediction value of the second prediction unit predicted by the prediction block is determined. The predicted values ​​of the first prediction unit and the second prediction unit are weighted using a preset weighting parameter group to obtain the predicted value of the image block to be encoded. Based on the predicted value of the image block to be encoded, the image block to be encoded is transformed and quantized to obtain a bitstream; When the MMVD indicator bit is 0, and the prediction mode identifier bit indicates that the prediction mode of the image block to be encoded is the triangular prediction unit mode, The image block to be encoded is composed of a first prediction unit and a second prediction unit; Obtain the MV and reference frame of the first prediction unit, and the MV and reference frame of the second prediction unit; The prediction block corresponding to the first prediction unit in the reference frame is determined based on the MV of the first prediction unit, and the prediction value of the first prediction unit is determined by the prediction block. The prediction block corresponding to the second prediction unit in the reference frame is determined based on the MV of the second prediction unit, and the prediction value of the second prediction unit predicted by the prediction block is determined. The predicted values ​​of the first prediction unit and the second prediction unit are weighted using a preset weighting parameter group to obtain the predicted value of the image block to be encoded. Based on the predicted value of the image block to be encoded, the image block to be encoded is transformed and quantized to obtain a bitstream.

12. The method according to claim 11, characterized in that, The MMVD indicator bit being 1 indicates that the motion information acquisition method for the image block to be encoded is MMVD.

13. The method according to claim 11 or 12, characterized in that, The MMVD identifier also includes an MMVD initial position identifier, and the method further includes: obtaining the MVP of the image block to be encoded based on the MMVD initial position identifier of the image block to be encoded.

14. The method according to claim 11 or 12, characterized in that, The method further includes: The MVP of the image block to be encoded is obtained based on the MVP information of the image block to be encoded; The MVP information of the image block to be encoded is encoded into the bitstream.

15. The method according to claim 11, characterized in that, The prediction mode also includes a joint inter-frame-intra-frame prediction mode.

16. A video encoding apparatus, characterized in that, include: The memory is used to store computer-executable instructions; A processor, the processor being configured to execute the computer-executable instructions to implement the video encoding method as described in any one of claims 11-15.

17. A computer-readable storage medium, characterized in that, Includes a computer program or instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-5 or 11-15.

18. A storage device, characterized in that, The storage device includes a storage unit and a transmission unit; the storage unit is used to store encoded video data, and the transmission unit is used to transmit the encoded video data; the encoded video data includes a bitstream generated by the encoding method according to any one of claims 11-15, or includes a bitstream processed by the decoding method according to any one of claims 1-5.

19. The storage device according to claim 18, characterized in that, The storage device is a file server.

Images