Video image prediction method and device

Abstract

The invention provides a video image prediction method and device, and provides a method for determining the maximum length of a candidate motion vector list of a sub-block fusion mode. The method comprises the following steps: analyzing a first identifier from a code stream; when the first identifier indicates that the candidate mode adopted by the to-be-processed block for inter-frame prediction comprises an affine mode, analyzing a second identifier from the code stream, the second identifier being used for indicating the maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the to-be-processed block adopts a sub-block fusion prediction mode; and determining the maximum length of the first candidate motion vector list according to the second identifier.

Description

[0001] This application is a divisional application. The application number of the original application is 201811642717.0, and the original application date is December 29, 2018. The entire contents of the original application are incorporated into this application by reference.

[0002] This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on October 29, 2018, with application number 201811268188.2 and invention name “A method and device for inter-frame prediction”, the entire contents of which are incorporated herein by reference; Technical Field

[0003] The present application relates to the field of image coding and decoding technology, and in particular to a video image prediction method and device. Background Art

[0004] With the development of information technology, video services such as high-definition television, online conferencing, IPTV, and 3D television have developed rapidly. Video signals have become the main way for people to obtain information in their daily lives due to their intuitiveness and efficiency. Since video signals contain a large amount of data, they require a large amount of transmission bandwidth and storage space. In order to effectively transmit and store video signals, video signals need to be compressed and encoded. Video compression technology is becoming an indispensable key technology in the field of video applications.

[0005] The basic principle of video coding compression is to use the correlation between the spatial domain, the temporal domain and the codewords to remove redundancy as much as possible. The current popular approach is to use a hybrid video coding framework based on image blocks to achieve video coding compression through prediction (including intra-frame prediction and inter-frame prediction), transformation, quantization, entropy coding and other steps.

[0006] In various video encoding / decoding schemes, motion estimation / motion compensation in inter-frame prediction is a key technology that affects encoding / decoding performance. Existing inter-frame prediction uses motion compensation (MC) prediction based on blocks according to a translational motion model, and adds sub-block fusion motion vector prediction. There is currently no feasible way to determine the maximum length of the candidate motion vector list of the sub-block fusion mode. Summary of the invention

[0007] The present application provides a video image prediction method and device, and provides a maximum length of a candidate motion vector list for determining a sub-block fusion mode.

[0008] In a first aspect, an embodiment of the present application provides a video image prediction method, comprising:

[0009] A first identifier (for example, sps_affine_enable_flag) is parsed from a bitstream; when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, a second identifier (for example, five_minus_max_num_subblock_merge_cand or six_minus_max_num_subblock_merge_cand) is parsed from the bitstream, wherein the second identifier is used to indicate a maximum length of a first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode; the maximum length of the first candidate motion vector list is determined according to the second identifier.

[0010] The above method provides a way to set the maximum length of a candidate motion vector list for a sub-block merging mode, which is simple and easy to implement.

[0011] In one possible design, before determining the maximum length of the first candidate motion vector list according to the second identifier, it also includes: parsing a third identifier (for example: sps_sbtmvp_enabled_flag) from the code stream, wherein the third identifier is used to indicate the existence status of the advanced time domain motion vector prediction mode in the sub-block fusion prediction mode.

[0012] In one possible design, the sub-block fusion prediction mode is composed of at least one of a plane motion vector prediction mode, the advanced time domain motion vector prediction mode and the affine mode. When the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, determining the maximum length of the first candidate motion vector list according to the second identifier includes: determining a first quantity value according to the third identifier; determining the maximum length of the first candidate motion vector list according to the second identifier and the first quantity value.

[0013] For example, when sps_sbtmvp_enabled_flag=0, it indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode. Exemplarily, the first quantity value is equal to the number of motion vectors supported for prediction by the advanced temporal motion vector prediction mode. sps_sbtmvp_enabled_flag=0, the first quantity value is equal to the number of motion vectors supported for prediction by the advanced temporal motion vector prediction mode.

[0014] In a possible design, before determining the maximum length of the first candidate motion vector list according to the second identifier, it also includes: parsing a fourth identifier (for example: sps_planar_enabled_flag) from the code stream, and the fourth identifier is used to indicate the existence status of the planar motion vector prediction mode in the sub-block fusion prediction mode.

[0015] In one possible design, when the third identifier indicates that the advanced temporal motion vector prediction mode exists in the sub-block fusion prediction mode and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, determining the maximum length of the first candidate motion vector list according to the second identifier includes: determining a second quantity value based on the fourth identifier; determining the maximum length of the first candidate motion vector list according to the second identifier and the second quantity value.

[0016] For example, when sps_planar_enabled_flag=0, it indicates that the planar motion vector prediction mode does not exist in the sub-block merging prediction mode. Exemplarily, the second quantity value is equal to the quantity of motion vectors supported for prediction by the planar motion vector prediction mode.

[0017] In one possible design, when the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode and the fourth identifier indicates that the plane motion vector prediction mode exists in the sub-block fusion prediction mode, determining the maximum length of the first candidate motion vector list according to the second identifier includes: determining the maximum length of the first candidate motion vector list according to the second identifier and the first quantity value.

[0018] In one possible design, when the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, determining the maximum length of the first candidate motion vector list according to the second identifier includes: determining the maximum length of the first candidate motion vector list according to the second identifier, the first quantity value and the second quantity value.

[0019] In one possible design, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0020] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand;

[0021] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second flag, and K is a preset non-negative integer.

[0022] In a possible design, when the maximum length of the first candidate motion vector list is determined according to the second identifier and the first quantity value, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0023] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1;

[0024] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L1 represents the first quantity value, and K is a preset non-negative integer.

[0025] In a possible design, when the maximum length of the first candidate motion vector list is determined according to the second identifier and the second quantity value, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0026] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L2;

[0027] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L2 represents the second quantity value, and K is a preset non-negative integer.

[0028] In a possible design, when the maximum length of the first candidate motion vector list is determined according to the second identifier and the first quantity value and the second quantity value, the maximum length of the first candidate motion vector list is obtained according to the following formula: MaxNumSubblockMergeCand = K-K_minus_max_num_subblock_merge_cand-L1-L2; wherein MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L1 represents the first quantity value, L2 represents the second quantity value, and K is a preset non-negative integer.

[0029] In a possible design, parsing the second identifier from the code stream includes:

[0030] The second identifier is parsed from a sequence parameter set of the code stream, or the second identifier is parsed from a slice header of a slice where the to-be-processed block of the code stream is located.

[0031] In one possible design, it also includes:

[0032] When the first identifier indicates that the candidate mode used for inter-frame prediction of the block to be processed only includes the translational motion vector prediction mode, and the third identifier (e.g., sps_sbtmvp_enabled_flag) indicates that the advanced temporal motion vector prediction mode exists in the sub-block fusion prediction mode, a third quantity value is determined according to the third identifier, and the maximum length of the first candidate motion vector list is determined according to the third quantity value. For example, sps_sbtmvp_enabled_flag = 1, indicating that the advanced temporal motion vector prediction mode exists in the sub-block fusion prediction mode, and the third quantity value is equal to the number of motion vectors supported for prediction by the advanced temporal motion vector prediction mode. Exemplarily, the maximum length of the first candidate motion vector list is equal to the third quantity value.

[0033] In a possible design, when the fourth flag (e.g., sps_planar_enabled_flag) indicates that the planar motion vector prediction mode exists in the sub-block fusion prediction mode, determining the maximum length of the first candidate motion vector list according to the first quantity value includes: determining a fourth quantity value according to the fourth flag; and determining the maximum length of the first candidate motion vector list according to the third quantity value and the fourth quantity value. Exemplarily, the maximum length of the first candidate motion vector list is equal to the sum of the third quantity value and the fourth quantity value.

[0034] For example: sps_planar_enabled_flag=1, to indicate that the planar motion vector prediction mode exists in the sub-block fusion prediction mode, and the fourth quantity value is equal to the number of motion vectors supported for prediction by the planar motion vector list.

[0035] In a possible design, when the first identifier indicates that the candidate mode used for inter-frame prediction of the block to be processed only includes the translational motion vector prediction mode, the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode exists in the sub-block fusion prediction mode, a fourth quantity value is determined according to the fourth identifier, and the maximum length of the first candidate motion vector list is determined according to the fourth quantity value. Exemplarily, the maximum length of the first candidate motion vector list is equal to the fourth quantity value.

[0036] In one possible design, when the first identifier indicates that the candidate modes used for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, and the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode, the maximum length of the first candidate motion vector list is zero.

[0037] In one possible design, when the first identifier indicates that the candidate modes adopted for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, the maximum length of the first candidate motion vector list is zero.

[0038] In a possible design, the third identifier is a first numerical value, and the first numerical value is 1.

[0039] In one possible design, the fourth identifier is a third numerical value, and the second quantity value is 1.

[0040] In a possible design, the third identifier is a second numerical value, and the third quantity value is 1.

[0041] In a possible design, the fourth identifier is a fourth numerical value, and the fourth quantity value is 1.

[0042] In a second aspect, an embodiment of the present application provides a video image prediction device, comprising:

[0043] a parsing unit, configured to parse a first identifier from a bitstream; and when the first identifier indicates that the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode, parse a second identifier from the bitstream, wherein the second identifier is used to indicate a maximum length of a first candidate motion vector list, wherein the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode;

[0044] A determining unit is used to determine the maximum length of the first candidate motion vector list according to the second identifier.

[0045] In one possible design, the parsing unit is further used to parse a third identifier from the code stream before determining the maximum length of the first candidate motion vector list based on the second identifier, and the third identifier is used to indicate the existence status of the advanced time domain motion vector prediction mode in the sub-block fusion prediction mode.

[0046] In a possible design, the sub-block fusion prediction mode is composed of at least one of a plane motion vector prediction mode, the advanced temporal motion vector prediction mode, and the affine mode, and when the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode, the determining unit is specifically configured to:

[0047] Determine a first quantity value according to the third identifier;

[0048] The maximum length of the first candidate motion vector list is determined according to the second identifier and the first quantity value.

[0049] In a possible design, the parsing unit, before determining the maximum length of the first candidate motion vector list according to the second identifier, is further configured to:

[0050] A fourth identifier is parsed from the bitstream, where the fourth identifier is used to indicate a presence status of the plane motion vector prediction mode in the sub-block fusion prediction mode.

[0051] In a possible design, when the third identifier indicates that the advanced temporal motion vector prediction mode exists in the sub-block fusion prediction mode and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, the determining unit is specifically used to:

[0052] Determine a second quantity value based on the fourth identifier;

[0053] The maximum length of the first candidate motion vector list is determined according to the second identifier and the second quantity value.

[0054] In a possible design, when the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode and the fourth identifier indicates that the plane motion vector prediction mode exists in the sub-block fusion prediction mode, the determining unit is specifically used to:

[0055] The maximum length of the first candidate motion vector list is determined according to the second identifier and the first quantity value.

[0056] In a possible design, when the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, the determining unit is specifically configured to:

[0057] The maximum length of the first candidate motion vector list is determined according to the second identifier, the first quantity value and the second quantity value.

[0058] In a possible design, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0059] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand;

[0060] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second flag, and K is a preset non-negative integer.

[0061] In a possible design, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0062] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1;

[0063] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L1 represents the first quantity value, and K is a preset non-negative integer.

[0064] In a possible design, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0065] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L2;

[0066] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L2 represents the second quantity value, and K is a preset non-negative integer.

[0067] In a possible design, the maximum length of the first candidate motion vector list is obtained according to the following formula:

[0068] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1-L2;

[0069] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L1 represents the first quantity value, L2 represents the second quantity value, and K is a preset non-negative integer.

[0070] In a possible design, the parsing unit, when parsing the second identifier from the code stream, is specifically configured to:

[0071] The second identifier is parsed from a sequence parameter set of the code stream, or the second identifier is parsed from a slice header of a slice where the to-be-processed block of the code stream is located.

[0072] In a possible design, the determining unit is further configured to:

[0073] When the first identifier indicates that the candidate modes adopted for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, and the third identifier indicates that the advanced time domain motion vector prediction mode exists in the sub-block fusion prediction mode, a third quantity value is determined according to the third identifier, and the maximum length of the first candidate motion vector list is determined according to the third quantity value.

[0074] In a possible design, when the fourth identifier indicates that the plane motion vector prediction mode exists in the sub-block fusion prediction mode, the determining unit determines the maximum length of the first candidate motion vector list according to the first quantity value, specifically for:

[0075] determining a fourth quantity value according to the fourth identifier;

[0076] The maximum length of the first candidate motion vector list is determined according to the first quantity value and the fourth quantity value.

[0077] In one possible design, the determination unit is also used to determine a fourth quantity value according to the fourth identifier, and determine the maximum length of the first candidate motion vector list according to the fourth quantity value, when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed only includes the translational motion vector prediction mode, the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode exists in the sub-block fusion prediction mode.

[0078] In one possible design, when the first identifier indicates that the candidate modes used for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, and the third identifier indicates that the advanced temporal motion vector prediction mode does not exist in the sub-block fusion prediction mode, the maximum length of the first candidate motion vector list is zero.

[0079] In one possible design, when the first identifier indicates that the candidate modes adopted for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, the maximum length of the first candidate motion vector list is zero.

[0080] In one possible design, the maximum length of the first candidate motion vector list is equal to the third quantity value.

[0081] In one possible design, a maximum length of the first candidate motion vector list is equal to the sum of the third quantity value and the fourth quantity value.

[0082] In one possible design, a maximum length of the first candidate motion vector list is equal to a fourth quantity value.

[0083] In a possible design, the third identifier is a first numerical value, and the first numerical value is 1.

[0084] In one possible design, the fourth identifier is a third numerical value, and the second quantity value is 1.

[0085] In a possible design, the third identifier is a second numerical value, and the third quantity value is 1.

[0086] In a possible design, the fourth identifier is a fourth numerical value, and the fourth quantity value is 1.

[0087] In a third aspect, an embodiment of the present application provides a device, which may be a decoder, comprising: a processor and a memory; the memory is used to store instructions, and when the device is running, the processor executes the instructions stored in the memory, so that the device executes the method provided by the first aspect or any design of the first aspect. It should be noted that the memory may be integrated into the processor or may be independent of the processor.

[0088] In a fourth aspect, an embodiment of the present application provides a video image prediction method, which is applied to the encoding side, including:

[0089] Encoding a first identifier into a bitstream;

[0090] When the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, a second identifier is encoded into the code stream, and the second identifier is used to indicate the maximum length of the first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode.

[0091] In a fifth aspect, an embodiment of the present application provides a video image prediction device, which may be an encoder, comprising: a processor and a memory; the memory is used to store instructions, and when the device is running, the processor executes the instructions stored in the memory, so that the device executes the method provided in the fourth aspect. It should be noted that the memory may be integrated into the processor or may be independent of the processor.

[0092] A sixth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the methods described in the above aspects.

[0093] A seventh aspect of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the methods described in the above aspects.

[0094] It should be understood that the second to seventh aspects of the present application are consistent with the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding implementable design methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0096] Figure 1Ais a block diagram of an example of a video encoding and decoding system 10 for implementing an embodiment of the present application;

[0097] Figure 1B is a block diagram of an example of a video decoding system 40 for implementing an embodiment of the present application;

[0098] Figure 2 is a block diagram of an example structure of an encoder 20 for implementing an embodiment of the present application;

[0099] Figure 3 is a block diagram of an example structure of a decoder 30 for implementing an embodiment of the present application;

[0100] Figure 4 is a block diagram of an example of a video decoding device 400 for implementing an embodiment of the present application;

[0101] Figure 5 is a block diagram of another encoding device or decoding device example for implementing the embodiments of the present application;

[0102] Fig. 6A is a schematic diagram of candidate positions of motion information for implementing an embodiment of the present application;

[0103] Figure 6B is a schematic diagram of control point motion vector prediction for implementing inheritance in an embodiment of the present application;

[0104] Figure 6C is a schematic diagram of control point motion vector prediction for implementing the structure of the embodiment of the present application;

[0105] Fig.6D It is a schematic diagram of a flow chart for combining motion information of control points to obtain constructed motion information of control points for implementing an embodiment of the present application;

[0106] Fig. 6E is a schematic diagram of an ATMVP prediction method for implementing an embodiment of the present application;

[0107] Figure 7 is a schematic diagram of a planar motion vector prediction method for implementing an embodiment of the present application;

[0108] Fig. 8A is a flow chart of an inter-frame prediction method for implementing an embodiment of the present application;

[0109] Figure 8B It is a schematic diagram of constructing a candidate motion vector list for implementing an embodiment of the present application;

[0110] Figure 8C is a schematic diagram of a motion compensation unit for implementing an embodiment of the present application;

[0111] Fig. 9It is a schematic flow chart of a video image prediction method for implementing an embodiment of the present application;

[0112] Fig.10 is a flowchart of another video image prediction method for implementing an embodiment of the present application;

[0113] Fig.11 It is a flowchart of another video image prediction method for implementing the embodiment of the present application;

[0114] Fig.12 It is a flowchart of another video image prediction method for implementing the embodiment of the present application;

[0115] Fig.13 is a schematic diagram of a device 1300 for implementing an embodiment of the present application;

[0116] Fig.14 is a schematic diagram of a device 1400 for implementing an embodiment of the present application;

[0117] Fig.15 It is a schematic diagram of an apparatus 1500 for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0118] The following is a description of the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. In the following description, reference is made to the drawings that form a part of the present disclosure and show the specific aspects of the embodiments of the present application or the specific aspects of the embodiments of the present application in an illustrative manner. It should be understood that the embodiments of the present application can be used in other aspects and may include structural or logical changes not depicted in the drawings. Therefore, the following detailed description should not be understood in a restrictive sense, and the scope of the present application is defined by the appended claims. For example, it should be understood that the disclosure of the described method can be equally applicable to the corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units such as functional units to perform the one or more method steps described (for example, one unit performs one or more steps, or multiple units, each of which performs one or more of the multiple steps), even if such one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific device is described based on one or more units such as functional units, the corresponding method may include a step to perform the functionality of the one or more units (e.g., a step to perform the functionality of the one or more units, or multiple steps, each of which performs the functionality of one or more of the multiple units), even if such one or more steps are not explicitly described or illustrated in the drawings. Further, it should be understood that, unless otherwise explicitly stated, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0119] The technical solutions involved in the embodiments of the present application may not only be applied to existing video coding standards (such as H.264, HEVC and other standards), but may also be applied to future video coding standards (such as H.266 standards). The terms used in the implementation method section of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in the embodiments of this application.

[0120] Video coding generally refers to processing a sequence of pictures that form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" can be used as synonyms. Video coding used in this article means video encoding or video decoding. Video encoding is performed on the source side and generally includes processing (e.g., by compression) the original video picture to reduce the amount of data required to represent the video picture, thereby more efficiently storing and / or transmitting. Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video picture. The "encoding" of video pictures involved in the embodiments should be understood as involving "encoding" or "decoding" of video sequences. The combination of the encoding part and the decoding part is also referred to as codec (encoding and decoding).

[0121] A video sequence includes a series of pictures, which are further divided into slices, and slices are further divided into blocks. Video coding is performed in blocks. In some new video coding standards, the concept of blocks is further expanded. For example, in the H.264 standard, there are macroblocks (MB), which can be further divided into multiple prediction blocks (partitions) that can be used for predictive coding. In the high-performance video coding (HEVC) standard, basic concepts such as coding units (CU), prediction units (PU) and transform units (TU) are used to functionally divide various block units and describe them using a new tree-based structure. For example, CU can be divided into smaller CUs according to a quadtree, and smaller CUs can be further divided to form a quadtree structure. CU is the basic unit for dividing and encoding coded images. There are similar tree structures for PU and TU. PU can correspond to prediction blocks and is the basic unit of predictive coding. CU is further divided into multiple PUs according to the division mode. TU may correspond to a transform block, which is a basic unit for transforming a prediction residual. However, no matter CU, PU or TU, they all belong to the concept of a block (or image block) in essence.

[0122] For example, in HEVC, a CTU is split into multiple CUs using a quadtree structure represented as a coding tree. A decision is made at the CU level whether to use inter-picture (temporal) or intra-picture (spatial) prediction to encode a picture area. Each CU can be further split into one, two or four PUs according to the PU split type. The same prediction process is applied within a PU, and relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU split type, the CU can be divided into transform units (TUs) according to other quadtree structures similar to the coding tree for the CU. In the latest developments in video compression technology, quadtree and binarytree (QTBT) are used to split frames to split coding blocks. In the QTBT block structure, the CU can be square or rectangular in shape.

[0123] In this article, for the convenience of description and understanding, the image block to be encoded in the current encoded image may be referred to as the current block. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded. The decoded image block in the reference image used to predict the current block is referred to as a reference block, that is, the reference block is a block that provides a reference signal for the current block, wherein the reference signal represents the pixel value in the image block. The block in the reference image that provides a prediction signal for the current block may be referred to as a prediction block, wherein the prediction signal represents the pixel value or sampling value or sampling signal in the prediction block. For example, after traversing multiple reference blocks, the best reference block is found. This best reference block will provide a prediction for the current block, and this block is referred to as a prediction block.

[0124] In the case of lossless video coding, the original video picture can be reconstructed, that is, the reconstructed video picture has the same quality as the original video picture (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed by, for example, quantization to reduce the amount of data required to represent the video picture, but the decoder side cannot fully reconstruct the video picture, that is, the quality of the reconstructed video picture is lower or worse than the quality of the original video picture.

[0125] Several video coding standards of H.261 belong to the category of "lossy hybrid video codecs" (i.e., combining spatial and temporal prediction in the sample domain with 2D transform coding in the transform domain for applying quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, which are usually encoded at the block level. In other words, the encoder side usually processes, i.e., encodes, the video at the block (video block) level, for example, by generating a prediction block through spatial (intra-picture) prediction and temporal (inter-picture) prediction, subtracting the prediction block from the current block (currently processed or to be processed) to obtain a residual block, transforming the residual block in the transform domain and quantizing the residual block to reduce the amount of data to be transmitted (compressed), while the decoder side applies the inverse processing part relative to the encoder to the coded or compressed block to reconstruct the current block for representation. In addition, the encoder replicates the decoder processing cycle, so that the encoder and decoder generate the same prediction (e.g., intra-frame prediction and inter-frame prediction) and / or reconstruction for processing, i.e., encoding subsequent blocks.

[0126] The following describes the system architecture used in the embodiments of the present application. Figure 1A , Figure 1A A schematic block diagram of a video encoding and decoding system 10 used in an embodiment of the present application is given as an example. Figure 1A As shown, the video encoding and decoding system 10 may include a source device 12 and a destination device 14, the source device 12 generating encoded video data, and therefore, the source device 12 may be referred to as a video encoding device. The destination device 14 may decode the encoded video data generated by the source device 12, and therefore, the destination device 14 may be referred to as a video decoding device. Various implementations of the source device 12, the destination device 14, or both may include one or more processors and a memory coupled to the one or more processors. The memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other media that can be used to store the desired program code in the form of instructions or data structures that can be accessed by a computer, as described herein. The source device 12 and the destination device 14 may include various devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, or the like.

[0127] Although Figure 1AThe source device 12 and the destination device 14 are depicted as separate devices, but device embodiments may also include both the source device 12 and the destination device 14 or the functionality of both, i.e., the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality. In such embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0128] The source device 12 and the destination device 14 may be communicatively connected via a link 13, and the destination device 14 may receive the encoded video data from the source device 12 via the link 13. The link 13 may include one or more media or devices capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the link 13 may include one or more communication media that enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. In this example, the source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol), and may transmit the modulated video data to the destination device 14. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 12 to the destination device 14.

[0129] The source device 12 includes an encoder 20. Optionally, the source device 12 may also include a picture source 16, a picture preprocessor 18, and a communication interface 22. In a specific implementation, the encoder 20, the picture source 16, the picture preprocessor 18, and the communication interface 22 may be hardware components in the source device 12, or may be software programs in the source device 12. They are described as follows:

[0130] The picture source 16 may include or may be any type of picture capture device, for example, for capturing real-world pictures, and / or any type of pictures or comments (for screen content encoding, some text on the screen is also considered to be part of the picture or image to be encoded) generating device, for example, a computer graphics processor for generating computer animation pictures, or any type of device for acquiring and / or providing real-world pictures, computer animation pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). The picture source 16 may be a camera for capturing pictures or a memory for storing pictures, and the picture source 16 may also include any type of (internal or external) interface for storing previously captured or generated pictures and / or acquiring or receiving pictures. When the picture source 16 is a camera, the picture source 16 may be, for example, a local or integrated camera integrated in the source device; when the picture source 16 is a memory, the picture source 16 may be, for example, a local or integrated memory integrated in the source device. When the picture source 16 includes an interface, the interface may be, for example, an external interface for receiving pictures from an external video source, such as an external picture capture device, such as a camera, an external memory, or an external picture generating device, such as an external computer graphics processor, a computer, or a server. The interface may be any type of interface according to any proprietary or standardized interface protocol, such as a wired or wireless interface, an optical interface.

[0131] Among them, the picture can be regarded as a two-dimensional array or matrix of pixels (picture element). The pixels in the array can also be called sampling points. The number of sampling points in the array or picture in the horizontal and vertical directions (or axes) defines the size and / or resolution of the picture. In order to represent the color, three color components are usually used, that is, the picture can be represented as or include three sampling arrays. For example, in the RBG format or color space, the picture includes corresponding red, green and blue sampling arrays. However, in video coding, each pixel is usually represented in a brightness / chroma format or color space, such as for a picture in the YUV format, including a brightness component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The brightness (luma) component Y represents the brightness or grayscale level intensity (for example, the two are the same in a grayscale picture), and the two chroma (chroma) components U and V represent the chroma or color information components. Accordingly, the picture in the YUV format includes a brightness sampling array of brightness sampling values ​​(Y), and two chroma sampling arrays of chroma values ​​(U and V). An image in RGB format may be converted or transformed into YUV format, and vice versa, which process is also referred to as color conversion or transformation. If the image is black and white, the image may only include a brightness sampling array. In the embodiment of the present application, the image transmitted from the image source 16 to the image processor may also be referred to as raw image data 17.

[0132] The image preprocessor 18 is used to receive the original image data 17 and perform preprocessing on the original image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by the image preprocessor 18 may include refurbishment, color format conversion (e.g., from RGB format to YUV format), color adjustment, or denoising.

[0133] The encoder 20 (or video encoder 20) is used to receive the pre-processed picture data 19, and process the pre-processed picture data 19 using a relevant prediction mode (such as the prediction mode in each embodiment of this invention), so as to provide encoded picture data 21 (hereinafter further based on Figure 2 or Figure 4 or Figure 5 ). In some embodiments, the encoder 20 may be used to execute various embodiments described below to implement the chroma block prediction method described in this application on the encoding side.

[0134] The communication interface 22 can be used to receive the encoded picture data 21 and transmit the encoded picture data 21 to the destination device 14 or any other device (such as a memory) through the link 13 for storage or direct reconstruction. The other device can be any device for decoding or storage. The communication interface 22 can, for example, be used to encapsulate the encoded picture data 21 into a suitable format, such as a data packet, for transmission on the link 13.

[0135] The destination device 14 includes a decoder 30. Optionally, the destination device 14 may also include a communication interface 28, a picture post-processor 32, and a display device 34. They are described as follows:

[0136] The communication interface 28 can be used to receive the encoded picture data 21 from the source device 12 or any other source, such as a storage device, such as an encoded picture data storage device. The communication interface 28 can be used to transmit or receive the encoded picture data 21 via the link 13 between the source device 12 and the destination device 14 or via any type of network, such as a direct wired or wireless connection, any type of network, such as a wired or wireless network or any combination thereof, or any type of private network and public network, or any combination thereof. The communication interface 28 can be used, for example, to decapsulate the data packet transmitted by the communication interface 22 to obtain the encoded picture data 21.

[0137] Both communication interface 28 and communication interface 22 may be configured as unidirectional communication interfaces or bidirectional communication interfaces and may be used, for example, to send and receive messages to establish connections, confirm and exchange any other information related to communication links and / or data transmissions such as encoded picture data transmissions.

[0138] The decoder 30 (or decoder 30) is configured to receive the encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (hereinafter further described based on Figure 3 or Figure 4 or Figure 5 ). In some embodiments, the decoder 30 may be used to execute various embodiments described below to implement the chrominance block prediction method described in this application on the decoding side.

[0139] The picture post-processor 32 is used to perform post-processing on the decoded picture data 31 (also called reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 may include: color format conversion (for example, from YUV format to RGB format), color adjustment, repair or resampling, or any other processing, and can also be used to transmit the post-processed picture data 33 to the display device 34.

[0140] A display device 34 is provided for receiving the post-processed picture data 33 to display the picture to, for example, a user or viewer. The display device 34 may be or may include any type of display for presenting the reconstructed picture, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0141] Although, Figure 1A The source device 12 and the destination device 14 are depicted as separate devices, but device embodiments may also include both the source device 12 and the destination device 14 or the functionality of both, i.e., the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality. In such embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0142] It is obvious to a person skilled in the art based on the description that the functionality or Figure 1A The presence and (exact) division of the functionality of the source device 12 and / or destination device 14 shown may vary depending on the actual device and application. The source device 12 and the destination device 14 may include any of a variety of devices, including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, a car device, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use or use any type of operating system.

[0143] Both the encoder 20 and the decoder 30 may be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is partially implemented in software, the device may store the instructions of the software in a suitable non-transitory computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered one or more processors.

[0144] In some cases, Figure 1A The video encoding and decoding system 10 shown in is only an example, and the technology of the present application can be applied to video encoding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between encoding and decoding devices. In other examples, data can be retrieved from local storage, streamed over a network, etc. A video encoding device can encode data and store the data in a memory, and / or a video decoding device can retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve data from a memory and decode data.

[0145] See also Figure 1B , Figure 1B According to an exemplary embodiment, Figure 2 The encoder 20 and / or Figure 3 30. The video decoding system 40 may implement a combination of various techniques of the embodiments of the present application. In the illustrated embodiment, the video decoding system 40 may include an imaging device 41, an encoder 20, a decoder 30 (and / or a video codec implemented by logic circuits 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0146] like Figure 1B As shown, imaging device 41, antenna 42, processing unit 46, logic circuit 47, encoder 20, decoder 30, processor 43, memory 44 and / or display device 45 can communicate with each other. As discussed, although video decoding system 40 is illustrated with encoder 20 and decoder 30, in different examples, video decoding system 40 may include only encoder 20 or only decoder 30.

[0147] In some examples, antenna 42 can be used to transmit or receive a coded bit stream of video data. In addition, in some examples, display device 45 can be used to present video data. In some examples, logic circuit 47 can be implemented by processing unit 46. Processing unit 46 can include application-specific integrated circuit (ASIC) logic, graphics processor, general processor, etc. Video decoding system 40 can also include optional processor 43, which can similarly include application-specific integrated circuit (ASIC) logic, graphics processor, general processor, etc. In some examples, logic circuit 47 can be implemented by hardware, such as video encoding dedicated hardware, etc., and processor 43 can be implemented by general software, operating system, etc. In addition, memory 44 can be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory 44 can be implemented by cache memory. In some examples, logic circuitry 47 may access memory 44 (eg, for implementing an image buffer). In other examples, logic circuitry 47 and / or processing unit 46 may include memory (eg, cache, etc.) for implementing an image buffer, etc.

[0148] In some examples, encoder 20 implemented by logic circuits may include an image buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include encoder 20 implemented by logic circuits 47 to implement reference Figure 2 The various modules discussed in connection with the encoder and / or any other encoder system or subsystem described herein. Logic circuits may be used to perform the various operations discussed herein.

[0149] In some examples, decoder 30 may be implemented in a similar manner by logic circuit 47 to implement reference Figure 3The decoder 30 and / or any other decoder system or subsystem described herein may include various modules discussed above. In some examples, the decoder 30 implemented by logic circuits may include an image buffer (implemented by processing unit 2820 or memory 44) and a graphics processing unit (implemented, for example, by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the decoder 30 implemented by logic circuits 47 to implement the reference Figure 3 and / or the various modules discussed above for any other decoder system or subsystem described herein.

[0150] In some examples, antenna 42 may be used to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoded video frames, indicators, index values, mode selection data, etc. discussed herein, such as data related to the encoded partitions (e.g., transform coefficients or quantized transform coefficients, (as discussed) optional indicators, and / or data defining the encoded partitions). Video decoding system 40 may also include decoder 30 coupled to antenna 42 and used to decode the encoded bitstream. Display device 45 is used to present the video frames.

[0151] It should be understood that in the embodiments of the present application, for the examples described with reference to encoder 20, decoder 30 can be used to perform the reverse process. With respect to signaling syntax elements, decoder 30 can be used to receive and parse such syntax elements and decode the associated video data accordingly. In some examples, encoder 20 can entropy encode the syntax elements into an encoded video bitstream. In such examples, decoder 30 can parse such syntax elements and decode the associated video data accordingly.

[0152] It should be noted that the method described in the embodiment of the present application is mainly used for the inter-frame prediction process, which exists in both the encoder 20 and the decoder 30. The encoder 20 and the decoder 30 in the embodiment of the present application can be encoders / decoders corresponding to video standard protocols such as H.263, H.264, HEVV, MPEG-2, MPEG-4, VP8, VP9, ​​or next-generation video standard protocols (such as H.266, etc.).

[0153] See also Figure 2 , Figure 2 Schematic / conceptual block diagram of an example of an encoder 20 for implementing an embodiment of the present application is shown. Figure 2In the example of , the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy encoding unit 270. The prediction processing unit 260 may include an inter-prediction unit 244, an intra-prediction unit 254, and a mode selection unit 262. The inter-prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The illustrated encoder 20 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0154] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 20, while for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder, wherein the backward signal path of the encoder corresponds to the signal path of the decoder (see Figure 3 Decoder 30 in ).

[0155] The encoder 20 receives a picture 201 or an image block 203 of the picture 201, e.g., a picture in a sequence of pictures forming a video or a video sequence, via, e.g., an input 202. The image block 203 may also be referred to as a current picture block or a picture block to be encoded, and the picture 201 may be referred to as a current picture or a picture to be encoded (particularly when the current picture is distinguished from other pictures in video coding, e.g., previously encoded and / or decoded pictures in the same video sequence, i.e., a video sequence that also includes the current picture).

[0156] An embodiment of the encoder 20 may include a segmentation unit ( Figure 2 201 into a plurality of blocks, such as image blocks 203, usually into a plurality of non-overlapping blocks. The segmentation unit may be used to use the same block size for all pictures in the video sequence and define a corresponding grid of block sizes, or to change the block size between pictures or subsets or groups of pictures, and to segment each picture into corresponding blocks.

[0157] In one example, prediction processing unit 260 of encoder 20 may be used to perform any combination of the above-described partitioning techniques.

[0158] Like the picture 201, the image block 203 is also or can be regarded as a two-dimensional array or matrix of sampling points with sample values, although its size is smaller than that of the picture 201. In other words, the image block 203 may include, for example, one sampling array (e.g., a luminance array in the case of the black and white picture 201) or three sampling arrays (e.g., one luminance array and two chrominance arrays in the case of a color picture) or any other number and / or type of arrays depending on the color format applied. The number of sampling points in the horizontal and vertical directions (or axes) of the image block 203 defines the size of the image block 203.

[0159] like Figure 2 The encoder 20 shown is used to encode the picture 201 block by block, for example, encoding and prediction are performed on each image block 203.

[0160] The residual calculation unit 204 is used to calculate the residual block 205 based on the picture image block 203 and the prediction block 265 (other details of the prediction block 265 are provided below), for example, by subtracting the sample value of the prediction block 265 from the sample value of the picture image block 203 sample by sample (pixel by pixel) to obtain the residual block 205 in the sample domain.

[0161] The transform processing unit 206 is used to apply a transform such as discrete cosine transform (DCT) or discrete sine transform (DST) on the sample values ​​of the residual block 205 to obtain a transform coefficient 207 in the transform domain. The transform coefficient 207 can also be called a transform residual coefficient and represents the residual block 205 in the transform domain.

[0162] The transform processing unit 206 can be used to apply an integer approximation of the DCT / DST, such as the transform specified for HEVC / H.265. Compared to the orthogonal DCT transform, this integer approximation is usually scaled by a certain factor. In order to maintain the norm of the residual block processed by the forward transform and the inverse transform, an additional scaling factor is applied as part of the transform process. The scaling factor is usually selected based on certain constraints, such as the scaling factor is a power of 2 for the shift operation, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform on the decoder 30 side by, for example, the inverse transform processing unit 212 (and for the corresponding inverse transform on the encoder 20 side by, for example, the inverse transform processing unit 212), and accordingly, a corresponding scaling factor can be specified for the forward transform on the encoder 20 side by the transform processing unit 206.

[0163] The quantization unit 208 is used to quantize the transform coefficients 207, for example, by applying scalar quantization or vector quantization to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. A suitable quantization step size may be indicated by a quantization parameter (QP). For example, the quantization parameter may be an index to a predefined set of suitable quantization step sizes. For example, a smaller quantization parameter may correspond to fine quantization (a smaller quantization step size), a larger quantization parameter may correspond to coarse quantization (a larger quantization step size), and vice versa. Quantization may include division by a quantization step size and corresponding quantization or inverse quantization, for example, performed by inverse quantization 210, or may include multiplication by a quantization step size. Embodiments according to some standards such as HEVC may use a quantization parameter to determine the quantization step size. In general, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of an equation containing division. Additional scaling factors may be introduced for quantization and inverse quantization to recover the norm of the residual block that may be modified due to the scale used in the fixed-point approximation of the equation for the quantization step size and the quantization parameter. In an example embodiment, the scales of the inverse transform and inverse quantization may be merged. Alternatively, a custom quantization table may be used and sent from an encoder to a decoder through a signal, for example in a bitstream. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

[0164] The inverse quantization unit 210 is used to apply the inverse quantization of the quantization unit 208 on the quantized coefficients to obtain inverse quantized coefficients 211, for example, applying an inverse quantization scheme of the quantization scheme applied by the quantization unit 208 based on or using the same quantization step size as the quantization unit 208. The inverse quantized coefficients 211 may also be referred to as inverse quantized residual coefficients 211, corresponding to the transform coefficients 207, although the loss due to quantization is generally different from that of the transform coefficients.

[0165] The inverse transform processing unit 212 is used to apply the inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform dequantized block 213 or an inverse transform residual block 213.

[0166] The reconstruction unit 214 (e.g., the summer 214) is used to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 213 to the sample values ​​of the prediction block 265.

[0167] Optionally, a buffer unit 216 (or simply "buffer" 216), such as a line buffer 216, is used to buffer or store the reconstructed blocks 215 and corresponding sample values ​​for, for example, intra-frame prediction. In other embodiments, the encoder can be used to use the unfiltered reconstructed blocks and / or corresponding sample values ​​stored in the buffer unit 216 for any type of estimation and / or prediction, such as intra-frame prediction.

[0168] For example, embodiments of the encoder 20 may be configured such that the buffer unit 216 is used not only to store the reconstructed blocks 215 used for the intra prediction 254, but also for the loop filter unit 220 (in Figure 2 ), and / or, for example, such that the buffer unit 216 and the decoded picture buffer unit 230 form a buffer. Other embodiments may be used to store the filtered block 221 and / or the blocks or samples ( Figure 2 (not shown) is used as input or basis for intra-frame prediction 254.

[0169] The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain a filtered block 221, so as to smoothly perform pixel conversion or improve video quality. The loop filter unit 220 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 220 is in Figure 2 2 is shown as an in-loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coding block after the loop filter unit 220 performs a filtering operation on the reconstructed coding block.

[0170] Embodiments of the encoder 20 (correspondingly, the loop filter unit 220) can be used to output loop filter parameters (e.g., sample adaptive offset information), for example, directly or after entropy encoding by the entropy encoding unit 270 or any other entropy encoding unit, such that the decoder 30 can receive and apply the same loop filter parameters for decoding.

[0171] The decoded picture buffer (DPB) 230 may be a reference picture memory for storing reference picture data for use by the encoder 20 in encoding video data. The DPB 230 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM)) or other types of memory devices. The DPB 230 and the buffer 216 may be provided by the same memory device or separate memory devices. In one example, the decoded picture buffer (DPB) 230 is used to store the filtered block 221. The decoded picture buffer 230 may further be used to store other previously filtered blocks of the same current picture or a different picture, such as a previously reconstructed picture, such as the previously reconstructed and filtered block 221, and may provide a complete previously reconstructed, i.e., decoded picture (and corresponding reference blocks and samples) and / or a portion of the reconstructed current picture (and corresponding reference blocks and samples), such as for inter-frame prediction. In one example, if the reconstructed block 215 is reconstructed without in-loop filtering, the decoded picture buffer (DPB) 230 is used to store the reconstructed block 215.

[0172] The prediction processing unit 260, also referred to as the block prediction processing unit 260, is used to receive or obtain an image block 203 (a current image block 203 of a current picture 201) and reconstructed picture data, such as reference samples of the same (current) picture from a buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from a decoded picture buffer 230, and to process such data for prediction, i.e. to provide a prediction block 265 which may be an inter-prediction block 245 or an intra-prediction block 255.

[0173] The mode selection unit 262 may be used to select a prediction mode (eg, intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 for use as a prediction block 265 to calculate the residual block 205 and reconstruct the reconstructed block 215 .

[0174] Embodiments of the mode selection unit 262 may be used to select a prediction mode (e.g., from those prediction modes supported by the prediction processing unit 260) that provides the best match or minimum residual (minimum residual means better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead means better compression in transmission or storage), or considers or balances both. The mode selection unit 262 may be used to determine the prediction mode based on rate distortion optimization (RDO), i.e., select a prediction mode that provides minimum rate distortion optimization, or select a prediction mode whose associated rate distortion at least satisfies the prediction mode selection criteria.

[0175] The prediction processing performed by an example of encoder 20 (eg, by prediction processing unit 260) and the mode selection performed (eg, by mode selection unit 262) will be explained in detail below.

[0176] As described above, the encoder 20 is used to determine or select the best or optimal prediction mode from a (predetermined) prediction mode set. The prediction mode set may include, for example, an intra-frame prediction mode and / or an inter-frame prediction mode.

[0177] The intra-frame prediction mode set may include 35 different intra-frame prediction modes, for example, non-directional modes such as DC (or mean) mode and planar mode, or directional modes as defined in H.265, or may include 67 different intra-frame prediction modes, for example, non-directional modes such as DC (or mean) mode and planar mode, or directional modes as defined in H.266 under development.

[0178] In a possible implementation, the inter-prediction mode set depends on the available reference picture (i.e., at least part of the decoded picture stored in the DBP 230 as described above) and other inter-prediction parameters, such as whether to use the entire reference picture or only a part of the reference picture, such as a search window area around the current block, to search for the best matching reference block, and / or, for example, whether to apply pixel interpolation such as half-pixel and / or quarter-pixel interpolation. The inter-prediction mode set may include, for example, an Advanced Motion Vector Prediction (AMVP) mode and a merge mode. In a specific implementation, the inter-prediction mode set may include an improved control point-based AMVP mode according to an embodiment of the present application, and an improved control point-based merge mode. In one example, the intra-prediction unit 254 may be used to perform any combination of the inter-prediction techniques described below.

[0179] In addition to the above prediction modes, embodiments of the present application may also apply skip mode and / or direct mode.

[0180] The prediction processing unit 260 can be further used to partition the image block 203 into smaller block partitions or sub-blocks, for example, by iteratively using quad-tree (QT) partitioning, binary-tree (BT) partitioning or triple-tree (TT) partitioning, or any combination thereof, and for performing prediction for each of the block partitions or sub-blocks, for example, wherein the mode selection includes selecting a tree structure of the partitioned image block 203 and selecting a prediction mode to be applied to each of the block partitions or sub-blocks.

[0181] The inter-frame prediction unit 244 may include a motion estimation (ME) unit ( Figure 2 ) and a motion compensation (MC) unit ( Figure 2 ). The motion estimation unit is used to receive or obtain a picture image block 203 (a current picture image block 203 of a current picture 201) and a decoded picture 231, or at least one or more previously reconstructed blocks, for example, reconstructed blocks of one or more other / different previously decoded pictures 231, to perform motion estimation. For example, a video sequence may include a current picture and a previously decoded picture 31, or in other words, the current picture and the previously decoded picture 31 may be part of a picture sequence forming a video sequence, or form the picture sequence.

[0182] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different pictures in a plurality of other pictures and to provide the reference block to the motion estimation unit ( Figure 2(not shown) provides a reference picture and / or provides an offset (spatial offset) between the position (X, Y coordinates) of a reference block and the position of a current block as an inter-frame prediction parameter. The offset is also called a motion vector (MV).

[0183] The motion compensation unit is used to obtain inter-frame prediction parameters, and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain the inter-frame prediction block 245. Figure 2 Motion compensation performed by the motion compensation unit 246 (not shown) may include fetching or generating a prediction block based on a motion / block vector determined by motion estimation (possibly performing interpolation to sub-pixel accuracy). Interpolation filtering may produce additional pixel samples from known pixel samples, thereby potentially increasing the number of candidate prediction blocks that may be used to encode the picture block. Upon receiving the motion vector for the PU of the current picture block, the motion compensation unit 246 may locate the prediction block to which the motion vector points in a reference picture list. The motion compensation unit 246 may also generate syntax elements associated with blocks and video slices for use by the decoder 30 when decoding picture blocks of the video slice.

[0184] Specifically, the inter-frame prediction unit 244 may transmit syntax elements to the entropy coding unit 270, wherein the syntax elements include inter-frame prediction parameters (such as indication information of selecting an inter-frame prediction mode for current block prediction after traversing multiple inter-frame prediction modes). In a possible application scenario, if there is only one inter-frame prediction mode, the inter-frame prediction parameters may not be carried in the syntax elements, and the decoding end 30 may directly use the default prediction mode for decoding. It can be understood that the inter-frame prediction unit 244 can be used to perform any combination of inter-frame prediction techniques.

[0185] The intra prediction unit 254 is used to obtain, for example, receive a picture block 203 (current picture block) of the same picture and one or more previously reconstructed blocks, for example, reconstructed adjacent blocks, for intra estimation. For example, the encoder 20 can be used to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[0186] Embodiments of the encoder 20 may be configured to select an intra prediction mode based on an optimization criterion, such as minimum residual (eg, the intra prediction mode that provides a prediction block 255 most similar to the current picture block 203) or minimum rate distortion.

[0187] The intra prediction unit 254 is further configured to determine an intra prediction block 255 based on the intra prediction parameters of the selected intra prediction mode. In any case, after selecting the intra prediction mode for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters, i.e., provide information indicating the selected intra prediction mode for the block, to the entropy encoding unit 270. In one example, the intra prediction unit 254 can be configured to perform any combination of intra prediction techniques.

[0188] Specifically, the intra prediction unit 254 may transmit a syntax element to the entropy coding unit 270, wherein the syntax element includes an intra prediction parameter (e.g., indication information of selecting an intra prediction mode for current block prediction after traversing multiple intra prediction modes). In a possible application scenario, if there is only one intra prediction mode, the intra prediction parameter may not be carried in the syntax element, and the decoding end 30 may directly use the default prediction mode for decoding.

[0189] The entropy coding unit 270 is used to apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC), a syntax-based context-adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to a single or all (or none) of the quantized residual coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, and / or loop filter parameters to obtain the encoded picture data 21 that can be output through the output 272 in the form of, for example, an encoded bitstream 21. The encoded bitstream can be transmitted to the video decoder 30, or archived for later transmission or retrieval by the video decoder 30. The entropy coding unit 270 can also be used to entropy encode other syntax elements of the current video slice being encoded.

[0190] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform based encoder 20 may directly quantize the residual signal without a transform processing unit 206 for certain blocks or frames. In another embodiment, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[0191] Specifically, in the embodiment of the present application, the encoder 20 may be used to implement the inter-frame prediction method described in the embodiments below.

[0192] It should be understood that other structural changes of the video encoder 20 can be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 20 can directly quantize the residual signal without processing by the transform processing unit 206, and accordingly, it does not need to be processed by the inverse transform processing unit 212; or, for some image blocks or image frames, the video encoder 20 does not generate residual data, and accordingly, it does not need to be processed by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210 and the inverse transform processing unit 212; or, the video encoder 20 can directly store the reconstructed image block as a reference block without being processed by the filter 220; or, the quantization unit 208 and the inverse quantization unit 210 in the video encoder 20 can be combined together. The loop filter 220 is optional, and in the case of lossless compression encoding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210 and the inverse transform processing unit 212 are optional. It should be understood that, according to different application scenarios, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 may be selectively enabled.

[0193] See also Figure 3 , Figure 3 A schematic / conceptual block diagram of an example of a decoder 30 for implementing an embodiment of the present application is shown. The video decoder 30 is used to receive encoded picture data (e.g., encoded bitstream) 21, for example, encoded by the encoder 20, to obtain a decoded picture 231. During the decoding process, the video decoder 30 receives video data from the video encoder 20, such as an encoded video bitstream representing picture blocks of an encoded video slice and associated syntax elements.

[0194] exist Figure 3 In the example of , decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter-prediction unit 344, an intra-prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform substantially the same as reference. Figure 2 The encoding pass described by video encoder 20 is the inverse of the decoding pass.

[0195] The entropy decoding unit 304 is used to perform entropy decoding on the encoded picture data 21 to obtain, for example, quantized coefficients 309 and / or decoded encoding parameters ( Figure 3The entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level.

[0196] The inverse quantization unit 310 may be functionally the same as the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally the same as the inverse transform processing unit 212, the reconstruction unit 314 may be functionally the same as the reconstruction unit 214, the buffer 316 may be functionally the same as the buffer 216, the loop filter 320 may be functionally the same as the loop filter 220, and the decoded picture buffer 330 may be functionally the same as the decoded picture buffer 230.

[0197] The prediction processing unit 360 may include an inter-prediction unit 344 and an intra-prediction unit 354, wherein the inter-prediction unit 344 may be functionally similar to the inter-prediction unit 244, and the intra-prediction unit 354 may be functionally similar to the intra-prediction unit 254. The prediction processing unit 360 is generally used to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and receive or obtain prediction related parameters and / or information about the selected prediction mode from, for example, the entropy decoding unit 304 (explicitly or implicitly).

[0198] When the video slice is encoded as an intra-coded (I) slice, the intra-prediction unit 354 of the prediction processing unit 360 is used to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra-prediction mode and data from a previously decoded block of the current frame or picture. When the video frame is encoded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., motion compensation unit) of the prediction processing unit 360 is used to generate a prediction block 365 for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. For inter-prediction, the prediction block may be generated from one of the reference pictures within one of the reference picture lists. Video decoder 30 may construct reference frame lists: List 0 and List 1 using a default construction technique based on the reference pictures stored in DPB 330.

[0199] The prediction processing unit 360 is configured to determine prediction information for a video block of a current video slice by parsing motion vectors and other syntax elements, and use the prediction information to generate a prediction block for the current video block being decoded. In one example of the present application, the prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra-frame or inter-frame prediction) for encoding a video block of a video slice, an inter-frame prediction slice type (e.g., B slice, P slice, or GPB slice), construction information of one or more of the reference picture lists for the slice, a motion vector for each inter-frame coded video block of the slice, an inter-frame prediction state of each inter-frame coded video block of the slice, and other information to decode the video block of the current video slice. In another example of the present disclosure, the syntax elements received by the video decoder 30 from the bitstream include syntax elements received in one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

[0200] Inverse quantization unit 310 may be used to inverse quantize (i.e., dequantize) quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each video block in a video slice to determine a degree of quantization that should be applied and likewise a degree of inverse quantization that should be applied.

[0201] The inverse transform processing unit 312 is used to apply an inverse transform (eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to generate a residual block in the pixel domain.

[0202] The reconstruction unit 314 (e.g., the summer 314) is used to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain the reconstructed block 315 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the prediction block 365.

[0203] The loop filter unit 320 is used to filter the reconstructed block 315 (during or after the encoding cycle) to obtain a filtered block 321, thereby smoothing pixel conversion or improving video quality. In one example, the loop filter unit 320 can be used to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is in Figure 3 3. Although shown as an in-loop filter in FIG. 3, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0204] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330 which stores reference pictures for subsequent motion compensation.

[0205] Decoder 30 is operable to output decoded picture 31 , eg, via output 332 , for presentation to or viewing by a user.

[0206] Other variations of the video decoder 30 may be used to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform based decoder 30 may directly inverse quantize the residual signal without the inverse transform processing unit 312 for certain blocks or frames. In another embodiment, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0207] Specifically, in the embodiment of the present application, the decoder 30 is used to implement the inter-frame prediction method described in the embodiments below.

[0208] It should be understood that other structural changes of the video decoder 30 can be used to decode the encoded video bitstream. For example, the video decoder 30 can generate an output video stream without being processed by the filter 320; or, for some image blocks or image frames, the entropy decoding unit 304 of the video decoder 30 does not decode the quantized coefficients, and accordingly does not need to be processed by the inverse quantization unit 310 and the inverse transform processing unit 312. The loop filter 320 is optional; and for lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are optional. It should be understood that, according to different application scenarios, the inter-frame prediction unit and the intra-frame prediction unit can be selectively enabled.

[0209] It should be understood that in the encoder 20 and decoder 30 of the present application, the processing result of a certain link can be output to the next link after further processing. For example, after the interpolation filtering, motion vector derivation or loop filtering links, the processing results of the corresponding links can be further subjected to operations such as Clip or shift.

[0210] For example, the motion vector of the control point of the current image block derived according to the motion vector of the adjacent affine coding block can be further processed, and this application does not limit this. For example, the value range of the motion vector is constrained so that it is within a certain bit width. Assuming that the bit width of the allowed motion vector is bitDepth, the range of the motion vector is -2^(bitDepth-1)~2^(bitDepth-1)-1, where the "^" symbol represents the power. If bitDepth is 16, the value range is -32768~32767. If bitDepth is 18, the value range is -131072~131071. Constraints can be made in the following two ways:

[0211] Method 1: remove the high bits of motion vector overflow:

[0212] ux=(vx+2 bitDepth )%2 bitDepth

[0213] vx=(ux>=2 bitDepth-1 )? (ux-2 bitDepth ):ux

[0214] uy=(vy+2 bitDepth )%2 bitDepth

[0215] vy=(uy>=2 bitDepth-1 )? (uy-2 bitDepth ):uy

[0216] For example, if the value of vx is -32769, the value obtained by the above formula is 32767. Because in computers, values ​​are stored in binary complement form, the binary complement of -32769 is 1,0111,1111,1111,1111 (17 bits), and the computer handles overflow by discarding the high bits, so the value of vx is 0111,1111,1111,1111, which is 32767, which is consistent with the result obtained by the formula.

[0217] Method 2: Clipping the motion vector, as shown in the following formula:

[0218] vx=Clip3(-2 bitDepth-1 ,2 bitDepth-1-1,vx)

[0219] vy=Clip3(-2 bitDepth-1 ,2 bitDepth-1 -1,vy)

[0220] The definition of Clip3 is to clamp the value of z to the interval [x, y]:

[0221]

[0222] See also Figure 4 , Figure 4 FIG. 4 is a schematic diagram of the structure of a video decoding device 400 (eg, a video encoding device 400 or a video decoding device 400) provided in an embodiment of the present application. The video decoding device 400 is suitable for implementing the embodiments described herein. In one embodiment, the video decoding device 400 may be a video decoder (eg, Figure 1A decoder 30) or a video encoder (e.g. Figure 1A In another embodiment, the video decoding device 400 may be the above-mentioned Figure 1A Decoder 30 or Figure 1A One or more components in encoder 20.

[0223] The video decoding device 400 includes: an inlet port 410 and a receiving unit (Rx) 420 for receiving data, a processor, a logic unit or a central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an outlet port 450 for transmitting data, and a memory 460 for storing data. The video decoding device 400 may also include an optical-to-electrical conversion component and an electro-optical (EO) component coupled to the inlet port 410, the receiver unit 420, the transmitter unit 440 and the outlet port 450 for the outlet or inlet of an optical signal or an electrical signal.

[0224] The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the inlet port 410, the receiver unit 420, the transmitter unit 440, the outlet port 450, and the memory 460. The processor 430 includes a decoding module 470 (e.g., an encoding module 470 or a decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed herein to implement the chrominance block prediction method provided in the embodiments of the present application. For example, the encoding / decoding module 470 implements, processes, or provides various encoding operations. Therefore, the encoding / decoding module 470 provides substantial improvements to the functions of the video decoding device 400 and affects the conversion of the video decoding device 400 to different states. Alternatively, the encoding / decoding module 470 is implemented with instructions stored in the memory 460 and executed by the processor 430.

[0225] The memory 460 includes one or more disks, tape drives, and solid-state drives, and can be used as an overflow data storage device for storing programs when these programs are selectively executed, and for storing instructions and data read during program execution. The memory 460 can be volatile and / or non-volatile, and can be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random access memory (SRAM).

[0226] See also Figure 5 , Figure 5 According to an exemplary embodiment, it can be used as Figure 1A 1 and 14. The device 500 is a simplified block diagram of an apparatus 500 for either or both of the source device 12 and the destination device 14. The apparatus 500 can implement the techniques of the present application. In other words, Figure 5 A schematic block diagram of an implementation of an encoding device or decoding device (referred to as a decoding device 500) of an embodiment of the present application. The decoding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and the memory are connected via a bus system, the memory is used to store instructions, and the processor is used to execute instructions stored in the memory. The memory of the decoding device stores program code, and the processor can call the program code stored in the memory to execute various video encoding or decoding methods described in this application. To avoid repetition, it is not described in detail here.

[0227] In the embodiment of the present application, the processor 510 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0228] The memory 530 may include a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may also be used as the memory 530. The memory 530 may include code and data 531 accessed by the processor 510 using the bus 550. The memory 530 may further include an operating system 533 and an application 535, which includes at least one program that allows the processor 510 to perform the video encoding or decoding method described in the present application. For example, the application 535 may include applications 1 to N, which further include a video encoding or decoding application (referred to as a video decoding application) that executes the video encoding or decoding method described in the present application.

[0229] The bus system 550 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus system 550 in the figure.

[0230] Optionally, the decoding device 500 may further include one or more output devices, such as a display 570. In one example, the display 570 may be a touch-sensitive display that combines a display with a touch-sensitive unit operable to sense touch input. The display 570 may be connected to the processor 510 via the bus 550.

[0231] The concepts involved in this application are first described below.

[0232] 1) Inter-frame prediction mode

[0233] In HEVC, two inter-frame prediction modes are used, namely advanced motion vector prediction (AMVP) mode and merge mode.

[0234] For the AMVP mode, first traverse the coded blocks (referred to as neighboring blocks) adjacent to the current block in the spatial domain or temporal domain, build a candidate motion vector list (also referred to as a motion information candidate list) based on the motion information of each neighboring block, and then determine the optimal motion vector from the candidate motion vector list by using the rate-distortion cost, and use the candidate motion information with the minimum rate-distortion cost as the motion vector predictor (MVP) of the current block. The positions of the neighboring blocks and their traversal order are predefined. The rate-distortion cost is calculated by formula (1), where J represents the rate-distortion cost RD Cost, SAD is the sum of absolute differences (SAD) between the predicted pixel value and the original pixel value obtained after motion estimation using the candidate motion vector predictor, R represents the code rate, and λ represents the Lagrange multiplier. The encoder transmits the index value of the selected motion vector predictor in the candidate motion vector list and the reference frame index value to the decoder. Furthermore, a motion search is performed in the neighborhood centered on the MVP to obtain the actual motion vector of the current block, and the encoder transmits the difference (motion vector difference) between the MVP and the actual motion vector to the decoder.

[0235] J=SAD+λR (1)

[0236] For the Merge mode, the motion information of the coded blocks adjacent to the current block in the spatial domain or temporal domain is first used to construct a candidate motion vector list, and then the optimal motion information is determined from the candidate motion vector list by calculating the rate-distortion cost as the motion information of the current block, and then the index value of the position of the optimal motion information in the candidate motion vector list (referred to as merge index, the same below) is passed to the decoder. The spatial and temporal candidate motion information of the current block is as follows: Fig. 6A As shown in the figure, the spatial candidate motion information comes from the five spatially adjacent blocks (A0, A1, B0, B1 and B2). If the adjacent block is not available (the adjacent block does not exist or the adjacent block is not encoded or the prediction mode used by the adjacent block is not the inter-frame prediction mode), the motion information of the adjacent block is not added to the candidate motion vector list. The temporal candidate motion information of the current block is obtained by scaling the MV of the corresponding position block in the reference frame according to the picture order count (POC) of the reference frame and the current frame. First, determine whether the block at position T in the reference frame is available. If not, select the block at position C.

[0237] Similar to the AMVP mode, the positions of neighbor blocks and their traversal order in the Merge mode are also predefined, and the positions of neighbor blocks and their traversal order may be different in different modes.

[0238] As you can see, in both AMVP mode and Merge mode, a candidate motion vector list needs to be maintained. Each time before adding new motion information to the candidate list, it will first check whether the same motion information already exists in the list. If so, the motion information will not be added to the list. We call this checking process pruning the candidate motion vector list. List pruning is to prevent the same motion information from appearing in the list and avoid redundant rate-distortion cost calculations.

[0239] In HEVC inter-frame prediction, all pixels in the coding block use the same motion information, and then motion compensation is performed based on the motion information to obtain the predicted value of the pixels in the coding block. However, not all pixels in the coding block have the same motion characteristics, and using the same motion information may lead to inaccurate motion compensation prediction, thereby increasing residual information.

[0240] Existing video coding standards use block matching motion estimation based on translational motion models and assume that the motion of all pixels in a block is consistent. However, in the real world, there are many kinds of motion, and there are many objects with non-translational motion, such as rotating objects, roller coasters rotating in different directions, fireworks, and some stunts in movies, especially moving objects in UGC scenes. If the block motion compensation technology based on the translational motion model in the current coding standard is used to encode them, the coding efficiency will be greatly affected. Therefore, non-translational motion models, such as affine motion models, are generated to further improve the coding efficiency.

[0241] Based on this, according to different motion models, the AMVP mode can be divided into the AMVP mode based on the translational model and the AMVP mode based on the non-translational model; the Merge mode can be divided into the Merge mode based on the translational model and the Merge mode based on the non-translational motion model.

[0242] 2) Non-translational motion model

[0243] Non-translational motion model prediction refers to using the same motion model at the codec end to derive the motion information of each sub-block in the current block, perform motion compensation based on the motion information of the sub-block, and obtain the predicted block, thereby improving the prediction efficiency. Commonly used non-translational motion models include 4-parameter affine motion models or 6-parameter affine motion models.

[0244] Among them, the sub-block involved in the embodiment of the present application can be a pixel point or a pixel block of size N1×N2 divided according to a specific method, wherein N1 and N2 are both positive integers, and N1 can be equal to N2 or not equal to N2.

[0245] The 4-parameter affine motion model is shown in formula (2):

[0246]

[0247] The 4-parameter affine motion model can be represented by the motion vectors of two pixels and their coordinates relative to the upper left vertex pixel of the current block. The pixels used to represent the motion model parameters are called control points. If the upper left vertex (0,0) and the upper right vertex (W,0) pixels are used as control points, the motion vectors (vx0,vy0) and (vx1,vy1) of the upper left vertex and upper right vertex control points of the current block are first determined, and then the motion information of each sub-block in the current block is obtained according to formula (3), where (x,y) is the coordinate of the sub-block relative to the upper left vertex pixel of the current block, and W is the width of the current block.

[0248]

[0249] The 6-parameter affine motion model is shown in formula (4):

[0250]

[0251] The 6-parameter affine motion model can be represented by the motion vectors of three pixels and their coordinates relative to the upper left vertex pixel of the current block. If the upper left vertex (0,0), upper right vertex (W,0) and lower left vertex (0,H) pixels are used as control points, the motion vectors of the upper left vertex, upper right vertex and lower left vertex control points of the current block are first determined as (vx0,vy0), (vx1,vy1) and (vx2,vy2), respectively. Then, the motion information of each sub-block in the current block is obtained according to formula (5), where (x,y) is the coordinate of the sub-block relative to the upper left vertex pixel of the current block, and W and H are the width and height of the current block, respectively.

[0252]

[0253] The coding block predicted by the affine motion model is called an affine coding block.

[0254] Generally, the motion information of the control points of the affine coding block can be obtained by using an Advanced Motion Vector Prediction (AMVP) mode based on an affine motion model or a Merge mode based on an affine motion model.

[0255] The motion information of the control point of the current coding block can be obtained by an inherited control point motion vector prediction method or a constructed control point motion vector prediction method.

[0256] 3) Inherited control point motion vector prediction method

[0257] The inherited control point motion vector prediction method refers to determining the candidate control point motion vectors of the current block by using the motion model of the adjacent encoded affine coding block.

[0258] by Figure 6B Taking the current block shown as an example, the adjacent position blocks around the current block are traversed in a set order, such as A1->B1->B0->A0->B2, and the affine coding block where the adjacent position block of the current block is located is found, and the control point motion information of the affine coding block is obtained. Then, the motion model constructed by the control point motion information of the affine coding block is used to derive the control point motion vector of the current block (for Merge mode) or the motion vector prediction value of the control point (for AMVP mode). A1->B1->B0->A0->B2 is only used as an example, and other combinations of orders are also applicable to this application. In addition, the adjacent position blocks are not limited to A1, B1, B0, A0, B2.

[0259] The adjacent position block may be a pixel point, a pixel block of a preset size divided according to a specific method, for example, a 4x4 pixel block, a 4x2 pixel block, or a pixel block of other sizes, without limitation.

[0260] The following takes A1 as an example to describe the determination process, and other cases are similar:

[0261] like Figure 4 As shown, if the coding block where A1 is located is a 4-parameter affine coding block, the motion vector (vx4, vy4) of the upper left vertex (x4, y4) and the motion vector (vx5, vy5) of the upper right vertex (x5, y5) of the affine coding block are obtained; the motion vector (vx0, vy0) of the upper left vertex (x0, y0) of the current affine coding block is calculated using formula (6), and the motion vector (vx1, vy1) of the upper right vertex (x1, y1) of the current affine coding block is calculated using formula (7).

[0262]

[0263] The combination of the motion vector (vx0, vy0) of the upper left vertex (x0, y0) of the current block and the motion vector (vx1, vy1) of the upper right vertex (x1, y1) obtained as above according to the affine coding block where A1 is located is the candidate control point motion vector of the current block.

[0264] If the coding block where A1 is located is a 6-parameter affine coding block, the motion vector (vx4, vy4) of the upper left vertex (x4, y4), the motion vector (vx5, vy5) of the upper right vertex (x5, y5), and the motion vector (vx6, vy6) of the lower left vertex (x6, y6) of the affine coding block are obtained; the motion vector (vx0, vy0) of the upper left vertex (x0, y0) of the current block is calculated using formula (8), the motion vector (vx1, vy1) of the upper right vertex (x1, y1) of the current block is calculated using formula (9), and the motion vector (vx2, vy2) of the lower left vertex (x2, y2) of the current block is calculated using formula (10).

[0265]

[0266] The combination of the motion vector (vx0, vy0) of the upper left vertex (x0, y0) of the current block, the motion vector (vx1, vy1) of the upper right vertex (x1, y1), and the motion vector (vx2, vy2) of the lower left vertex (x2, y2) of the current block obtained as above according to the affine coding block where A1 is located is the candidate control point motion vector of the current block.

[0267] It should be noted that other motion models, candidate positions, and search traversal orders may also be applicable to the present application, and the embodiments of the present application will not go into details therein.

[0268] It should be noted that the method of using other control points to represent the motion model of adjacent and current coding blocks can also be applied to the present application, which will not be described in detail here.

[0269] 4) Constructed control point motion vectors prediction method 1:

[0270] The constructed control point motion vector prediction method refers to combining the motion vectors of the neighboring coded blocks around the control point of the current block as the motion vector of the control point of the current affine coded block, without considering whether the neighboring coded blocks are affine coded blocks.

[0271] The motion information of the coded blocks adjacent to the current coded block is used to determine the motion vectors of the upper left and upper right vertices of the current block. Figure 6C The control point motion vector prediction method constructed is described as an example. It should be noted that Figure 6C Just as an example.

[0272] like Figure 6CAs shown, the motion vectors of the coded blocks A2, B2 and B3 adjacent to the upper left vertex are used as candidate motion vectors for the motion vector of the upper left vertex of the current block; the motion vectors of the coded blocks B1 and B0 adjacent to the upper right vertex are used as candidate motion vectors for the motion vector of the upper right vertex of the current block. The candidate motion vectors of the upper left vertex and the upper right vertex are combined to form multiple binary groups. The motion vectors of the two coded blocks included in the binary group can be used as candidate control point motion vectors of the current block, as shown in the following formula (11A):

[0273] {v A2 ,v B1},{v A2 ,v B0},{v B2 ,v B1},{v B2 ,v B0},{v B3 ,v B1},{v B3 ,v B0}(11A);

[0274] Among them, v A2 represents the motion vector of A2, v B1 represents the motion vector of B1, v B0 represents the motion vector of B0, v B2 represents the motion vector of B2, v B3 Indicates the motion vector of B3.

[0275] like Figure 6C As shown, the motion vectors of the coded blocks A2, B2 and B3 adjacent to the upper left vertex are used as candidate motion vectors for the motion vector of the upper left vertex of the current block; the motion vectors of the coded blocks B1 and B0 adjacent to the upper right vertex are used as candidate motion vectors for the motion vector of the upper right vertex of the current block, and the motion vectors of the coded blocks A0 and A1 adjacent to the lower vertex are used as candidate motion vectors for the motion vector of the lower left vertex of the current block. The candidate motion vectors of the upper left vertex, the upper right vertex and the lower left vertex are combined to form a triple. The motion vectors of the three coded blocks included in the triple can be used as candidate control point motion vectors of the current block, as shown in the following formulas (11B) and (11C): {v A2 ,v B1 ,v A0},{v A2 ,v B0 ,v A0},{v B2 ,v B1 ,v A0},{v B2 ,v B0 ,vA0},{v B3 ,v B1 ,v A0},{v B3 ,v B0 ,v A0}(11B)

[0276] {v A2 ,v B1 ,v A1},{v A2 ,v B0 ,v A1},{v B2 ,v B1 ,v A1},{v B2 ,v B0 ,v A1},{v B3 ,v B1 ,v A1},{v B3 ,v B0 ,v A1}(11C);

[0277] Among them, v A2 represents the motion vector of A2, v B1 represents the motion vector of B1, v B0 represents the motion vector of B0, v B2 represents the motion vector of B2, v B3 represents the motion vector of B3, v A0 represents the motion vector of A0, v A1 Indicates the motion vector of A1.

[0278] It should be noted that other methods of combining control point motion vectors may also be applicable to the present application and will not be described in detail here.

[0279] It should be noted that the method of using other control points to represent the motion model of adjacent and current coding blocks can also be applied to the present application, which will not be described in detail here.

[0280] 5) Constructed control point motion vectors prediction method 2, see Fig.6D shown.

[0281] Step 601: Obtain motion information of each control point of the current block.

[0282] by Figure 6CAs shown in the figure, CPk (k = 1, 2, 3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial neighboring positions of the current block, which are used to predict CP1, CP2 or CP3; T is the temporal neighboring position of the current block, which is used to predict CP4.

[0283] Assume that the coordinates of CP1, CP2, CP3 and CP4 are (0,0), (W,0), (H,0) and (W,H), respectively, where W and H are the width and height of the current block.

[0284] For each control point, its motion information is obtained in the following order:

[0285] (1) For CP1, the checking order is B2->A2->B3. If B2 is available, the motion information of B2 is used. Otherwise, A2 and B3 are checked. If the motion information of the three positions is not available, the motion information of CP1 cannot be obtained.

[0286] (2) For CP2, the checking order is B0->B1; if B0 is available, CP2 uses the motion information of B0. Otherwise, B1 is checked. If the motion information of both positions is not available, the motion information of CP2 cannot be obtained.

[0287] (3) For CP3, the detection order is A0->A1.

[0288] (4) For CP4, the motion information of T is used.

[0289] Here, X is available, which means that the block including position X (X is A0, A1, A2, B0, B1, B2, B3 or T) has been encoded and uses the inter-frame prediction mode; otherwise, position X is not available.

[0290] It should be noted that other methods for obtaining motion information of control points may also be applicable to the present application and will not be described in detail here.

[0291] Step 602: Combine the motion information of the control points to obtain constructed motion information of the control points.

[0292] The motion information of two control points is combined into a binary group to construct a 4-parameter affine motion model. The combination of two control points can be {CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}. For example, a 4-parameter affine motion model constructed using a binary group consisting of CP1 and CP2 control points can be recorded as Affine(CP1, CP2).

[0293] The motion information of the three control points is combined to form a triplet to construct a 6-parameter affine motion model. The three control points can be combined in the form of {CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}. For example, a 6-parameter affine motion model constructed using a triplet consisting of CP1, CP2, and CP3 control points can be recorded as Affine(CP1, CP2, CP3).

[0294] The quaternary group formed by combining the motion information of the four control points is used to construct an 8-parameter bilinear model. The 8-parameter bilinear model constructed by the quaternary group composed of the CP1, CP2, CP3 and CP4 control points is denoted as Bilinear(CP1, CP2, CP3, CP4).

[0295] In the embodiment of the present application, for the convenience of description, the combination of motion information of two control points (or two encoded blocks) is referred to as a binary group, the combination of motion information of three control points (or two encoded blocks) is referred to as a triplet, and the combination of motion information of four control points (or four encoded blocks) is referred to as a quadruple.

[0296] Traverse these models in a preset order. If the motion information of a control point corresponding to the combined model is not available, the model is considered to be unavailable. Otherwise, determine the reference frame index of the model and scale the motion vector of the control point. If the motion information of all control points after scaling is consistent, the model is illegal. If it is determined that the motion information of the control points controlling the model is available and the model is legal, the motion information of the control points that construct the model is added to the motion information candidate list.

[0297] The method for scaling the motion vector of the control point is shown in formula (12):

[0298]

[0299] Among them, CurPoc represents the POC number of the current frame, DesPoc represents the POC number of the reference frame of the current block, SrcPoc represents the POC number of the reference frame of the control point, MV s represents the motion vector obtained by scaling, and MV represents the motion vector of the control point.

[0300] It should be noted that a combination of different control points can also be converted into control points at the same position.

[0301] For example, the 4-parameter affine motion model obtained by combining {CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4} is converted to be represented by {CP1, CP2} or {CP1, CP2, CP3}. The conversion method is to substitute the motion vector and coordinate information of the control point into formula (2) to obtain the model parameters, and then substitute the coordinate information of {CP1, CP2} into formula (3) to obtain its motion vector.

[0302] More directly, the conversion can be performed according to the following formulas (13)-(21), where W represents the width of the current block, H represents the height of the current block, and in formulas (13)-(21), (vx0, vy0) represents the motion vector of CP1, (vx1, vy1) represents the motion vector of CP2, (vx2, vy2) represents the motion vector of CP3, and (vx3, vy3) represents the motion vector of CP4.

[0303] The conversion of {CP1, CP2} to {CP1, CP2, CP3} can be achieved by the following formula (13), that is, the motion vector of CP3 in {CP1, CP2, CP3} can be determined by formula (13):

[0304]

[0305] The conversion of {CP1, CP3} to {CP1, CP2} or {CP1, CP2, CP3} can be achieved by the following formula (14):

[0306]

[0307] The conversion of {CP2, CP3} to {CP1, CP2} or {CP1, CP2, CP3} can be achieved by the following formula (15):

[0308]

[0309] The conversion of {CP1, CP4} to {CP1, CP2} or {CP1, CP2, CP3} can be achieved by the following formula (16) or (17):

[0310]

[0311]

[0312] The conversion of {CP2, CP4} to {CP1, CP2} can be achieved by the following formula (18), and the conversion of {CP2, CP4} to {CP1, CP2, CP3} can be achieved by formulas (18) and (19):

[0313]

[0314] The conversion of {CP3, CP4} to {CP1, CP2} can be achieved by the following formula (20), and the conversion of {CP3, CP4} to {CP1, CP2, CP3} can be achieved by the following formulas (20) and (21):

[0315]

[0316] For example, the 6-parameter affine motion model composed of {CP1, CP2, CP4}, {CP2, CP3, CP4}, {CP1, CP3, CP4} is converted to control points {CP1, CP2, CP3}. The conversion method is to substitute the motion vector and coordinate information of the control point into formula (4) to obtain the model parameters, and then substitute the coordinate information of {CP1, CP2, CP3} into formula (5) to obtain its motion vector.

[0317] More directly, the conversion can be performed according to the following formulas (22)-(24), where W represents the width of the current block and H represents the height of the current block. In formulas (13)-(21), (vx0, vy0) represents the motion vector of CP1, (vx1, vy1) represents the motion vector of CP2, (vx2, vy2) represents the motion vector of CP3, and (vx3, vy3) represents the motion vector of CP4.

[0318] The conversion of {CP1, CP2, CP4} to {CP1, CP2, CP3} can be achieved by formula (22):

[0319]

[0320] The conversion of {CP2, CP3, CP4} to {CP1, CP2, CP3} can be achieved by formula (23):

[0321]

[0322] The conversion of {CP1, CP3, CP4} to {CP1, CP2, CP3} can be achieved by formula (24):

[0323]

[0324] 6) Advanced temporal motion vector prediction (ATMVP):

[0325] In the inter-frame prediction of HEVC, all pixels in the current block use the same motion information for motion compensation to obtain the predicted values ​​of the pixels in the block to be processed. However, the pixels in the block to be processed do not necessarily have the same motion characteristics. Therefore, using the same motion information to predict all pixels in the block to be processed may reduce the accuracy of motion compensation, thereby increasing the residual information.

[0326] In order to solve the above problems, the existing solution proposes an advanced temporal motion vector prediction (ATMVP) technology.

[0327] The process of using ATMVP technology to predict mainly includes: Fig. 6E As shown:

[0328] (1) Determine the offset vector of the block to be processed.

[0329] (2) According to the position of the sub-block to be processed in the block to be processed and the above-mentioned offset vector, a corresponding sub-block of the sub-block to be processed is determined in a corresponding target image; the target image is one of the encoded images.

[0330] (3) Determine the motion vector of the current sub-block to be processed based on the motion vector of the corresponding sub-block.

[0331] For example, a scaling method may be used to determine the motion vector of the current sub-block to be processed. For example, the scaling method is implemented by formula (25):

[0332]

[0333] Among them, CPoc represents the POC number of the frame where the block to be processed is located, DPoc represents the POC number of the frame where the corresponding sub-block is located, SrcPoc represents the POC number of the reference frame of the corresponding sub-block, and MV c Represents the scaled motion vector, MV g Indicates the motion vector of the corresponding sub-block.

[0334] (4) Performing motion compensation prediction on the sub-block to be processed according to the motion vector of the sub-block to be processed to obtain a predicted pixel value of the sub-block to be processed.

[0335] 7) Planar Motion Vector Prediction (PLANAR):

[0336] PLANAR obtains the motion information of the upper spatial adjacent position, the left spatial adjacent position, the right and the bottom position of each sub-block to be processed in the block to be processed, calculates the average value, and converts it into the motion information of each sub-block to be processed.

[0337] For the sub-block to be processed with coordinates (x, y), the motion vector P(x, y) of the sub-block to be processed uses the horizontal interpolation motion vector P h (x, y) and the horizontal interpolation motion vector P v (x, y) is calculated by the following formula (26):

[0338] P(x, y) = (H × P h (x, y)+W×P v (x, y)+H×W) / (2×H×W)(26).

[0339] Wherein, H represents the height of the block to be processed, and W represents the width of the block to be processed.

[0340] Horizontal interpolation motion vector P h (x, y and horizontal interpolation motion vector P v (x, y) can be calculated using the motion vectors of the sub-blocks on the left, right, above, and below the current sub-block to be processed by the following formulas (27) and (28):

[0341] P h (x,y)=(W-1-x)×L(-1,y)+(x+1)×R(w,y)(27)

[0342] P v (x, y)=(H-1-y)×A(x,-1)+(y+1)×B(x,H)(28)

[0343] Among them, L(-1,y) represents the motion vector of the left side of the sub-block to be processed, R(w,y) represents the motion vector of the right side of the sub-block to be processed, A(x,-1) represents the motion vector above the sub-block to be processed, and B(x,H) represents the motion vector below the sub-block to be processed.

[0344] The left motion vector L and the upper motion vector A are obtained from the spatial neighboring blocks of the current coding block. The motion vectors L(-1, y) and A(x, -1) of the coding blocks at the preset positions (-1, y) and (x, -1) are obtained according to the coordinates (x, y) of the sub-block to be processed.

[0345] See also Figure 7 As shown, the right motion vector R(w,y) and the lower motion vector B(x,H) can be extracted by the following method:

[0346] (1) Extract the temporal motion information BR of the spatial neighboring position in the lower right corner of the block to be processed;

[0347] (2) The motion vector AR of the upper right spatial neighboring position and the temporal motion information BR of the lower right spatial neighboring position are used for weighted calculation to obtain the right motion vector R(w,y), as shown in formula (29):

[0348] R(w,y)=((Hy-1)*AR+(y+1)*BR) / H(29)

[0349] (3) The motion vector BL of the lower left spatial neighboring position and the temporal motion information BR of the lower right spatial neighboring position are weighted to obtain the lower motion vector B(x,H), as shown in formula (30):

[0350] B(x,H)=((Wx-1)*BL+(x+1)*BR) / H(30)

[0351] It should be noted that the motion vector used in the calculation refers to the motion vector obtained after being scaled to point to the first reference frame in the specific reference frame queue.

[0352] 8) Advanced motion vector prediction mode based on affine motion model (Affine AMVP mode):

[0353] (1) Constructing a list of candidate motion vectors

[0354] A candidate motion vector list of the AMVP mode according to the affine motion model is constructed by using the inherited control point motion vector prediction method and / or the constructed control point motion vector prediction method. In the embodiment of the present application, the candidate motion vector list of the AMVP mode according to the affine motion model may be referred to as a control point motion vector predictor candidate list, and the motion vector predictor of each control point includes the motion vectors of 2 (4-parameter affine motion model) control points or includes the motion vectors of 3 (6-parameter affine motion model) control points.

[0355] Optionally, the control point motion vector prediction value candidate list is pruned and sorted according to specific rules, and may be truncated or padded to a specific number.

[0356] (2) Determine the optimal control point motion vector prediction value

[0357] At the encoding end, the motion vector of each sub-block in the current coding block is obtained by using the motion vector prediction value of each control point in the candidate list of the motion vector prediction value of the control point through formula (3) / (5), and then the pixel value of the corresponding position in the reference frame pointed to by the motion vector of each sub-block is obtained as its prediction value, and motion compensation using the affine motion model is performed. The average value of the difference between the original value and the prediction value of each pixel point in the current coding block is calculated, and the control point motion vector prediction value corresponding to the minimum average value is selected as the optimal control point motion vector prediction value, and is used as the motion vector prediction value of 2 / 3 control points in the current coding block. The index number representing the position of the control point motion vector prediction value in the candidate list of the control point motion vector prediction value is encoded into the bitstream and sent to the decoder.

[0358] At the decoding end, the index number is parsed, and a control point motion vector predictor (CPMVP) is determined from a control point motion vector predictor candidate list according to the index number.

[0359] (3) Determine the motion vector of the control point

[0360] At the encoding end, the control point motion vector prediction value is used as the search starting point to perform motion search within a certain search range to obtain the control point motion vectors (CPMV), and the difference between the control point motion vector and the control point motion vector prediction value (CPMVD) is passed to the decoding end.

[0361] At the decoding end, the control point motion vector difference is analyzed and added to the control point motion vector prediction value to obtain the control point motion vector.

[0362] 9) Sub-block based merging mode:

[0363] A sub-blockbased merging candidate list is constructed by using at least one of the advanced temporal motion vector prediction, the inherited control point motion vector prediction method, the constructed control point motion vector prediction method and the PLANAR prediction method.

[0364] Optionally, the sub-block fusion candidate list is pruned and sorted according to specific rules, and may be truncated or padded to a specific number.

[0365] At the encoding end, if advanced temporal motion vector prediction is used, the motion vector of each sub-block (pixel point or pixel block of size N1×N2 divided by a specific method) is obtained according to the method shown in 7). If PLANAR prediction is used, the motion vector of each sub-block is obtained according to the method shown in 8).

[0366] If the control point motion vector prediction method is inherited or constructed, the motion vector of each sub-block (pixel point or pixel block of size N1×N2 divided by a specific method) in the current block is obtained by formula (3) / (5). After obtaining the motion vector of each sub-block, the pixel value of the position in the reference frame pointed to by the motion vector of each sub-block is obtained as its prediction value, and affine motion compensation is performed. The average value of the difference between the original value and the prediction value of each pixel point in the current coding block is calculated, and the control point motion vector corresponding to the minimum average value of the difference is selected as the motion vector of the 2 / 3 control points of the current coding block. The index number representing the position of the control point motion vector in the candidate list is encoded into the bitstream and sent to the decoder.

[0367] At the decoding end, the index number is parsed, and the control point motion vectors (CPMV) are determined from the control point motion vector fusion candidate list according to the index number.

[0368] In addition, it should be noted that, in the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0369] In the present application, when the inter prediction mode is used to decode the current block, a syntax element may be used to signal the inter prediction mode.

[0370] Part of the syntax structure currently used to parse the inter-frame prediction mode used by the current block can be shown in Table 1. It should be noted that the syntax elements in the syntax structure can also be represented by other identifiers, which are not specifically limited in this application.

[0371] Table 1

[0372]

[0373]

[0374]

[0375] The variable treeType indicates the coding tree type used for encoding the current block.

[0376] The variable slice_type is used to indicate the type of the slice where the current block is located, such as P type, B type or I type.

[0377] The syntax element cu_skip_flag[x0][y0] can be used to indicate whether the current block has a residual. For example, cu_skip_flag[x0][y0]=1 indicates that the current block has a residual, and cu_skip_flag[x0][y0]=0 indicates that the current block has no residual.

[0378] It should be noted that the skip mode is a special case of the merge mode. After the merge mode finds the motion vector, if the encoder determines that the current block and the reference block are basically the same based on some method, then there is no need to transmit the residual data, only the index of the motion vector and a cu_skip_flag need to be transmitted. Therefore, if cu_skip_flag[x0][y0]=0, it indicates that the current block has no residual and does not need to parse the residual data.

[0379] The syntax element pred_mode_flag[x0][y0] is used to indicate whether the prediction mode of the current block is inter-frame prediction or intra-frame prediction.

[0380] The variable CuPredMode[x0][y0] is determined by pre_mode_falg[x0][y0]. MODE_INTRA indicates intra prediction.

[0381] The syntax element merge_flag[x0][y0] can be used to indicate whether the merge mode is used for the current block. For example, when merge_flag[x0][y0]=1, it indicates that the merge mode is used for the current block, and when merge_flag[x0][y0]=0, it indicates that the merge mode is not used for the current block. x0, y0 represent the coordinates of the current block in the video image.

[0382] cbWidth represents the width of the current block, and cbHeight represents the height of the current block.

[0383] The syntax element sps_affine_enabled_flag can be used to indicate whether an affine motion model can be used when inter-frame prediction is performed on image blocks included in a video image. For example, sps_affine_enabled_flag = 0 indicates that an affine motion model cannot be used when inter-frame prediction is performed on image blocks included in a video image, and sps_affine_enabled_flag = 1 indicates that an affine motion model can be used when inter-frame prediction is performed on image blocks included in a video image.

[0384] The syntax element merge_subblock_flag[x0][y0] may be used to indicate whether the merge mode based on subblocks is used for the current block. The type (slice_type) of the slice where the current block is located is P type or B type. For example, merge_subblock_flag[x0][y0]=1 indicates that the merge mode based on subblocks is used for the current block, and merge_subblock_flag[x0][y0]=0 indicates that the merge mode based on subblocks is not used for the current block, but the merge mode of the translation motion model may be used.

[0385] The syntax element merge_idx[x0][y0] may be used to indicate an index value for the merge candidate list.

[0386] The syntax element merge_subblock_idx[x0][y0] may be used to indicate an index value for a merge candidate list according to a sub-block.

[0387] sps_sbtmvp_enabled_flag can be used to indicate whether ATMVP can be used when inter-frame prediction is performed on image blocks included in a video image. For example, sps_sbtmvp_enabled_flag = 1 indicates that ATMVP can be used when inter-frame prediction is performed on image blocks included in a video image, and sps_sbtmvp_enabled_flag = 0 indicates that ATMVP cannot be used when inter-frame prediction is performed on image blocks included in a video image.

[0388] The syntax element inter_affine_flag[x0][y0] may be used to indicate whether the AMVP mode based on the affine motion model is used for the current block when the slice where the current block is located is a P-type slice or a B-type slice. For example, inter_affine_flag[x0][y0]=0 indicates that the AMVP mode based on the affine motion model is used for the current block, and inter_affine_flag[x0][y0]=1 indicates that the AMVP mode based on the affine motion model is not used for the current block, and the AMVP mode of the translational motion model may be used.

[0389] The syntax element cu_affine_type_flag[x0][y0] can be used to indicate whether to use the 6-parameter affine motion model for motion compensation for the current block when the slice where the current block is located is a P-type slice or a B-type slice. cu_affine_type_flag[x0][y0] = 0 indicates that the 6-parameter affine motion model is not used for motion compensation for the current block, and only the 4-parameter affine motion model can be used for motion compensation; cu_affine_type_flag[x0][y0] = 1 indicates that the 6-parameter affine motion model is used for motion compensation for the current block.

[0390] As shown in Table 2, MotionModelIdc[x0][y0]=1 indicates that a 4-parameter affine motion model is used, MotionModelIdc[x0][y0]=2 indicates that a 6-parameter affine motion model is used, and MotionModelIdc[x0][y0]=0 indicates that a translational motion model is used.

[0391] Table 2

[0392] MotionModelIdc[x0][y0] motion model for motion compensation 0 Translational motion 1 4-parameter affine motion 2 6-parameter affine motion

[0393] The variable MaxNumMergeCand is used to indicate the maximum length of the merge candidate motion vector list, and MaxNumSubblockMergeCand is used to indicate the maximum length of the merge candidate motion vector list based on the sub-block. inter_pred_idc[x0][y0] is used to indicate the prediction direction. PRED_L0 indicates forward prediction. num_ref_idx_l0_active_minus1 indicates the number of reference frames in the forward reference frame list, and ref_idx_l0[x0][y0] indicates the forward reference frame index value of the current block. mvd_coding(x0,y0,0,0) indicates the first motion vector difference. mvp_l0_flag[x0][y0] indicates the forward MVP candidate list index value. PRED_L1 is used to indicate backward prediction. num_ref_idx_l1_active_minus1 indicates the number of reference frames in the backward reference frame list. ref_idx_l1[x0][y0] indicates the backward reference frame index value of the current block, and mvp_l1_flag[x0][y0] indicates the backward MVP candidate list index value.

[0394] In Table 1, ae(v) represents a syntax element encoded according to context-based adaptive binary arithmetic coding (cabac).

[0395] The following is a detailed description of the inter-frame prediction process. Fig. 8A shown.

[0396] Step 801: According to the syntax structure shown in Table 1, parse the bitstream and determine the inter-frame prediction mode of the current block.

[0397] If it is determined that the inter prediction mode of the current block is the AMVP mode based on the affine motion model, step 802a is executed.

[0398] That is, the syntax elements sps_affine_enabled_flag=1, merge_flag=0, and inter_affine_flag=1 indicate that the inter prediction mode of the current block is the AMVP mode according to the affine motion model.

[0399] If it is determined that the inter-frame prediction mode of the current block is a sub-block merge mode, step 802b is executed.

[0400] That is, one of the syntax elements sps_affine_enabled_flag or sps_sbtmvp_enabled_flag is equal to 1, merge_flag=1, and merge_subblock_flag=1, indicating that the inter prediction mode of the current block is the sub-block merging mode.

[0401] Step 802a: Construct a candidate motion vector list corresponding to the AMVP mode according to the affine motion model, and execute step 803a.

[0402] The candidate control point motion vectors of the current block are derived by using the inherited control point motion vector prediction method and / or the constructed control point motion vector prediction method to be added to the candidate motion vector list.

[0403] The candidate motion vector list may include a binary list (the current coding block is a 4-parameter affine motion model) or a triple list. The binary list includes one or more binary lists for constructing a 4-parameter affine motion model. The triple list includes one or more triplets for constructing a 6-parameter affine motion model.

[0404] Optionally, the candidate motion vector binary / triplet list is pruned and sorted according to specific rules, and may be truncated or padded to a specific number.

[0405] A1: The process of constructing a candidate motion vector list using the inherited control motion vector prediction method is described.

[0406] by Figure 4 As shown in the example, for example, according to Figure 4 The adjacent blocks around the current block are traversed in the order of A1->B1->B0->A0->B2, and the affine coding block where the adjacent block is located is found, and the control point motion information of the affine coding block is obtained. Then, the motion model is constructed through the control point motion information of the affine coding block, and the candidate control point motion information of the current block is derived. For details, please refer to the relevant description of the control point motion vector prediction method inherited from 3) above, which will not be repeated here.

[0407] Exemplarily, the affine motion model used in the current block is a 4-parameter affine motion model (i.e., MotionModelIdc=1). If the adjacent affine decoding block is a 4-parameter affine motion model, the motion vectors of the two control points of the affine decoding block are obtained: the motion vector value (vx4, vy4) of the upper left control point (x4, y4) and the motion vector value (vx5, vy5) of the upper right control point (x5, y5). The affine decoding block is an affine coding block predicted by the affine motion model in the encoding stage.

[0408] Using the 4-parameter affine motion model composed of 2 control points of adjacent affine decoding blocks, the motion vectors of the 2 control points at the upper left and upper right of the current block are derived according to the 4-parameter affine motion model formulas (6) and (7).

[0409] If the adjacent affine decoding block is a 6-parameter affine motion model, the motion vectors of the three control points of the adjacent affine decoding block are obtained, such as Figure 4 In the figure, the motion vector value (vx4,vy4) of the upper left control point (x4,y4), the motion vector value (vx5,vy5) of the upper right control point (x5,y5), and the motion vector (vx6,vy6) of the lower left vertex (x6,y6).

[0410] Using the 6-parameter affine motion model composed of 3 control points of adjacent affine decoding blocks, the motion vectors of the upper left and upper right control points of the current block are derived according to the 6-parameter motion model formulas (8) and (9).

[0411] Exemplarily, the affine motion model of the current decoding block is a 6-parameter affine motion model (ie, MotionModelIdc=2),

[0412] If the affine motion model used by the adjacent affine decoding block is a 6-parameter affine motion model, the motion vectors of the three control points of the adjacent affine decoding block are obtained, such as Figure 4 In the figure, the motion vector value (vx4,vy4) of the upper left control point (x4,y4), the motion vector value (vx5,vy5) of the upper right control point (x5,y5), and the motion vector (vx6,vy6) of the lower left vertex (x6,y6).

[0413] Using the 6-parameter affine motion model composed of 3 control points of adjacent affine decoding blocks, the motion vectors of the top left, top right and bottom left control points of the current block are derived according to formulas (8), (9) and (10) corresponding to the 6-parameter affine motion model.

[0414] If the affine motion model adopted by the adjacent affine decoding block is a 4-parameter affine motion model, the motion vectors of the two control points of the affine decoding block are obtained: the motion vector value (vx4, vy4) of the upper left control point (x4, y4) and the motion vector value (vx5, vy5) of the upper right control point (x5, y5).

[0415] Using the 4-parameter affine motion model composed of 2 control points of adjacent affine decoding blocks, the motion vectors of the top left, top right and bottom left control points of the current block are derived according to the 4-parameter affine motion model formulas (6) and (7).

[0416] It should be noted that other motion models, candidate positions, and search orders may also be applicable to this application, which will not be described in detail here. It should be noted that methods using other control points to represent motion models of adjacent and current coding blocks may also be applicable to this application, which will not be described in detail here.

[0417] A2: The process of constructing a candidate motion vector list using the constructed controlled motion vector prediction method is described.

[0418] Exemplarily, the affine motion model used by the current decoding block is a 4-parameter affine motion model (i.e., MotionModelIdc is 1), and the motion information of the coded blocks adjacent to the current coding block is used to determine the motion vectors of the upper left vertex and the upper right vertex of the current coding block. Specifically, the constructed control point motion vector prediction method 1 or the constructed control point motion vector prediction method 2 can be used to construct a candidate motion vector list. For specific methods, see the description in 4) and 5) above, which will not be repeated here.

[0419] Exemplarily, the current decoding block affine motion model is a 6-parameter affine motion model (i.e., MotionModelIdc is 2), and the motion information of the adjacent coded blocks around the current coding block is used to determine the motion vectors of the upper left vertex, the upper right vertex, and the lower left vertex of the current coding block. Specifically, the constructed control point motion vector prediction method 1 or the constructed control point motion vector prediction method 2 can be used to construct a candidate motion vector list. For specific methods, see the description in 4) and 5) above, which will not be repeated here.

[0420] It should be noted that other control point motion information combination methods may also be applicable to the present application and will not be described in detail here.

[0421] Step 803a: parse the bitstream, determine the optimal control point motion vector prediction value, and execute step 804a.

[0422] B1, if the affine motion model used by the current decoding block is a 4-parameter affine motion model (MotionModelIdc is 1), the index number is parsed, and the optimal motion vector prediction value of the two control points is determined from the candidate motion vector list according to the index number.

[0423] Exemplarily, the index number is mvp_l0_flag or mvp_l1_flag.

[0424] B2: If the affine motion model used by the current decoding block is a 6-parameter affine motion model (MotionModelIdc is 2), the index number is parsed, and the optimal motion vector prediction value of the three control points is determined from the candidate motion vector list according to the index number.

[0425] Step 804a: parse the code stream to determine the motion vector of the control point.

[0426] C1, the affine motion model used by the current decoding block is a 4-parameter affine motion model (MotionModelIdc is 1). The motion vector difference of the two control points of the current block is obtained by decoding from the bitstream, and the motion vector value of the control point is obtained according to the motion vector difference and motion vector prediction value of each control point. Taking forward prediction as an example, the motion vector difference of the two control points is mvd_coding(x0,y0,0,0) and mvd_coding(x0,y0,0,1).

[0427] Exemplarily, the motion vector differences of the upper left position control point and the upper right position control point are decoded from the bitstream and added to the motion vector prediction values ​​to obtain the motion vector values ​​of the upper left position control point and the upper right position control point of the current block.

[0428] C2, the current decoded block affine motion model is a 6-parameter affine motion model (MotionModelIdc is 2)

[0429] The motion vector differences of the three control points of the current block are obtained by decoding the bitstream, and the motion vector values ​​of the control points are obtained according to the motion vector differences and motion vector prediction values ​​of each control point. Taking forward prediction as an example, the motion vector differences of the three control points are mvd_coding(x0,y0,0,0), mvd_coding(x0,y0,0,1), and mvd_coding(x0,y0,0,2).

[0430] Exemplarily, the motion vector differences of the upper left control point, upper right control point and lower left control point are decoded from the bitstream and added to the motion vector prediction values ​​respectively to obtain the motion vector values ​​of the upper left control point, upper right control point and lower left control point of the current block.

[0431] Step 805a: Obtain the motion vector value of each sub-block in the current block according to the motion information of the control point and the affine motion model adopted by the current decoding block.

[0432] For each sub-block of the current affine decoding block (a sub-block can also be equivalent to a motion compensation unit, and the width and height of the sub-block are smaller than the width and height of the current block), the motion information of the preset position pixel in the motion compensation unit can be used to represent the motion information of all pixels in the motion compensation unit. Assuming that the size of the motion compensation unit is MxN, the preset position pixel can be the center point of the motion compensation unit (M / 2, N / 2), the upper left pixel (0, 0), the upper right pixel (M-1, 0), or a pixel at another position. The following takes the center point of the motion compensation unit as an example, see Figure 8C shown. Figure 8CV0 represents the motion vector of the upper left control point, and V1 represents the motion vector of the upper right control point. Each small box represents a motion compensation unit.

[0433] The coordinates of the center point of the motion compensation unit relative to the upper left vertex pixel of the current affine decoding block are calculated using formula (25), where i is the i-th motion compensation unit in the horizontal direction (from left to right), j is the j-th motion compensation unit in the vertical direction (from top to bottom), (x (i,j) ,y (i,j) ) represents the coordinates of the center point of the (i, j)th motion compensation unit relative to the upper left control point pixel of the current affine decoding block.

[0434] If the affine motion model used by the current affine decoding block is a 6-parameter affine motion model, (x (i,j) ,y (i,j) ) is substituted into the 6-parameter affine motion model formula (26) to obtain the motion vector of the center point of each motion compensation unit as the motion vector of all pixels in the motion compensation unit (vx (i,j) ,vy (i,j) ).

[0435] If the affine motion model used by the current affine decoding block is the 4-affine motion model, (x (i,j) ,y (i,j) ) is substituted into the 4-parameter affine motion model formula (27) to obtain the motion vector of the center point of each motion compensation unit as the motion vector of all pixels in the motion compensation unit (vx (i,j) ,vy (i,j) ).

[0436]

[0437]

[0438] Step 806a: Perform motion compensation on each sub-block according to the determined motion vector value of the sub-block to obtain a pixel prediction value of the sub-block.

[0439] Step 802b: construct a motion information candidate list of the sub-block merge mode.

[0440] Specifically, one or more of the advanced temporal motion vector prediction, the inherited control point motion vector prediction method, the constructed control point motion vector prediction method or the PLANAR method may be used to construct a sub-block based merging candidate list. In the embodiment of the present application, the sub-block based merging candidate list may be referred to as a sub-block merging candidate list.

[0441] Optionally, the motion information candidate list is pruned and sorted according to specific rules, and may be truncated or padded to a specific number.

[0442] D0: If sps_sbtmvp_enabled_flag is 1, use ATMVP to construct candidate motion vectors and add them to the sub-block fusion candidate list, see 6).

[0443] D1: If sps_affine_enabled_flag is 1, the inherited control point motion vector prediction method is used to derive the candidate control point motion information of the current block and add it to the sub-block fusion candidate list. See 3)

[0444] For example, according to Figure 6C The surrounding adjacent position blocks of the current block are traversed in the order of A1, B1, B0, A0, B2 to find the affine coding block at the position, obtain the control point motion information of the affine coding block, and then derive the candidate control point motion information of the current block through its motion model.

[0445] If the sub-block fusion candidate list is empty at this time, the candidate control point motion information is added to the sub-block fusion candidate list; otherwise, the motion information in the sub-block fusion candidate list is traversed in turn to check whether there is motion information in the sub-block fusion candidate list that is the same as the candidate control point motion information. If there is no motion information in the sub-block fusion candidate list that is the same as the candidate control point motion information, the candidate control point motion information is added to the sub-block fusion candidate list.

[0446] To determine whether two candidate motion information are the same, it is necessary to determine in turn whether their forward and backward reference frames, and the horizontal and vertical components of each forward and backward motion vector are the same. Only when all the above elements are different, the two motion information are considered different.

[0447] If the number of motion information in the sub-block fusion candidate list reaches the maximum list length MaxNumSubblockMergeCand (MaxNumSubblockMergeCand is a positive integer, such as 1, 2, 3, 4, 5, etc., 5 is used as an example for description below, and no further details are given), the candidate list is constructed, otherwise the next adjacent position block is traversed.

[0448] D2: If sps_affine_enabled_flag is 1, the constructed control point motion vector prediction method is used to derive the candidate control point motion information of the current block and add it to the sub-block fusion candidate list, see Figure 8B shown.

[0449] Step 801c, obtaining motion information of each control point of the current block. For example, see step 601 in the control point motion vector prediction method 2 constructed in 5), which will not be described in detail here.

[0450] Step 802c, combining the motion information of the control points to obtain constructed motion information of the control points, which can be seen in Fig.6D Step 602 is not described again here.

[0451] Step 803c: Add the constructed control point motion information to the sub-block fusion candidate list.

[0452] If the length of the sub-block fusion candidate list is less than the maximum list length MaxNumSubblockMergeCand, then these combinations are traversed in a preset order to obtain a legal combination as the candidate control point motion information. If the sub-block fusion candidate list is empty, the candidate control point motion information is added to the sub-block fusion candidate list; otherwise, the motion information in the candidate motion vector list is traversed in turn to check whether there is the same motion information as the candidate control point motion information in the sub-block fusion candidate list. If there is no motion information in the sub-block fusion candidate list that is the same as the candidate control point motion information, the candidate control point motion information is added to the sub-block fusion candidate list.

[0453] Exemplarily, a preset order is as follows: Affine(CP1,CP2,CP3)->Affine(CP1,CP2,CP4)->Affine(CP1,CP3,CP4)->Affine(CP2,CP3,CP4)->Affine(CP1,CP2)->Affine(CP1,CP3), a total of 6 combinations.

[0454] Exemplarily, if sps_affine_type_flag is 1, a preset order is as follows: Affine(CP1, CP2, CP3)->Affine(CP1, CP2, CP4)->Affine(CP1, CP3, CP4)->Affine(CP2, CP3, CP4)->Affine(CP1, CP2)->Affine(CP1, CP3), a total of 6 combinations. The embodiment of the present application does not specifically limit the order in which the 6 combinations are added to the candidate motion vector list.

[0455] If sps_affine_type_flag is 0, a preset order is as follows: Affine (CP1, CP2) -> Affine (CP1, CP3), a total of 2 combinations. The embodiment of the present application does not specifically limit the order of adding the 2 combinations to the candidate motion vector list.

[0456] If the motion information of the control points corresponding to the combination is not available, the combination is considered unavailable. If the combination is available, determine the reference frame index of the combination (when there are two control points, select the reference frame index with the smallest index as the reference frame index of the combination; when there are more than two control points, first select the reference frame index with the largest number of occurrences. If there are multiple reference frame indexes with the same number of occurrences, select the reference frame index with the smallest index as the reference frame index of the combination), and scale the motion vector of the control point. If the motion information of all control points after scaling is consistent, the combination is illegal.

[0457] D3: Optional, if sps_planar_enabled_flag is 1, the motion information constructed using ATMVP is added to the candidate motion information list, see 7).

[0458] Optionally, the embodiment of the present application may also fill the candidate motion vector list. For example, after the above-mentioned traversal process, if the length of the candidate motion vector list is less than the maximum list length MaxNumSubblockMergeCand, the candidate motion vector list may be filled until the length of the list is equal to MaxNumSubblockMergeCand.

[0459] It can be filled by adding zero motion vectors, or by combining and weighted averaging the candidate motion information already in the existing list. It should be noted that other methods for obtaining candidate motion vector list filling are also applicable to this application and will not be described in detail here.

[0460] Step 803b: Analyze the code stream to determine the optimal control point motion information.

[0461] Parse the index number merge_subblock_idx and determine the optimal motion information from the sub-block fusion candidate list according to the index number.

[0462] The binarization of merge_subblock_idx usually adopts TR code (truncated unary), that is, it is mapped to different binary numbers according to the maximum index value. The maximum index value is pre-configured or transmitted. For example, if the maximum index value is 4, it is binarized according to the following Table 2.

[0463] Table 2

[0464]

[0465] When the maximum index value is 2, binarization is performed according to Table 3 below.

[0466] Table 3

[0467]

[0468]

[0469] merge_subblock_idx is transmitted in binary format, and the decoder can determine the index number through the maximum index value and Table 2 or Table 3. For example, if the maximum index value is 4, when decoding the index number, if 0 is encountered or the index number is equal to the maximum index value, the index number is determined. For example, when decoding the index number, if the first bit is 0, the index number is stopped from being decoded, that is, the index number is determined to be 0. For another example, if the first bit is 1 and the second bit is 0, the index number is stopped from being decoded, that is, the index number is determined to be 1.

[0470] The embodiment of the present application does not specifically limit the setting of the maximum index value and the binarization table.

[0471] Step 804b: If the optimal motion information is ATMVP or PLANR motion information, the motion vector of each sub-block is obtained directly by the ATMVP or PLANR method.

[0472] If the optimal motion information is an affine mode, the motion vector value of each sub-block in the current block is obtained according to the optimal control point motion information and the affine motion model adopted by the current decoding block. This is the same as step 805a.

[0473] Step 805b: Perform motion compensation on each sub-block according to the determined motion vector value of the sub-block to obtain a pixel prediction value of the sub-block.

[0474] There is currently no feasible way to determine the maximum length (MaxNumSubblockMergeCand) of the candidate motion vector list of the sub-block merging mode.

[0475] Based on this, the embodiment of the present application provides a video image prediction method and device, providing a method for determining the maximum length (MaxNumSubblockMergeCand) of the candidate motion vector list of the sub-block fusion mode. Among them, the method and the device are based on the same invention concept, and since the principles of solving the problem by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0476] The following describes in detail the method for determining the maximum length of the candidate motion vector list of the sub-block merging mode in two cases.

[0477] The first case is:

[0478] The sub-block merging mode may include at least one of an affine mode, an advanced temporal motion vector prediction mode, and a planar motion vector prediction mode.

[0479] The second case is that the sub-block fusion mode does not consider the existence of the planar motion vector prediction mode, that is, the sub-block fusion mode may include at least one of the affine mode and the advanced temporal motion vector prediction mode.

[0480] The implementation of the present application is described in detail from the decoding side in conjunction with the accompanying drawings. Specifically, it can be performed by the video decoder 30, or by the motion compensation module in the video decoder, or by the processor.

[0481] For the first case, several possible implementation modes are illustrated, including the first implementation mode to the fifth implementation mode.

[0482] See also Fig. 9 As shown, the first embodiment is described.

[0483] Step 901: parse a first identifier from a bitstream. Execute S902 or S904.

[0484] The first flag is used to indicate whether the candidate modes used for inter-frame prediction of the processed block include the affine mode. In other words, flag 1 is used to indicate whether the affine mode can be (or is allowed to be) used when motion compensation is performed on the processed block.

[0485] Exemplarily, the first identifier may be configured in the SPS of the code stream. Based on this, parsing the first identifier from the code stream may be implemented in the following manner: parsing identifier 1 from the SPS of the code stream. Alternatively, the first identifier may also be configured in the stripe header of the stripe where the block to be processed of the code stream is located. Based on this, parsing the first identifier from the code stream may be implemented in the following manner: parsing the first identifier from the stripe header of the stripe where the block to be processed of the code stream is located.

[0486] Exemplarily, the first flag may be represented by a syntax element sps_affine_enabled_flag. sps_affine_enabled_flag is used to indicate whether an affine mode can be used when inter-frame prediction is performed on image blocks included in a video image. For example, sps_affine_enabled_flag = 0 indicates that an affine mode cannot be used when inter-frame prediction is performed on image blocks included in a video image. sps_affine_enabled_flag = 1 indicates that an affine motion model can be used when inter-frame prediction is performed on image blocks included in a video image.

[0487] S902, when the first identifier indicates that the candidate mode used for inter-frame prediction of the block to be processed includes an affine mode, a second identifier is parsed from the code stream, the second identifier is used to indicate (or determine) the maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode. The first candidate motion vector list can be called MaxNumSubblockMergeCand.

[0488] Exemplarily, the second identifier can be configured in the SPS or PPS or the slice header. Based on this, the parsing of the second identifier from the code stream can be achieved in the following manner: parsing the second identifier from the sequence parameter set of the code stream, or parsing the second identifier from the slice header of the slice where the block to be processed of the code stream is located.

[0489] Exemplarily, the second flag may be represented by K_minus_max_num_subblock_merge_cand.

[0490] Exemplarily, the allowed value range of K_minus_max_num_subblock_merge_cand is 0-4.

[0491] Exemplarily, the maximum value allowed for MaxNumSubblockMergeCand is 5.

[0492] When the maximum value allowed by MaxNumSubblockMergeCand is 5, the second flag may be represented by five_minus_max_num_subblock_merge_cand.

[0493] See Table 4, which illustrates a grammatical structure for parsing the second identifier.

[0494] Table 4

[0495]

[0496] S903: Determine a maximum length of the first candidate motion vector list according to the second identifier.

[0497] As an example, when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, the maximum length of the first candidate motion vector list (MaxNumSubblockMergeCand) can be obtained according to the following formula: MaxNumSubblockMergeCand = K-K_minus_max_num_subblock_merge_cand.

[0498] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second flag, and K is a preset non-negative integer.

[0499] S904, when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed only includes the translational motion vector prediction mode, and when the third identifier is used to indicate that the ATMVP exists in the sub-block fusion prediction mode, a third quantity value is determined according to the third identifier, and the maximum length of the first candidate motion vector list is determined according to the third quantity value.

[0500] The candidate mode used for inter-frame prediction of the processed block only includes the translational motion vector prediction mode, that is, the candidate mode used for inter-frame prediction of the processed block cannot (is allowed to) use the affine mode. Among them, the third identifier is used to indicate whether the ATMVP exists in the sub-block fusion prediction mode. In other words, the third identifier is used to indicate whether the ATMVP is allowed to be used for inter-frame prediction of the processed block. The third identifier can be configured in the SPS or PPS or slice header.

[0501] In one example, when the first identifier indicates that the candidate modes used for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, and the third identifier is used to indicate that the ATMVP exists in the sub-block fusion prediction mode, the maximum length of the first candidate motion vector list (MaxNumSubblockMergeCand) is equal to the third quantity value.

[0502] Exemplarily, the third flag may be represented by sps_sbtmvp_enabled_flag. For example, when sps_sbtmvp_enabled_flag is a first value, it indicates that ATMVP does not exist in the sub-block fusion prediction mode, and when sps_sbtmvp_enabled_flag is a second value, it indicates that ATMVP exists in the sub-block fusion prediction mode. For example, the first value is 0 and the second value is 1.

[0503] Exemplarily, the third quantity value can be used to indicate the maximum number of motion vectors supported for prediction by ATMVP. For example, when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed only includes the translational motion vector prediction mode, when sps_sbtmvp_enabled_flag=0, the third quantity value is 0, and when sps_sbtmvp_enabled_flag=1, the third quantity value is the maximum number of motion vectors supported for prediction by ATMVP. In addition, when the maximum number of motion vectors supported for prediction by ATMVP is 1, the third quantity value can be equal to the value of sps_sbtmvp_enabled_flag, for example, if sps_sbtmvp_enabled_flag=0, the third quantity value=0, and if sps_sbtmvp_enabled_flag=1, the third quantity value=1.

[0504] As an example, the maximum value allowed by MaxNumSubblockMergeCand is 5. If sps_affine_enable_flag is 0, MaxNumSubblockMergeCand is obtained by the following formula:

[0505] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag.

[0506] If sps_affine_enable_flag is 1, MaxNumSubblockMergeCand is obtained by the following formula:

[0507] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand.

[0508] For example, five_minus_max_num_subblock_merge_cand can be defined as 5 minus the maximum length of the subblock merging motion vector prediction list in the slice (five_minus_max_num_subblock_merge_cand specifies the maximum number of sublock merging motion vector prediction (MVP) candidates supported in the slice subtracted from 5).

[0509] The maximum number of subblock merging MVP candidates,MaxNumSubblockMergeCand is derived as follows:

[0510] -If sps_affine_enabled_flag is equal to 0:

[0511] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag;

[0512] -Otherwise(sps_affine_enabled_flag is equal to 1):

[0513] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand;

[0514] The value of MaxNumSubblockMergeCand shall be in the range of 0 to 5, inclusive.

[0515] See also Fig.10 As shown, the second implementation mode is described below.

[0516] S1001, see S901, which will not be described in detail here. Execute S1002 or S1004.

[0517] S1002, when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, the second identifier and the third identifier are parsed from the code stream, the second identifier is used to indicate (or determine) the maximum length of the first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode.

[0518] S1003, when the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, determine a first quantity value according to the third identifier, and determine the maximum length of the first candidate motion vector list according to the second identifier and the first quantity value.

[0519] For explanations of the first, second, and third identifiers, see Fig. 9 The corresponding embodiments will not be described in detail here.

[0520] When the third identifier indicates that ATMVP does not exist in the sub-block fusion prediction mode, the first quantity value may be equal to the maximum number of motion vectors supported for prediction by ATMVP. Taking the example that the third identifier can be represented by sps_sbtmvp_enabled_flag, when sps_sbtmvp_enabled_flag=0, it indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the first quantity value is the maximum number of motion vectors supported for prediction by ATMVP. On the contrary, when sps_sbtmvp_enabled_flag=1, it indicates that the advanced time domain motion vector prediction mode exists in the sub-block fusion prediction mode, and the first quantity value is 0. Exemplarily, the maximum number of motion vectors supported for prediction by ATMVP may be 1. In this case, the first quantity value may be equal to the value of the third identifier. For details, see Fig. 9 The description of the corresponding embodiments will not be repeated here.

[0521] In a possible example, when the first identifier indicates that the candidate mode adopted by the to-be-processed block for inter-frame prediction includes an affine mode, the maximum length of the first candidate motion vector list may be obtained according to the following formula:

[0522] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1;

[0523] Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second identifier, L1 represents the first quantity value, and K is a preset non-negative integer.

[0524] Exemplarily, the allowed value range of K_minus_max_num_subblock_merge_cand may be 0-3.

[0525] Exemplarily, the maximum value allowed for MaxNumSubblockMergeCand is 5.

[0526] When the maximum value allowed by MaxNumSubblockMergeCand is 5, the second flag may be represented by five_minus_max_num_subblock_merge_cand.

[0527] In one example, L1 can be obtained by the following formula:

[0528] L1=sps_sbtmvp_enabled_flag==1?0:1. If sps_sbtmvp_enabled_flag=1, then L1=0. If sps_sbtmvp_enabled_flag=0, then L1=1.

[0529] S1004, see S904, which will not be described again here.

[0530] As an example, the maximum value allowed by MaxNumSubblockMergeCand is 5. If sps_affine_enable_flag is 0, MaxNumSubblockMergeCand is obtained by the following formula:

[0531] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag.

[0532] If sps_affine_enable_flag is 1, MaxNumSubblockMergeCand is obtained by the following formula:

[0533] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand-(sps_sbtmvp_enabled_flag==1?0:1).

[0534] For example, five_minus_max_num_subblock_merge_cand can be defined as 5 minus the maximum length of the subblock merging motion vector prediction list in the slice (five_minus_max_num_subblock_merge_cand specifies the maximum number of sublock merging motion vector prediction (MVP) candidates supported in the slice subtracted from 5). The maximum number of subblock merging MVP candidates, MaxNumSubblockMergeCand is derived as follows:

[0535] -If sps_affine_enabled_flag is equal to 0:

[0536] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag;

[0537] -Otherwise(sps_affine_enabled_flag is equal to 1):

[0538] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand-(sps_sbtmvp_enabled_flag==1?0:1);

[0539] The value of MaxNumSubblockMergeCand shall be in the range of 0 to 5, inclusive.

[0540] See also Fig.11 As shown, the third implementation mode is described below.

[0541] S1101-S1103, see S901-S903, which will not be repeated here.

[0542] It should be noted that, in the third implementation, the maximum value allowed by MaxNumSubblockMergeCand may be 5. Exemplarily, the second flag may be represented by K_minus_max_num_subblock_merge_cand, and the allowed value range of K_minus_max_num_subblock_merge_cand is 0-4. When the maximum value allowed by MaxNumSubblockMergeCand is 5, the second flag may be represented by five_minus_max_num_subblock_merge_cand.

[0543] S1104: When the first identifier indicates that the candidate mode used for inter-frame prediction of the block to be processed only includes the translational motion vector prediction mode, the third identifier and the fourth identifier are parsed from the bitstream, and S1105 or S1106 or S1107 are executed.

[0544] The third identifier is used to indicate the existence state of the ATMVP mode in the sub-block fusion prediction mode. Fig. 9 The description in the corresponding embodiments will not be repeated here.

[0545] The fourth flag is used to indicate the existence of the translational motion vector prediction (PLANAR) mode in the sub-block fusion prediction mode. In other words, the fourth flag is used to indicate whether the PLANAR mode is allowed to be used for inter-frame prediction of the block to be processed.

[0546] S1105, when the third identifier indicates the existence of the ATMVP mode in the sub-block fusion prediction mode, and the fourth identifier indicates the non-existence of PLANAR in the sub-block fusion prediction mode, determine a third quantity value according to the third identifier, and determine the maximum length of the first candidate motion vector list only according to the third quantity value.

[0547] Exemplarily, when the fourth identifier is the third value, it indicates that the PLANAR mode does not exist in the sub-block fusion prediction mode; when the fourth identifier is the fourth value, it indicates that the PLANAR mode exists in the sub-block fusion prediction mode. For example, the third value is 0 and the fourth value is 1. Exemplarily, the fourth identifier can be configured in the SPS or PPS or slice header. The fourth identifier can be represented by sps_planar_enabled_flag.

[0548] S1106, when the third identifier indicates that the ATMVP mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the PLANAR mode exists in the sub-block fusion prediction mode, a fourth quantity value is determined according to the fourth identifier, and the maximum length of the first candidate motion vector list is determined according to the fourth quantity value.

[0549] Exemplarily, when the fourth identifier indicates that PLANAR exists in the sub-block fusion prediction mode, the fourth quantity value is the maximum number of motion vectors supported for prediction by the PLANAR mode.

[0550] Exemplarily, in S1106, the maximum length of the first candidate motion vector list is equal to the fourth quantity value. For example, the maximum number of motion vectors supported by the PLANAR mode for prediction is 1, and the maximum length of the first candidate motion vector list is 1. For another example, when the fourth identifier is 1, it indicates that the PLANAR mode exists in the sub-block fusion prediction mode, and the maximum length of the first candidate motion vector list is equal to the fourth identifier.

[0551] S1107, when the third identifier indicates that the ATMVP mode exists in the sub-block fusion prediction mode, and the fourth identifier indicates that PLANAR exists in the sub-block fusion prediction mode, a third quantity value is determined according to the third identifier and a fourth quantity value is determined according to the fourth identifier, and the maximum length of the first candidate motion vector list is determined according to the third quantity value and the fourth quantity value.

[0552] As an example, when the first identifier indicates that the candidate modes adopted for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, taking the maximum number of motion vectors supported for prediction by the PLANAR mode as 1 and the maximum number of motion vectors supported for prediction by the ATMVP mode as 1 as an example, when the third identifier is 1, it indicates that the ATMVP mode exists in the sub-block fusion prediction mode; when the third identifier is 0, it indicates that the ATMVP mode does not exist in the sub-block fusion prediction mode; when the fourth identifier is 1, it indicates the existence of the PLANAR mode in the sub-block fusion prediction mode; when the fourth identifier is 0, it indicates the existence of the PLANAR mode in the sub-block fusion prediction mode; at this time, the maximum length of the first candidate motion vector list can be equal to the sum of the third identifier and the fourth identifier.

[0553] With the third flag represented by sps_sbtmvp_enabled_flag and the fourth flag represented by sps_planar_enabled_flag, the maximum length of the first candidate motion vector list can be obtained by the following formula:

[0554] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

[0555] When sps_sbtmvp_enabled_flag = 1 (indicating that the ATMVP mode exists in the sub-block fusion prediction mode), sps_planar_enabled_flag = 0 (indicating that the planar mode does not exist in the sub-block fusion prediction mode), MaxNumSubblockMergeCand = sps_sbtmvp_enabled_flag = 1, corresponding to S1105. When sps_sbtmvp_enabled_flag = 0 (indicating that the ATMVP mode does not exist in the sub-block fusion prediction mode), sps_planar_enabled_flag = 1 (indicating that the planar mode exists in the sub-block fusion prediction mode), MaxNumSubblockMergeCand = sps_planar_enabled_flag = 1, corresponding to S1106. When sps_sbtmvp_enabled_flag=1, sps_planar_enabled_flag=1, then MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag=2, corresponding to S1107.

[0556] Of course, when the first identifier indicates that the candidate modes adopted for inter-frame prediction of the block to be processed only include the translational motion vector prediction mode, when the third identifier indicates that the ATMVP mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that PLANAR does not exist in the sub-block fusion prediction mode, MaxNumSubblockMergeCand=0.

[0557] As an example, the maximum value allowed by MaxNumSubblockMergeCand is 5.

[0558] If sps_affine_enable_flag is 0, MaxNumSubblockMergeCand is obtained by the following formula:

[0559] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

[0560] If sps_affine_enable_flag is 1, MaxNumSubblockMergeCand is obtained by the following formula:

[0561] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand.

[0562] For example, five_minus_max_num_subblock_merge_cand can be defined as 5 minus the maximum length of the subblock merging motion vector prediction list in the slice (five_minus_max_num_subblock_merge_cand specifies the maximum number of sublock merging motion vector prediction (MVP) candidates supported in the slice subtracted from 5).

[0563] The maximum number of subblock merging MVP candidates,MaxNumSubblockMergeCand is derived as follows:

[0564] -If sps_affine_enabled_flag is equal to 0:

[0565] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag;

[0566] -Otherwise(sps_affine_enabled_flag is equal to 1):

[0567] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand;

[0568] The value of MaxNumSubblockMergeCand shall be in the range of 0 to 5, inclusive.

[0569] See also Fig.12 As shown, the fourth implementation mode is described below.

[0570] S1201, see S901, which will not be described in detail here. Execute S1202 or S1206.

[0571] S1202: when the first identifier indicates that the candidate mode used for inter-frame prediction of the block to be processed includes an affine mode, parse the second identifier, the third identifier, and the fourth identifier from the code stream, wherein the second identifier is used to indicate (or determine) the maximum length of the first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode. Execute S1203 or S1204 or S1205.

[0572] S1203, when the third identifier indicates that the advanced time domain motion vector prediction mode exists in the sub-block fusion prediction mode and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, determine a second quantity value according to the fourth identifier; determine the maximum length of the first candidate motion vector list according to the second identifier and the second quantity value.

[0573] S1204, when the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode and the fourth identifier indicates that the plane motion vector prediction mode exists in the sub-block fusion prediction mode, the first quantity value is determined according to the third identifier, and the maximum length of the first candidate motion vector list is determined according to the second identifier and the first quantity value.

[0574] S1205, when the third identifier indicates that the advanced time domain motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the fourth identifier indicates that the plane motion vector prediction mode does not exist in the sub-block fusion prediction mode, the first quantity value is determined according to the third identifier and the second quantity value is determined according to the fourth identifier, and the maximum length of the first candidate motion vector list is determined according to the second identifier, the first quantity value and the second quantity value.

[0575] Among them, when the fourth identifier indicates that the planar mode does not exist in the sub-block fusion prediction mode, the second quantity value may be equal to the maximum number of motion vectors supported for prediction by the planar mode. Taking the fourth identifier as an example that can be represented by sps_planar_enabled_flag, when sps_planar_enabled_flag=0, it indicates that the planar motion vector prediction mode does not exist in the sub-block fusion prediction mode, and the second quantity value is the maximum number of motion vectors supported for prediction by ATMVP. On the contrary, when sps_planar_enabled_flag=1, it indicates that the planar motion vector prediction mode exists in the sub-block fusion prediction mode, and the second quantity value is 0. Exemplarily, the maximum number of motion vectors supported for prediction by the planar mode may be 1, in which case the second quantity value may be equal to the value of sps_planar_enabled_flag.

[0576] In a possible example, when the first identifier indicates that the candidate mode adopted by the to-be-processed block for inter-frame prediction includes an affine mode, the maximum length of the first candidate motion vector list may be obtained according to the following formula:

[0577] MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand-L1-L2;

[0578] Illustratively, in the fourth implementation manner, the allowed value range of K_minus_max_num_subblock_merge_cand may be 0-2, or 0-3.

[0579] Exemplarily, the maximum value allowed by MaxNumSubblockMergeCand may be 5 or 6.

[0580] When the maximum value allowed by MaxNumSubblockMergeCand is 5, the second flag may be represented by five_minus_max_num_subblock_merge_cand. When the maximum value allowed by MaxNumSubblockMergeCand is 6, the second flag may be represented by six_minus_max_num_subblock_merge_cand.

[0581] In one example, L1 can be obtained by the following formula:

[0582] L1=sps_sbtmvp_enabled_flag==1?0:1. If sps_sbtmvp_enabled_flag=1, then L1=0. If sps_sbtmvp_enabled_flag=0, then L1=1.

[0583] In one example, L2 can be obtained by the following formula:

[0584] L2=sps_planar_enabled_flag==1?0:1. If sps_planar_enabled_flag=1, then L2=0. If sps_sbtmvp_enabled_flag=0, then L2=1.

[0585] S1206-S1209, see S1104-S1107, which will not be repeated here.

[0586] For example, the maximum value allowed by MaxNumSubblockMergeCand is 5. The value range of five_minus_max_num_subblock_merge_cand is 0-2.

[0587] If sps_affine_enable_flag is 0, MaxNumSubblockMergeCand is obtained by the following formula:

[0588] MaxNumSubblockMergeCand=sps_sbtmvp_enabled_flag+sps_planar_enabled_flag.

[0589] Otherwise (sps_affine_enable_flag is 1), MaxNumSubblockMergeCand is obtained by the following formula:

[0590] MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand-(sps_sbtmvp_enabled_flag==1?0:1)-(sps_planar_enabled_flag==1?0:1).

[0591] (five_minus_max_num_subblock_merge_cand specifies the maximum number of sublock merging motion vector prediction (MVP) candidates supported in the slice subtracted from 5. The maximum number of subblock merging MVP candidates, MaxNumSubblockMergeCand is derived as follows:

[0592] - If sps_affine_enabled_flag is equal to 0:

[0593] MaxNumSubblockMergeCand = sps_sbtmvp_enabled_flag + sps_planar_enabled_flag

[0594] - Otherwise (sps_affine_enabled_flag is equal to 1):

[0595] MaxNumSubblockMergeCand = 5 - five_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1) - (sps_planar_enabled_flag == 1? 0 : 1)

[0596] The value of MaxNumSubblockMergeCand shall be in the range of 0 to 5, inclusive.)。

[0597] Take the example where the maximum value allowed by MaxNumSubblockMergeCand can be 6. Among them, the value range of five_minus_max_num_subblock_merge_cand is 0 - 3.

[0598] If sps_affine_enable_flag is 0, then MaxNumSubblockMergeCand is obtained through the following formula:

[0599] MaxNumSubblockMergeCand = sps_sbtmvp_enabled_flag + sps_planar_enabled_flag;

[0600] Otherwise (if sps_affine_enable_flag is 1), MaxNumSubblockMergeCand is obtained by the following formula:

[0601] MaxNumSubblockMergeCand = 6 - six_minus_max_num_subblock_merge_cand - (sps_sbtmvp_enabled_flag == 1? 0 : 1) - (sps_planar_enabled_flag == 1? 0 : 1).

[0602] (six_minus_max_num_subblock_merge_cand specifies the maximum number of sublock merging motion vector prediction (MVP) candidates supported in the slice subtracted from 6. The maximum number of subblock merging MVP candidates, MaxNumSubblockMergeCand is derived as follows:

[0603] - If sps_affine_enabled_flag is equal to 0:

[0604] MaxNumSubblockMergeCand = sps_sbtmvp_enabled_flag + sps_planar_enabled_flag;

[0605] - Otherwise (sps_affine_enabled_flag is equal to 1):

[0606] MaxNumSubblockMergeCand=6-six_minus_max_num_subblock_merge_cand-(sps_sbtmvp_enabled_flag==1?0:1)-(sps_planar_enabled_flag==1?0:1);

[0607] The value of MaxNumSubblockMergeCand shall be in the range of 0to 6,inclusive.).

[0608] For the second case, the sub-block fusion mode does not consider the existence of the planar motion vector prediction mode, that is, the sub-block fusion mode may include at least one of the affine mode and the advanced time domain motion vector prediction mode. The above-mentioned first implementation or the second implementation may be adopted, which will not be repeated here.

[0609] According to the same inventive concept as the method embodiment, the present application embodiment also provides a device, see Fig.13 As shown, the device 1300 may specifically be a processor in a video decoder, or a chip or a chip system, or a module in a video decoder, such as an entropy decoding unit 304 and / or an inter-frame prediction unit 344 .

[0610] Illustratively, the apparatus may include a parsing unit 1301 and a determining unit 1302. The parsing unit 1301 and the determining unit 1302 execute Figure 9-12 The method steps shown in the corresponding embodiment. For example, the parsing unit 1301 can be used to parse each identifier (such as the first identifier, the second identifier, the third identifier, or the fourth identifier, etc.) included in the bitstream, and the determining unit 1302 is used to determine the maximum length of the first candidate motion vector list.

[0611] The embodiment of the present application also provides another structure of the device for a decoder, such as Fig.14 As shown, the device 1400 may include a communication interface 1410 and a processor 1420. Optionally, the device 1400 may also include a memory 1430. The memory 1430 may be disposed inside the device or outside the device. Fig.13 The parsing unit 1301 and the determining unit 1302 shown in FIG. 1 can be implemented by the processor 1420. The processor 1420 sends or receives a video stream or a bit stream through the communication interface 1410, and is used to implement Figure 9-12 In the implementation process, each step of the processing flow can be completed by the hardware integrated logic circuit in the processor 1420 or the instructions in the form of software. Figure 9-12 The method described in .

[0612] In the embodiment of the present application, the communication interface 1410 may be a circuit, a bus, a transceiver or any other device that can be used for information exchange. Exemplarily, the other device may be a device connected to the device 1400, for example, when the device is a video encoder, the other device may be a video decoder.

[0613] In the embodiment of the present application, the processor 1420 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiment of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software units in the processor. The program code executed by the processor 1420 to implement the above method may be stored in the memory 1430. The memory 1430 is coupled to the processor 1420.

[0614] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.

[0615] The processor 1420 may operate in conjunction with the memory 1430. The memory 1430 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory 1430 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0616] The specific connection medium between the communication interface 1410, the processor 1420 and the memory 1430 is not limited in the embodiment of the present application. Fig.14 The memory 1430, the processor 1420 and the communication interface 1410 are connected via a bus. Fig.14 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.14Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0617] Above Figure 9-12 The relevant feasible implementation methods and specific examples describe one or more devices for decoding video data of the present application. It should be understood that, according to the foregoing, under normal circumstances, the encoding end will determine the inter-frame prediction mode and encode it into the bitstream. After the inter-frame prediction mode finally selected is selected, the indication of the inter-frame prediction mode (such as the first identifier, the second identifier, the third identifier, or the fourth identifier, etc. mentioned above) will be encoded into the bitstream (corresponding to the parsed first identifier, the second identifier, the third identifier, or the fourth identifier in the decoding process, respectively) in accordance with the encoding process that is completely opposite to the above-mentioned decoding method. It should be understood that the maximum length of the first candidate motion vector list is determined, which is completely consistent for the encoding and decoding end. The specific embodiments of the encoding end will not be repeated, but it should be understood that the video image prediction method described in this application is also applicable to the encoding device.

[0618] The embodiment of the present application also provides the device for the encoder, such as Fig.15 As shown, the device 1500 may include a communication interface 1510 and a processor 1520. Optionally, the device 1500 may also include a memory 1530. The memory 1530 may be disposed inside the device or outside the device. The processor 1520 sends or receives a video stream or a bit stream through the communication interface 1510.

[0619] On the one hand, processor 1520 is used to encode a first identifier into a bitstream; when the first identifier indicates that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, a second identifier is encoded into the bitstream, and the second identifier is used to indicate the maximum length of a first candidate motion vector list, and the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion prediction mode.

[0620] In the embodiment of the present application, the communication interface 1510 may be a circuit, a bus, a transceiver or any other device that can be used for information exchange. Exemplarily, the other device may be a device connected to the device 1500, for example, when the device is a video encoder, the other device may be a video decoder.

[0621] In the embodiment of the present application, the processor 1520 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software units in the processor can be executed. The program code executed by the processor 1520 to implement the above method can be stored in the memory 1530. The memory 1530 is coupled to the processor 1520.

[0622] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.

[0623] The processor 1520 may operate in conjunction with the memory 1530. The memory 1530 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory 1530 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0624] The specific connection medium between the communication interface 1510, the processor 1520 and the memory 1530 is not limited in the embodiment of the present application. Fig.15 The memory 1530, the processor 1520 and the communication interface 1510 are connected via a bus. Fig.15 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0625] According to the above embodiments, the embodiments of the present application further provide a computer storage medium, in which a software program is stored, and when the software program is read and executed by one or more processors, the method provided by any one or more of the above embodiments can be implemented. The computer storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0626] According to the above embodiments, the embodiments of the present application further provide a chip, which includes a processor for implementing the functions involved in any one or more of the above embodiments, such as obtaining or processing the information or messages involved in the above methods. Optionally, the chip also includes a memory, which is used for the necessary program instructions and data executed by the processor. The chip can be composed of a chip, or it can include a chip and other discrete devices.

[0627] Although certain aspects of the present application have been described with respect to video encoder 20 and video decoder 30, it should be understood that the techniques of the present application may be applied by many other video encoding and / or decoding units, processors, processing units, decoding units based on hardware such as encoder / decoders (CODECs), and the like. Furthermore, it should be understood that the present application is provided only as a possible implementation. Figure 8A-Figure 12 The steps shown and described. That is, Figure 8A-Figure 12 The steps shown in the feasible implementation mode do not necessarily need to be followed. Figure 8A-Figure 12 The steps are performed in the order shown in FIG. , and fewer, additional, or alternative steps may be performed.

[0628] Furthermore, it should be understood that, depending on the possible implementation, certain actions or events of any of the methods described herein may be performed in a different sequence, may be added, combined, or omitted altogether (e.g., not all described actions or events are necessary to practice the methods). Furthermore, in certain possible implementations, actions or events may be performed simultaneously rather than sequentially, for example, via multithreading, interrupt processing, or multiple processors. Additionally, while certain aspects of the present application are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of the present application may be performed by a combination of units or modules associated with a video decoder.

[0629] In one or more possible implementations, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes, and executed by a processing unit according to hardware. Computer-readable media may include computer-readable storage media or communication media, where computer-readable storage media corresponds to tangible media such as data storage media, and communication media includes any media that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol.

[0630] In this manner, computer-readable media may correspond, for example, to (1) non-transitory, tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

[0631] As a possible implementation and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store the desired code in the form of instructions or data structures and can be accessed by a computer. Likewise, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of media.

[0632] However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead are directed to non-transitory tangible storage media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks usually reproduce data magnetically, while optical discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0633] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, as used herein, the term "processor" may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Likewise, the techniques may be fully implemented in one or more circuits or logic elements.

[0634] The techniques of the present application may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or collections of ICs (e.g., chipsets). Various components, modules, or units are described in this application to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperating hardware units (including one or more processors as described above) in conjunction with suitable software and / or firmware.

[0635] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video image encoding method, characterized in that: include: Determining whether the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode; When the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode, determining the maximum length of a first candidate motion vector list according to the second identifier, the first candidate motion vector list being a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the sub-block fusion mode includes at least one of an affine mode, an advanced temporal motion vector prediction mode, and a planar motion vector prediction mode; The first identifier and the second identifier are encoded into a bitstream to obtain an encoded bitstream, wherein the first identifier is used to indicate whether the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode.

2. The method according to claim 1, characterized in that The determining the maximum length of the first candidate motion vector list according to the second identifier comprises: The maximum length of the first candidate motion vector list is obtained according to the following formula: MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand; Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, K_minus_max_num_subblock_merge_cand represents the second flag, and K is a preset non-negative integer.

3. The method according to claim 1 or 2, characterized in that When the candidate modes adopted by the block to be processed for inter-frame prediction do not include an affine mode, the maximum length of the first candidate motion vector list is determined according to the third identifier.

4. The method according to claim 3, characterized in that The third flag is used to indicate the existence status of the advanced temporal motion vector prediction mode in the sub-block fusion mode.

5. The method according to claim 3 or 4, characterized in that The method further includes encoding the third identifier into the code stream.

6. A video image encoding method, characterized in that: include: When the value of the first flag sps_affine_enabled_flag is the first value, determining the maximum length of the first candidate motion vector list according to the value of the second flag, the first candidate motion vector list being a candidate motion vector list constructed when the block to be processed adopts the sub-block fusion mode; When the value of the first flag sps_affine_enabled_flag is a second value, determining the maximum length of the first candidate motion vector list according to the value of the third flag sps_sbtmvp_enabled_flag, wherein the first value is 1 and the second value is 0; The first identifier, the second identifier, and the third identifier are encoded into a sequence parameter set SPS of a code stream to obtain an encoded code stream.

7. The method according to claim 6, characterized in that Determining the maximum length of the first candidate motion vector list according to the value of the second identifier includes: The maximum length of the first candidate motion vector list = 5 - the value of the second identifier.

8. The method according to claim 6 or 7, characterized in that The determining the maximum length of the first candidate motion vector list according to the value of the third identifier includes: When the value of the third flag is 1, it is determined that the maximum length of the first candidate motion vector list is 1.

9. The method according to claim 8, characterized in that The value of the third flag sps_sbtmvp_enabled_flag being 1 indicates that an advanced temporal motion vector prediction mode exists in the sub-block merging mode.

10. The method according to claim 6 or 7, characterized in that: The determining the maximum length of the first candidate motion vector list according to the value of the third identifier includes: When the value of the third identifier is 0, it is determined that the maximum length of the first candidate motion vector list is 0.

11. The method according to claim 10, characterized in that The value of the third flag sps_sbtmvp_enabled_flag being 0 indicates that there is no advanced temporal motion vector prediction mode in the sub-block fusion prediction mode.

12. The method according to claim 9 or 11, characterized in that The advanced temporal motion vector prediction mode is a prediction performed according to the temporal motion vector of the sub-block.

13. The method according to any one of claims 6 to 12, characterized in that The value of the first identifier is used to indicate whether the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode.

14. A video image encoding method, characterized in that: include: Determine whether an affine mode can be used when performing inter-frame prediction on a block to be processed; When the affine mode can be used for inter-frame prediction of the block to be processed, determining the maximum length of the first candidate motion vector list according to the second identifier K_minus_max_num_subblock_merge_cand, where the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts the sub-block fusion mode; The first identifier, the second identifier, and the third identifier are encoded into a bitstream to obtain an encoded bitstream, wherein the first identifier is used to indicate whether an affine mode can be used when performing inter-frame prediction on the block to be processed, and the third identifier is used to indicate the existence status of the advanced time domain motion vector prediction mode in the sub-block fusion mode.

15. The method according to claim 14, characterized in that The maximum length of the first candidate motion vector list is obtained according to the following formula: MaxNumSubblockMergeCand=K-K_minus_max_num_subblock_merge_cand; Among them, MaxNumSubblockMergeCand represents the maximum length of the first candidate motion vector list, and K is a preset non-negative integer.

16. An encoder, characterized in that: include: Memory and processor; The memory is used to store program instructions; The processor is used to call and execute the program instructions stored in the memory to implement the method according to any one of claims 1-15.

17. A chip, characterized in that: The chip is connected to the memory and is used to read and execute the software program stored in the memory to implement the method according to any one of claims 1-15.

18. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 15.

19. A computer-readable storage medium, wherein a code stream is stored in the computer-readable storage medium, wherein the code stream includes a first identifier, wherein the first identifier is used to indicate whether the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode. When the first identifier is used to indicate that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, the code stream also includes a second identifier, wherein the second identifier is used to indicate the maximum length of a first candidate motion vector list, wherein the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the sub-block fusion mode includes at least one of an affine mode, an advanced time domain motion vector prediction mode, and a plane motion vector prediction mode.

20. The computer-readable storage medium of claim 19, wherein: The code stream further includes a third identifier, and when the first identifier is used to indicate that the candidate mode used for inter-frame prediction of the block to be processed does not include an affine mode, the third identifier is used to indicate the maximum length of the first candidate motion vector list.

21. A computer-readable storage medium, wherein a code stream is stored in the computer-readable storage medium, wherein the code stream includes a first identifier, sps_affine_enabled_flag; when the first identifier is a first value, the code stream further includes a second identifier, wherein the second identifier is used to indicate a maximum length of a first candidate motion vector list; the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode; when the first identifier is a second value, the code stream further includes a third identifier, wherein the third identifier is used to indicate a maximum length of the first candidate motion vector list; the first value is 1, and the second value is 0.

22. A computer-readable storage medium, wherein a code stream is stored in the computer-readable storage medium, wherein the code stream includes a first identifier, wherein the first identifier is used to indicate that an affine mode can be used for inter-frame prediction of image blocks included in a video image, and the code stream also includes a second identifier and a third identifier, wherein the second identifier is used to indicate a maximum length of a first candidate motion vector list, wherein the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the third identifier is used to indicate the existence status of an advanced time domain motion vector prediction mode in the sub-block fusion mode.

23. A code stream, the code stream comprising a first identifier, the first identifier being used to indicate whether the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, when the first identifier is used to indicate that the candidate mode adopted for inter-frame prediction of the block to be processed includes an affine mode, the code stream further comprises a second identifier, the second identifier being used to indicate the maximum length of a first candidate motion vector list, the first candidate motion vector list being a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, the sub-block fusion mode including at least one of an affine mode, an advanced time domain motion vector prediction mode and a plane motion vector prediction mode.

24. The code stream as described in claim 23, wherein the code stream further includes a third identifier, and when the first identifier is used to indicate that the candidate mode used for inter-frame prediction of the block to be processed does not include an affine mode, the third identifier is used to indicate the maximum length of the first candidate motion vector list.

25. A code stream, the code stream comprising a first identifier sps_affine_enabled_flag, when the first identifier is a first value, the code stream further comprising a second identifier, the second identifier is used to indicate the maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, when the first identifier is a second value, the code stream further comprises a third identifier, the third identifier is used to indicate the maximum length of the first candidate motion vector list, the first value is 1, and the second value is 0.

26. A code stream, the code stream comprising a first identifier, the first identifier being used to indicate that an affine mode can be used for inter-frame prediction of image blocks included in a video image, the code stream further comprising a second identifier and a third identifier, the second identifier being used to indicate a maximum length of a first candidate motion vector list, the first candidate motion vector list being a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the third identifier being used to indicate the existence status of an advanced time domain motion vector prediction mode in the sub-block fusion mode.

27. A device for storing a code stream, characterized in that: comprising a receiver and at least one memory; The receiver is used to receive a code stream; The at least one memory is used to store the code stream, the code stream includes a first identifier, the first identifier is used to indicate whether the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode, when the first identifier is used to indicate that the candidate mode adopted by the block to be processed for inter-frame prediction includes an affine mode, the code stream also includes a second identifier, the second identifier is used to indicate the maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the sub-block fusion mode includes at least one of an affine mode, an advanced temporal motion vector prediction mode and a planar motion vector prediction mode.

28. The device according to claim 27, characterized in that The code stream further includes a third identifier, and when the first identifier is used to indicate that the candidate mode used for inter-frame prediction of the block to be processed does not include an affine mode, the third identifier is used to indicate the maximum length of the first candidate motion vector list.

29. A device for transmitting a code stream, characterized in that: comprising a transmitter and one or more memories; The one or more memories are used to store a code stream, the code stream includes a first identifier, the first identifier is used to indicate whether a candidate mode adopted by the to-be-processed block for inter-frame prediction includes an affine mode, when the first identifier is used to indicate that the candidate mode adopted by the to-be-processed block for inter-frame prediction includes an affine mode, the code stream further includes a second identifier, the second identifier is used to indicate a maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the to-be-processed block adopts a sub-block fusion mode, the sub-block fusion mode includes at least one of an affine mode, an advanced temporal motion vector prediction mode, and a planar motion vector prediction mode; The transmitter is used to send the code stream.

30. The device according to claim 29, characterized in that The code stream further includes a third identifier, and when the first identifier is used to indicate that the candidate mode used for inter-frame prediction of the block to be processed does not include an affine mode, the third identifier is used to indicate the maximum length of the first candidate motion vector list.

31. A device for storing a code stream, characterized in that: comprising a receiver and at least one memory; The receiver is used to receive a code stream; The at least one memory is used to store the code stream, the code stream includes a first identifier sps_affine_enabled_flag, when the first identifier is a first value, the code stream includes a second identifier, the second identifier is used to indicate a maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, when the first identifier is a second value, the code stream includes a third identifier, the third identifier is used to indicate a maximum length of the first candidate motion vector list, the first value is 1, and the second value is 0.

32. A device for transmitting a code stream, characterized in that: comprising a transmitter and one or more memories; The one or more memories are used to store a code stream, the code stream includes a first identifier sps_affine_enabled_flag, when the first identifier is a first value, the code stream includes a second identifier, the second identifier is used to indicate a maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the to-be-processed block adopts a sub-block fusion mode, when the first identifier is a second value, the code stream includes a third identifier, the third identifier is used to indicate a maximum length of the first candidate motion vector list, the first value is 1, and the second value is 0; The transmitter is used to send the code stream.

33. A device for storing a code stream, characterized in that: comprising a receiver and at least one memory; The receiver is used to receive a code stream; The at least one memory is used to store the code stream, the code stream includes a first identifier, the first identifier is used to indicate that an affine mode can be used when performing inter-frame prediction on an image block included in a video image, the code stream also includes a second identifier and a third identifier, the second identifier is used to indicate a maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the third identifier is used to indicate the existence status of an advanced temporal motion vector prediction mode in the sub-block fusion mode.

34. A device for transmitting a code stream, characterized in that: comprising a transmitter and one or more memories; The one or more memories are used to store a code stream, the code stream includes a first identifier, the first identifier is used to indicate that an affine mode can be used when inter-frame prediction is performed on an image block included in a video image, the code stream also includes a second identifier and a third identifier, the second identifier is used to indicate a maximum length of a first candidate motion vector list, the first candidate motion vector list is a candidate motion vector list constructed when the block to be processed adopts a sub-block fusion mode, and the third identifier is used to indicate the existence state of the advanced temporal motion vector prediction mode in the sub-block fusion mode; The transmitter is used to send the code stream.

Images