Image prediction method and device and computer readable storage medium

Abstract

The embodiment of the invention discloses an image prediction method and device. The image prediction method comprises the following steps: determining whether a current image block uses a fusion mode, and determining whether a first-level fusion mode is available for the current image block when the current image block uses the fusion mode; if the first fusion mode is unavailable and a high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is prohibited from being used, directly determining a second fusion mode as a target fusion mode suitable for the current image block, both the first fusion mode and the second fusion mode belonging to a second-level fusion mode; and predicting the current image block according to the target fusion mode. According to the embodiment of the invention, redundancy generated in the fusion mode determination process can be reduced as much as possible.

Description

[0001] This application is a divisional application. The application number of the original application is 201910474007.X, and the original application date is June 2, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of video coding and decoding technology, and more specifically, to an image prediction method, device and computer-readable storage medium. Background Art

[0003] Digital video capabilities may be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, electronic book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio telephones (so-called "smart phones"), video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video compression techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Coding (AVC), video coding standard H.265 / high efficiency video coding (HEVC) standard, and extensions of such standards. Video devices may transmit, receive, encode, decode, and / or store digital video information more efficiently by implementing such video compression techniques.

[0004] Video compression techniques perform spatial (intra-image) prediction and / or temporal (inter-image) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video frame or a portion of a video frame) may be partitioned into several image blocks, which may also be referred to as tree blocks, coding units (CUs), and / or coding nodes. Image blocks in an intra-coded (I) slice of an image are encoded using spatial predictions with respect to reference samples in neighboring blocks in the same image. Image blocks in an inter-coded (P or B) slice of an image may use spatial predictions with respect to reference samples in neighboring blocks in the same image or temporal predictions with respect to reference samples in other reference images. Images may be referred to as frames, and reference images may be referred to as reference frames.

[0005] When using a fusion mode to predict an image block, there are generally multiple optional fusion modes. The traditional solution generally determines the fusion mode applicable to the current image block one by one from multiple candidate fusion modes. When a fusion mode is not available, continue to confirm whether the next fusion mode is available. The traditional solution has a certain redundancy when determining the fusion mode applicable to the current block from the last two remaining fusion modes. Summary of the invention

[0006] The present application provides an image prediction method, device and computer-readable storage medium to reduce redundancy in the image prediction process as much as possible.

[0007] In a first aspect, an image prediction method is provided, the method comprising: determining whether a current image block uses a fusion mode, and continuing to determine whether a first-level fusion mode is available when the current image block uses the fusion mode; when the first-level fusion mode is not available and a high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is prohibited from use, determining a second fusion mode as a target fusion mode applicable to the current image block; and predicting the current image block according to the above-mentioned target fusion mode.

[0008] Among them, the first fusion mode and the second fusion mode are both secondary fusion modes, and the secondary fusion mode is composed of the first fusion mode and the second fusion mode. In addition, for the current image block, the primary fusion mode and the secondary fusion mode already include all the optional fusion modes of the current image block. For the current image block, it is necessary to determine the final target fusion mode from the primary fusion mode and the secondary fusion mode.

[0009] Optionally, the priority of the first-level fusion mode is higher than the priority of the second-level fusion mode.

[0010] The priority of the first-level fusion mode is higher than that of the second-level fusion mode, which means that in the process of determining the target fusion mode of the current image block, the first-level fusion mode is given priority. If there is no available fusion mode in the first-level fusion mode, it is determined from the second-level fusion mode.

[0011] Optionally, the above determining whether the current image block uses the fusion mode includes: when merge_flag corresponding to the current image block is 1, determining that the current image block uses the fusion mode; when merge_flag corresponding to the current image block is 0, determining that the current image block does not use the fusion mode.

[0012] It should be understood that, when it is determined that the current image block does not use the fusion mode, the current image block can be predicted using other modes other than the fusion mode. For example, when it is determined that the current image block does not use the fusion mode, the advanced motion vector AMVP mode can be used to predict the current image block.

[0013] In the present application, when the high-level syntax elements of the first fusion mode indicate that the first fusion mode is prohibited from use, there is no need to parse the remaining available status information of the second fusion mode, and the second fusion mode can be directly determined as the final target fusion mode, which can minimize the process redundancy caused by determining the target fusion mode during the image prediction process.

[0014] Optionally, the above method also includes: determining whether the first-level fusion mode is available.

[0015] Specifically, whether the first-level fusion mode is available is determined according to the high-level syntax elements corresponding to the first-level fusion mode and / or the available status information corresponding to the first-level fusion mode.

[0016] In combination with the first aspect, in certain implementations of the first aspect, when the first-level fusion mode is not available and the high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is allowed to be used, the target fusion mode is determined based on the high-level syntax element corresponding to the second fusion mode and / or the available status information of the second fusion mode.

[0017] The available status information of the second fusion mode is used to indicate whether to use the second fusion mode when predicting the current image block.

[0018] For example, the second fusion mode is the CIIP mode, and the available status information of the second fusion mode is the value of ciip_flag. When ciip_flag is 0, the CIIP mode is unavailable for the current image block, and when ciip_flag is 1, the CIIP mode is available for the current image block.

[0019] It should be understood that for the above-mentioned CIIP mode, if the CIIP mode is to be selected as the target fusion mode, it must be satisfied that the high-level syntax elements corresponding to the CIIP indicate that the CIIP mode is allowed to be used, and at the same time, the available status information expressing the available status of the CIIP mode indicates that the CIIP is available.

[0020] For example, when sps_ciip_enabled_flag=1 and ciip_flag=1, the CIIP mode may be determined as the target fusion mode of the current image block.

[0021] In combination with the first aspect, in certain implementations of the first aspect, the above-mentioned determination of the target fusion mode based on the high-level syntax elements corresponding to the second fusion mode and / or the available status information of the second fusion mode includes: when the high-level syntax elements corresponding to the second fusion mode and / or the available status information of the second fusion mode indicate that the second fusion mode is prohibited from use, determining the first fusion mode as the target fusion mode.

[0022] The high-level syntax element corresponding to the second fusion mode and / or the available status information of the second fusion mode indicates that the second fusion mode is prohibited from being used, including:

[0023] The high-level syntax element corresponding to the second fusion mode indicates that the second fusion mode is prohibited from use, and the available status information of the second fusion mode indicates that the second fusion mode cannot be used; the high-level syntax element corresponding to the second fusion mode indicates that the second fusion mode is allowed to be used, and the available status information of the second fusion mode indicates that the second fusion mode cannot be used.

[0024] Optionally, the above-mentioned determining the target fusion mode according to the high-level syntax elements corresponding to the second fusion mode and / or the available status information of the second fusion mode also includes: when the high-level syntax elements corresponding to the second fusion mode indicate that the second fusion mode is allowed to be used, and the available status information of the second fusion mode indicates that the second fusion mode is available, determining the second fusion mode as the target fusion mode.

[0025] In combination with the first aspect, in certain implementations of the first aspect, before determining the target fusion mode based on the high-level syntax elements corresponding to the second fusion mode and / or the available status information of the second fusion mode, the above method also includes: determining that at least one of the following conditions is met: the size of the current image block meets a preset condition; the current image block is not predicted using a skip mode.

[0026] That is to say, before determining the target fusion mode, it is also necessary to ensure that the size of the current image block meets the conditions and the current image block does not use the skip mode, otherwise other modes other than the fusion mode may be used to predict the current image block.

[0027] In combination with the first aspect, in some implementations of the first aspect, the size of the current image block satisfies a preset condition, including: the current image block satisfies the following three conditions:

[0028] (cbWidth*cbHeight)>=64;

[0029] cbWidth<128;

[0030] cbHeight<128;

[0031] Among them, cbWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0032] In combination with the first aspect, in some implementations of the first aspect, the first fusion mode includes a triangulation TPM mode, and the second fusion mode includes a joint intra-frame and inter-frame prediction CIIP mode.

[0033] Optionally, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, the CIIP mode is determined as the target fusion mode.

[0034] In the present application, when the high-level syntax elements corresponding to the TPM mode indicate that the TPM mode is prohibited from use, it is no longer necessary to determine whether the CIIP mode is available by parsing the high-level syntax corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode. Instead, the CIIP mode can be directly determined as the target fusion mode, which can reduce redundancy in the process of determining the target fusion mode.

[0035] Optionally, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, the target fusion mode is determined according to the high-level syntax element corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode.

[0036] Optionally, when the high-level syntax element corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, the TPM mode is determined as the target fusion mode.

[0037] Optionally, when the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used and the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is available, the CIIP mode is determined as the target fusion mode.

[0038] In combination with the first aspect, in certain implementations of the first aspect, before determining the target fusion mode based on the high-level syntax elements corresponding to the second fusion mode and / or the available status information of the second fusion mode, the above method also includes: determining that the type of the slice or slice group where the current image block is located is B; determining that the maximum candidate number of TPM modes supported by the slice or slice group where the current image block is located is greater than or equal to 2.

[0039] Optionally, before determining the target fusion mode based on the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode, the above method also includes: determining that the type of the slice or slice group where the current image block is located is B; determining that the maximum candidate number of TPM modes supported by the slice or slice group where the current image block is located is greater than or equal to 2.

[0040] In combination with the first aspect, in certain implementations of the first aspect, the first fusion mode is the triangulation TPM mode, and the second fusion mode is the joint intra-frame and inter-frame prediction CIIP mode. The above method also includes: when the first-level fusion mode is not available, the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, but the current image block does not meet at least one of conditions A and condition B, the CIIP mode is determined as the target fusion mode.

[0041] Among them, the above conditions A and B are respectively:

[0042] Condition A: The type of the slice where the current image block is located is B;

[0043] Condition B: The maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located is greater than or equal to 2.

[0044] Only when both conditions A and B are met can the TPM mode be selected as the target fusion mode ultimately adopted when predicting the current image block.

[0045] On the one hand, if only one of condition A and condition B is not satisfied, the CIIP mode is determined as the target fusion mode.

[0046] On the other hand, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from use, if only one of the conditions A and B is not satisfied, the CIIP mode is determined as the target fusion mode.

[0047] On the other hand, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, if only one of the conditions A and B is not satisfied, the CIIP mode is determined as the target fusion mode.

[0048] That is, as long as sps_trangle_enabled_flag=1 and one of condition A and condition B is not satisfied, the CIIP mode can be determined as the target fusion mode.

[0049] On the other hand, if sps_trangle_enabled_flag=1, and both conditions A and B are met, then it is necessary to combine several conditions of the prior art and determine the target fusion mode according to ciip_flag.

[0050] In combination with the first aspect, in some implementations of the first aspect, the high-level syntax elements are syntax elements at at least one level of sequence level, picture level, slice level, and slice group level.

[0051] In combination with the first aspect, in some implementations of the first aspect, the above-mentioned first-level fusion mode includes: a normal fusion mode, a fused motion vector difference MMVD mode and a sub-block fusion mode.

[0052] When determining whether the primary fusion mode is available, it may be determined whether these modes are available in the order of the conventional fusion mode, the MMVD mode, and the sub-block fusion mode.

[0053] For example, you can first determine whether the conventional fusion mode is available. If the conventional fusion mode is not available (if the conventional mode is available, you can directly use the conventional fusion mode as the final target fusion mode), continue to determine whether the MMVD mode is available. If the MMVD mode is not available, continue to determine whether the sub-block fusion mode is available.

[0054] In combination with the first aspect, in some implementations of the first aspect, the method further includes: when the first-level fusion mode is not available, determining a target fusion mode from the second-level fusion mode, where the second-level fusion mode includes a TPM mode and a CIIP mode; wherein, when the CIIP mode is allowed to be used and any of the following conditions is not met, determining the CIIP mode as the target fusion mode;

[0055] Condition D: TPM mode is allowed;

[0056] Condition E: The current image block does not use skip mode for prediction;

[0057] Condition F: (cbWidth*cbHeight)>=64;

[0058] Condition G: cbWidth<128;

[0059] Condition H: cbHeight < 128;

[0060] Among them, cbWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0061] In combination with the first aspect, in some implementations of the first aspect, the above prediction method is applied at the encoding end to implement encoding of the current image block.

[0062] In combination with the first aspect, in some implementations of the first aspect, the above prediction method is applied at the decoding end to implement decoding of the current image block.

[0063] In a second aspect, an image prediction method is provided, the method comprising: determining whether a current image block uses a fusion mode; if the current image block uses the fusion mode, determining whether a primary fusion mode is available; if the primary fusion mode is not available, determining a target fusion mode from a secondary fusion mode, the secondary fusion mode including a TPM mode and a CIIP mode; wherein, when the CIIP mode is allowed to be used, if any one of the following conditions (conditions 1 to 5) is not met, the CIIP mode is determined as the target fusion mode;

[0064] Condition 1: TPM mode is allowed to be used;

[0065] Condition 2: The type of the slice or slice group where the current image block is located is B;

[0066] Condition 3: Determine that the maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located is greater than or equal to 2;

[0067] Condition 4: The size of the current image block meets the preset conditions;

[0068] Condition 5: The current image block does not use skip mode for prediction.

[0069] The first condition can be specifically represented by sps_triangle_enabled_flag=1, the second condition can be specifically represented by slice_type==B, and the third condition can be specifically represented by MaxNumTriangleMergeCand>=2. MaxNumTriangleMergeCand represents the maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located.

[0070] In addition, for the current image block, the primary fusion mode and the secondary fusion mode may include all optional fusion modes of the current image block, and for the current image block, the final target fusion mode needs to be determined from the primary fusion mode and the secondary fusion mode.

[0071] Optionally, the priority of the first-level fusion mode is higher than the priority of the second-level fusion mode.

[0072] The priority of the above-mentioned first-level fusion mode is higher than that of the second-level fusion mode, which means that in the process of determining the target fusion mode of the current image block, the target fusion mode is determined from the first-level fusion mode first. If there is no available fusion mode in the first-level fusion mode, it is determined from the second-level fusion mode.

[0073] Optionally, the size of the current image block satisfies a preset condition, including: the current image block satisfies the following three conditions:

[0074] (cbWidth*cbHeight)>=64;

[0075] cbWidth<128;

[0076] cbHeight<128.

[0077] Optionally, the above determining whether the current image block uses the fusion mode includes: when merge_flag corresponding to the current image block is 1, determining that the current image block uses the fusion mode; when merge_flag corresponding to the current image block is 0, determining that the current image block does not use the fusion mode.

[0078] It should be understood that, when it is determined that the current image block does not use the fusion mode, the current image block can be predicted using other modes other than the fusion mode. For example, when it is determined that the current image block does not use the fusion mode, the advanced motion vector AMVP mode can be used to predict the current image block.

[0079] Optionally, the above-mentioned first-level fusion mode includes: a normal fusion mode, an MMVD mode and a sub-block fusion mode.

[0080] When determining whether the primary fusion mode is available, the availability of the conventional fusion mode, the MMVD mode and the sub-block fusion mode may be determined in sequence. When none of these modes are available, it is determined that the primary fusion mode is unavailable.

[0081] In the present application, when the first-level fusion mode is not available, it is possible to determine whether to select the CIIP mode as the final fusion mode based on some preset conditions, and as long as any of the preset conditions is not met, the CIIP mode can be directly determined as the target fusion mode, which can reduce the process redundancy generated in the process of determining the target fusion.

[0082] In combination with the second aspect, in certain implementations of the second aspect, a target fusion mode is determined from the secondary fusion mode, including: when any one of conditions one to five is not met, the value of the available status information expressing the available status of the CIIP mode is set to a first value, wherein when the value of the available status information expressing the available status of the CIIP mode is the first value, the current image block adopts the CIIP mode for image prediction.

[0083] It should be understood that here, setting the value of the available status information expressing the available status of the CIIP mode to the first value is equivalent to determining CIIP as the target fusion mode.

[0084] Optionally, the available status information expressing the available status of the CIIP mode is ciip_flag.

[0085] Setting the value of the available status information expressing the available status of the CIIP mode to the first value may specifically be setting ciip_flag to 1.

[0086] In addition, when the value of the available state information expressing the available state of the CIIP mode is set to the second value, it may mean that the current image block does not use the CIIP mode for image prediction. For example, the available state information expressing the available state of the CIIP mode is ciip_flag, and when ciip_flag=0, the current image block does not use the CIIP mode for image prediction.

[0087] In combination with the second aspect, in certain implementations of the second aspect, a target fusion mode is determined from the secondary fusion mode, including: when all conditions from conditions one to five are met, the target fusion mode is determined according to the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode, wherein the available status information expressing the available status of the CIIP mode is used to indicate whether the CIIP mode is used when predicting the current image block.

[0088] For example, the available status information expressing the available status of the CIIP mode is the value of ciip_flag. When ciip_flag is 0, the CIIP mode is unavailable for the current image block. When ciip_flag is 1, the CIIP mode is available for the current image block.

[0089] In the present application, the target fusion mode is determined based on the high-level syntax elements of the CIIP mode and / or the available status information expressing the available status of the CIIP mode only when five preset conditions are met. Compared with the traditional scheme, more conditions need to be met before the target fusion mode is further determined in combination with the high-level syntax elements and available status information of the CIIP mode. Otherwise, the CIIP mode can be directly determined as the target fusion mode, which can reduce some redundant processes in the process of determining the target fusion mode.

[0090] In combination with the second aspect, in certain implementations of the second aspect, the target fusion mode is determined based on the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode, including: when the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicate that the CIIP mode is prohibited from use, determining the TPM mode as the target fusion mode.

[0091] In conjunction with the second aspect, in some implementations of the second aspect, when the high-level syntax element corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is prohibited from use, determining the TPM mode as the target fusion mode includes:

[0092] When the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicate that the CIIP mode is prohibited from use, the value of the available status information expressing the available status of the TPM is set to a first value, wherein when the value of the available status information expressing the available status of the TPM mode is the first value, the current image block adopts the TPM mode for image prediction.

[0093] It should be understood that setting the value of the available state information expressing the available state of the TPM mode to the first value here is equivalent to determining the TPM as the target fusion mode.

[0094] Optionally, the available state information expressing the available state of the TPM mode is MergeTriangleFlag.

[0095] Setting the value of the available state information expressing the available state of the TPM mode to the first value may specifically be setting MergeTriangleFlag to 1.

[0096] In conjunction with the second aspect, in some implementations of the second aspect, before determining the target fusion mode according to the high-level syntax element corresponding to the CIIP mode and / or the available state information expressing the available state of the CIIP mode, the method further includes:

[0097] Make sure at least one of the following conditions is met:

[0098] The size of the current image block meets the preset conditions;

[0099] The current image block does not use skip mode for prediction.

[0100] Among them, cbWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0101] In a third aspect, an image prediction method is provided, the method comprising: determining whether a current image block uses a fusion mode; if the current image block uses the fusion mode, determining whether a primary fusion mode is available; if the primary fusion mode is not available, determining a target fusion mode from a secondary fusion mode, the secondary fusion mode including a TPM mode and a CIIP mode; wherein, when the CIIP mode is allowed to be used, when all of the following conditions (conditions 1 to 3) are met, obtaining available status information of the CIIP mode by parsing a bitstream, and determining the target fusion mode according to the available status information of the CIIP mode;

[0102] Condition 1: TPM mode is allowed to be used;

[0103] Condition 2: The size of the current image block meets the preset conditions;

[0104] Condition three: The current image block does not use skip mode for prediction.

[0105] According to a possible implementation manner of the third aspect, if the available status information of the CIIP mode obtained by parsing the code stream indicates that the CIIP mode is not available, TPM is used as the target fusion mode.

[0106] In the present application, when the first-level fusion mode is not available, it is possible to determine whether to select the CIIP mode as the final fusion mode based on some preset conditions, and as long as any of the preset conditions is not met, the CIIP mode can be directly determined as the target fusion mode, which can reduce the process redundancy generated in the process of determining the target fusion.

[0107] In a fourth aspect, an image prediction method is provided, the method comprising: determining whether a current image block uses a fusion mode; if the current image block uses the fusion mode, determining whether a primary fusion mode is available; if the primary fusion mode is not available, determining a target fusion mode from a secondary fusion mode, the secondary fusion mode including a TPM mode and a CIIP mode; wherein, when the CIIP mode is allowed to be used, when all of the following conditions (conditions 1 to 5) are met, obtaining available status information of the CIIP mode by parsing a bitstream, and determining the target fusion mode according to the available status information of the CIIP mode;

[0108] Condition 1: TPM mode is allowed to be used;

[0109] Condition 2: The type of the slice or slice group where the current image block is located is B;

[0110] Condition 3: Determine that the maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located is greater than or equal to 2;

[0111] Condition 4: The size of the current image block meets the preset conditions;

[0112] Condition 5: The current image block does not use skip mode for prediction.

[0113] According to a possible implementation of the fourth aspect, if the available status information of the CIIP mode obtained by parsing the code stream indicates that the CIIP mode is not available, TPM is used as the target fusion mode

[0114] In the present application, when the first-level fusion mode is not available, it is possible to determine whether to select the CIIP mode as the final fusion mode based on some preset conditions, and as long as any of the preset conditions is not met, the CIIP mode can be directly determined as the target fusion mode, which can reduce the process redundancy generated in the process of determining the target fusion.

[0115] In a fifth aspect, an image prediction method is provided, the method comprising: determining whether a current image block uses a fusion mode, and continuing to determine whether a first-level fusion mode is available when the current image block uses the fusion mode; when the first-level fusion mode is not available and the high-level syntax element corresponding to the first fusion mode set indicates that the fusion mode in the first fusion mode set is prohibited from use, determining a target fusion mode applicable to the current image block from the second fusion mode set; and predicting the current image block according to the above-mentioned target fusion mode.

[0116] Among them, the above-mentioned first fusion mode set and the second fusion mode set both belong to the second-level fusion mode, that is, the second-level fusion mode is composed of the first fusion mode set and the second fusion mode set. In addition, for the current image block, the first-level fusion mode and the second-level fusion mode already include all the optional fusion modes of the current image block. For the current image block, it is necessary to determine the final target fusion mode from the first-level fusion mode and the second-level fusion mode.

[0117] Optionally, the first fusion mode set includes at least one fusion mode, and the second fusion mode set includes at least one fusion mode.

[0118] It should be understood that the above-mentioned first fusion mode set and second fusion mode set are concepts introduced only for the convenience of description, mainly to distinguish different fusion modes. In the process of actually determining the final target fusion mode, the above-mentioned first fusion mode set and second fusion mode set may not exist.

[0119] In the present application, when the high-level syntax elements of certain fusion modes indicate that these fusion modes are prohibited from use, there is no need to parse the available status information of these fusion modes. Instead, the final target fusion mode can be directly determined from the remaining optional fusion modes, which can minimize the process redundancy caused by determining the target fusion mode during the image prediction process.

[0120] In combination with the fifth aspect, in certain implementations of the fifth aspect, when the first-level fusion mode is not available and the high-level syntax element corresponding to the first fusion mode set indicates that the fusion mode in the first fusion mode set is allowed to be used, the target fusion mode is determined based on the high-level syntax element corresponding to the second fusion mode set and / or the available status information of the second fusion mode set.

[0121] The available status information of the second fusion mode set is used to indicate whether to use a fusion mode in the second fusion mode set when predicting the current image block.

[0122] For example, the second fusion mode set consists of the CIIP mode, then the available state information of the second fusion mode set may be the value of ciip_flag. When ciip_flag is 0, the CIIP mode is unavailable for the current image block, and when ciip_flag is 1, the CIIP mode is available for the current image block.

[0123] In combination with the fifth aspect, in some implementations of the fifth aspect, the first fusion mode set includes a triangulation TPM mode, and the second fusion mode set includes a joint intra-frame and inter-frame prediction CIIP mode.

[0124] Optionally, the first fusion mode set consists of TPM modes, and the second mode set consists of CIIP modes.

[0125] When the above-mentioned first fusion mode set and the second fusion mode consist of only one fusion mode, if the fusion mode in the first fusion mode set is disabled, then the fusion mode in the second fusion mode set can be determined as the target fusion mode; if the fusion mode in the second fusion mode set is disabled, then the fusion in the first fusion mode set can be determined as the target fusion mode.

[0126] When the first fusion mode set and the second fusion mode set are both composed of only one fusion mode, as long as the fusion mode in one of the fusion mode sets is prohibited from use, then the fusion mode in the other fusion mode set can be directly determined as the final target fusion mode.

[0127] In combination with the fifth aspect, in certain implementations of the fifth aspect, when the first-level fusion mode is not available and the high-level syntax element corresponding to the first fusion mode set indicates that the fusion mode in the first fusion mode set is prohibited from use, a target fusion mode applicable to the current image block is determined from the second fusion mode set, including: when the first-level fusion mode is not available and the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from use, the CIIP mode is determined as the target fusion mode.

[0128] In the sixth aspect, an image prediction device is provided, which includes a module corresponding to the method of any one of the first to fifth aspects above, and the corresponding module can implement each step of the method of any one of the first to fifth aspects above.

[0129] The image prediction device in the sixth aspect mentioned above may include one or more modules, and any one of the one or more modules may be composed of any one of a circuit, a field programmable gate array FPGA, a special application integrated circuit ASIC and a general-purpose processor.

[0130] The image prediction device in the sixth aspect mentioned above may be located in an encoder device or a decoding device.

[0131] In a seventh aspect, an image prediction device is provided, comprising a memory and a processor, wherein the processor calls a program code stored in the memory to execute the method of any one of the first aspect, the second aspect and the third aspect.

[0132] The image prediction device in the seventh aspect described above may be located in an image encoding device or an image decoding device.

[0133] In an eighth aspect, an image encoding / decoding device is provided, which includes a module corresponding to the method of any one of the first to fifth aspects above, and the corresponding module can implement each step of the method of any one of the first to fifth aspects.

[0134] In a ninth aspect, an image encoding / decoding device is provided, comprising a memory and a processor, wherein the processor calls a program code stored in the memory to execute a method of any one of the first to fifth aspects.

[0135] Optionally, the above-mentioned memory is a non-volatile memory.

[0136] Optionally, the memory and the processor are coupled to each other.

[0137] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and the instructions enable one or more processors to execute the method in any one of the first to fifth aspects.

[0138] Any one of the one or more processors mentioned above may be composed of any one of a circuit, a field programmable gate array FPGA, an application specific integrated circuit ASIC and a general-purpose processor.

[0139] In the eleventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute part or all of the steps of the method in any one of the first to fifth aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0140] Figure 1 is a schematic block diagram of an example of a video encoding system for implementing an embodiment of the present application;

[0141] Figure 2 is a schematic structural block diagram of an example of a video encoder for implementing an embodiment of the present application;

[0142] Figure 3 is a schematic structural block diagram of an example of a video decoder for implementing an embodiment of the present application;

[0143] Figure 4 is a schematic structural block diagram of an example of a video decoding system for implementing an embodiment of the present application;

[0144] Figure 5 is a schematic structural block diagram of an example of a video decoding device for implementing an embodiment of the present application;

[0145] Figure 6 is a schematic block diagram of an example of an encoding device or a decoding device for implementing an embodiment of the present application;

[0146] Figure 7 is a schematic diagram of the spatial and temporal candidate motion information of the current coding unit;

[0147] Figure 8 is a schematic diagram of an MMVD search point for implementing an embodiment of the present application;

[0148] Fig. 9 is another schematic diagram of an MMVD search point for implementing an embodiment of the present application;

[0149] Fig.10 is an example graph of triangulation;

[0150] Fig.11 It is a schematic diagram of the prediction method under the triangulation method;

[0151] Fig.12 is a schematic block diagram of a video communication system according to an embodiment of the present application;

[0152] Fig.13 is a schematic flow chart of an image prediction method according to an embodiment of the present application;

[0153] Fig.14 is a schematic flow chart of an image prediction method according to an embodiment of the present application;

[0154] Fig.15 is a schematic flow chart of an image prediction method according to an embodiment of the present application;

[0155] Fig.16 is a schematic block diagram of an image prediction device according to an embodiment of the present application;

[0156] Fig.17 is a schematic block diagram of an image prediction device according to an embodiment of the present application;

[0157] Fig.18 It is a schematic block diagram of an image encoding / decoding device according to an embodiment of the present application. DETAILED DESCRIPTION

[0158] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0159] In the following description, reference will be made to the drawings that form a part of the present application and illustrate specific aspects of the embodiments of the present application or specific aspects of the embodiments of the present application that can be used by way of illustration. It should be understood that the embodiments of the present application can also be used in other aspects and may include structural or logical changes not depicted in the drawings. Therefore, the following detailed description should not be understood in a restrictive sense, and the scope of the present application should be defined by the appended claims.

[0160] For example, it should be understood that the disclosure in conjunction with a described method may also be applicable to a corresponding device or system that performs the method, and vice versa.

[0161] For another example, if one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, to perform the described one or more method steps (for example, one unit performs one or more steps, or multiple units, each of which performs one or more of the multiple steps), even if such one or more units are not explicitly described or illustrated in the drawings.

[0162] In addition, if a specific device is described based on one or more units such as functional units, the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps, each of which performs the function of one or more units in multiple units), even if such one or more steps are not explicitly described or illustrated in the drawings. Further, it should be understood that unless otherwise explicitly stated, the features of the various exemplary embodiments and / or aspects described herein can be combined with each other.

[0163] The technical solutions involved in the embodiments of the present application can be applied to the H.266 standard and future video coding standards. The terms used in the implementation method of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. The following is a brief introduction to some concepts that may be involved in the embodiments of the present application.

[0164] Video coding generally refers to processing a sequence of pictures that form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" can be used as synonyms. Video coding used in this article means video encoding or video decoding. Video encoding is performed on the source side and generally includes processing (e.g., by compression) the original video picture to reduce the amount of data required to represent the video picture, thereby more efficiently storing and / or transmitting. Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video picture. The "encoding" of video pictures involved in the embodiments should be understood as involving "encoding" or "decoding" of video sequences. The combination of the encoding part and the decoding part is also referred to as codec (encoding and decoding).

[0165] A video sequence includes a series of pictures, which are further divided into slices, and slices are further divided into blocks. Video coding is performed in blocks. In some new video coding standards, the concept of blocks is further expanded. For example, in the H.264 standard, there are macroblocks (MB), which can be further divided into multiple prediction blocks (partitions) that can be used for predictive coding. In the high-performance video coding (HEVC) standard, basic concepts such as coding units (CU), prediction units (PU) and transform units (TU) are used to functionally divide various block units and describe them using a new tree-based structure. For example, CU can be divided into smaller CUs according to a quadtree, and smaller CUs can be further divided to form a quadtree structure. CU is the basic unit for dividing and encoding coded images. There are similar tree structures for PU and TU. PU can correspond to prediction blocks and is the basic unit of predictive coding. CU is further divided into multiple PUs according to the division mode. TU may correspond to a transform block, which is a basic unit for transforming a prediction residual. However, no matter CU, PU or TU, they all belong to the concept of a block (or image block) in essence.

[0166] For example, in HEVC, a CTU is split into multiple CUs using a quadtree structure represented as a coding tree. A decision is made at the CU level whether to use inter-picture (temporal) or intra-picture (spatial) prediction to encode a picture area. Each CU can be further split into one, two, or four PUs according to the PU split type. The same prediction process is applied within a PU, and relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU split type, the CU can be divided into transform units (TUs) according to other quadtree structures similar to the coding tree for the CU. In the latest developments in video compression technology, quad-tree and binarytree (QTBT) are used to split frames to split coding blocks. In the QTBT block structure, the CU can be square or rectangular in shape.

[0167] In this document, for ease of description and understanding, the image block to be encoded in the current encoded image may be referred to as the current image block. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded. The decoded image block in the reference image used to predict the current image block is referred to as a reference block, that is, the reference block is a block that provides a reference signal for the current image block, wherein the reference signal represents the pixel value in the image block. The block in the reference image that provides a prediction signal for the current image block may be referred to as a prediction block, wherein the prediction signal represents the pixel value or sampling value or sampling signal in the prediction block. For example, after traversing multiple reference blocks, the best reference block is found. This best reference block will provide a prediction for the current image block, and this block is referred to as a prediction block.

[0168] In the case of lossless video coding, the original video picture can be reconstructed, that is, the reconstructed video picture has the same quality as the original video picture (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed by, for example, quantization to reduce the amount of data required to represent the video picture, but the decoder side cannot fully reconstruct the video picture, that is, the quality of the reconstructed video picture is lower or worse than the quality of the original video picture.

[0169] Several video coding standards of H.261 belong to the category of "lossy hybrid video codecs" (i.e., combining spatial and temporal prediction in the sample domain with 2D transform coding in the transform domain for applying quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, which are usually encoded at the block level. In other words, the encoder side usually processes, i.e., encodes the video at the block (video block) level, for example, by generating a prediction block through spatial (intra-picture) prediction and temporal (inter-picture) prediction, subtracting the prediction block from the current image block (currently processed or to be processed block) to obtain a residual block, transforming the residual block in the transform domain and quantizing the residual block to reduce the amount of data to be transmitted (compressed), while the decoder side applies the inverse processing part relative to the encoder to the coded or compressed block to reconstruct the current image block for representation. In addition, the encoder replicates the decoder processing cycle, so that the encoder and decoder generate the same prediction (e.g., intra-frame prediction and inter-frame prediction) and / or reconstruction for processing, i.e., encoding subsequent blocks.

[0170] The following describes the system architecture used in the embodiments of the present application. Figure 1 , Figure 1 A schematic block diagram of a video encoding and decoding system 10 used in an embodiment of the present application is given as an example. Figure 1As shown, the video encoding and decoding system 10 may include a source device 12 and a destination device 14. The source device 12 generates encoded video data, and therefore, the source device 12 may be referred to as a video encoding device. The destination device 14 may decode the encoded video data generated by the source device 12, and therefore, the destination device 14 may be referred to as a video decoding device. Various implementations of the source device 12, the destination device 14, or both may include one or more processors and a memory coupled to the one or more processors. The memory may include, but is not limited to, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), a flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, as described herein. Source device 12 and destination device 14 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, or the like.

[0171] Although Figure 1 The source device 12 and the destination device 14 are illustrated as separate devices, but device embodiments may also include both the source device 12 and the destination device 14 or the functions of both, that is, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions. In such embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0172] The source device 12 and the destination device 14 may be connected in communication via a link 13, and the destination device 14 may receive the encoded video data from the source device 12 via the link 13. The link 13 may include one or more media or devices capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the link 13 may include one or more communication media that enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. In this example, the source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol), and may transmit the modulated video data to the destination device 14. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 12 to the destination device 14.

[0173] The source device 12 includes an encoder 20. Optionally, the source device 12 may also include a picture source 16, a picture preprocessor 18, and a communication interface 22. In a specific implementation, the encoder 20, the picture source 16, the picture preprocessor 18, and the communication interface 22 may be hardware components in the source device 12, or may be software programs in the source device 12. They are described as follows:

[0174] The picture source 16 may include or may be any type of picture capture device, for example, for capturing real-world pictures, and / or any type of pictures or comments (for screen content encoding, some text on the screen is also considered to be part of the picture or image to be encoded) generating device, for example, a computer graphics processor for generating computer animation pictures, or any type of device for acquiring and / or providing real-world pictures, computer animation pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). The picture source 16 may be a camera for capturing pictures or a memory for storing pictures, and the picture source 16 may also include any type of (internal or external) interface for storing previously captured or generated pictures and / or acquiring or receiving pictures. When the picture source 16 is a camera, the picture source 16 may be, for example, a local or integrated camera integrated in the source device; when the picture source 16 is a memory, the picture source 16 may be, for example, a local or integrated memory integrated in the source device. When the picture source 16 includes an interface, the interface may be, for example, an external interface for receiving pictures from an external video source, such as an external picture capture device, such as a camera, an external memory, or an external picture generating device, such as an external computer graphics processor, a computer, or a server. The interface may be any type of interface according to any proprietary or standardized interface protocol, such as a wired or wireless interface, an optical interface.

[0175] Among them, the picture can be regarded as a two-dimensional array or matrix of pixels (picture element). The pixels in the array can also be called sampling points. The number of sampling points in the array or picture in the horizontal and vertical directions (or axes) defines the size and / or resolution of the picture. In order to represent the color, three color components are usually used, that is, the picture can be represented as or include three sampling arrays. For example, in the RBG format or color space, the picture includes corresponding red, green and blue sampling arrays. However, in video coding, each pixel is usually represented in a brightness / chroma format or color space, such as for a picture in the YUV format, including a brightness component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The brightness (luma) component Y represents the brightness or grayscale level intensity (for example, the two are the same in a grayscale picture), and the two chroma (chroma) components U and V represent the chroma or color information components. Accordingly, the picture in the YUV format includes a brightness sampling array of brightness sampling values ​​(Y), and two chroma sampling arrays of chroma values ​​(U and V). An image in RGB format may be converted or transformed into YUV format, and vice versa, which process is also referred to as color conversion or transformation. If the image is black and white, the image may only include a brightness sampling array. In the embodiment of the present application, the image transmitted from the image source 16 to the image processor may also be referred to as raw image data 17.

[0176] The image preprocessor 18 is used to receive the original image data 17 and perform preprocessing on the original image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by the image preprocessor 18 may include refurbishment, color format conversion (e.g., from RGB format to YUV format), color adjustment, or denoising.

[0177] The encoder 20 (or video encoder 20) is used to receive the pre-processed picture data 19, and process the pre-processed picture data 19 using a relevant prediction mode (such as the prediction mode in each embodiment of this invention), so as to provide encoded picture data 21 (hereinafter further based on Figure 2 or Figure 4 or Figure 5 ). In some embodiments, the encoder 20 may be used to execute various embodiments described below to implement the application of the image prediction method described in this application on the encoding side.

[0178] The communication interface 22 can be used to receive the encoded picture data 21 and transmit the encoded picture data 21 to the destination device 14 or any other device (such as a memory) through the link 13 for storage or direct reconstruction. The other device can be any device for decoding or storage. The communication interface 22 can, for example, be used to encapsulate the encoded picture data 21 into a suitable format, such as a data packet, for transmission on the link 13.

[0179] The destination device 14 includes a decoder 30. Optionally, the destination device 14 may also include a communication interface 28, a picture post-processor 32, and a display device 34. They are described as follows:

[0180] The communication interface 28 can be used to receive the encoded picture data 21 from the source device 12 or any other source, such as a storage device, such as an encoded picture data storage device. The communication interface 28 can be used to transmit or receive the encoded picture data 21 via the link 13 between the source device 12 and the destination device 14 or via any type of network, such as a direct wired or wireless connection, any type of network, such as a wired or wireless network or any combination thereof, or any type of private network and public network, or any combination thereof. The communication interface 28 can be used, for example, to decapsulate the data packet transmitted by the communication interface 22 to obtain the encoded picture data 21.

[0181] Both communication interface 28 and communication interface 22 may be configured as unidirectional communication interfaces or bidirectional communication interfaces and may be used, for example, to send and receive messages to establish connections, confirm and exchange any other information related to communication links and / or data transmissions such as encoded picture data transmissions.

[0182] The decoder 30 (or decoder 30) is configured to receive the encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (hereinafter further described based on Figure 3 or Figure 4 or Figure 5 ). In some embodiments, the decoder 30 may be used to execute various embodiments described below to implement the application of the image prediction method described in this application on the decoding side.

[0183] The picture post-processor 32 is used to perform post-processing on the decoded picture data 31 (also called reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 may include: color format conversion (for example, from YUV format to RGB format), color adjustment, repair or resampling, or any other processing, and can also be used to transmit the post-processed picture data 33 to the display device 34.

[0184] A display device 34 is provided for receiving the post-processed picture data 33 to display the picture to, for example, a user or viewer. The display device 34 may be or may include any type of display for presenting the reconstructed picture, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0185] Although, Figure 1 1 and 14 are shown as separate devices, but device embodiments may also include both the source device 12 and the destination device 14 or both functionalities, that is, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality. In such embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.

[0186] It is obvious to a person skilled in the art based on the description that the functionality or Figure 1 The presence and (exact) division of the functionality of the source device 12 and / or destination device 14 shown may vary depending on the actual device and application. The source device 12 and the destination device 14 may include any of a variety of devices, including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, a car device, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may not use or use any type of operating system.

[0187] The encoder 20 and the decoder 30 may be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is partially implemented in software, the device may store the instructions of the software in a suitable non-transitory computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to perform the technology of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors.

[0188] In some cases, Figure 1 The video encoding and decoding system 10 shown in is only an example, and the technology of the present application can be applied to video encoding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between encoding and decoding devices. In other examples, data can be retrieved from local storage, streamed over a network, etc. A video encoding device can encode data and store the data in a memory, and / or a video decoding device can retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve data from a memory and decode data.

[0189] See also Figure 2 , Figure 2 Schematic / conceptual block diagram of an example of an encoder 20 for implementing an embodiment of the present application is shown. Figure 2 In the example of , the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy encoding unit 270. The prediction processing unit 260 may include an inter-prediction unit 244, an intra-prediction unit 254, and a mode selection unit 262. The inter-prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The illustrated encoder 20 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0190] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 20, while for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder, wherein the backward signal path of the encoder corresponds to the signal path of the decoder (see Figure 3 Decoder 30 in ).

[0191] The encoder 20 receives a picture 201 or an image block 203 of the picture 201, e.g., a picture in a sequence of pictures forming a video or a video sequence, via, e.g., an input 202. The image block 203 may also be referred to as a current picture block or a picture block to be encoded, and the picture 201 may be referred to as a current picture or a picture to be encoded (particularly when the current picture is distinguished from other pictures in video coding, e.g., previously encoded and / or decoded pictures in the same video sequence, i.e., a video sequence that also includes the current picture).

[0192] An embodiment of the encoder 20 may include a segmentation unit ( Figure 2 201 into a plurality of blocks, such as image blocks 203, typically into a plurality of non-overlapping blocks. The segmentation unit may be used to use the same block size for all pictures in the video sequence and define a corresponding grid of block sizes, or to change the block size between pictures or subsets or groups of pictures and to segment each picture into corresponding blocks.

[0193] In one example, prediction processing unit 260 of encoder 20 may be used to perform any combination of the above-described partitioning techniques.

[0194] Like the picture 201, the image block 203 is also or can be regarded as a two-dimensional array or matrix of sampling points with sample values, although its size is smaller than that of the picture 201. In other words, the image block 203 may include, for example, one sampling array (e.g., a luminance array in the case of the black and white picture 201) or three sampling arrays (e.g., one luminance array and two chrominance arrays in the case of a color picture) or any other number and / or type of arrays depending on the color format applied. The number of sampling points in the horizontal and vertical directions (or axes) of the image block 203 defines the size of the image block 203.

[0195] like Figure 2 The encoder 20 shown is used to encode the picture 201 block by block, for example, encoding and prediction are performed on each image block 203.

[0196] The residual calculation unit 204 is used to calculate the residual block 205 based on the picture image block 203 and the prediction block 265 (other details of the prediction block 265 are provided below), for example, by subtracting the sample value of the prediction block 265 from the sample value of the picture image block 203 sample by sample (pixel by pixel) to obtain the residual block 205 in the sample domain.

[0197] The transform processing unit 206 is used to apply a transform such as discrete cosine transform (DCT) or discrete sine transform (DST) on the sample values ​​of the residual block 205 to obtain a transform coefficient 207 in the transform domain. The transform coefficient 207 can also be called a transform residual coefficient and represents the residual block 205 in the transform domain.

[0198] The transform processing unit 206 can be used to apply an integer approximation of the DCT / DST, such as the transform specified for HEVC / H.265. Compared to the orthogonal DCT transform, this integer approximation is usually scaled by a certain factor. In order to maintain the norm of the residual block processed by the forward transform and the inverse transform, an additional scaling factor is applied as part of the transform process. The scaling factor is usually selected based on certain constraints, such as the scaling factor is a power of 2 for the shift operation, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform on the decoder 30 side by, for example, the inverse transform processing unit 212 (and for the corresponding inverse transform on the encoder 20 side by, for example, the inverse transform processing unit 212), and accordingly, a corresponding scaling factor can be specified for the forward transform on the encoder 20 side by the transform processing unit 206.

[0199] The quantization unit 208 is used to quantize the transform coefficients 207, for example, by applying scalar quantization or vector quantization to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. A suitable quantization step size may be indicated by a quantization parameter (QP). For example, the quantization parameter may be an index to a predefined set of suitable quantization step sizes. For example, a smaller quantization parameter may correspond to fine quantization (a smaller quantization step size), a larger quantization parameter may correspond to coarse quantization (a larger quantization step size), and vice versa. Quantization may include division by a quantization step size and corresponding quantization or inverse quantization, for example, performed by inverse quantization 210, or may include multiplication by a quantization step size. Embodiments according to some standards such as HEVC may use a quantization parameter to determine the quantization step size. In general, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of an equation containing division. Additional scaling factors may be introduced for quantization and inverse quantization to recover the norm of the residual block that may be modified due to the scale used in the fixed-point approximation of the equation for the quantization step size and the quantization parameter. In an example embodiment, the scales of the inverse transform and inverse quantization may be merged. Alternatively, a custom quantization table may be used and sent from an encoder to a decoder through a signal, for example in a bitstream. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

[0200] The inverse quantization unit 210 is used to apply the inverse quantization of the quantization unit 208 on the quantized coefficients to obtain inverse quantized coefficients 211, for example, applying an inverse quantization scheme of the quantization scheme applied by the quantization unit 208 based on or using the same quantization step size as the quantization unit 208. The inverse quantized coefficients 211 may also be referred to as inverse quantized residual coefficients 211, corresponding to the transform coefficients 207, although the loss due to quantization is generally different from that of the transform coefficients.

[0201] The inverse transform processing unit 212 is used to apply the inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform dequantized block 213 or an inverse transform residual block 213.

[0202] The reconstruction unit 214 (e.g., the summer 214) is used to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 213 to the sample values ​​of the prediction block 265.

[0203] Optionally, a buffer unit 216 (or simply "buffer" 216), such as a line buffer 216, is used to buffer or store the reconstructed blocks 215 and corresponding sample values ​​for, for example, intra-frame prediction. In other embodiments, the encoder can be used to use the unfiltered reconstructed blocks and / or corresponding sample values ​​stored in the buffer unit 216 for any type of estimation and / or prediction, such as intra-frame prediction.

[0204] For example, embodiments of the encoder 20 may be configured such that the buffer unit 216 is used not only to store the reconstructed blocks 215 used for the intra prediction 254, but also for the loop filter unit 220 (in Figure 2 ), and / or, for example, such that the buffer unit 216 and the decoded picture buffer unit 230 form a buffer. Other embodiments may be used to store the filtered block 221 and / or the blocks or samples ( Figure 2 (not shown) is used as input or basis for intra-frame prediction 254.

[0205] The loop filter unit 220 (or simply loop filter 220) is used to filter the reconstructed block 215 to obtain a filtered block 221, so as to smoothly perform pixel conversion or improve video quality. The loop filter unit 220 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 220 is in Figure 2 2 is shown as an in-loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coding block after the loop filter unit 220 performs a filtering operation on the reconstructed coding block.

[0206] Embodiments of the encoder 20 (correspondingly, the loop filter unit 220) can be used to output loop filter parameters (e.g., sample adaptive offset information), for example, directly or after entropy encoding by the entropy encoding unit 270 or any other entropy encoding unit, such that the decoder 30 can receive and apply the same loop filter parameters for decoding.

[0207] The decoded picture buffer (DPB) 230 may be a reference picture memory for storing reference picture data for use by the encoder 20 in encoding video data. The DPB 230 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM)) or other types of memory devices. The DPB 230 and the buffer 216 may be provided by the same memory device or separate memory devices. In one example, the decoded picture buffer (DPB) 230 is used to store the filtered block 221. The decoded picture buffer 230 may further be used to store other previously filtered blocks of the same current picture or a different picture, such as a previously reconstructed picture, such as the previously reconstructed and filtered block 221, and may provide a complete previously reconstructed, i.e., decoded picture (and corresponding reference blocks and samples) and / or a portion of the reconstructed current picture (and corresponding reference blocks and samples), such as for inter-frame prediction. In one example, if the reconstructed block 215 is reconstructed without in-loop filtering, the decoded picture buffer (DPB) 230 is used to store the reconstructed block 215.

[0208] The prediction processing unit 260, also referred to as the block prediction processing unit 260, is used to receive or obtain an image block 203 (a current image block 203 of a current picture 201) and reconstructed picture data, such as reference samples of the same (current) picture from a buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from a decoded picture buffer 230, and to process such data for prediction, i.e. to provide a prediction block 265 which may be an inter-prediction block 245 or an intra-prediction block 255.

[0209] The mode selection unit 262 may be used to select a prediction mode (eg, intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 for use as a prediction block 265 to calculate the residual block 205 and reconstruct the reconstructed block 215 .

[0210] Embodiments of the mode selection unit 262 may be used to select a prediction mode (e.g., from those prediction modes supported by the prediction processing unit 260) that provides the best match or minimum residual (minimum residual means better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead means better compression in transmission or storage), or considers or balances both. The mode selection unit 262 may be used to determine the prediction mode based on rate distortion optimization (RDO), i.e., select a prediction mode that provides minimum rate distortion optimization, or select a prediction mode whose associated rate distortion at least satisfies the prediction mode selection criteria.

[0211] The prediction processing performed by an example of encoder 20 (eg, by prediction processing unit 260) and the mode selection performed (eg, by mode selection unit 262) will be explained in detail below.

[0212] As described above, the encoder 20 is used to determine or select the best or optimal prediction mode from a (predetermined) prediction mode set. The prediction mode set may include, for example, an intra-frame prediction mode and / or an inter-frame prediction mode.

[0213] The intra-frame prediction mode set may include 35 different intra-frame prediction modes, for example, non-directional modes such as DC (or mean) mode and planar mode, or directional modes as defined in H.265, or may include 67 different intra-frame prediction modes, for example, non-directional modes such as DC (or mean) mode and planar mode, or directional modes as defined in H.266 under development.

[0214] In a possible implementation, the inter-frame prediction mode set depends on the available reference picture (i.e., at least part of the decoded picture stored in the DBP 230 as described above) and other inter-frame prediction parameters, such as whether to use the entire reference picture or only a part of the reference picture, such as a search window area around the area of ​​the current image block, to search for the best matching reference block, and / or, for example, whether to apply pixel interpolation such as half-pixel and / or quarter-pixel interpolation. The inter-frame prediction mode set may include, for example, an advanced motion vector prediction (AMVP) mode and a merge mode. In a specific implementation, the inter-frame prediction mode set may include an improved control point-based AMVP mode according to an embodiment of the present application, and an improved control point-based merge mode. In one example, the intra-frame prediction unit 254 may be used to perform any combination of the inter-frame prediction techniques described below.

[0215] In addition to the above prediction modes, embodiments of the present application may also apply skip mode and / or direct mode.

[0216] The prediction processing unit 260 can be further used to partition the image block 203 into smaller block partitions or sub-blocks, for example, by iteratively using quad-tree (QT) partitioning, binary-tree (BT) partitioning or triple-tree (TT) partitioning, or any combination thereof, and for performing prediction for each of the block partitions or sub-blocks, for example, wherein the mode selection includes selecting a tree structure of the partitioned image block 203 and selecting a prediction mode to be applied to each of the block partitions or sub-blocks.

[0217] The inter-frame prediction unit 244 may include a motion estimation (ME) unit ( Figure 2 ) and a motion compensation (MC) unit ( Figure 2 ). The motion estimation unit is used to receive or obtain a picture image block 203 (a current picture image block 203 of a current picture 201) and a decoded picture 231, or at least one or more previously reconstructed blocks, for example, reconstructed blocks of one or more other / different previously decoded pictures 231, to perform motion estimation. For example, a video sequence may include a current picture and a previously decoded picture 31, or in other words, the current picture and the previously decoded picture 31 may be part of a picture sequence forming a video sequence, or form the picture sequence.

[0218] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different pictures in a plurality of other pictures and to provide the reference block to the motion estimation unit ( Figure 2(not shown) provides a reference picture and / or provides an offset (spatial offset) between the position (X, Y coordinates) of a reference block and the position of a current image block as an inter-frame prediction parameter. The offset is also called a motion vector (MV).

[0219] The motion compensation unit is used to obtain inter-frame prediction parameters, and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain the inter-frame prediction block 245. Figure 2 Motion compensation performed by the motion compensation unit 246 (not shown) may include fetching or generating a prediction block based on a motion / block vector determined by motion estimation (possibly performing interpolation to sub-pixel accuracy). Interpolation filtering may produce additional pixel samples from known pixel samples, thereby potentially increasing the number of candidate prediction blocks that may be used to encode the picture block. Upon receiving the motion vector for the PU of the current picture block, the motion compensation unit 246 may locate the prediction block to which the motion vector points in a reference picture list. The motion compensation unit 246 may also generate syntax elements associated with blocks and video slices for use by the decoder 30 when decoding picture blocks of the video slice.

[0220] Specifically, the inter-frame prediction unit 244 may transmit syntax elements to the entropy coding unit 270, wherein the syntax elements include inter-frame prediction parameters (such as indication information of selecting an inter-frame prediction mode for current image block prediction after traversing multiple inter-frame prediction modes). In possible application scenarios, if there is only one inter-frame prediction mode, the inter-frame prediction parameters may not be carried in the syntax elements, and the decoding end 30 may directly use the default prediction mode for decoding. It is understandable that the inter-frame prediction unit 244 may be used to perform any combination of inter-frame prediction techniques.

[0221] The intra prediction unit 254 is used to obtain, for example, receive a picture block 203 (current picture block) of the same picture and one or more previously reconstructed blocks, for example, reconstructed adjacent blocks, for intra estimation. For example, the encoder 20 can be used to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[0222] Embodiments of the encoder 20 may be configured to select an intra prediction mode based on an optimization criterion, such as minimum residual (eg, the intra prediction mode that provides a prediction block 255 most similar to the current picture block 203) or minimum rate distortion.

[0223] The intra prediction unit 254 is further configured to determine an intra prediction block 255 based on the intra prediction parameters of the selected intra prediction mode. In any case, after selecting the intra prediction mode for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters, i.e., provide information indicating the selected intra prediction mode for the block, to the entropy encoding unit 270. In one example, the intra prediction unit 254 can be configured to perform any combination of intra prediction techniques.

[0224] Specifically, the intra prediction unit 254 may transmit syntax elements to the entropy coding unit 270, wherein the syntax elements include intra prediction parameters (such as indication information of selecting an intra prediction mode for current image block prediction after traversing multiple intra prediction modes). In a possible application scenario, if there is only one intra prediction mode, the intra prediction parameters may not be carried in the syntax elements, and the decoding end 30 may directly use the default prediction mode for decoding.

[0225] The entropy coding unit 270 is used to apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC), a syntax-based context-adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to a single or all (or none) of the quantized residual coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, and / or loop filter parameters to obtain the encoded picture data 21 that can be output through the output 272 in the form of, for example, an encoded bitstream 21. The encoded bitstream can be transmitted to the video decoder 30, or archived for later transmission or retrieval by the video decoder 30. The entropy coding unit 270 can also be used to entropy encode other syntax elements of the current video slice being encoded.

[0226] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform based encoder 20 may directly quantize the residual signal without a transform processing unit 206 for certain blocks or frames. In another embodiment, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[0227] Specifically, in the embodiment of the present application, the encoder 20 may be used to implement the video encoding process described in the embodiments below.

[0228] It should be understood that the video encoder in the present application may include only some modules in the video encoder 20. For example, the video encoder in the present application may include an image decoding unit and a partitioning unit. The image decoding unit may be composed of one or more units of an entropy decoding unit, a prediction unit, an inverse transform unit, and an inverse quantization unit.

[0229] In addition, other structural changes of the video encoder 20 can be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 20 can directly quantize the residual signal without processing by the transform processing unit 206, and accordingly, it does not need to be processed by the inverse transform processing unit 212; or, for some image blocks or image frames, the video encoder 20 does not generate residual data, and accordingly, it does not need to be processed by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210 and the inverse transform processing unit 212; or, the video encoder 20 can directly store the reconstructed image block as a reference block without processing by the filter 220; or, the quantization unit 208 and the inverse quantization unit 210 in the video encoder 20 can be combined together. The loop filter 220 is optional, and in the case of lossless compression encoding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210 and the inverse transform processing unit 212 are optional. It should be understood that according to different application scenarios, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 can be selectively enabled.

[0230] See also Figure 3 , Figure 3 A schematic / conceptual block diagram of an example of a decoder 30 for implementing an embodiment of the present application is shown. The video decoder 30 is used to receive encoded picture data (e.g., encoded bitstream) 21, for example, encoded by the encoder 20, to obtain a decoded picture 231. During the decoding process, the video decoder 30 receives video data from the video encoder 20, such as an encoded video bitstream representing picture blocks of an encoded video slice and associated syntax elements.

[0231] exist Figure 3 In the example of , decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter-prediction unit 344, an intra-prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform substantially the same as reference. Figure 2 The encoding pass described by video encoder 20 is the inverse of the decoding pass.

[0232] The entropy decoding unit 304 is used to perform entropy decoding on the encoded picture data 21 to obtain, for example, quantized coefficients 309 and / or decoded encoding parameters ( Figure 3 The entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 30 may receive syntax elements at the video slice level and / or the video block level.

[0233] The inverse quantization unit 310 may be functionally the same as the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally the same as the inverse transform processing unit 212, the reconstruction unit 314 may be functionally the same as the reconstruction unit 214, the buffer 316 may be functionally the same as the buffer 216, the loop filter 320 may be functionally the same as the loop filter 220, and the decoded picture buffer 330 may be functionally the same as the decoded picture buffer 230.

[0234] The prediction processing unit 360 may include an inter-prediction unit 344 and an intra-prediction unit 354, wherein the inter-prediction unit 344 may be functionally similar to the inter-prediction unit 244, and the intra-prediction unit 354 may be functionally similar to the intra-prediction unit 254. The prediction processing unit 360 is generally used to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and receive or obtain prediction related parameters and / or information about the selected prediction mode from, for example, the entropy decoding unit 304 (explicitly or implicitly).

[0235] When the video slice is encoded as an intra-coded (I) slice, the intra-prediction unit 354 of the prediction processing unit 360 is used to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra-prediction mode and data from a previously decoded block of the current frame or picture. When the video frame is encoded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., motion compensation unit) of the prediction processing unit 360 is used to generate a prediction block 365 for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. For inter-prediction, the prediction block may be generated from one of the reference pictures within one of the reference picture lists. Video decoder 30 may construct reference frame lists: List 0 and List 1 using a default construction technique based on the reference pictures stored in DPB 330.

[0236] The prediction processing unit 360 is configured to determine prediction information for a video block of a current video slice by parsing motion vectors and other syntax elements, and use the prediction information to generate a prediction block for the current video block being decoded. In one example of the present application, the prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra-frame or inter-frame prediction) for encoding a video block of a video slice, an inter-frame prediction slice type (e.g., B slice, P slice, or GPB slice), construction information of one or more of the reference picture lists for the slice, a motion vector for each inter-frame coded video block of the slice, an inter-frame prediction state of each inter-frame coded video block of the slice, and other information to decode the video block of the current video slice. In another example of the present application, the syntax elements received by the video decoder 30 from the bitstream include syntax elements in one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

[0237] Inverse quantization unit 310 may be used to inverse quantize (i.e., dequantize) quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each video block in a video slice to determine a degree of quantization that should be applied and likewise a degree of inverse quantization that should be applied.

[0238] The inverse transform processing unit 312 is used to apply an inverse transform (eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to generate a residual block in the pixel domain.

[0239] The reconstruction unit 314 (e.g., the summer 314) is used to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain the reconstructed block 315 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the prediction block 365.

[0240] The loop filter unit 320 is used to filter the reconstructed block 315 (during or after the encoding cycle) to obtain a filtered block 321, thereby smoothing pixel conversion or improving video quality. In one example, the loop filter unit 320 can be used to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is in Figure 3 3. Although shown as an in-loop filter in FIG. 3, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0241] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330 which stores reference pictures for subsequent motion compensation.

[0242] Decoder 30 is operable to output decoded picture 31 , eg, via output 332 , for presentation to or viewing by a user.

[0243] Other variations of the video decoder 30 may be used to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform based decoder 30 may directly inverse quantize the residual signal without the inverse transform processing unit 312 for certain blocks or frames. In another embodiment, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0244] Specifically, in the embodiment of the present application, the decoder 30 is used to implement the video decoding method described in the embodiments below.

[0245] It should be understood that the video encoder in the present application may include only some modules in the video encoder 30. For example, the video encoder in the present application may include a partition unit and an image encoding unit. The image encoding unit may be composed of one or more units of a prediction unit, a transform unit, a quantization unit, and an entropy encoding unit.

[0246] In addition, other structural changes of the video decoder 30 can be used to decode the encoded video bitstream. For example, the video decoder 30 can generate an output video stream without being processed by the filter 320; or, for some image blocks or image frames, the entropy decoding unit 304 of the video decoder 30 does not decode the quantized coefficients, and accordingly does not need to be processed by the inverse quantization unit 310 and the inverse transform processing unit 312. The loop filter 320 is optional; and for lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are optional. It should be understood that, according to different application scenarios, the inter-frame prediction unit and the intra-frame prediction unit can be selectively enabled.

[0247] It should be understood that in the encoder 20 and decoder 30 of the present application, the processing result of a certain link can be further processed and output to the next link. For example, after the interpolation filtering, motion vector derivation or loop filtering links, the processing result of the corresponding link can be further clamped (clip) or shifted (shift) and other operations.

[0248] For example, the motion vector of the control point of the current image block derived from the motion vector of the adjacent affine coding block (the coding block predicted by the affine motion model can be called an affine coding block), or the motion vector of the sub-block of the current image block derived, can be further processed, and this application does not limit this. For example, the value range of the motion vector is constrained so that it is within a certain bit width. Assuming that the bit width of the allowed motion vector is bitDepth, the range of the motion vector is -2^(bitDepth-1)~2^(bitDepth-1)-1, where the "^" symbol represents the power. If bitDepth is 16, the value range is -32768~32767. If bitDepth is 18, the value range is -131072~131071.

[0249] For example, the value of the motion vector (for example, the motion vector MV of four 4x4 sub-blocks in an 8x8 image block) can also be constrained so that the maximum difference between the integer parts of the four 4x4 sub-blocks MV does not exceed N (for example, N can be 1) pixels.

[0250] See also Figure 4 , Figure 4 According to an exemplary embodiment, Figure 2 The encoder 20 and / or Figure 330. The video decoding system 40 may implement a combination of various techniques of the embodiments of the present application. In the illustrated embodiment, the video decoding system 40 may include an imaging device 41, an encoder 20, a decoder 30 (and / or a video codec implemented by logic circuits 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0251] like Figure 4 As shown, imaging device 41, antenna 42, processing unit 46, logic circuit 47, encoder 20, decoder 30, processor 43, memory 44 and / or display device 45 can communicate with each other. As discussed, although video decoding system 40 is illustrated with encoder 20 and decoder 30, in different examples, video decoding system 40 may include only encoder 20 or only decoder 30.

[0252] In some examples, antenna 42 can be used to transmit or receive a coded bit stream of video data. In addition, in some examples, display device 45 can be used to present video data. In some examples, logic circuit 47 can be implemented by processing unit 46. Processing unit 46 can include application-specific integrated circuit (ASIC) logic, graphics processor, general processor, etc. Video decoding system 40 can also include optional processor 43, which can similarly include application-specific integrated circuit (ASIC) logic, graphics processor, general processor, etc. In some examples, logic circuit 47 can be implemented by hardware, such as video encoding dedicated hardware, etc., and processor 43 can be implemented by general software, operating system, etc. In addition, memory 44 can be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory 44 can be implemented by cache memory. In some examples, logic circuitry 47 may access memory 44 (eg, for implementing an image buffer). In other examples, logic circuitry 47 and / or processing unit 46 may include memory (eg, cache, etc.) for implementing an image buffer, etc.

[0253] In some examples, encoder 20 implemented by logic circuits may include an image buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include encoder 20 implemented by logic circuits 47 to implement reference Figure 2 The various modules discussed in connection with the encoder and / or any other encoder system or subsystem described herein. Logic circuits may be used to perform the various operations discussed herein.

[0254] In some examples, decoder 30 may be implemented in a similar manner by logic circuit 47 to implement reference Figure 3 The decoder 30 and / or any other decoder system or subsystem described herein may include various modules discussed above. In some examples, the decoder 30 implemented by logic circuits may include an image buffer (implemented by processing unit 2820 or memory 44) and a graphics processing unit (implemented, for example, by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the decoder 30 implemented by logic circuits 47 to implement the reference Figure 3 and / or the various modules discussed above for any other decoder system or subsystem described herein.

[0255] In some examples, antenna 42 may be used to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoded video frames, indicators, index values, mode selection data, etc. discussed herein, such as data related to the encoded partitions (e.g., transform coefficients or quantized transform coefficients, (as discussed) optional indicators, and / or data defining the encoded partitions). Video decoding system 40 may also include decoder 30 coupled to antenna 42 and used to decode the encoded bitstream. Display device 45 is used to present the video frames.

[0256] It should be understood that in the embodiments of the present application, for the examples described with reference to encoder 20, decoder 30 can be used to perform the reverse process. With respect to signaling syntax elements, decoder 30 can be used to receive and parse such syntax elements and decode the associated video data accordingly. In some examples, encoder 20 can entropy encode the syntax elements into an encoded video bitstream. In such examples, decoder 30 can parse such syntax elements and decode the associated video data accordingly.

[0257] See also Figure 5 , Figure 5FIG. 4 is a schematic diagram of the structure of a video decoding device 400 (eg, a video encoding device 400 or a video decoding device 400) provided in an embodiment of the present application. The video decoding device 400 is suitable for implementing the embodiments described herein. In one embodiment, the video decoding device 400 may be a video decoder (eg, Figure 3 decoder 30) or a video encoder (e.g. Figure 2 In another embodiment, the video decoding device 400 may be the above-mentioned Figure 3 Decoder 30 or Figure 2 One or more components in encoder 20.

[0258] The video decoding device 400 includes: an inlet port 410 and a receiving unit (Rx) 420 for receiving data, a processor, a logic unit or a central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an outlet port 450 for transmitting data, and a memory 460 for storing data. The video decoding device 400 may also include an optical-to-electrical conversion component and an electro-optical (EO) component coupled to the inlet port 410, the receiver unit 420, the transmitter unit 440 and the outlet port 450 for the outlet or inlet of an optical signal or an electrical signal.

[0259] The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the inlet port 410, the receiver unit 420, the transmitter unit 440, the outlet port 450, and the memory 460. The processor 430 includes a decoding module 470 (e.g., an encoding module 470 or a decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed herein to implement the image prediction method provided in the embodiments of the present application. For example, the encoding / decoding module 470 implements, processes, or provides various encoding operations. Therefore, the encoding / decoding module 470 provides substantial improvements to the functions of the video decoding device 400 and affects the conversion of the video decoding device 400 to different states. Alternatively, the encoding / decoding module 470 is implemented with instructions stored in the memory 460 and executed by the processor 430.

[0260] The memory 460 includes one or more disks, tape drives, and solid-state drives, and can be used as an overflow data storage device for storing programs when these programs are selectively executed, and for storing instructions and data read during program execution. The memory 460 can be volatile and / or non-volatile, and can be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random access memory (SRAM).

[0261] See also Figure 6 , Figure 6 According to an exemplary embodiment, it can be used as Figure 1 A simplified block diagram of an apparatus 500 for either or both of the source device 12 and the destination device 14 in FIG. The apparatus 500 can implement the image prediction method of the embodiment of the present application. In other words, Figure 6 A schematic block diagram of an implementation method of an encoding device or decoding device (referred to as a decoding device 500) of an embodiment of the present application. The decoding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and the memory are connected via a bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory. The memory of the decoding device stores program code, and the processor can call the program code stored in the memory to execute various video encoding or decoding methods described in this application, especially various new image block division methods. To avoid repetition, it is not described in detail here.

[0262] In the embodiment of the present application, the processor 510 may be a central processing unit (CPU), and the processor 510 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0263] The memory 530 may include a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may also be used as the memory 530. The memory 530 may include code and data 531 accessed by the processor 510 using the bus 550. The memory 530 may further include an operating system 533 and an application 535, which includes at least one program that allows the processor 510 to perform the video encoding or decoding method described in the present application. For example, the application 535 may include applications 1 to N, which further include a video encoding or decoding application (referred to as a video decoding application) that executes the video encoding or decoding method described in the present application.

[0264] The bus system 550 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as the bus system 550 in the figure.

[0265] Optionally, the decoding device 500 may further include one or more output devices, such as a display 570. In one example, the display 570 may be a touch-sensitive display that combines a display with a touch-sensitive unit operable to sense touch input. The display 570 may be connected to the processor 510 via the bus 550.

[0266] In order to better understand the image prediction method in the embodiment of the present application, some relevant concepts and basic contents of inter-frame prediction are first introduced in detail below.

[0267] Inter-frame prediction refers to finding a matching reference block for the current image block in the current image in the reconstructed image, and using the pixel values ​​of the pixels in the reference block as the predicted values ​​of the pixel values ​​of the pixels in the current image block (this process is called motion estimation (ME)).

[0268] Motion estimation is to try multiple reference blocks in the reference image for the current image block, and then use rate-distortion optimization (RDO) or other methods to finally determine one or two reference blocks (two reference blocks are required for bidirectional prediction) from the multiple reference blocks, and use the reference blocks to perform inter-frame prediction on the current image block.

[0269] The motion information of the current image block includes the indication information of the prediction direction (usually forward prediction, backward prediction or bidirectional prediction), one or two motion vectors (MV) pointing to the reference block, and the indication information of the image where the reference block is located (usually represented by a reference frame index).

[0270] Forward prediction means that the current image block selects a reference image from the forward reference image set to obtain a reference block. Backward prediction means that the current image block selects a reference image from the backward reference image set to obtain a reference block. Bidirectional prediction means that a reference image is selected from each of the forward reference image set and the backward reference image set to obtain a reference block. When the bidirectional prediction method is used, there will be two reference blocks in the current coding block, each of which requires a motion vector and a reference frame index to indicate it, and then the predicted value of the pixel value in the current image block is determined based on the pixel values ​​in the two reference blocks.

[0271] In HEVC, there are two inter-frame prediction modes, namely AMVP mode and merge mode.

[0272] In the AMVP mode, the coded blocks adjacent to the current coding block in the spatial or temporal domain (referred to as neighboring blocks) are first traversed, and a candidate motion vector list is constructed according to the motion information of each neighboring block. Then, the optimal motion vector is determined from the candidate motion information list through the rate-distortion cost, and the candidate motion information with the smallest rate-distortion cost is used as the motion vector predictor (MVP) of the current coding block.

[0273] Among them, the positions of neighboring blocks and their traversal order are predefined. The rate-distortion cost can be calculated according to formula (1), where J is the rate-distortion cost, SAD is the sum of absolute differences (SAD) between the pixel prediction value obtained after motion estimation using the candidate motion vector prediction value and the original pixel value, R is the bit rate, λ is the Lagrange multiplier, and the encoder passes the index value of the selected motion vector prediction value in the candidate motion vector list and the reference frame index value to the decoder. Furthermore, the encoder can perform motion search in the neighborhood centered on the MVP to obtain the actual motion vector of the current coding block, and then pass the difference (motion vector difference) between the MVP and the actual motion vector to the decoder.

[0274] J=SAD+λR (1)

[0275] In addition, according to different motion models, the AMVP mode can be divided into an AMVP mode based on a translational model and an AMVP mode based on a non-translational model.

[0276] In merge mode, a candidate motion information list is first constructed through the motion information of the coded units adjacent to the current coding unit in the spatial domain or temporal domain, and then the optimal motion information is determined from the candidate motion information list through the rate-distortion cost as the motion information of the current coding unit. Finally, the index value of the position of the optimal motion information in the candidate motion information list (denoted as merge index, the same below) is passed to the decoding end.

[0277] In merge mode, the spatial and temporal candidate motion information of the current coding unit can be Figure 7 As shown, the spatial candidate motion information comes from five spatially adjacent blocks (A0, A1, B0, B1 and B2). If the adjacent block is not available or the prediction mode is intra-frame prediction, the adjacent block will not be added to the candidate motion information list.

[0278] The temporal candidate motion information of the current coding unit can be obtained by scaling the MV of the corresponding position block in the reference frame according to the picture order count (POC) of the reference frame and the current frame. When obtaining the corresponding position block of the reference frame, it can be determined whether the block at position T in the reference frame is available. If not, the block at position C is selected.

[0279] When the translation model is used for prediction, all pixels in the coding unit use the same motion information, and then motion compensation is performed based on the motion information to obtain the predicted value of the pixel of the coding unit. However, in the real world, there are many kinds of motion, and there are many objects with non-translational motion, such as rotating objects, roller coasters rotating in different directions, fireworks, and some stunts in movies, especially moving objects in user generated content (UGC) scenes. If the block motion compensation technology based on the translation motion model in the current coding standard is used for their encoding, the coding efficiency will be greatly affected. Therefore, in order to improve the coding effect, a prediction based on the non-translational motion model is proposed.

[0280] Non-translational motion model prediction refers to using the same motion model at the codec end to derive the motion information of each sub-motion compensation unit in the current coding block, and then performing motion compensation based on the motion information of the sub-motion compensation unit to obtain the predicted sub-block of each sub-block, thereby improving the prediction efficiency. Commonly used non-translational motion models include 4-parameter affine motion models or 6-parameter affine motion models.

[0281] In addition, the skip mode is a special mode of the fusion mode. The difference is that the skip mode has no residual when transmitted, but only transmits the fusion candidate index (merge index), and the merge index is used to indicate the best or target candidate motion information in the fusion candidate motion information list.

[0282] Different modes can be used when predicting images. These common modes are introduced in detail below.

[0283] Fusion Motion Vector Difference Mode:

[0284] The merge with motion vector difference (MMVD) mode selects one or more candidate motion vectors from the merge candidate motion vector list, and then performs motion vector (MV) expansion based on the candidate motion vectors. The MV expansion includes the MV starting point, motion step, and motion direction.

[0285] Generally speaking, the type of candidate motion vector selected in MMVD mode is the default merge type (eg, MRG_TYPE_DEFAULT_N). The selected candidate motion vector is the starting point of the MV. In other words, the selected candidate motion vector is used to determine the initial position of the MV.

[0286] As shown in Table 1, the base candidate index (Base candidate IDX) indicates which candidate motion vector in the candidate motion vector list is selected as the optimal candidate motion vector. If the number of candidate motion vectors available for selection in the merge candidate motion vector list is 1, the Base candidate IDX may not be determined.

[0287] Table 1

[0288] Basic candidate index 0 1 2 3 <![CDATA[N th MVP]]> <![CDATA[1 st MVP]]> <![CDATA[2 nd MVP]]> <![CDATA[3 rd MVP]]> <![CDATA[4 th MVP]]>

[0289] For example, if the basic candidate index is 1, then the selected candidate motion vector is the second motion vector in the merge candidate motion vector list.

[0290] The step identifier (Distance IDX) represents the offset distance information of the motion vector. The value of the step identifier represents the distance of the offset initial position (for example, a preset distance). The distance here is represented by pixel distance (Pixel distance), which can also be abbreviated as pel. The corresponding relationship between the step identifier and the pixel distance can be shown in Table 2.

[0291] Table 2

[0292] Step size indicator 0 1 2 3 4 5 6 7 Pixel distance 1 / 4-pel 1 / 2-pel 1-pel 2-pel 4-pel 8-pel 16-pel 32-pel

[0293] The direction identifier (Direction IDX) is used to indicate the direction based on the initial position motion vector difference (MVD). The direction identifier may include four cases in total, and the specific definitions may be shown in Table 3.

[0294] Table 3

[0295] Direction signs 00 01 10 11 X-axis + – N / A N / A Y-axis N / A N / A + –

[0296] The process of determining the predicted pixel value of the current image block according to the MMVD method includes the following steps:

[0297] Step 1: Determine the MV starting point according to the Base candidate IDX.

[0298] like Figure 8 and Fig. 9 As shown, Figure 8 A schematic diagram showing the MMVD search point of an embodiment of the present application is shown. Fig. 9 A schematic diagram of the MMVD search process of an embodiment of the present application is shown.

[0299] For example, the MV starting point can be found in Figure 8 The hollow dot in the center of Fig. 9 The position corresponding to the solid line in .

[0300] Step 2: Determine the direction to offset based on the starting point of the MV based on Direction IDX.

[0301] Step 3: Determine how many pixels to offset in the direction indicated by Direction IDX based on Distance IDX.

[0302] For example, Direction IDX==00, Distance IDX=2, then it means that the motion vector offset by one pixel in the positive X direction is used as the motion vector of the current image block to predict or obtain the predicted pixel value of the current image block. Combined intra-frame prediction mode and inter-frame prediction mode:

[0303] In a coding block / CU encoded using the merge mode, a flag (e.g., ciip_flag) may be transmitted to indicate whether the current image block uses the combined intra prediction and inter prediction (CIIP) mode. When the CIIP mode is used, an intra prediction block may be generated by selecting an intra prediction mode from an intra candidate list according to relevant syntax elements, an inter prediction block may be generated using a traditional inter prediction method, and finally, an adaptive weighted method may be used to combine intra prediction coding and inter prediction coding prediction blocks to generate a final prediction block.

[0304] For luma blocks, the intra candidate mode list can be selected from four modes: DC, planar, horizontal, and vertical. The size of the intra candidate mode list is selected according to the shape of the current coding block, which may be 3 or 4. When the width of the current coding block / CU is greater than twice the height, the intra candidate mode list does not contain horizontal modes. When the height of the current coding block / CU is greater than twice the width, the intra candidate mode list does not contain vertical modes.

[0305] In the weighted method of combining intra prediction coding and inter prediction coding, different weighting coefficients are used for different intra prediction modes. Specifically, when intra prediction coding uses DC or planar mode, or when the length or width of the current coding block is less than or equal to 4, the prediction values ​​obtained by intra prediction and inter prediction use the same weight value / weighting coefficient. Otherwise, the weight value / weighting coefficient can be determined according to the intra prediction mode used by the current block and / or the position of the prediction sample in the current block, for example, a variable weighting coefficient is adopted when the intra prediction coding adopts horizontal and vertical modes.

[0306] Triangle prediction unit mode:

[0307] The triangle prediction unit mode (triangle PU) may also be referred to as the triangle partition mode (triangle partition mode, TPM) or the fused triangle mode. For the sake of description convenience, this application will refer to the triangle prediction unit mode or the triangle partition mode as TPM, which will also apply in the following descriptions.

[0308] like Fig.11As shown, the current image block is divided into two triangular prediction units, and each triangular prediction unit selects a motion vector and a reference frame index from the unidirectional prediction candidate list. Then a prediction value is obtained for each of the two triangular prediction units. Then the pixels included in the hypotenuse area are adaptively weighted to obtain a prediction value. Then the entire current image block is transformed and quantized. It should be noted that the triangular prediction unit method is generally only applied to skip mode or merge mode. Fig.10 The left side is divided into upper left and lower right (i.e. divided from upper left to lower right). Fig.10 The right side is divided into upper right and lower left (i.e., divided from upper right to lower left).

[0309] The unidirectional prediction candidate list in the triangular prediction unit mode generally includes 5 candidate prediction motion vectors. These candidate prediction motion vectors are as follows: Figure 5 The 7 surrounding neighboring blocks (5 spatial neighboring blocks and 2 temporal corresponding blocks) are obtained. By searching the motion information of the 7 neighboring blocks, and putting them into the unidirectional prediction candidate list in order, for example, the order can be the bidirectional prediction motion vector of L0, the bidirectional prediction motion vector of L1, and the motion vector average of L0 and L1. If the number of candidates is less than 5, add the zero motion vector 0 to the unidirectional prediction candidate list. During encoding, the unidirectional prediction candidate list is obtained in the above manner. Exemplarily, the forward predicted motion information in the unidirectional prediction candidate list is used to predict the pixel prediction value of a triangular PU, and the backward predicted motion information is used to predict the pixel prediction value of another triangular PU. The encoder selects the best motion vector by traversal. For example, it can be done in the following way {m,i,j}:

[0310] {0,1,0},{1,0,1},{1,0,2},{0,0,1},{0,2,0}

[0311] {1,0,3},{1,0,4},{1,1,0},{0,3,0},{0,4,0}

[0312] {0,0,2},{0,1,2},{1,1,2},{0,0,4},{0,0,3}

[0313] {0,1,3},{0,1,4},{1,1,4},{1,1,3},{1,2,1}

[0314] {1,2,0},{0,2,1},{0,4,3},{1,3,0},{1,3,2}

[0315] {1,3,4},{1,4,0},{1,3,1},{1,2,3},{1,4,1}

[0316] {0,4,1},{0,2,3},{1,4,2},{0,3,2},{1,4,3}

[0317] {0,3,1},{0,2,4},{1,2,4},{0,4,2},{0,3,4}

[0318] The first bit m in {m,i,j} represents the division mode from upper left to lower right or from lower left to upper right. The second bit represents the forward motion information of the i-th candidate prediction motion vector for the first triangular PU, and the third bit represents the backward motion information of the j-th candidate prediction motion vector for the second triangular PU.

[0319] The adaptive weighting process based on the predicted values ​​of the pixels included in the slant edge area can be seen in Fig.11 After completing the prediction of the triangle prediction units P1 and P2, an adaptive weighting process is performed on the pixels included in the oblique edge area to obtain the final prediction value of the current image block.

[0320] For example, Fig.11 In the left image, the predicted value of the pixel at position 2 is Among them, P1 represents Fig.11 The predicted value of the pixel in the upper right area, P2 represents Fig.11 The predicted value of the pixel in the lower left area.

[0321] Among them, the two sets of weighted parameters are as follows:

[0322] The first set of weighted parameters: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for brightness and chromaticity points respectively;

[0323] The second set of weighted parameters: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for brightness and chromaticity points respectively.

[0324] One set of weighting parameters will be used for the encoding and decoding implementation of the current image block. When the reference images of the two prediction units are different or the difference between their motion vectors is greater than 16 pixels, the second set of weighting parameters is selected, otherwise the first set of weighting parameters is used.

[0325] Fig.12 It is a schematic block diagram of a video communication system according to an embodiment of the present application.

[0326] Fig.12The video communication system 500 shown includes a source device 600 and a destination device 700, wherein the source device 600 can encode the acquired video and transmit the encoded video stream to the receiving device 700, and the destination device 700 can parse the received video stream to obtain a video image and display the video through a display device.

[0327] The image prediction method of the embodiment of the present application may be performed by the source device 600 or the destination device 700. Specifically, the image prediction method of the embodiment of the present application may be performed by the video encoder 603 or the video decoder 702.

[0328] The video communication system 500 may also be referred to as a video coding and decoding system, the source device 600 may also be referred to as a video coding device or a video coding apparatus, and the destination device 700 may also be referred to as a video decoding device or a video decoding apparatus.

[0329] exist Fig.12 , source device 600 includes a video capture device 601, a video memory 602, a video encoder 603, and a transmitter 604. Video memory 602 can store the video obtained by video capture device 601, and video encoder 603 can encode video data from video memory 602 and video capture device 601. In some examples, source device 600 transmits the encoded video data directly to destination device 700 via transmitter 604. The encoded video data may also be stored on a storage medium or a file server for later retrieval by destination device 700 for decoding and / or playback.

[0330] exist Fig.12 8, destination device 700 includes a receiver 701, a video decoder 702, and a display device 703. In some examples, receiver 701 may receive encoded video data via channel 800. Display device 703 may be integrated with destination device 700 or may be external to destination device 700. In general, display device 700 displays decoded video data. Display device 700 may include a variety of display devices, such as a liquid crystal display, a plasma display, an organic light emitting diode display, or other types of display devices.

[0331] The specific implementation form of the source device 600 and the destination device 700 can be any of the following devices: a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a smart phone, a handheld device, a television, a camera, a display device, a digital media player, a video game console, a car computer, or other similar devices.

[0332] The destination device 700 may receive the encoded video data from the source device 600 via the channel 800. The channel 800 may include one or more media and / or devices capable of moving the encoded video data from the source device 600 to the destination device 700. In one example, the channel 800 may include one or more communication media that enable the source device 600 to transmit the encoded video data directly to the destination device 700 in real time, in which case the source device 600 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to the destination device 700. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network (e.g., the Internet)). The one or more communication media may include routers, switches, base stations, or other devices that enable communication from the source device 600 to the destination device 700.

[0333] In another example, channel 800 may include a storage medium storing the encoded video data generated by source device 600. In this example, destination device 700 may access the storage medium via disk access or card access. The storage medium may include a variety of locally accessible data storage media, such as Blu-ray discs, high-density digital video discs (DVDs), compact disc read-only memories (CD-ROMs), flash memory, or other suitable digital storage media for storing encoded video data.

[0334] In another example, channel 800 may include a file server or another intermediate storage device storing the encoded video data generated by source device 600. In this example, destination device 700 may access the encoded video data stored at the file server or other intermediate storage device via streaming or downloading. The file server may be a type of server capable of storing the encoded video data and transmitting the encoded video data to destination device 700. For example, the file server may include a world wide web (Web) server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, and a local disk drive.

[0335] Destination device 700 can access the encoded video data via a standard data connection (e.g., an Internet connection). Example types of data connections include a wireless channel suitable for accessing the encoded video data stored on a file server, a wired connection (e.g., a cable modem, etc.), or a combination of both. The transmission of the encoded video data from the file server can be a streaming transmission, a download transmission, or a combination of both.

[0336] The image prediction method of the embodiment of the present application is described in detail below with reference to specific drawings.

[0337] Fig.13 It is a schematic flow chart of the image prediction method of an embodiment of the present application. Fig.13 The image prediction method shown may be performed by an image prediction device (the image prediction device may be located in an image decoding device (system) or an image encoding device (system)). Specifically, Fig.13 The method shown may be executed by an image encoding device or an image decoding device. Fig.13 The method shown can be executed at the encoding end or the decoding end. Fig.13 The method shown includes steps 1001 to 1008, and these steps are described in detail below.

[0338] 1001. Start.

[0339] Step 1001 indicates starting image prediction.

[0340] 1002. Determine whether the current image block uses a fusion mode.

[0341] Optionally, Fig.13 The method shown also includes: obtaining a current image block before step 1002 .

[0342] The current image block may be an image block in an image to be currently encoded or decoded.

[0343] It should be understood that in the present application, the current image block may also be acquired during or after the process of determining the target fusion mode of the current image block.

[0344] For the decoding end, in step 1002, it can be determined whether the current image block uses the fusion mode according to the syntax element merge_flag[x0][y0] at the CU level.

[0345] If merge_flag[x0][y0]=1, it is determined that the current image block uses the fusion mode for prediction, and if merge_flag[x0][y0]=0, it is determined that the current image block does not use the fusion mode for prediction. Where x0 and y0 represent the coordinate position of the brightness pixel point of the upper left vertex of the current image block relative to the brightness pixel point of the upper left vertex of the current image.

[0346] After determining the fusion mode used by the current image block according to the CU-level syntax element merge_flag[x0][y0], the target fusion mode to be finally used can be determined by parsing the specific information in the CU-level syntax element merge_flag[x0][y0].

[0347] When it is determined in step 1002 that the current image block does not use the fusion mode, the current image block may be predicted using other modes other than the fusion mode. For example, when it is determined that the current image block does not use the fusion mode, the AMVP mode may be used to predict the current image block.

[0348] After it is determined in step 1002 that the current image block uses a fusion mode, step 1003 is continued to be performed to determine a target fusion mode suitable for the current image block.

[0349] 1003. Determine whether to use the first-level fusion mode.

[0350] Specifically, whether the first-level fusion mode is available may be determined according to the high-level syntax elements corresponding to the first-level fusion mode and / or the available status information corresponding to the first-level fusion mode.

[0351] In a possible implementation of the present invention, assuming that the first-level fusion mode includes two fusion modes, fusion mode A and fusion mode B, then it is determined one by one whether there are available fusion modes in the first-level fusion mode, and if there are available fusion modes, step 1005 is executed. If no fusion mode is available in the first-level fusion mode, it is determined that the first-level fusion mode is unavailable, and at this time, it is necessary to determine the target fusion mode from the second-level fusion mode, that is, step 1004 is executed.

[0352] 1004. Determine whether the high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is prohibited from being used.

[0353] Among them, the above-mentioned first fusion mode belongs to a secondary fusion mode, and the secondary fusion mode also includes a second fusion mode.

[0354] When it is determined in step 1004 that the high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is prohibited from being used, step 1006 is executed to determine the second fusion mode as the target fusion mode.

[0355] In the present application, when the high-level syntax elements of the first fusion mode indicate that the first fusion mode is prohibited from use, there is no need to parse the remaining available status information of the second fusion mode, and the second fusion mode can be directly determined as the final target fusion mode, which can minimize the process redundancy caused by determining the target fusion mode during the image prediction process.

[0356] When it is determined in step 1004 that the high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is allowed to be used, step 1007 is executed to further determine the target fusion mode.

[0357] 1005. Predict the current image block according to the first-level fusion mode.

[0358] It should be understood that, in step 1005 , if fusion mode A in the first-level fusion mode is available, then the current image block is predicted according to fusion mode A.

[0359] 1006. Determine the second fusion mode as a target fusion mode suitable for the current image block.

[0360] When the high-level syntax element corresponding to the first fusion mode indicates that the first fusion mode is prohibited, it is not necessary to parse the high-level syntax element and / or available status information corresponding to the second fusion mode, and the second fusion mode can be directly determined as the target fusion mode.

[0361] 1007. Determine a target fusion mode according to a high-level syntax element corresponding to the second fusion mode and / or available state information of the second fusion mode.

[0362] The available status information of the second fusion mode is used to indicate whether to use the second fusion mode when predicting the current image block.

[0363] For example, the second fusion mode is the CIIP mode, and the available state information of the second fusion mode is the value of ciip_flag. When ciip_flag is 0, the CIIP mode is not used for the current image block. When ciip_flag is 1, the CIIP mode is used for prediction of the current image block.

[0364] It should be understood that for the above-mentioned CIIP mode, if the CIIP mode is to be selected as the target fusion mode, it must be satisfied that the high-level syntax elements corresponding to the CIIP indicate that the CIIP mode is allowed to be used, and at the same time, the available status information expressing the available status of the CIIP mode indicates that the CIIP is available.

[0365] For example, when sps_ciip_enabled_flag=1 and ciip_flag=1, the CIIP mode may be determined as the target fusion mode of the current image block.

[0366] In step 1007, since the first fusion mode is allowed to be used, both the first fusion mode and the second fusion mode may be used as the target fusion mode of the current image block. Therefore, the target fusion mode can be determined from the secondary fusion modes by means of the high-level syntax corresponding to one of the fusion modes and the available state information.

[0367] Optionally, the first fusion mode is a TPM mode, and the second fusion mode is a CIIP mode.

[0368] The following is a detailed introduction on how to determine the target fusion mode when the first fusion mode is the TPM mode and the second fusion mode is the CIIP mode.

[0369] Optionally, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, the CIIP mode is determined as the target fusion mode.

[0370] For example, when sps_triangle_enabled_flag corresponding to the TPM mode is 0, the TPM mode is prohibited from use. In this case, there is no need to parse the specific value of ciip_flag, but the CIIP mode can be directly determined as the target fusion mode. This can reduce unnecessary parsing processes and reduce the redundancy of the solution.

[0371] In the present application, when the high-level syntax elements corresponding to the TPM mode indicate that the TPM mode is prohibited from use, it is no longer necessary to determine whether the CIIP mode is available by parsing the high-level syntax corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode. Instead, the CIIP mode can be directly determined as the target fusion mode, which can reduce redundancy in the process of determining the target fusion mode.

[0372] Optionally, when the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, the target fusion mode is determined according to the high-level syntax element corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode.

[0373] For example, when sps_triangle_enabled_flag=1 corresponding to the above TPM mode, the TPM mode is allowed to be used. In this case, both the TPM mode and the CIIP mode may be used as target fusion modes, and further judgment is required to select either the TPM mode or the CIIP mode as the target fusion mode.

[0374] Optionally, when the high-level syntax element corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, the TPM mode is determined as the target fusion mode.

[0375] The high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicate that the CIIP mode is prohibited from use, including cases 1 to 3:

[0376] Case 1: the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is prohibited from being used, and the available state information expressing the available state of the CIIP mode indicates that the CIIP mode is unavailable;

[0377] Case 2: the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used, and the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is not available;

[0378] Case 3: The available status information expressing the available status of the CIIP mode indicates that the CIIP mode is not available.

[0379] Optionally, when the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used and the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is available, the CIIP mode is determined as the target fusion mode.

[0380] 1008. Predict the current image block according to the target fusion mode.

[0381] exist Fig.13 In the method shown, before executing step 1007, Fig.13 The method shown also includes: determining that the type of the slice or slice group where the current image block is located is B; and determining that the maximum candidate number of the TPM mode supported by the slice or slice group where the current image block is located is greater than or equal to 2.

[0382] Optionally, before determining the target fusion mode based on the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode, the above method also includes: determining that the type of the slice or slice group where the current image block is located is B; determining that the maximum candidate number of TPM modes supported by the slice or slice group where the current image block is located is greater than or equal to 2.

[0383] Optionally, as an embodiment, Fig.13 The method shown also includes: when the first-level fusion mode is not available, the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, but the current image block does not meet at least one of conditions A and condition B, determining the CIIP mode as the target fusion mode.

[0384] Among them, condition A and condition B are:

[0385] Condition A: The type of the slice where the current image block is located is B;

[0386] Condition B: The maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located is greater than or equal to 2.

[0387] The above conditions A and B can be expressed in some specific ways. For example, the above condition A can be expressed by slice_type==B, and the above condition B can be expressed by MaxNumTriangleMergeCand>=2. MaxNumTriangleMergeCand indicates the maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located.

[0388] Only when both conditions A and B are met can the TPM mode be selected as the target fusion mode ultimately adopted when predicting the current image block.

[0389] If only one of condition A and condition B is not satisfied, the CIIP mode is determined as the target fusion mode.

[0390] In the case where the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, if only one of the conditions A and B is not satisfied, the CIIP mode is determined as the target fusion mode.

[0391] In the case where the high-level syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, if only one of the conditions A and B is not satisfied, the CIIP mode is determined as the target fusion mode.

[0392] That is, if sps_trangle_enabled_flag=1 and any one of condition A and condition B is not satisfied, the CIIP mode can be determined as the target fusion mode.

[0393] Conversely, if sps_trangle_enabled_flag=1, and both condition A and condition B are met, then it is necessary to combine several conditions of the prior art and determine the target fusion mode according to ciip_flag.

[0394] Optionally, the high-level syntax elements are syntax elements of at least one level among sequence level, picture level, slice level and slice group level.

[0395] Optionally, the above-mentioned first-level fusion mode includes: a normal fusion mode, an MMVD mode and a sub-block fusion mode.

[0396] When determining whether the primary fusion mode is available, it may be determined whether these modes are available in the order of the conventional fusion mode, the MMVD mode, and the sub-block fusion mode.

[0397] For example, you can first determine whether the conventional fusion mode is available. If the conventional fusion mode is not available (if the conventional mode is available, you can directly use the conventional fusion mode as the final target fusion mode), continue to determine whether the MMVD mode is available. If the MMVD mode is not available, continue to determine whether the sub-block fusion mode is available.

[0398] Optionally, Fig.13 The method shown can be applied at the encoding end to implement encoding of the current image block.

[0399] Optionally, Fig.13 The method shown can be applied at the decoding end to achieve decoding of the current image block.

[0400] In order to better understand the specific process of the image prediction method of the embodiment of the present application, the process of determining the image fusion mode in the image prediction method of the embodiment of the present application is described in detail below with reference to specific examples.

[0401] Combine the following Fig.14 And Table 4 provides a detailed introduction to the image prediction method of the embodiment of the present application.

[0402] Fig.14 The figure shows the process of determining the fusion mode of the embodiment of the present application. Fig.14 The process shown includes steps 3001 to 3007, each of which is described in detail below.

[0403] 3001. Start.

[0404] Step 3001 indicates starting image prediction.

[0405] 3002. Determine whether the current image block uses a fusion mode.

[0406] Specifically, when step 3002 is executed by the decoding end, in step 3002, it can be determined whether the current image block uses the fusion mode according to the value of the CU-level syntax element merge_flag[x0][y0] corresponding to the current image block.

[0407] For example, as shown in Table 4, when merge_flag[x0][y0]=0, the current image block does not use the fusion mode. In this case, the current image block can be predicted using other methods, for example, the current image block can be predicted using the AMVP mode.

[0408] When merge_flag[x0][y0]=1, the current image block uses the fusion mode. Next, it can be further determined which fusion mode the current image block uses for prediction.

[0409] It should be understood that when merge_flag[x0][y0] does not appear in the code stream, merge_flag[x0][y0] defaults to 0.

[0410] The above (x0, y0) represents the coordinate position of the brightness pixel point of the upper left vertex of the current image block relative to the brightness pixel point of the upper left vertex of the current image. The meaning of (x0, y0) in the following syntax elements is the same as this, and will not be repeated here.

[0411] Optionally, Fig.14 The method shown also includes: obtaining the current image block before step 3002.

[0412] The current image block may be an image block in an image to be currently encoded or decoded.

[0413] It should be understood that in the present application, the current image block may also be acquired during or after the process of determining the target fusion mode of the current image block.

[0414] 3003. Determine whether the current image block uses a traditional fusion mode.

[0415] Specifically, in step 3003, whether the current image block adopts the traditional fusion mode can be determined by parsing the value of the syntax element regular_merge_flag[x0][y0].

[0416] When regular_merge_flag[x0][y0]=1, it is determined that the current image block uses the traditional fusion mode. At this time, step 3007 can be executed, that is, the current image block is predicted according to the traditional fusion mode.

[0417] When regular_merge_flag[x0][y0]=0, it is determined that the current image block does not use the traditional fusion mode. At this time, it is necessary to continue to execute step 3004 to further determine the fusion mode used by the current image block.

[0418] It should be understood that when regular_merge_flag[x0][y0] does not appear in the code stream, regular_merge_flag[x0][y0] defaults to 0.

[0419] 3004. Determine whether the current image block uses the MMVD mode.

[0420] Specifically, in step 3004, when the high-level syntax element corresponding to the MMVD mode indicates that MMVD is allowed to be used and the area of ​​the current image block is not equal to 32, it can be determined whether the current image block uses MMVD by parsing the value of the syntax element mmvd_flag[x0][y0].

[0421] When mmvd_flag[x0][y0]=1, it is determined that the current image block uses the MMVD mode, and step 3007 can be executed at this time, that is, predicting the current image block according to the MMVD mode.

[0422] When mmvd_flag[x0][y0]=0, it is determined that the current image block does not use the MMVD mode. At this time, it is necessary to continue to execute step 3005 to further determine the fusion mode used by the current image block.

[0423] It should be understood that when mmvd_flag[x0][y0] does not appear in the code stream, mmvd_flag[x0][y0] defaults to 0.

[0424] 3005. Determine whether the current image block uses the sub-block fusion mode.

[0425] In step 3004, it can be determined whether the current image block uses the sub-block merging mode according to the value of the syntax element merge_subblock_flag[x0][y0] parsed from the bitstream.

[0426] When merge_subblock_flag[x0][y0]=1, it is determined that the current image block uses the sub-block fusion mode, and step 3007 can be executed at this time, that is, predicting the current image block according to the sub-block fusion mode.

[0427] When merge_subblock_flag[x0][y0]=0, it is determined that the current image block does not use the sub-block merging mode. At this time, it is necessary to continue to execute step 3006 to further determine the merging mode used by the current image block.

[0428] It should be understood that when merge_subblock_flag[x0][y0] does not appear in the bitstream, merge_subblock_flag[x0][y0] defaults to 0.

[0429] Further, in step 3004, the value of the syntax element merge_subblock_flag[x0][y0] may be parsed only when the maximum length of the sub-block fusion candidate list is greater than 0 and the width and height of the current image block are greater than or equal to 8, and when the obtained value of merge_subblock_flag[x0][y0] is 0, continue to execute step 3007.

[0430] 3006. Determine a fusion mode used by the current image block from the CIIP mode and the TPM mode.

[0431] Specifically, in step 3006, if all six of the following conditions (1) to (6) are met, then ciip_flag[x0][y0] is parsed from the bitstream, and the fusion mode used by the current image block is determined according to the value of ciip_flag[x0][y0]. When ciip_flag[x0][y0]=1, the current image block is predicted using the CIIP mode.

[0432] In addition, when the following condition (1) is met, if any one of the following conditions (2) to (6) is not met, the current image block is predicted using the CIIP mode.

[0433] (1)sps_ciip_enabled_flag=1;

[0434] (2)sps_triangle_enabled_flag=1;

[0435] (3)cu_skip_flag[x0][y0]==0;

[0436] (4)(cbWidth*cbHeight)>=64;

[0437] (5)cbWidth<128;

[0438] (6)cbHeight<128.

[0439] Among them, cbWidth and cbHeight are the width and height of the current image block respectively.

[0440] Optionally, more judgment conditions may be added when determining the fusion mode used by the current block in step 3006 .

[0441] On the basis of the above conditions (1) to (6), conditions (7) and (8) may be further added.

[0442] (7) slice_type == B;

[0443] (8)MaxNumTriangleMergeCand>=2.

[0444] Optionally, in step 3006, if all eight conditions in the above conditions (1) to (8) are met, then ciip_flag[x0][y0] is parsed from the bitstream, and the fusion mode used by the current image block is determined according to the value of ciip_flag[x0][y0]. When ciip_flag[x0][y0]=1, the current image block is predicted using the CIIP mode.

[0445] In addition, when the above condition (1) is met, if any one of the following conditions (2) to (8) is not met, the current image block is predicted using the CIIP mode.

[0446] 3007. Predict the current image block according to the fusion mode used by the current image block.

[0447] When it is determined in step 3003 that the current image block uses the traditional fusion mode, the current image block is predicted according to the traditional fusion mode in step 3007; when it is determined in step 3004 that the current image block uses the MMVD mode, the current image block is predicted according to the MMVD mode in step 3007; when it is determined in step 3005 that the current image block uses the sub-block fusion mode, the current image block is predicted according to the sub-block fusion mode in step 3007.

[0448] Table 4 shows how to determine the fusion mode used by the current image block according to the corresponding syntax elements when the fusion mode is adopted. The following is a detailed introduction to determining the fusion mode of the current image block in conjunction with Table 4.

[0449] Table 4

[0450]

[0451]

[0452] When regular_merge_flag[x0][y0]=1 as shown in Table 4, it is determined that the current image block uses the traditional fusion mode. At this time, the parameters of the traditional fusion mode can be obtained by parsing the syntax element merge_idx[x0][y0]. When regular_merge_flag[x0][y0]=0 as shown in Table 4, it is determined that the current image block does not use the traditional fusion mode, and it is necessary to further determine the fusion mode used by the current image block.

[0453] When sps_mmvd_enabled_flag=1 as shown in Table 4, and cbWidth*cbHeight!=32, it means that the current image block may use the MMVD mode. At this time, the fusion mode of the current image block can be determined according to the value of mmvd_flag[x0][y0]. If mmvd_flag[x0][y0]=1, it is determined that the current image block uses the MMVD mode, and the parameters of the MMVD mode can be determined by parsing these syntax elements mmvd_merge_flag[x0][y0], mmvd_distance_idx[x0][y0], and mmvd_direction_idx[x0][y0]. If mmvd_flag[x0][y0]=0, it is necessary to further determine the fusion mode used by the current image block.

[0454] When merge_subblock_flag[x0][y0]=1 as shown in Table 4, it is determined that the current image block uses the sub-block fusion mode; when merge_subblock_flag[x0][y0]=0 as shown in Table 4, it is determined that the current image block does not use the sub-block fusion mode, and it is necessary to further determine the fusion mode used by the current image block.

[0455] When sps_ciip_enabled_flag=0 as shown in Table 4, it can be directly determined that the current image block uses the TPM mode. When sps_ciip_enabled_flag=1, sps_ciip_enabled_flag=0 as shown in Table 4, it can be directly determined that the current image block uses the CIIP mode.

[0456] As shown in Table 4, when all six conditions (1) to (6) below are met, it is necessary to obtain the available status information indication of the Ciip mode from the code stream, that is, the value of ciip_flag[x0][y0]. If ciip_flag[x0][y0]=1, it is determined that the current image block uses the CIIP mode. If ciip_flag[x0][y0]=0, it is determined that the current image block uses the TPM mode.

[0457] (1)sps_ciip_enabled_flag=1;

[0458] (2)sps_triangle_enabled_flag=1;

[0459] (3)cu_skip_flag[x0][y0]==0;

[0460] (4)(cbWidth*cbHeight)>=64;

[0461] (5)cbWidth<128;

[0462] (6)cbHeight<128.

[0463] Optionally, if(sps_ciip_enabled_flag&&sps_triangle_enabled_flag&&

[0464] cu_skip_flag[x0][y0]==0&&(cbWidth*cbHeight)>=64&&cbWidth<128&&cbHeight<128) can also be replaced by if(sps_triangle_enabled_flag&&sps_ciip_enabled_flag&&cu_skip_flag[x0][y0]==0&&(cbWidth*cbHeight)>=64&&cbWidth<128&&cbHeight<128), that is, the order of sps_ciip_enabled_flag and sps_triangle_enabled_flag can be adjusted, and the specific results can be shown in Table 5.

[0465] Table 5

[0466]

[0467]

[0468] It should be noted that in the above Tables 4 and 5, the timing of the judgment of CIIP is earlier than the timing of the judgment of TPM, that is, CIIP is judged first, and the prediction mode finally used by the current block is determined according to the state of CIIP. If CIIP is true, there is no need to determine the information of TPM. If CIIP is false, it means that only TPM is available, so the final prediction mode of the current block can be set to TPM mode. The above logic or priority setting of the judgment timing is only an example, which can also be adjusted as needed, for example, making the timing of TPM earlier than the timing of CIIP, but at the same time, it is also necessary to adjust the judgment condition of whether the TPM mode is applicable as needed.

[0469] Table 6

[0470]

[0471]

[0472] As shown in Table 6, when the following conditions (1) to (8) are met, it is necessary to obtain the available status information indication of the CIIP mode from the code stream, that is, the value of ciip_flag[x0][y0]. If ciip_flag[x0][y0]=1, it is determined that the current image block uses the CIIP mode. If ciip_flag[x0][y0]=0, it is determined that the current image block uses the TPM mode.

[0473] (1)sps_ciip_enabled_flag=1;

[0474] (2)sps_triangle_enabled_flag=1;

[0475] (3)cu_skip_flag[x0][y0]==0;

[0476] (4)(cbWidth*cbHeight)>=64;

[0477] (5)cbWidth<128;

[0478] (6)cbHeight<128.

[0479] Optionally, if(sps_ciip_enabled_flag&&sps_triangle_enabled_flag&&slice_type==B&&MaxNumTriangleMergeCand>=2&&cu_skip_flag[x0][y0]==0&&(cbWidth*cbHeight)>=64&&cbWidth<128&&cbHeight<128) in Table 6 may also be replaced by:

[0480] if(sps_triangle_enabled_flag&&sps_ciip_enabled_flag&&slice_type==B&&MaxNumTriangleMergeCand>=2&&cu_skip_flag[x0][y0]==0&&(cbWidth*cbHeight)>=64&&cbWidth<128&&cbHeight<128).

[0481] That is to say, the order of sps_ciip_enabled_flag and sps_triangle_enabled_flag can be adjusted, and the specific results can be shown in Table 7.

[0482] Table 7

[0483]

[0484]

[0485] Combination of the above Fig.13 and Fig.14 The image prediction method of the embodiment of the present application is described in detail. Fig.15 The image prediction method of the embodiment of the present application is described.

[0486] Fig.15 It is a schematic flow chart of the image prediction method of an embodiment of the present application. Fig.15 The image prediction method shown may be performed by an image prediction device (the image prediction device may be located in an image decoding device (system) or an image encoding device (system)). Specifically, Fig.15 The method shown may be executed by an image encoding device or an image decoding device. Fig.15 The method shown can be executed at the encoding end or the decoding end. Fig.15 The method shown includes steps 4001 to 4007, and these steps are described in detail below.

[0487] 4001. Start.

[0488] Step 4001 indicates the start of image prediction.

[0489] 4002. Determine whether the current image block uses a fusion mode.

[0490] For the decoding end, in step 4002, it can be determined whether the current image block uses the merge mode according to the syntax element merge_flag[x0][y0] at the CU level. For the specific determination process, please refer to the relevant description below step 1003 above.

[0491] When it is determined in step 4002 that the current image block does not use the fusion mode, the current image block may be predicted using other modes other than the fusion mode. For example, when it is determined that the current image block does not use the fusion mode, the AMVP mode may be used to predict the current image block.

[0492] After it is determined in step 4002 that the current image block uses a fusion mode, step 4003 is continued to be executed to determine a target fusion mode suitable for the current image block.

[0493] Optionally, Fig.15 The method shown also includes: obtaining the current image block before step 4002.

[0494] The current image block may be an image block in an image to be currently encoded or decoded.

[0495] It should be understood that in the present application, the current image block may also be acquired during or after the process of determining the target fusion mode of the current image block.

[0496] 4003. Determine whether to use the first-level fusion mode.

[0497] Specifically, whether the first-level fusion mode is available may be determined according to the high-level syntax elements corresponding to the first-level fusion mode and / or the available status information corresponding to the first-level fusion mode.

[0498] Optionally, the first-level fusion mode in step 4003 includes: a normal fusion mode, an MMVD mode and a sub-block fusion mode.

[0499] When determining whether the primary fusion mode is available, the availability of the conventional fusion mode, the MMVD mode and the sub-block fusion mode may be determined in sequence. When none of these modes are available, it is determined that the primary fusion mode is unavailable.

[0500] When it is determined in step 4003 that the primary fusion mode is not available, step 4004 may be continued to determine the target fusion mode from the secondary fusion modes.

[0501] for Fig.15 For the image prediction method shown, the primary fusion mode and the secondary fusion mode may include all optional fusion modes of the current image block, and for the current image block, the final target fusion mode needs to be determined from the primary fusion mode and the secondary fusion mode.

[0502] Optionally, the priority of the first-level fusion mode is higher than the priority of the second-level fusion mode.

[0503] The priority of the above-mentioned first-level fusion mode is higher than that of the second-level fusion mode, which means that in the process of determining the target fusion mode of the current image block, the target fusion mode is determined from the first-level fusion mode first. If there is no available fusion mode in the first-level fusion mode, it is determined from the second-level fusion mode.

[0504] 4004. Determine whether conditions one to five are met.

[0505] The above conditions 1 to 5 are:

[0506] Condition 1: TPM mode is allowed to be used;

[0507] Condition 2: The type of the slice or slice group where the current image block is located is B;

[0508] Condition 3: Determine that the maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located is greater than or equal to 2;

[0509] Condition 4: The size of the current image block meets the preset conditions;

[0510] Condition 5: The current image block does not use skip mode for prediction.

[0511] The above condition 1 can be specifically represented by sps_triangle_enabled_flag=1, the above condition 2 can be specifically represented by slice_type==B, and the above condition 3 can be specifically represented by MaxNumTriangleMergeCand>=2. Wherein, MaxNumTriangleMergeCand represents the maximum number of candidates for the TPM mode supported by the slice or slice group where the current image block is located.

[0512] When it is determined in step 4004 that any one of conditions one to five is not met, the CIIP mode can be directly determined as the target fusion mode, that is, step 4005 is executed; when it is determined in step 4004 that all five conditions, one to five, are met, it is necessary to further determine the target fusion mode based on the relevant information of the CIIP mode, that is, step 4006 is executed.

[0513] 4005. When the CIIP mode is allowed to be used, the CIIP mode is determined as the target fusion mode.

[0514] That is, in step 4005, when the CIIP mode is allowed to be used, if any one of conditions one to five is not met, the CIIP mode is determined as the target fusion mode.

[0515] Optionally, when any one of conditions one to five is not met, the value of the available status information expressing the available status of the CIIP mode is set to a first value, wherein when the value of the available status information expressing the available status of the CIIP mode is the first value, the current image block adopts the CIIP mode for image prediction.

[0516] It should be understood that here, setting the value of the available status information expressing the available status of the CIIP mode to the first value is equivalent to determining CIIP as the target fusion mode.

[0517] Optionally, the available status information expressing the available status of the CIIP mode is ciip_flag.

[0518] Setting the value of the available status information expressing the available status of the CIIP mode to the first value may specifically be setting ciip_flag to 1.

[0519] In addition, when the value of the available state information expressing the available state of the CIIP mode is set to the second value, it may mean that the current image block does not use the CIIP mode for image prediction. For example, the available state information expressing the available state of the CIIP mode is ciip_flag, and when ciip_flag=0, the current image block does not use the CIIP mode for image prediction.

[0520] 4006. Determine a target fusion mode according to high-level syntax elements corresponding to the CIIP mode and / or available status information expressing available status of the CIIP mode.

[0521] The available status information expressing the available status of the CIIP mode is used to indicate whether the CIIP mode is used when predicting the current image block.

[0522] That is, in step 4006, when all conditions from condition 1 to condition 5 are met, it is necessary to further determine the target fusion mode based on the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode.

[0523] The available status information expressing the available status of the CIIP mode may be the value of ciip_flag. When ciip_flag is 0, the CIIP mode is unavailable for the current image block. When ciip_flag is 1, the CIIP mode is available for the current image block.

[0524] Optionally, the above-mentioned determining the target fusion mode based on the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode includes: when the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicate that the CIIP mode is prohibited from use, determining the TPM mode as the target fusion mode.

[0525] The high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicate that the CIIP mode is prohibited from use, including Case 1 and Case 3:

[0526] Case 1: the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is prohibited from being used, and the available state information expressing the available state of the CIIP mode indicates that the CIIP mode is unavailable;

[0527] Case 2: the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used, and the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is not available;

[0528] Case 3: The available status information expressing the available status of the CIIP mode indicates that the CIIP mode is not available.

[0529] It should be understood that when the high-level syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used and the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is available, the CIIP mode is determined as the target fusion mode.

[0530] Optionally, when the high-level syntax element corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, determining the TPM mode as the target fusion mode includes:

[0531] When the high-level syntax elements corresponding to the CIIP mode and / or the available status information expressing the available status of the CIIP mode indicate that the CIIP mode is prohibited from use, the value of the available status information expressing the available status of the TPM mode is set to a first value, wherein when the value of the available status information expressing the available status of the TPM mode is the first value, the current image block adopts the TPM mode for image prediction.

[0532] It should be understood that setting the value of the available state information expressing the available state of the TPM mode to the first value here is equivalent to determining the TPM as the target fusion mode.

[0533] Optionally, the available state information expressing the available state of the TPM mode is MergeTriangleFlag.

[0534] Setting the value of the available state information expressing the available state of the TPM mode to the first value may specifically be setting MergeTriangleFlag to 1.

[0535] In the present application, the target fusion mode is determined based on the high-level syntax elements of the CIIP mode and / or the available status information expressing the available status of the CIIP mode only when five preset conditions are met. Compared with the traditional scheme, more conditions need to be met before the target fusion mode is further determined in combination with the high-level syntax elements and available status information of the CIIP mode. Otherwise, the CIIP mode can be directly determined as the target fusion mode, which can reduce some redundant processes in the process of determining the target fusion mode.

[0536] From another perspective, when the first-level fusion mode is not available, it is possible to determine whether to select the CIIP mode as the final fusion mode based on some preset conditions, and as long as any of the preset conditions is not met, the CIIP mode can be directly determined as the target fusion mode, which can reduce the process redundancy generated in determining the target fusion process.

[0537] 4007. Predict the current image block according to the target fusion mode.

[0538] Optionally, before determining the target fusion mode according to the high-level syntax element corresponding to the CIIP mode and / or the available state information expressing the available state of the CIIP mode, Fig.15 The method shown also includes:

[0539] Make sure at least one of the following conditions is met:

[0540] The size of the current image block meets the preset conditions;

[0541] The current image block does not use skip mode for prediction.

[0542] Optionally, the size of the current image block satisfies a preset condition, including: the current image block satisfies the following three conditions:

[0543] (cbWidth*cbHeight)>=64;

[0544] cbWidth<128;

[0545] cbHeight<128;

[0546] Among them, cbWidth is the width of the current image block, and cbHeight is the height of the current image block.

[0547] The above describes the image prediction method of the embodiment of the present application in detail with reference to the accompanying drawings. Fig.16 The image prediction device of the embodiment of the present application is introduced, and it should be understood that: Fig.16 The image prediction device shown can execute each step of the image prediction method of the embodiment of the present application. In order to avoid unnecessary repetition, repeated descriptions are appropriately omitted when introducing the image prediction device of the embodiment of the present application.

[0548] Fig.16 It is a schematic block diagram of an image prediction device according to an embodiment of the present application.

[0549] Fig.16 The image prediction device 5000 shown includes: a determination unit 5001 and a prediction unit 5002 .

[0550] Fig.16 The image prediction device 5000 shown is used to perform the image prediction method of the embodiment of the present application. Specifically, the determination unit 5001 in the image prediction device 5000 can be used to perform Figures 13 to 15 In the process of determining the target fusion mode in the image prediction method shown in FIG. 1 , the prediction unit 5002 in the image prediction device 5000 is used to perform Figures 13 to 15The image prediction method shown is a process of performing image prediction on the current image block according to the target fusion mode.

[0551] Fig.17 It is a schematic diagram of the hardware structure of the image prediction device provided in an embodiment of the present application. Fig.17 The image prediction device 6000 shown (the device 6000 may be a computer device) includes a memory 6001, a processor 6002, a communication interface 6003 and a bus 6004. The memory 6001, the processor 6002 and the communication interface 6003 are connected to each other through the bus 6004.

[0552] The memory 6001 may be a read only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 6001 may store a program. When the program stored in the memory 6001 is executed by the processor 6002, the processor 6002 is used to execute each step of the image prediction method of the embodiment of the present application.

[0553] Processor 6002 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the image prediction method of the method embodiment of the present application.

[0554] The processor 6002 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the image prediction method of the present application may be completed by an integrated logic circuit of hardware in the processor 6002 or by instructions in the form of software.

[0555] The processor 6002 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 6001, and the processor 6002 reads the information in the memory 6001, and completes the functions required to be performed by the units included in the image prediction device in combination with its hardware, or executes the image prediction method of the method embodiment of the present application.

[0556] The communication interface 6003 uses a transceiver such as, but not limited to, a transceiver to implement communication between the device 6000 and other devices or communication networks. For example, the information of the neural network to be constructed and the training data required in the process of constructing the neural network can be obtained through the communication interface 6003.

[0557] The bus 6004 may include a path for transmitting information between various components of the device 6000 (eg, the memory 6001 , the processor 6002 , and the communication interface 6003 ).

[0558] The determination unit 5001 and the prediction unit 5002 in the above-mentioned image prediction device 5000 are equivalent to the processor 6002 in the image prediction device 6000.

[0559] Fig.18 It is a schematic diagram of the hardware structure of the image encoding / decoding device provided in an embodiment of the present application. Fig.18 The image encoding / decoding device 7000 shown (the device 7000 may be a computer device) includes a memory 7001, a processor 7002, a communication interface 7003 and a bus 7004. The memory 7001, the processor 7002 and the communication interface 7003 are connected to each other through the bus 7004.

[0560] The above definitions and explanations of the various modules in the image prediction device 6000 are also applicable to the image encoding / decoding device 7000 and will not be described in detail here.

[0561] The above-mentioned memory 7001 can be used to store programs, and the processor 7002 is used to execute the programs stored in the memory 7001. When the program stored in the memory 7001 is executed, the processor 7002 is used to execute the various steps of the image prediction method of the embodiment of the present application.

[0562] In addition, when the image encoding / decoding device 7000 encodes a video image, the video image can be obtained through the communication interface, and then the obtained video image is encoded to obtain encoded video data, and the encoded video data can be transmitted to the video decoding device through the communication interface 7003.

[0563] When the image encoding / decoding device 7000 decodes a video image, the video image can be acquired through the communication interface, and then the acquired video image can be decoded to obtain a video image to be displayed.

[0564] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0565] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0566] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0567] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0568] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0569] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0570] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a code stream, and the code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

2. The computer-readable storage medium of claim 1, wherein: In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

3. The computer-readable storage medium according to claim 1 or 2, wherein: The first-level fusion mode is a conventional fusion mode.

4. The computer-readable storage medium according to any one of claims 1 to 3, wherein: The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

5. A device for storing a code stream, characterized in that: The device comprises at least one storage medium and at least one communication interface, wherein the at least one communication interface is used to receive or transmit a code stream, and the at least one storage medium is used to store the code stream, wherein the code stream comprises: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

6. The device according to claim 5, characterized in that In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

7. The device according to claim 5 or 6, characterized in that The first-level fusion mode is a conventional fusion mode.

8. The device according to any one of claims 5 to 7, characterized in that The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

9. A method for storing a code stream, characterized in that: The method comprises: Receiving or transmitting code streams through a communication interface; The code stream is stored in one or more storage media, wherein the code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

10. The method according to claim 9, characterized in that In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

11. The method according to claim 9 or 10, characterized in that The first-level fusion mode is a conventional fusion mode.

12. The method according to any one of claims 9 to 11, characterized in that The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

13. A device for transmitting a code stream, characterized in that: The device comprises: At least one storage medium, used for storing a code stream; At least one processor is used to obtain the code stream from the at least one storage medium and transmit the code stream to a target device, wherein the code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

14. The device according to claim 13, characterized in that In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

15. The device according to claim 13 or 14, characterized in that The first-level fusion mode is a conventional fusion mode.

16. The device according to any one of claims 13 to 15, characterized in that The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

17. A method for transmitting a code stream, characterized in that: The method comprises: storing the code stream in at least one storage medium; Obtain the code stream from the at least one storage medium, and transmit the code stream to a target device, wherein the code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

18. The method according to claim 17, characterized in that In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

19. The method according to claim 17 or 18, characterized in that The first-level fusion mode is a conventional fusion mode.

20. The method according to any one of claims 17 to 19, characterized in that The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

21. A system for processing a code stream, characterized in that: The system comprises: A source device for providing video data; An encoding device, used for obtaining the video data of the source device through an interface, and encoding the video data to obtain a code stream; A storage medium, used for storing the code stream; The target device is used to decode the code stream, wherein the code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

22. The system of claim 21, wherein: In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

23. The system according to claim 21 or 22, characterized in that The first-level fusion mode is a conventional fusion mode.

24. The system according to any one of claims 21 to 23, characterized in that The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

25. A method for processing a code stream, characterized in that: The method comprises: Providing video data; Obtaining video data of the source device through an interface, and encoding the video data to obtain a bit stream; storing the code stream; Decoding the code stream, wherein the code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

26. The method of claim 25, wherein: In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

27. The method according to claim 25 or 26, characterized in that The first-level fusion mode is a conventional fusion mode.

28. The method according to any one of claims 25 to 27, characterized in that The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

29. An encoded bit stream, characterized in that The code stream includes: Encoded data and identification information representing a current image, the current image comprising a current image block; The identification information includes a first syntax element, and the first syntax element indicates whether the current image block uses a fusion mode; When the first syntax element indicates that the current image block uses a fusion mode, the identification information further includes a second syntax element, and the second syntax element indicates whether the first-level fusion mode is available; The code stream further includes a first high-level syntax element corresponding to a first fusion mode in a secondary fusion mode, wherein the first high-level syntax element indicates whether the first fusion mode is disabled, and the secondary fusion mode includes the first fusion mode and the second fusion mode; When the first-level fusion mode is unavailable and the first fusion mode is available, the identification information further includes a third syntax element, wherein when the third syntax element is a first value, it indicates that the second fusion mode is applicable to the target fusion mode of the current image block, and when the third syntax element is a second value, it indicates that the second fusion mode is not applicable to the target fusion mode of the current image block; When the primary fusion mode is not available and the first fusion mode is disabled, the identification information does not include a third syntax element.

30. The code stream according to claim 29, characterized in that In a case where the identification information does not include a third syntax element, the third syntax element is inferred to be the first value.

31. The code stream according to claim 29 or 30, characterized in that: The first-level fusion mode is a conventional fusion mode.

32. The code stream according to any one of claims 29 to 31, characterized in that: The first fusion mode is a triangulation TPM mode, and the second fusion mode is a joint intra-frame and inter-frame prediction CIIP mode.

Images