Picture prediction method and apparatus, and computer-readable storage medium

Abstract

An embodiment of this application discloses a picture prediction method and apparatus. The picture prediction method includes the steps of: determining whether a merge mode is used for a current picture block; and, when the merge mode is used for the current picture block, determining whether a level 1 merge mode is available for the current picture block; if a first merge mode is unavailable and an upper layer syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used, directly determining a second merge mode as a target merge mode applicable to the current picture block, where both the first merge mode and the second merge mode belong to level 2 merge modes; and predicting the current picture block based on the target merge mode. In the embodiment of this application, redundancy generated in the merge mode determination process can be reduced as much as possible.

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 201910341218.6, entitled "Video Picture Encoding / Decoding Method and Apparatus," filed with the State Intellectual Property Office of the People's Republic of China on April 25, 2019, and is incorporated herein by reference in its entirety.

[0002] This application claims priority to Chinese Patent Application No. 201910474007.X, entitled "Picture prediction method and apparatus, and computer-readable storage medium," filed with the China National Intellectual Property Office on June 2, 2019, and is incorporated herein by reference in its entirety.

[0003] The present application relates to the field of video coding technologies, and more particularly to a picture prediction method and apparatus, and a computer-readable storage medium. [Background technology]

[0004] Digital video capabilities may be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, mobile or satellite radio telephones (i.e., "smartphones"), video conferencing devices, video streaming devices, and the like. Digital video devices implement video compression techniques such as those described in MPEG-2, MPEG-4, ITU-T H.263, and ITU-T H.264 / MPEG-4 Part 10 Advanced Video Coding (AVC), video coding standard H.265 / high efficiency video coding (HEVC) standards, and standards defined in extensions to such standards. By implementing such video compression techniques, video devices may more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0005] Video compression techniques are used to perform spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove inherent redundancy in a video sequence. In block-based video coding, a video slice (i.e., a video frame or a portion of a video frame) may be partitioned into picture blocks, which may also be referred to as tree blocks, coding units (CUs), and / or coding nodes. Picture blocks in an intra-coded (I) slice of a picture are coded through spatial prediction based on reference samples in neighboring blocks in the same picture. For picture blocks in an inter-coded (P or B) slice of a picture, spatial prediction based on reference samples in neighboring blocks in the same picture or temporal prediction based on reference samples in another reference picture may be used. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0006] When a merge mode is used to predict a picture block, there are generally multiple optional merge modes. In conventional solutions, the merge mode applicable to the current picture block is usually determined one by one from multiple candidate merge modes. When a merge mode is not available, it is followed by determining whether the next merge mode is available. In conventional solutions, redundancy exists when the merge mode applicable to the current block is determined from the last two remaining merge modes. Summary of the Invention [Means for solving the problem]

[0007] This application provides a picture prediction method and apparatus, and a computer-readable storage medium, to reduce redundancy as much as possible in the picture prediction process.

[0008] According to a first aspect, a picture prediction method is provided, the method including: determining whether a merge mode is used for a current picture block; when the merge mode is used for the current picture block, determining whether a level 1 merge mode is available; when the level 1 merge mode is not available and an upper layer syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used, determining a second merge mode as a target merge mode applicable to the current picture block; and predicting the current picture block based on the target merge mode.

[0009] Both the first merge mode and the second merge mode belong to the level 2 merge mode, and the level 2 merge mode includes the first merge mode and the second merge mode. In addition, for the current picture block, the level 1 merge mode and the level 2 merge mode already include all optional merge modes of the current picture block, and a final target merge mode needs to be determined from the level 1 merge mode and the level 2 merge mode for the current picture block.

[0010] Optionally, the priority of a level 1 merge mode is higher than the priority of a level 2 merge mode.

[0011] The priority of the merge modes in level 1 is higher than the priority of the merge modes in level 2, which means that in the process of determining the target merge mode for the current picture block, the target merge mode is preferentially determined from the merge modes in level 1. If there is no available merge mode in the merge modes in level 1, the target merge mode is then determined from the merge modes in level 2.

[0012] Optionally, the step of determining whether a merge mode is used for the current picture block includes the steps of determining that a merge mode is used for the current picture block when a merge_flag corresponding to the current picture block is 1, and determining that a merge mode is not used for the current picture block when a merge_flag corresponding to the current picture block is 0.

[0013] It should be understood that when it is determined that the merge mode is not to be used for the current picture block, another mode other than the merge mode may be used to predict the current picture block. For example, when it is determined that the merge mode is not to be used for the current picture block, the advanced motion vector AMVP mode may be used to predict the current picture block.

[0014] In this application, when the upper layer syntax element of the first merge mode indicates that the first merge mode is prohibited from being used, there is no need to analyze the available status information of the remaining second merge modes, and the second merge mode can be directly determined as the final target merge mode, which can reduce the redundancy generated by the determination of the target merge mode in the picture prediction process as much as possible.

[0015] Optionally, the method further comprises determining whether a level 1 merge mode is available.

[0016] Specifically, whether the level 1 merge mode is available is determined based on higher layer syntax elements corresponding to the level 1 merge mode and / or available status information corresponding to the level 1 merge mode.

[0017] With reference to the first aspect, in some implementations of the first aspect, when a level 1 merge mode is not available and an upper layer syntax element corresponding to the first merge mode indicates that the first merge mode is permitted to be used, the target merge mode is determined based on an upper layer syntax element corresponding to a second merge mode and / or available status information of the second merge mode.

[0018] The availability status information of the second merge mode is used to indicate whether the second merge mode is used when the current picture block is predicted.

[0019] For example, the second merge mode is a CIIP mode, and the available status information of the second merge mode is a value of ciip_flag. When ciip_flag is 0, the CIIP mode is not available for the current picture block. When ciip_flag is 1, the CIIP mode is available for the current picture block.

[0020] For the CIIP mode, it should be understood that if the CIIP mode is to be selected as the target merge mode, the upper layer syntax element corresponding to CIIP needs to indicate that the CIIP mode is allowed to be used, and the available status information indicating the available status of the CIIP mode needs to indicate that CIIP is available.

[0021] For example, when sps_ciip_enabled_flag=1 and ciip_flag=1, the CIIP mode may be determined as the target merge mode of the current picture block.

[0022] With reference to the first aspect, in some implementations of the first aspect, determining the target merge mode based on an upper layer syntax element corresponding to the second merge mode and / or available status information of the second merge mode includes determining the first merge mode as the target merge mode when the upper layer syntax element corresponding to the second merge mode and / or the available status information of the second merge mode indicates that the second merge mode is prohibited from being used.

[0023] The higher layer syntax element corresponding to the second merge mode and / or the available status information of the second merge mode indicating that the second merge mode is prohibited from being used includes:

[0024] The upper layer syntax element corresponding to the second merge mode indicates that the second merge mode is prohibited from being used, the available status information of the second merge mode indicates that the second merge mode is not allowed to be used, and the upper layer syntax element corresponding to the second merge mode indicates that the second merge mode is permitted to be used, and the available status information of the second merge mode indicates that the second merge mode is not allowed to be used.

[0025] Optionally, determining the target merge mode based on an upper layer syntax element corresponding to the second merge mode and / or availability status information of the second merge mode further includes determining the second merge mode as the target merge mode when the upper layer syntax element corresponding to the second merge mode indicates that the second merge mode is allowed to be used and the availability status information of the second merge mode indicates that the second merge mode is available.

[0026] With reference to the first aspect, in some implementations of the first aspect, before the target merge mode is determined based on an upper layer syntax element corresponding to the second merge mode and / or available status information of the second merge mode, the method further includes a step of determining that at least one of the following conditions is satisfied: the size of the current picture block meets a preset condition, and a skip mode is not used to predict the current picture block.

[0027] In other words, before the target merge mode is determined, it is necessary to further ensure that the size of the current picture block meets the condition and the skip mode is not used for the current picture block, otherwise, another mode other than the merge mode may be used to predict the current picture block.

[0028] With reference to the first aspect, in some implementations of the first aspect, the size of the current picture block satisfies the preset condition when the size of the current picture block satisfies the following three conditions: (cdWidth*cbHeight)≧64, cbWidth<128, and cbHeight<128 This includes satisfying the following:

[0029] cdWidth is the width of the current picture block, and cbHeight is the height of the current picture block.

[0030] With reference to the first aspect, in some implementations of the first aspect, the first merge mode includes a triangle partition mode TPM, and the second merge mode includes a combined intra and inter prediction CIIP mode.

[0031] Optionally, the CIIP mode is determined as the target merge mode when the higher layer syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used.

[0032] In this application, when the upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, it is not necessary to determine whether the CIIP mode is available by analyzing the upper layer syntax corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode. Instead, the CIIP mode can be directly determined as the target merge mode. This can reduce redundancy in the process of determining the target merge mode.

[0033] Optionally, when an upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is permitted to be used, the target merge mode is determined based on an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating the available status of the CIIP mode.

[0034] Optionally, the TPM mode is determined as the target merge mode when the higher layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used.

[0035] Optionally, the CIIP mode is determined as the target merge mode when the higher layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is permitted to be used and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is available.

[0036] With reference to the first aspect, in some implementations of the first aspect, before the target merge mode is determined based on upper layer syntax elements corresponding to the second merge mode and / or available status information of the second merge mode, the method further includes a step of determining that the type of the slice or slice group in which the current picture block is located is B, and a step of determining that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is two or greater.

[0037] Optionally, before the target merge mode is determined based on the upper layer syntax element corresponding to the CIIP mode and / or available status information indicating the available status of the CIIP mode, the method further includes a step of determining that the type of the slice or slice group in which the current picture block is located is B, and a step of determining that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is greater than or equal to 2.

[0038] With reference to the first aspect, in some implementations of the first aspect, the first merge mode is a triangular partition mode TPM, and the second merge mode is a combined intra and inter prediction CIIP mode. The method further includes determining a CIIP mode as a target merge mode when the level 1 merge mode is not available and the upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, but the current picture block does not satisfy at least one of the conditions A and B.

[0039] Conditions A and B are as follows.

[0040] Condition A: The slice in which the current picture block is located is of type B.

[0041] Condition B: The maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is 2 or greater.

[0042] The TPM mode can be selected as the target merging mode that is ultimately used to predict the current picture block only when both condition A and condition B are satisfied.

[0043] On the other hand, if either condition A or condition B is not met, the CIIP mode is determined as the target merge mode.

[0044] On the other hand, when the upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, if either condition A or condition B is not met, the CIIP mode is determined as the target merge mode.

[0045] On the other hand, when the upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, if either condition A or condition B is not met, the CIIP mode is determined as the target merge mode.

[0046] In other words, the CIIP mode may be determined as the target merge mode, provided that sps_trangle_enabled_flag=1, condition A, and one of condition B are not satisfied.

[0047] On the other hand, if sps_trangle_enabled_flag=1, condition A, and condition B are all satisfied, the target merge mode needs to be determined based on ciip_flag according to some conditions in the prior art.

[0048] With reference to the first aspect, in some implementations of the first aspect, the higher layer syntax element is a syntax element at at least one of a sequence level, a picture level, a slice level, and a slice group level.

[0049] Referring to the first aspect, in some implementations of the first aspect, the level 1 merge modes include a normal merge mode, a merge using motion vector differential MMVD mode, and a sub-block merge mode.

[0050] When determining whether level 1 merge modes are available, whether these modes are available may be determined sequentially in the sequence of normal merge mode, MMVD mode, and sub-block merge mode.

[0051] For example, it may first be determined whether a normal merge mode is available. When the normal merge mode is not available, it is followed by determining whether an MMVD mode is available (if the normal merge mode is available, it may be directly used as the final target merge mode). When the MMVD mode is not available, it is followed by determining whether a sub-block merge mode is available.

[0052] With reference to the first aspect, in some implementations of the first aspect, the method further includes a step of determining a target merge mode from a level 2 merge mode when the level 1 merge mode is not available, the level 2 merge modes including a TPM mode and a CIIP mode, and a step of determining the CIIP mode as the target merge mode when the CIIP mode is allowed to be used and any one of the following conditions is not met:

[0053] Condition D: TPM mode is permitted to be used.

[0054] Condition E: Skip mode is not used to predict the current picture block.

[0055] Condition F: (cbWidth*cbHeight)≧64.

[0056] Condition G:cbWidth<128.

[0057] Condition H:cbHeight<128.

[0058] cbWidth is the width of the current picture block, and cbHeight is the height of the current picture block.

[0059] Referring to the first aspect, in some implementations of the first aspect, the prediction method is applied on the encoder side to encode the current picture block.

[0060] Referring to the first aspect, in some implementations of the first aspect, the prediction method is applied on the decoder side to decode the current picture block.

[0061] According to a second aspect, a picture prediction method is provided, which includes the steps of: determining whether a merge mode is used for a current picture block; when the merge mode is used for the current picture block, determining whether a level 1 merge mode is available; when the level 1 merge mode is not available, determining a target merge mode from a level 2 merge mode, where the level 2 merge mode includes a TPM mode and a CIIP mode; and when the CIIP mode is allowed to be used and any one of the following conditions (condition 1 to condition 5) is not satisfied, determining the CIIP mode as the target merge mode.

[0062] Condition 1: TPM mode is allowed to be used.

[0063] Condition 2: The slice or slice group in which the current picture block is located is of type B.

[0064] Condition 3: It is determined that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is two or greater.

[0065] Condition 4: The size of the current picture block meets a preset condition.

[0066] Condition 5: Skip mode is not used to predict the current picture block.

[0067] The first condition may be specifically expressed by sps_triangle_enabled_flag=1, the second condition may be specifically expressed by slice_type==B, and the third condition may be specifically expressed by MaxNumTriangleMergeCand≧2. MaxNumTriangleMergeCand indicates the maximum quantity of candidate TPM modes supported by the slice or group of slices in which the current picture block is located.

[0068] In addition, for the current picture block, the level 1 merge mode and the level 2 merge mode may include all optional merge modes of the current picture block, and the final target merge mode for the current picture block needs to be determined from the level 1 merge mode and the level 2 merge mode.

[0069] Optionally, the priority of a level 1 merge mode is higher than the priority of a level 2 merge mode.

[0070] The priority of the merge modes in level 1 is higher than the priority of the merge modes in level 2, which means that in the process of determining the target merge mode for the current picture block, the target merge mode is preferentially determined from the merge modes in level 1. If there is no available merge mode in the merge modes in level 1, the target merge mode is then determined from the merge modes in level 2.

[0071] Optionally, the size of the current picture block satisfies the preset condition when the size of the current picture block satisfies the following three conditions: (cdWidth*cbHeight)≧64, cbWidth<128, and cbHeight<128 This includes satisfying the following:

[0072] Optionally, the step of determining whether a merge mode is used for the current picture block includes the steps of determining that a merge mode is used for the current picture block when a merge_flag corresponding to the current picture block is 1, and determining that a merge mode is not used for the current picture block when a merge_flag corresponding to the current picture block is 0.

[0073] It should be understood that when it is determined that the merge mode is not to be used for the current picture block, another mode other than the merge mode may be used to predict the current picture block. For example, when it is determined that the merge mode is not to be used for the current picture block, the advanced motion vector AMVP mode may be used to predict the current picture block.

[0074] Optionally, level 1 merge modes include normal merge mode, MMVD mode, and sub-block merge mode.

[0075] When it is determined whether the level 1 merge modes are available, the availability of these modes may be determined sequentially in the sequence of the normal merge mode, the MMVD mode, and the sub-block merge mode. When all the modes are not available, it is determined that the level 1 merge mode is not available.

[0076] In this application, when the level 1 merge mode is not available, it may be determined whether to select the CIIP mode as the final merge mode based on some preset conditions, and the CIIP mode may be directly determined as the target merge mode on condition that any one of the preset conditions is not met, which reduces the redundancy generated in the process of determining the target merge mode.

[0077] With reference to the second aspect, in some implementations of the second aspect, the step of determining the target merge mode from the level 2 merge mode includes a step of setting a value of available status information indicating the available status of the CIIP mode to a first value when any one of conditions 1 to 5 is not satisfied, wherein when the value of the available status information indicating the available status of the CIIP mode is the first value, the CIIP mode is used to perform picture prediction on the current picture block.

[0078] It should be understood that the step of setting the value of the available status information indicating the available status of the CIIP mode to a first value is equivalent to the step of determining CIIP as the target merge mode.

[0079] Optionally, available status information indicating the available status of the CIIP mode is ciip_flag.

[0080] The step of setting the value of the available status information indicating the available status of the CIIP mode to a first value may specifically be a step of setting ciip_flag to 1.

[0081] In addition, when the value of the available status information indicating the available status of the CIIP mode is set to a second value, it may mean that the CIIP mode is not used to perform picture prediction on the current picture block. For example, when the available status information indicating the available status of the CIIP mode is ciip_flag, and ciip_flag=0, the CIIP mode is not used to perform picture prediction on the current picture block.

[0082] With reference to the second aspect, in some implementations of the second aspect, the step of determining a target merge mode from the level 2 merge mode includes a step of determining the target merge mode based on an upper layer syntax element corresponding to a CIIP mode and / or available status information indicating an available status of the CIIP mode when all conditions 1 to 5 are satisfied, wherein the available status information indicating the available status of the CIIP mode is used to indicate whether the CIIP mode is used when the current picture block is predicted.

[0083] For example, the available status information indicating the available status of the CIIP mode is the value of ciip_flag. When ciip_flag is 0, the CIIP mode is not available for the current picture block. When ciip_flag is 1, the CIIP mode is available for the current picture block.

[0084] In this application, the target merge mode can be determined based on the upper layer syntax elements of the CIIP mode and / or the available status information indicating the available status of the CIIP mode only when five preset conditions are met. Compared with the conventional solution, more conditions need to be met before the target merge mode is further determined based on the upper layer syntax elements of the CIIP mode and the available status information. Otherwise, the CIIP mode can be directly determined as the target merge mode. This can reduce some redundant processes in the process of determining the target merge mode.

[0085] With reference to the second aspect, in some implementations of the second aspect, the step of determining the target merge mode based on an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating an available status of the CIIP mode includes a step of determining the TPM mode as the target merge mode when the upper layer syntax element corresponding to the CIIP mode and / or the available status information indicating an available status of the CIIP mode indicates that the CIIP mode is prohibited from being used.

[0086] With reference to the second aspect, in some implementations of the second aspect, when the higher layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, the step of determining the TPM mode as the target merge mode includes: The method includes a step of setting a value of the available status information indicating the available status of the TPM mode to a first value when an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, wherein when the value of the available status information indicating the available status of the TPM mode is the first value, the TPM mode is used to perform picture prediction on the current picture block.

[0087] It should be understood that the step of setting the value of the available status information indicating the available status of the TPM mode to a first value is equivalent to the step of determining the TPM as the target merge mode.

[0088] Optionally, the available status information indicating the available status of the TPM mode is MergeTriangleFlag.

[0089] The step of setting the value of the available status information indicating the available status of the TPM mode to a first value may specifically be a step of setting MergeTriangleFlag to 1.

[0090] With reference to the second aspect, in some implementations of the second aspect, before the target merge mode is determined based on an upper layer syntax element corresponding to a CIIP mode and / or available status information indicating an available status of the CIIP mode, the method includes: The method further includes determining that at least one of the following conditions is met:

[0091] The size of the current picture block meets a preset condition, and The skip mode is not used to predict the current picture block.

[0092] cdWidth is the width of the current picture block, and cbHeight is the height of the current picture block.

[0093] According to a third aspect, a picture prediction method is provided. The method includes: determining whether a merge mode is used for a current picture block; when the merge mode is used for the current picture block, determining whether a level 1 merge mode is available; and when the level 1 merge mode is not available, determining a target merge mode from a level 2 merge mode. The level 2 merge modes include a TPM mode and a CIIP mode. When the CIIP mode is allowed to be used and all the following conditions (condition 1 to condition 3) are satisfied, available status information of the CIIP mode is obtained by analyzing a bitstream, and a target merge mode is determined based on the available status information of the CIIP mode.

[0094] Condition 1: TPM mode is allowed to be used.

[0095] Condition 2: The size of the current picture block meets a preset condition.

[0096] Condition 3: Skip mode is not used to predict the current picture block.

[0097] In a possible implementation form of the third aspect, if the CIIP mode availability status information obtained by analyzing the bitstream indicates that the CIIP mode is not available, the TPM is used as the target merge mode.

[0098] In this application, when the level 1 merge mode is not available, it may be determined whether to select the CIIP mode as the final merge mode based on some preset conditions, and the CIIP mode may be directly determined as the target merge mode on condition that any one of the preset conditions is not met, which reduces the redundancy generated in the process of determining the target merge mode.

[0099] According to a fourth aspect, a picture prediction method is provided. The method includes: determining whether a merge mode is used for a current picture block; when the merge mode is used for the current picture block, determining whether a level 1 merge mode is available; and when the level 1 merge mode is not available, determining a target merge mode from a level 2 merge mode. The level 2 merge modes include a TPM mode and a CIIP mode. When the CIIP mode is allowed to be used and all the following conditions (condition 1 to condition 5) are satisfied, available status information of the CIIP mode is obtained by analyzing a bitstream, and a target merge mode is determined based on the available status information of the CIIP mode.

[0100] Condition 1: TPM mode is allowed to be used.

[0101] Condition 2: The slice or slice group in which the current picture block is located is of type B.

[0102] Condition 3: It is determined that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is two or greater.

[0103] Condition 4: The size of the current picture block meets a preset condition.

[0104] Condition 5: Skip mode is not used to predict the current picture block.

[0105] In a possible implementation form of the fourth aspect, if the CIIP mode availability status information obtained by analyzing the bitstream indicates that the CIIP mode is not available, the TPM is used as the target merge mode.

[0106] In this application, when the level 1 merge mode is not available, it may be determined whether to select the CIIP mode as the final merge mode based on some preset conditions, and the CIIP mode may be directly determined as the target merge mode on condition that any one of the preset conditions is not met, which reduces the redundancy generated in the process of determining the target merge mode.

[0107] According to a fifth aspect, a picture prediction method is provided, the method including: determining whether a merge mode is used for a current picture block; when the merge mode is used for the current picture block, determining whether a level 1 merge mode is available; when the level 1 merge mode is not available and an upper layer syntax element corresponding to a first merge mode set indicates that a merge mode in the first merge mode set is prohibited from being used, determining a target merge mode applicable to the current picture block from a second merge mode set; and predicting the current picture block by using the target merge mode.

[0108] Both the first merge mode set and the second merge mode set belong to the merge mode of level 2. In other words, the merge mode of level 2 includes the first merge mode set and the second merge mode set. In addition, for a current picture block, the merge modes of level 1 and the merge modes of level 2 already include all optional merge modes of the current picture block, and a final target merge mode for the current picture block needs to be determined from the merge modes of level 1 and the merge modes of level 2.

[0109] Optionally, the first merge mode set includes at least one merge mode and the second merge mode set includes at least one merge mode.

[0110] It should be understood that the first merge mode set and the second merge mode set are merely concepts introduced for ease of explanation and are primarily used to distinguish between different merge modes. In the actual process of determining a final target merge mode, the first merge mode set and the second merge mode set may not exist.

[0111] In this application, when the upper layer syntax elements of some merge modes indicate that these merge modes are prohibited from being used, there is no need to analyze the available status information of these merge modes. Instead, the final target merge mode can be directly determined from the remaining optional merge modes. This can reduce the redundancy generated by the target merge mode determination in the picture prediction process as much as possible.

[0112] With reference to the fifth aspect, in some implementations of the fifth aspect, when a level 1 merge mode is not available and an upper layer syntax element corresponding to a first merge mode set indicates that a merge mode in the first merge mode set is permitted to be used, the target merge mode is determined based on an upper layer syntax element corresponding to a second merge mode set and / or available status information of the second merge mode set.

[0113] The availability status information of the second merge mode set is used to indicate whether a merge mode in the second merge mode set is used when the current picture block is predicted.

[0114] For example, if the second merge mode set includes a CIIP mode, the availability status information of the second merge mode set may be the value of ciip_flag. When ciip_flag is 0, the CIIP mode is not available for the current picture block. When ciip_flag is 1, the CIIP mode is available for the current picture block.

[0115] With reference to the fifth aspect, in some implementations of the fifth aspect, the first merge mode set includes a triangle partition mode TPM, and the second merge mode set includes a combined intra- and inter-prediction CIIP mode.

[0116] Optionally, the first merged mode set comprises a TPM mode and the second mode set comprises a CIIP mode.

[0117] When the first merge mode set and the second merge mode set each include only one merge mode, if a merge mode in the first merge mode set is prohibited from being used, a merge mode in the second merge mode set may be determined as the target merge mode, and if a merge mode in the second merge mode set is prohibited from being used, a merge mode in the first merge mode set may be determined as the target merge mode.

[0118] When the first merge mode set and the second merge mode set each contain only one merge mode, as long as a merge mode in one of the merge mode sets is prohibited from being used, a merge mode in the other merge mode set can be directly determined as the final target merge mode.

[0119] With reference to the fifth aspect, in some implementations of the fifth aspect, when a level 1 merge mode is not available and an upper layer syntax element corresponding to the first merge mode set indicates that a merge mode in the first merge mode set is prohibited from being used, determining a target merge mode from a second merge mode set applicable to the current picture block includes determining a CIIP mode as the target merge mode when a level 1 merge mode is not available and an upper layer syntax element corresponding to a TPM mode indicates that the TPM mode is prohibited from being used.

[0120] According to a sixth aspect, a picture prediction device is provided, the device includes a module corresponding to the method according to any one of the first to fifth aspects, and the corresponding module can realize the steps of the method according to any one of the first to fifth aspects.

[0121] The picture prediction device in the sixth aspect may include one or more modules, any one of which may include any one of a circuit, a field programmable gate array FPGA, an application specific integrated circuit ASIC, and a general purpose processor.

[0122] The picture prediction device in the sixth aspect may be disposed in an encoder device or a decoder device.

[0123] According to a seventh aspect, a picture prediction device is provided, the picture prediction device including a memory and a processor, the processor calling a program code stored in the memory to execute a method according to any one of the first aspect, the second aspect, and the third aspect.

[0124] The picture prediction device in the seventh aspect may be disposed in a picture encoding device or a picture decoding device.

[0125] According to an eighth aspect, there is provided a picture encoding / decoding device, the device including a module corresponding to a method according to any one of the first to fifth aspects, the corresponding module being capable of implementing steps of the method according to any one of the first to fifth aspects.

[0126] According to a ninth aspect, there is provided a picture encoding / decoding device, including a memory and a processor, the processor calling a program code stored in the memory to execute a method according to any one of the first to fifth aspects.

[0127] Optionally, the memory is a non-volatile memory.

[0128] Optionally, the memory and the processor are coupled to each other.

[0129] According to a tenth aspect, an embodiment of the present application provides a computer-readable storage medium storing instructions, the instructions enabling one or more processors to execute a method according to any one of the first to fifth aspects.

[0130] Any one of the one or more processors may include any one of a circuit, a field programmable gate array FPGA, an application specific integrated circuit ASIC, and a general purpose processor.

[0131] According to an eleventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to perform some or all steps of a method according to any one of the first to fifth aspects. [Brief description of the drawings]

[0132] [Figure 1] 1 is a schematic block diagram of an example of a video coding system for implementing an embodiment of the present application; [Diagram 2] FIG. 2 is a schematic structural block diagram of an example of a video encoder for implementing an embodiment of the present application; [Diagram 3] FIG. 2 is a schematic structural block diagram of an example of a video decoder for implementing an embodiment of the present application; [Figure 4] FIG. 1 is a schematic structural block diagram of an example of a video coding system for implementing an embodiment of this application; [Diagram 5] FIG. 2 is a schematic structural block diagram of an example of a video coding device for implementing an embodiment of this application; [Figure 6] 1 is a schematic block diagram of an example of an encoding device or a decoding device for implementing an embodiment of the present application; [Figure 7] A schematic diagram of spatial and temporal candidate motion information for a current coding unit. [Figure 8] FIG. 2 is a schematic diagram of an MMVD search point used to implement one embodiment of this application. [Figure 9] FIG. 2 is another schematic diagram of an MMVD search point used to implement an embodiment of the present application. [Figure 10] FIG. 2 is a schematic diagram of a triangular section. [Figure 11] FIG. 1 is a schematic diagram of a prediction method in a triangular partitioning scheme. [Figure 12] 1 is a schematic block diagram of a video communication system according to an embodiment of the present application. [Figure 13] 1 is a schematic flowchart of a picture prediction method according to an embodiment of the present application; [Figure 14] 1 is a schematic flowchart of a picture prediction method according to an embodiment of the present application; [Figure 15] 1 is a schematic flowchart of a picture prediction method according to an embodiment of the present application; [Figure 16] 1 is a schematic block diagram of a picture prediction device according to an embodiment of the present application; [Figure 17] 1 is a schematic block diagram of a picture prediction device according to an embodiment of the present application; [Figure 18] 1 is a schematic block diagram of a picture encoding / decoding device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0133] The following describes the technical solutions of this application with reference to the accompanying drawings.

[0134] In the following description, reference is made to the accompanying drawings which form a part of this application and which illustrate, by way of example, specific aspects of the embodiments of this application or specific aspects in which the embodiments of this application may be used. It should be understood that the embodiments of this application may also be used in other aspects and may include structural or logical changes not depicted in the accompanying drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of this application should be defined by the appended claims.

[0135] For example, it should be understood that disclosure related to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa.

[0136] For another example, if one or more particular method steps are described, a corresponding device may include one or more units, such as functional units (e.g., one unit that performs one or more steps, or multiple units that each perform one or more of the multiple steps), to perform the described method steps or steps, even if such one or more units are not explicitly described or illustrated in the accompanying drawings.

[0137] Additionally, if a particular apparatus has been described based on one or more units, such as functional units, a corresponding method may include a step used to perform the function of the one or more units (e.g., a step used to perform the function of the one or more units, or multiple steps each used to perform the function of one or more of the multiple units), even if such step or steps are not explicitly described or illustrated in the accompanying drawings. Furthermore, it should be understood that features of the various example embodiments and / or aspects described in this specification may be combined with each other, unless otherwise noted.

[0138] The technical solutions in the embodiments of this application can be applied to H.266 standard and future video coding standards. The terms used in the implementation of this application are merely intended to describe the specific embodiments of this application, and are not intended to limit this application. The following will first briefly describe the relevant concepts in the embodiments of this application.

[0139] Video coding typically refers to processing a sequence of pictures that constitute a video or video sequence. In the field of video coding, the terms "picture", "frame" and "image" may be used synonymously. Video coding as used in this specification includes video encoding and video decoding. Video encoding is performed at the source side and typically involves processing an original video picture (e.g., by compressing) to reduce the amount of data to represent the video picture for more efficient storage and / or transmission. Video decoding is performed at the destination side and typically involves the inverse process to the encoder to reconstruct the video picture. The "coding" of a video picture in the embodiments should be understood as the "encoding" or "decoding" of a video sequence. The combination of the encoding and decoding parts is also called a codec (encoding and decoding).

[0140] A video sequence includes a series of pictures, and a picture is further divided into slices, which are further divided into blocks. Video coding is performed on a block-by-block basis. In some new video coding standards, the concept "block" is further extended. For example, macroblocks (MB) are introduced in the H.264 standard. A macroblock may be further divided into multiple prediction blocks (partitions), which can be used for predictive coding. In the high efficiency video coding (HEVC) standard, basic concepts such as "coding unit (CU)", "prediction unit (PU)" and "transform unit (TU)" are used. Multiple block units are obtained through functional partitioning and described by using a new tree-based structure. For example, a CU may be divided into smaller CUs based on a quadtree to generate a quadtree structure, and the smaller CUs may be further divided. A CU is a basic unit for dividing and encoding a coded picture. PUs and TUs also have similar tree structures. A PU may correspond to a prediction block and is a basic unit for predictive coding. A CU is further divided into multiple PUs based on a division pattern. A TU may correspond to a transform block and is a basic unit for transforming a prediction residual. However, in essence, all of the CU, PU, ​​and TU are conceptually blocks (or picture blocks).

[0141] For example, in HEVC, a CTU is divided into multiple CUs by using a quad-tree structure represented as a coding tree. The decision on whether to code a picture area by using inter-picture (temporal) or intra-picture (spatial) prediction is made at the CU level. Each CU may be further divided into one, two, or four PUs based on the PU partitioning pattern. Within one PU, the same prediction process is applied, and related information is sent to the decoder based on the PU. After obtaining the residual block by applying the prediction process based on the PU partitioning pattern, the CU may be partitioned into transform units (TUs) based on another quad-tree structure similar to the coding tree used for the CU. In recent developments in video compression technology, a quad-tree and binary tree (QTBT) partitioned frame is used to partition the coding block. In the QTBT block structure, the CU may be square or rectangular.

[0142] In this specification, for ease of explanation and understanding, a picture block to be coded in a current coded picture may be called a current picture block. For example, in coding, the current picture block is a block currently being coded, and in decoding, the current picture block is a block currently being decoded. A decoded picture block in a reference picture that is used to predict a current picture block is called a reference block. In other words, a reference block is a block that provides a reference signal for a current picture block, where the reference signal represents pixel values ​​in a picture block. A block that provides a prediction signal for a current picture block in a reference picture may be called a prediction block, where the prediction signal represents pixel values, sampling values, or sampling signals in the prediction block. For example, after traversing multiple reference blocks, an optimal reference block is found, where the optimal reference block provides a prediction for the current picture block, and this block is called a prediction block.

[0143] In the case of lossless video coding, the original video picture can be reconstructed, which means that the reconstructed video picture has the same quality as the original video picture (assuming that no transmission losses or other data losses occur during storage or transmission). In the case of lossy video coding, further compression is performed, for example through quantization, to reduce the amount of data required to represent the video picture, and the video picture cannot be completely reconstructed at the decoder side, which means that the quality of the reconstructed video picture is lower or inferior to that of the original video picture.

[0144] Some H.261 video coding standards are for a "lossy hybrid video codec" (specifically, spatial and temporal prediction in the sample domain is combined with 2D transform coding to apply quantization in the transform domain). Each picture of a video sequence is usually partitioned into a set of non-overlapping blocks, and coding is usually performed at the block level. In other words, at the encoder side, the video is usually processed, i.e., coded, at the block (video block) level. For example, a predictive block is generated through spatial (intra-picture) prediction and temporal (inter-picture) prediction, the predictive block is subtracted from a current picture block (a block currently being or to be processed) to obtain a residual block, and the residual block is transformed and quantized in the transform domain to reduce the amount of data to be transmitted (compressed). At the decoder side, the inverse processing part to the encoder is applied to the coded or compressed block to reconstruct the current picture block for representation. Furthermore, the encoder duplicates the decoder processing loop so that the encoder and decoder generate the same predictions (e.g., intra-prediction and inter-prediction) and / or reconstructions for processing, i.e., coding, subsequent blocks.

[0145] The following describes a system architecture applicable to an embodiment of the present invention. Figure 1 is a schematic block diagram of an example of a video encoding and decoding system 10 applicable to an embodiment of this application. As shown in Figure 1, the video encoding and decoding system 10 may include a source device 12 and a destination device 14. The source device 12 generates encoded video data, and therefore the source device 12 may be referred to as a video encoding device. The destination device 14 is capable of decoding the encoded video data generated by the source device 12, and therefore the destination device 14 may be referred to as a video decoding device. In various implementation solutions, the source device 12, the destination device 14, or both the source device 12 and the destination device 14 may include one or more processors and memories coupled to the one or more processors. Memory may include, but is not limited to, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), flash memory, or any other medium capable of being configured to store the required program code in the form of instructions or data structures and capable of being accessed by a computer as described herein. Source device 12 and destination device 14 may include a variety of devices including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, or the like.

[0146] 1 depicts source device 12 and destination device 14 as separate devices, a device embodiment may alternatively include both source device 12 and destination device 14, or the functionality of both source device 12 and destination device 14, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be realized by using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0147] A communication connection between source device 12 and destination device 14 may be realized through link 13. Destination device 14 may receive encoded video data from source device 12 through link 13. Link 13 may include one or more media or devices capable of moving encoded video data from source device 12 to destination device 14. In one example, link 13 may include one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real time. In this example, source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to destination device 14. The one or more communication media may include wireless communication media and / or wired communication media, e.g., a radio frequency (RF) spectrum, or one or more physical transmission cables. The one or more communication media may be part of a packet-based network, e.g., a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication mediums may include a router, a switch, a base station, or another device that facilitates communication from source device 12 to destination device 14.

[0148] The source device 12 includes an encoder 20. Optionally, the source device 12 may further include a picture source 16, a picture pre-processor 18, and a communication interface 22. In a specific implementation form, the encoder 20, the picture source 16, the picture pre-processor 18, and the communication interface 22 may be hardware components in the source device 12, or may be software programs in the source device 12. Descriptions are provided separately as follows.

[0149] Picture source 16 may include or be, for example, any type of picture capture device configured to capture real-world pictures, and / or any type of device for generating pictures or comments (for encoding screen content, any text on the screen is also considered as part of the picture or image to be encoded), for example, a computer graphics processor configured to generate computer animated pictures, or any type of device configured to obtain and / or provide real-world or computer animated pictures (e.g., screen content or virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). Picture source 16 may be a camera configured to capture pictures, or a memory configured to store pictures. Picture source 16 may further include any type of (internal or external) interface through which previously captured or generated pictures are stored and / or through which pictures are obtained or received. When picture source 16 is a camera, picture source 16 may be, for example, a local camera, or an integrated camera integrated into the source device. When the picture source 16 is a memory, it may be a local memory or, for example, an integrated memory integrated in the source device. When the picture source 16 includes an interface, the interface may be, for example, an external interface for receiving pictures from an external video source. The external video source may be, for example, an external picture capture device such as a camera, an external memory, or an external picture generation device. The external picture generation device may be, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface, for example, a wired or wireless interface, or an optical interface, following any proprietary or standardized interface protocol.

[0150] A picture may be considered as a two-dimensional array or matrix of pixel elements. The pixel elements in the array may also be called samples. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, three color components are usually used. Specifically, a picture may be represented as or contain three sample arrays. For example, in an RGB format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is usually represented in a luma / chroma format or color space. For example, a picture in a YUV format contains a luma component denoted by Y (alternatively sometimes denoted by L) and two chroma components denoted by U and V. The luma component Y represents the brightness or gray level intensity (e.g., in a grayscale picture, both are the same), and the two chroma components U and V represent the chroma or color information components. Correspondingly, a picture in YUV format includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(U and V). A picture in RGB format may be converted or translated to YUV format, or vice versa. This process is also called color conversion or transformation. If the picture is monochrome, the picture may include only a luma sample array. In this embodiment of the application, the picture transmitted by the picture source 16 to the picture processor may also be called raw picture data.

[0151] The picture pre-processor 18 is configured to receive raw picture data 17 and perform pre-processing on the raw picture data 17 to obtain a pre-processed picture 19 or pre-processed picture data 19. For example, the pre-processing performed by the picture pre-processor 18 may include cropping, color format conversion (e.g., from RGB format to YUV format), color correction, or noise removal.

[0152] The encoder 20 (also referred to as video encoder 20) is configured to receive pre-processed picture data 19 and process the pre-processed picture data 19 by using an associated prediction mode (such as a prediction mode in the embodiments of this specification) to provide encoded picture data 21 (structural details of the encoder 20 are further described below based on FIG. 2, FIG. 4, or FIG. 5). In some embodiments, the encoder 20 may be configured to perform the embodiments described below to realize an encoder-side application of the picture prediction methods described in this specification.

[0153] The communication interface 22 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 over the link 13 to the destination device 14 or any other device (e.g., memory) for storage or direct reconstruction. The other device may be any device used for decoding or storage. The communication interface 22 may be configured, for example, to encapsulate the encoded picture data 21 into an appropriate format, e.g., data packets, for transmission over the link 13.

[0154] The destination device 14 includes a decoder 30. Optionally, the destination device 14 may further include a communication interface 28, a picture post-processor 32, and a display device 34. Descriptions are provided separately below.

[0155] The communication interface 28 may be configured to receive the encoded picture data 21 from the source device 12 or any other source. The any other source is, for example, a storage device. The storage device is, for example, an encoded picture data storage device. The communication interface 28 may be configured to transmit or receive the encoded picture data 21 through a link 13 between the source device 12 and the destination device 14 or through any type of network. The link 13 is, for example, a direct wired or wireless connection. The any type of network is, for example, a wired or wireless network, or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may be configured to decapsulate data packets transmitted through the communication interface 22, for example, to obtain the encoded picture data 21.

[0156] Both communication interface 28 and communication interface 22 may be configured as unidirectional or bidirectional communication interfaces, for example, to send and receive messages to establish a connection, verify and exchange any other information related to the communication link and / or data transmission, such as encoded picture data transmission.

[0157] The decoder 30 (also referred to as decoder 30) is configured to receive the encoded picture data 21 and provide decoded picture data 31 or decoded pictures 31 (structural details of the decoder 30 are further described below on the basis of FIG. 3, FIG. 4 or FIG. 5). In some embodiments, the decoder 30 may be configured to perform the embodiments described below to realize a decoder-side application of the picture prediction methods described in this application.

[0158] The picture post-processor 32 is configured to post-process the decoded picture data 31 (also referred to as reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 may include color format conversion (e.g., from YUV format to RGB format), color correction, cropping, resampling, or any other processing. The picture post-processor 32 may be further configured to transmit the post-processed picture data 33 to a display device 34.

[0159] A display device 34 is configured to receive the post-processed picture data 33, for example, to display the picture to a user or viewer. The display device 34 may be or include any type of display, for example, an integrated or external display or monitor, for presenting the reconstructed picture. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any type of other display.

[0160] 1 depicts source device 12 and destination device 14 as separate devices, a device embodiment may alternatively include both source device 12 and destination device 14, or the functionality of both source device 12 and destination device 14, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be realized by using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0161] 1 may vary depending on the actual device and application. Source device 12 and destination device 14 may include any of a wide range of devices, including any type of handheld or stationary device, e.g., a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, an in-vehicle device, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, or a broadcast transmitter device, and may or may not use any type of operating system.

[0162] The encoder 20 and the decoder 30 may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques are implemented in part by using software, the device may store the software instructions in a suitable non-transitory computer-readable storage medium and execute the instructions by using hardware, such as one or more processors, to perform the techniques of this specification. Any of the foregoing (including hardware, software, combinations of hardware and software, and the like) may be considered as one or more processors.

[0163] In some cases, the video encoding and decoding system 10 depicted in FIG. 1 is merely an example, and the techniques of this application may be applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In another example, data may be retrieved from a local memory, streamed over a network, or the like. A video encoding device may encode data and store the data in a memory, and / or a video decoding device may retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data in a memory and / or retrieve data from a memory and decode data.

[0164] Figure 2 is a schematic / conceptual block diagram of an example of an encoder 20 configured to implement an embodiment of the present invention. In the example of Figure 2, the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown in the figure). The encoder 20 shown in Figure 2 may also be referred to as a hybrid video encoder, or a video encoder based on a hybrid video codec.

[0165] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 20, while for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder. The backward signal path of the encoder corresponds to the signal path of the decoder (see decoder 30 in FIG. 3).

[0166] Encoder 20 receives, for example, via input 202, a picture 201 or a picture block 203 of picture 201, for example a picture in a sequence of pictures forming a video or a video sequence. Picture block 203 may also be referred to as a current picture block or a picture block to be coded, and picture 201 may also be referred to as a current picture or a picture to be coded (particularly in video coding, to distinguish the current picture from other pictures, for example previously coded and / or decoded pictures in the same video sequence, i.e. a video sequence that also includes the current picture).

[0167] An embodiment of the encoder 20 may include a partitioning unit (not shown in FIG. 2) configured to partition a picture 201 into multiple blocks, such as picture blocks 203. The picture 201 is typically partitioned into multiple non-overlapping blocks. The partitioning unit may be configured to use the same block size for all pictures in a video sequence and a corresponding grid that defines the block sizes, or to vary the block size between pictures, or between subsets or groups of pictures, and to partition each picture into corresponding blocks.

[0168] In one example, prediction processing unit 260 of encoder 20 may be configured to perform any combination of the partitioning techniques described above.

[0169] The size of the picture block 203 is smaller than that of the picture 201, but like the picture 210, the picture block 203 may also be or be considered as a two-dimensional array or matrix of samples having sample values. In other words, the picture block 203 may include, for example, one sample array (e.g., a luma array in the case of a monochrome picture 201), three sample arrays (e.g., one luma array and two chroma arrays in the case of a color picture), or any other quantity and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of the picture block 203 defines the size of the picture block 203.

[0170] The encoder 20 depicted in FIG. 2 is configured to code a picture 201 block-by-block, eg, performing encoding and prediction for each picture block 203 .

[0171] The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture image block 203 and the prediction block 265 (details regarding the prediction block 265 are provided further below), for example by subtracting sample values ​​of the prediction block 265 from sample values ​​of the picture image block 203 on a sample-by-sample (pixel-by-pixel) basis to obtain the residual block 205 in the sample domain.

[0172] The transform processing unit 206 is configured to apply a transform, for example a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207, which may also be referred to as transform residual coefficients, represent the residual block 205 in the transform domain.

[0173] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as the transform specified in HEVC / H.265. Compared to an orthogonal DCT transform, such an integer approximation is typically scaled by a factor. In order to maintain the norm of the residual block processed by using the forward transform and the inverse transform, an additional scale factor is applied as part of the transform process. The scale factor is typically selected based on some constraints, e.g., the scale factor being a power of two due to shift operations, the bit depth of the transform coefficients, and a trade-off between accuracy and implementation cost. For example, a particular scale factor may be specified for the inverse transform, e.g., by the inverse transform processing unit 212 at the decoder 30 side (and the corresponding inverse transform, e.g., by the inverse transform processing unit 212 at the encoder 20 side), and correspondingly, a corresponding scale factor may be specified for the forward transform, e.g., by the transform processing unit 206 at the encoder 20 side.

[0174] The quantization unit 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example, by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce a bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, various scales may be applied to achieve finer or coarser quantization. A smaller quantization step corresponds to a finer quantization and a larger quantization step corresponds to a coarser quantization. The appropriate quantization step may be indicated by a quantization parameter (QP). For example, the quantization parameter may be an index to a predefined set of appropriate quantization steps. For example, a smaller quantization parameter may correspond to a finer quantization (smaller quantization step), and a larger quantization parameter may correspond to a coarser quantization (larger quantization step), and vice versa. Quantization may include a division by the quantization step and a corresponding quantization or inverse quantization, or may include a multiplication by the quantization step, performed, for example, by the inverse quantization unit 210. An embodiment according to some standards, such as HEVC, may use the quantization parameter to determine the quantization step. In general, the quantization step may be calculated based on the quantization parameter by using a fixed-point approximation of an equation that includes a division. To restore the norm of the residual block, an additional scale factor may be introduced for quantization and dequantization, and the norm of the residual block may be modified for the scale used in the fixed-point approximation of the equation for the quantization step and the quantization parameter. In one example implementation, the scale of the inverse transform may be combined with the scale of the dequantization. Alternatively, a customized quantization table may be used. It may be used and signaled, for example, in the bitstream, from the encoder to the decoder. Quantization is a lossy operation, with larger quantization steps indicating larger losses.

[0175] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients, e.g., by applying the inverse of the quantization scheme applied by the quantization unit 208, based on or using the same quantization step as the quantization unit 208, to obtain dequantized coefficients 211. The dequantized coefficients 211, also referred to as dequantized residual coefficients 211, may correspond to the transform coefficients 207, although they typically differ from the transform coefficients due to losses caused by quantization.

[0176] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, for example, an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transformed dequantized block 213 or an inverse transformed residual block 213.

[0177] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265, for example, by adding sample values ​​of the reconstructed residual block 213 and sample values ​​of the prediction block 265, to obtain a reconstructed block 215 in the sample domain.

[0178] Optionally, a buffer unit 216 (or "buffer" 216 for short), for example of the line buffer 216, is configured to buffer or store the reconstructed block 215 and corresponding sample values, for example for intra prediction. In another embodiment, the encoder may be configured to use the unfiltered reconstructed block and / or corresponding sample values ​​stored in the buffer unit 216 to perform any type of estimation and / or prediction, for example, intra prediction.

[0179] For example, in one embodiment, the encoder 20 may be configured such that the buffer unit 216 is configured to store reconstructed blocks 215 that are used not only for intra prediction 254 but also for loop filter unit 220 (not shown in FIG. 2), and / or such that, for example, the buffer unit 216 and the decoded picture buffer 230 form one buffer. In another embodiment, the filtered blocks 211 and / or blocks or samples from the decoded picture buffer 230 (not shown in FIG. 2) are used as input or basis for intra prediction 254.

[0180] The loop filter unit 220 ("loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221 to smooth pixel transitions or improve video quality. The loop filter unit 220 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, e.g., a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 200 is represented as an in-loop filter in FIG. 2, in another implementation, the loop filter unit 200 may be realized as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coded block after the loop filter unit 220 performs a filtering operation on the reconstructed coded block.

[0181] In one embodiment, the encoder 20 (correspondingly, the loop filter unit 220) may be configured to output loop filter parameters (e.g., sample adaptive offset information), e.g., directly or after entropy coding performed by the entropy coding unit 270 or any other entropy coding unit, so that the decoder 30 can receive and apply the same loop filter parameters for decoding.

[0182] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in video data encoding by the encoder 20. The DPB 230 may be formed by any one of a number of types of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM)), or another type of memory device. The DPB 230 and the buffer 216 may be provided by the same memory device or separate memory devices. In one example, the decoded picture buffer (DPB) 230 is configured to store the filtered block 221. The decoded picture buffer 230 may further be configured to store other previously filtered blocks, e.g., previously reconstructed filtered block 221, of the same current picture, or a different picture, e.g., a previously reconstructed picture, and may provide a fully previously reconstructed, i.e., decoded picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), e.g., for inter prediction. In one example, if the reconstructed block 215 is reconstructed without in-loop filtering, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215.

[0183] The prediction processing unit 260, also referred to as block prediction processing unit 260, is configured to receive or obtain a picture block 203 (current picture block 203 of current picture 201) and reconstructed picture data, e.g., reference samples of the same (current) picture from buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from decoded picture buffer 230, and to process such data for prediction to provide a prediction block 265, which may be an inter prediction block 245 or an intra prediction block 255.

[0184] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 to be used as the prediction block 265 for calculation of the residual block 205 and for reconstruction of the reconstructed block 215.

[0185] In one embodiment, mode selection unit 262 may be configured to select a prediction mode (e.g., from prediction modes supported by prediction processing unit 260) that provides the best match, or in other words, the smallest residual (smallest residual means better compression for transmission or storage), or the smallest signaling overhead (smallest signaling overhead means better compression for transmission or storage), or considers or balances both. Mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), specifically, to select a prediction mode that provides the smallest rate distortion optimization, or to select a prediction mode whose associated rate distortion at least satisfies a prediction mode selection criterion.

[0186] The following describes in detail the prediction process performed (eg, by prediction processing unit 260) and the mode selection performed (eg, by mode selection unit 262) by an example of encoder 20.

[0187] As described above, the encoder 20 is configured to determine or select an optimal or best prediction mode from a (predetermined) set of prediction modes, which may include, for example, intra-prediction modes and / or inter-prediction modes.

[0188] The set of intra-prediction modes may include 35 different intra-prediction modes, e.g., omni-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or may include 67 different intra-prediction modes, e.g., omni-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in the currently developing H.266.

[0189] In a possible implementation, the set of inter prediction modes depends on the available reference pictures (i.e., at least partially decoded pictures stored in DBP 230, for example, as described above) and other inter prediction parameters, for example, whether the entire reference picture is used to search for the best matching reference block, or only a part of the reference picture, for example, a search window area around the area of ​​the current picture block, is used, and / or whether pixel interpolation, such as half-pel and / or quarter-pel interpolation, is applied. The set of inter prediction modes may include, for example, an Advanced Motion Vector Prediction (AMVP) mode and a merge mode. In a specific implementation, the set of inter prediction modes may include, in an embodiment of this application, an improved control point based AMVP mode and an improved control point based merge mode. In one example, the intra prediction unit 254 may be configured to perform any combination of the inter prediction techniques described below.

[0190] In addition to the aforementioned prediction modes, skip mode and / or direct mode may also be applied in the embodiments of this application.

[0191] Prediction processing unit 260 may be further configured to partition picture block 203 into smaller block partitions or sub-blocks, e.g., by repeatedly using quad-tree (QT) partitions, binary-tree (BT) partitions, triple-tree (TT) partitions, or any combination thereof, and perform prediction on each of the block partitions or sub-blocks, e.g., Mode selection includes selecting a tree structure of the partitioned picture block 203 and selecting a prediction mode to be applied to each of the block partitions or sub-blocks.

[0192] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain a picture image block 203 (the current picture image block 203 of the current picture 201) and a decoded picture 231 or at least one or more previously reconstructed blocks, e.g., one or more reconstructed blocks of other / different previously decoded pictures 231, for motion estimation. For example, a video sequence may include the current picture and the previously decoded picture 31, or in other words, the current picture and the previously decoded picture 31 may be part of or form a sequence of pictures forming a video sequence.

[0193] For example, the encoder 20 may be configured to select a reference block from multiple reference blocks of different pictures of the same picture or multiple other pictures, and provide the reference picture and / or an offset (spatial offset) between the position (X, Y coordinates) of the reference block and the position of the current picture block as an inter-prediction parameter to a motion estimation unit (not shown in FIG. 2). The offset is also called a motion vector (MV).

[0194] The motion compensation unit is configured to obtain inter prediction parameters and perform inter prediction based on or by using the inter prediction parameters to obtain inter prediction block 245. The motion compensation performed by the motion compensation unit (not shown in FIG. 2) may include fetching or generating a predictive block (possibly performing interpolation with sub-pixel accuracy) based on a motion / block vector determined through motion compensation. Interpolation filtering can generate additional pixel samples from known pixel samples, thereby potentially increasing the number of candidate predictive blocks that can be used to code the picture block. Upon receiving a motion vector for the PU of the current picture block, the motion compensation unit 246 may locate the predictive block to which the motion vector points in one of the reference picture lists. The motion compensation unit 246 may also generate syntax elements associated with the block and the video slice, such that the decoder 30 uses the syntax elements to decode the picture block in the video slice.

[0195] Specifically, the inter prediction unit 244 may send a syntax element to the entropy encoding unit 270. The syntax element includes inter prediction parameters (such as index information of the selection of the inter prediction mode used for prediction of the current picture block after traversing multiple inter prediction modes). In a possible application scenario, if there is only one inter prediction mode, the inter prediction parameters may alternatively not be held in the syntax element. In this case, the decoder side 30 may directly perform decoding by using a default prediction mode. It may be understood that the inter prediction unit 244 may be configured to perform any combination of inter prediction techniques.

[0196] The intra prediction unit 254 is configured to obtain, e.g., receive, the picture block 203 (current picture block) and one or more previously reconstructed blocks of the same picture, e.g., reconstructed neighboring blocks, for intra estimation. The encoder 20 may be configured, for example, to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[0197] In one embodiment, the encoder 20 is configured to select an intra prediction mode according to an optimization criterion, for example based on a minimum residual (e.g., the intra prediction mode that provides a predicted block 255 that is most similar to the current picture block 203) or a minimum rate distortion.

[0198] The intra prediction unit 254 is further configured to determine the intra prediction block 255 based on, for example, intra prediction parameters in the selected intra prediction mode. In either case, after selecting an intra prediction mode for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters, i.e., information indicating the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform any combination of intra prediction techniques.

[0199] Specifically, the intra prediction unit 254 may send a syntax element to the entropy encoding unit 270. The syntax element includes intra prediction parameters (such as index information of the selection of the intra prediction mode used for prediction of the current picture block after traversing multiple intra prediction modes). In a possible application scenario, if there is only one intra prediction mode, the intra prediction parameters may alternatively not be held in the syntax element. In this case, the decoder side 30 may directly perform decoding by using a default prediction mode.

[0200] The entropy encoding unit 270 is configured to apply (or not) an entropy encoding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, a context adaptive binary arithmetic coding (CABAC), a syntax-based context-adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding, or another entropy encoding methodology or technique) to one or all of the quantized residual coefficients 209, the inter prediction parameters, the intra prediction parameters, and / or the loop filter parameters to obtain encoded picture data 21, which may be output via an output 272 in the form of, for example, an encoded bitstream 21. The encoded bitstream may be transmitted to the video decoder 30 or archived for later transmission or retrieval by the video decoder 30. Entropy encoding unit 270 may be further configured to entropy encode another syntax element for the current video slice being encoded.

[0201] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform-based encoder 20 may directly quantize the residual signal without the transform processing unit 206 for some blocks or frames. In another implementation, the encoder 20 may have the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0202] Specifically, in this embodiment of the application, the encoder 20 may be configured to implement the video encoding process described in the following embodiments.

[0203] It should be understood that the video encoder in this application may include only some modules in the video encoder 20. For example, the video encoder in this application may include a picture decoding unit and a partition unit. The picture decoding unit may include one or more of an entropy decoding unit, a prediction unit, an inverse transform unit, and an inverse quantization unit.

[0204] In addition, another structural variant of the video encoder 20 may be used to encode the video stream. For example, for some picture blocks or picture frames, the video encoder 20 may directly quantize the residual signal, and no processing by the transform processing unit 206 is required, and correspondingly, no processing by the inverse transform processing unit 212 is required. Alternatively, for some picture blocks or picture frames, the video encoder 20 does not generate residual data, and correspondingly, no processing by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 is required. Alternatively, the video encoder 20 may directly store the reconstructed picture block as a reference block, and no processing by the filter 220 is required. Alternatively, the quantization unit 208 and the inverse quantization unit 210 in the video encoder 20 may be combined. The loop filter 220 is optional. In addition, for lossless compression encoding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are optional. It should be understood that in different application scenarios, the inter prediction unit 244 and the intra prediction unit 254 may be selectively used.

[0205] 3 is a schematic / conceptual block diagram of an example of a decoder 30 configured to implement an embodiment of the present invention. The video decoder 30 is configured to receive encoded picture data (e.g., an encoded bitstream) 21, e.g., encoded by the encoder 20, to obtain a decoded picture 231. In the decoding process, the video decoder 30 receives video data from the video encoder 20, e.g., an encoded video bitstream representing picture blocks of an encoded video slice and associated syntax elements.

[0206] 3, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, the video decoder 30 may perform a decoding process that is generally reciprocal to the encoding process described with reference to the video encoder 20 in FIG.

[0207] The entropy decoding unit 304 is configured to perform entropy decoding on the encoded picture data 21, e.g., to obtain quantized coefficients 309 and / or decoded coding parameters (not depicted in FIG. 3), e.g., any one or all of (decoded) inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 30 may receive the syntax elements at a video slice level and / or a video block level.

[0208] The inverse quantization unit 310 may have the same functionality as the inverse quantization unit 110. The inverse transform processing unit 312 may have the same functionality as the inverse transform processing unit 212. The reconstruction unit 314 may have the same functionality as the reconstruction unit 214. The buffer 316 may have the same functionality as the buffer 216. The loop filter 320 may have the same functionality as the loop filter 220. The decoded picture buffer 330 may have the same functionality as the decoded picture buffer 230.

[0209] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. The inter prediction unit 344 may be similar in function to the inter prediction unit 244, and the intra prediction unit 354 may be similar in function to the intra prediction unit 254. Prediction processing unit 360 typically performs block prediction and / or obtains a prediction block 365 from the coded data 21, and is configured to receive or obtain (explicitly or implicitly) prediction-related parameters and / or information regarding a selected prediction mode, e.g., from the entropy decoding unit 304.

[0210] When a video slice is coded into an intra-coded (I) slice, the intra prediction unit 354 of the prediction processing unit 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra prediction mode and data of a previously decoded block of the current frame or picture. When a video frame is coded into an inter-coded (i.e., B or P) slice, the inter prediction unit 344 (e.g., a motion compensation unit) in the prediction processing unit 360 is configured to generate a prediction block 365 for a video block in the current video slice based on a motion vector and another syntax element received from the entropy decoding unit 304. For inter prediction, the prediction block may be generated from one of the reference pictures in one reference picture list. The video decoder 30 may construct the reference frame lists, List 0 and List 1, based on the reference pictures stored in the DPB 330 by using a default construction technique.

[0211] Prediction processing unit 360 is configured to determine prediction information for video blocks of the current video slice by analyzing the motion vectors and other syntax elements, and use the prediction information to generate predictive blocks for the current video block being decoded. In one example of this application, prediction processing unit 360 determines a prediction mode (e.g., intra or inter prediction) for encoding video blocks in the video slice, an inter prediction slice type (e.g., B slice, P slice, or GPB slice), construction information of one or more of a reference picture list for the slice, a motion vector of each inter coded video block for the slice, an inter prediction status of each inter coded video block in the slice, and other information, by using some received syntax elements to decode video blocks in the current video slice. In another example of this application, the syntax elements received by video decoder 30 from the bitstream include syntax elements in one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

[0212] Inverse quantization unit 310 may be configured to perform inverse quantization (i.e., dequantization) on quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each video block in a video slice to determine the degree of quantization to be applied, and similarly, the degree of inverse quantization to be applied.

[0213] Inverse transform processing unit 312 is configured to apply an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to produce residual blocks in the pixel domain.

[0214] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365, for example, by adding sample values ​​of the reconstructed residual block 313 and sample values ​​of the prediction block 365, to obtain a reconstructed block 315 in the sample domain.

[0215] The loop filter unit 320 (in the coding loop or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321 to smooth pixel transitions or improve video quality. In an example, the loop filter unit 320 may be configured to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, e.g., a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is depicted as an in-loop filter in FIG. 3, in another implementation, the loop filter unit 320 may be realized as a post-loop filter.

[0216] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0217] The decoder 30 is arranged to output the decoded pictures 31 via an output 332, for example for presentation or viewing to a user.

[0218] Other variations of the video decoder 30 may be used to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform-based decoder 30 may directly inverse quantize the residual signal for some blocks or frames, without the inverse transform processing unit 312. In another implementation, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0219] Specifically, in this embodiment of the application, the decoder 30 is configured to implement the video decoding methods described in the following embodiments.

[0220] It should be understood that the video encoder in this application may include only some modules in the video encoder 30. For example, the video encoder in this application may include a partition unit and a picture coding unit. The picture coding unit may include one or more of a prediction unit, a transform unit, a quantization unit, and an entropy coding unit.

[0221] In addition, another structural variant of the video decoder 30 may be used to decode the encoded video bitstream. For example, the video decoder 30 may generate an output video stream without processing by the filter 320. Alternatively, for some picture blocks or picture frames, the entropy decoding unit 304 of the video decoder 30 does not obtain quantized coefficients through decoding, and correspondingly, the inverse quantization unit 310 and the inverse transform processing unit 312 do not need to perform processing. The loop filter 320 is optional. In addition, in the case of lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are also optional. It should be understood that in different application scenarios, the inter prediction unit and the intra prediction unit may be selectively used.

[0222] It should be understood that in the encoder 20 and the decoder 30 in this application, the processing result of a procedure may be output to the following procedure after further processing. For example, after a procedure such as interpolation filtering, motion vector derivation, or loop filtering, an operation such as clip or shift is further performed on the processing result of the corresponding procedure.

[0223] For example, a motion vector derived based on the motion vector of a neighboring affine coding block (a coding block predicted by using an affine motion model may be called an affine coding block) of a control point of a current picture block, or a motion vector derived based on the motion vector of a neighboring affine coding block of a subblock of a current picture block, may be further processed. This is not limited in this application. For example, the value of a motion vector is constrained to be within a certain bit width range. The allowed bit width of a motion vector is bitDepth, and the value of a motion vector ranges from -2^(bitDepth-1) to 2^(bitDepth-1)-1, and the symbol "^" is assumed to represent a power. If bitDepth is 16, the value ranges from -32768 to 32767. If bitDepth is 18, the value ranges from -131072 to 131071.

[0224] For another example, the values ​​of the motion vectors (e.g., the motion vectors MV of four 4x4 sub-blocks in an 8x8 picture block) may be further constrained such that the maximum difference between the integer parts of the MVs of the four 4x4 sub-blocks does not exceed N (e.g., N may be set to 1) pixels.

[0225] 4 is an illustrative diagram of an example of a video coding system 40 including the encoder 20 in FIG. 2 and / or the decoder 30 in FIG. 3 according to an example embodiment. The video coding system 40 may implement a combination of various techniques in the embodiments of this application. In the illustrated implementation, the video coding system 40 may include an imaging device 41, the encoder 20, the decoder 30 (and / or a video encoder / decoder implemented by logic circuitry 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0226] 4, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, encoder 20, decoder 30, processor 43, memory 44, and / or display device 45 may be in communication with one another. As described, video coding system 40 is illustrated with encoder 20 and decoder 30, but video coding system 40 may include only encoder 20 or only decoder 30 in different examples.

[0227] In some examples, the antenna 42 may be configured to transmit or receive an encoded bitstream of video data. Additionally, in some examples, the display device 45 may be configured to present the video data. In some examples, the logic circuitry 47 may be realized by the processing unit 46. The processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. The video coding system 40 may also include an optional processor 43. The optional processor 43 may likewise include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. In some examples, the logic circuitry 47 may be realized by hardware, e.g., dedicated hardware for video coding. The processor 43 may be realized by general-purpose software, an operating system, or the like. Additionally, the memory 44 may be any type of memory, e.g., a volatile memory (e.g., a static random access memory (SRAM) or a dynamic random access memory (DRAM)) or a non-volatile memory (e.g., a flash memory). In a non-limiting example, memory 44 may be implemented as a cache memory. In some examples, logic circuitry 47 may access memory 44 (e.g., for implementing a picture buffer). In another example, logic circuitry 47 and / or processing unit 46 may include a memory (e.g., a cache) for implementing a picture buffer or the like.

[0228] In some examples, the encoder 20 implemented by logic circuitry may include a picture buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include the encoder 20 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 2 and / or any other encoder system or subsystems described herein. The logic circuitry may be configured to perform various operations described herein.

[0229] In some examples, the decoder 30 may be implemented by logic circuitry 47 in a similar manner to implement the various modules described with reference to the decoder 30 in FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, the decoder 30 implemented by logic circuitry may include a picture buffer (e.g., implemented by processing unit 2820 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include the decoder 30 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 3 and / or any other decoder system or subsystem described herein.

[0230] In some examples, antenna 42 may be configured to receive an encoded bitstream of video data. As described, the encoded bitstream may include data, indicators, index values, mode selection data, or the like described herein related to video frame coding, such as data related to a coding partition (e.g., transform coefficients or quantized transform coefficients, optional indicators (as described), and / or data defining the coding partition). Video coding system 40 may further include a decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0231] In this embodiment of the application, it should be understood that for the examples described with reference to the encoder 20, the decoder 30 may be configured to perform an inverse process. With respect to signaling syntax elements, the decoder 30 may be configured to receive and parse such syntax elements and, in response, decode the associated video data. In some examples, the encoder 20 may entropy encode the syntax elements into the encoded video bitstream. In such examples, the decoder 30 may parse the syntax elements and, in response, decode the associated video data.

[0232] 5 is a schematic structural diagram of a video coding device 400 (e.g., video encoding device 400 or video decoding device 400) according to an embodiment of the present invention. Video coding device 400 is suitable for implementing embodiments described in this specification. In one embodiment, video coding device 400 may be a video decoder (e.g., decoder 30 in FIG. 3) or a video encoder (e.g., encoder 20 in FIG. 2). In another embodiment, video coding device 400 may be one or more components of decoder 30 in FIG. 3 or encoder 20 in FIG. 2.

[0233] The video coding device 400 includes an input port 410 and a receiving unit (Rx) 420 configured to receive data, a processor, logic unit, or central processing unit (CPU) 430 configured to process the data, a transmitter unit (Tx) 440 and an output port 450 configured to transmit the data, and a memory 460 configured to store the data. The video coding device 400 may further include optical-electrical and electro-optical (EO) components coupled to the input port 410, the receiver unit 420, the transmitter unit 440, and the output port 450 for outputting or inputting optical or electrical signals.

[0234] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the input port 410, the receiver unit 420, the transmitter unit 440, the output port 450, and the memory 460. The processor 430 includes a coding module 470 (e.g., the encoding module 470 or the decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed in this specification to realize the picture prediction method provided in the embodiments of the present invention. For example, the encoding / decoding module 470 performs, processes, or provides various coding operations. Thus, the encoding / decoding module 470 substantially improves the functionality of the video coding device 400 and affects the transformation of the video coding device 400 into different states. Alternatively, the encoding / decoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0235] Memory 460 may include one or more disks, tape drives, and solid state devices, and may be used as an overflow data storage device for storing programs when such programs are selected for execution, and for storing instructions and data read during program execution. Memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), random access memory (ternary content-addressable memory, TCAM), and / or static random access memory (SRAM).

[0236] FIG. 6 is a simplified block diagram of an apparatus 500 that may be used as any one or two of the source device 12 and the destination device 14 in FIG. 1 according to an example embodiment. The apparatus 500 may realize the picture prediction method in the embodiment of this application. In other words, FIG. 6 is a schematic block diagram of an implementation of an encoding or decoding device (shortly called coding device 500) according to an embodiment of this application. The coding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and the memory are connected through the bus system. The memory is configured to store instructions. The processor is configured to execute the instructions stored in the memory. The memory of the coding device stores program code. The processor may call the program code stored in the memory to execute various video encoding or decoding methods described in this application, in particular various new picture block partitioning methods. To avoid repetition, details will not be described again here.

[0237] In this embodiment of the application, the processor 510 may be a central processing unit (CPU). Alternatively, the processor 510 may be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.

[0238] The memory 530 may include a read-only memory (ROM) device, or a random access memory (RAM) device. Alternatively, any other suitable type of storage device may be used as the memory 530. The memory 530 may include code and data 531 accessed by the processor 510 through the bus 550. The memory 530 may further include an operating system 533 and an application program 535. The application program 535 includes at least one program that enables the processor 510 to perform the video encoding or decoding method described in this application. For example, the application program 535 may include applications 1 to N and may further include a video encoding or decoding application (shortly called a video coding application) that performs the video encoding or decoding method described in this application.

[0239] In addition to a data bus, the bus system 550 may further include a power bus, a control bus, a status signal bus, and the like. However, for clarity of illustration, the various types of buses in the figures are marked as the bus system 550.

[0240] Optionally, coding device 500 may further include one or more output devices, such as a display 570. In one example, display 570 may be a touch display that combines a display with a touch unit that operatively senses touch input. Display 570 may be connected to processor 510 through bus 550.

[0241] In order to better understand the picture prediction method in the embodiment of this application, the following first describes in detail some related concepts and basic contents of inter prediction.

[0242] Inter prediction means searching the reconstructed picture for a matching reference block for a current picture block in the current picture and using the pixel values ​​of pixel elements in the reference block as predictors of pixel values ​​of pixel elements in the current picture block (this process is called Motion Estimation (ME)).

[0243] Motion estimation is to try multiple reference blocks in a reference picture for a current picture block, and then finally determine one or two reference blocks (two reference blocks are required for bi-prediction) from the multiple reference blocks by using rate-distortion optimization (RDO) or another method. The reference blocks are used to perform inter prediction for the current picture block.

[0244] The motion information of the current picture block includes index information of the prediction direction (which is usually forward prediction, backward prediction, or bi-prediction), one or two motion vectors (MV) pointing to the reference block, and index information of the picture in which the reference block is located (which is usually represented by using a reference index).

[0245] Forward prediction means selecting a reference picture from a forward reference picture set to obtain a reference block for the current picture block. Backward prediction means selecting a reference picture from a backward reference picture set to obtain a reference block for the current picture block. Bi-prediction means selecting a reference picture from a forward reference picture set and selecting a reference picture from a backward reference picture set to obtain a reference block. When a bi-prediction method is used, there are two reference blocks in the current coding block. Each reference block needs to be indicated by using a motion vector and a reference index. Then, a predictor of the pixel value of the pixel element in the current picture block is determined based on the pixel values ​​of the pixel elements in the two reference blocks.

[0246] In HEVC, there are two inter prediction modes: AMVP mode and merge mode.

[0247] In AMVP mode, the spatially or temporally adjacent coded blocks (denoted as neighboring blocks) of the current coding block are first traversed. Based on the motion information of the neighboring blocks, a candidate motion vector list is constructed. Then, based on the rate-distortion cost, an optimal motion vector is determined from the candidate motion information list, and the candidate motion information with the minimum rate-distortion cost is used as the motion vector predictor (MVP) of the current coding block.

[0248] The position and traversal order of adjacent blocks are predefined. The rate-distortion cost can be obtained through calculation by using formula (1), where J is the rate-distortion cost, SAD is the sum of absolute differences (SAD) between the original pixel value and the predicted pixel value obtained through motion estimation performed by using the candidate motion vector predictor, R is the bit rate, and λ is the Lagrange multiplier. The encoder side transfers the index value of the selected motion vector predictor in the candidate motion vector list and the reference index value to the decoder side. In addition, the encoder side may perform a motion search in a neighborhood centered on the MVP to obtain the actual motion vector of the current coding block, and then transfer the motion vector difference between the MVP and the actual motion vector to the decoder side. J = SAD + λR (1)

[0249] In addition, with respect to different motion models, the AMVP modes can be classified into AMVP modes based on a translational model and AMVP modes based on a non-translational model.

[0250] In merge mode, first, a candidate motion information list is constructed based on the motion information of the spatially or temporally coded units of the current coding unit. Then, based on the rate-distortion cost, the best motion information is determined from the candidate motion information list as the motion information of the current coding unit. Finally, the index value of the position of the best motion information in the candidate motion information list (hereinafter referred to as merge index) is transmitted to the decoder side.

[0251] In merge mode, the spatial and temporal candidate motion information of the current coding unit may be represented in Figure 7. The spatial candidate motion information is from five spatially neighboring blocks (A0, A1, B0, B1, and B2). If the neighboring blocks are not available or the prediction mode is intra prediction, the neighboring blocks are not added to the candidate motion information list.

[0252] The temporal candidate motion information of the current coding unit may be obtained by scaling the MV of a block at a corresponding position in the reference frame based on a picture order count (POC) of the reference frame and the current frame. When the block at the corresponding position in the reference frame is obtained, it may first be determined whether a block at a position T in the reference frame is available. If the block at the position T is not available, a block at a position C is selected.

[0253] When a translation model is used for prediction, the same motion information is used for all pixels in a coding unit, and then motion compensation is performed based on the motion information to obtain a predictor of pixels in the coding unit. However, in the real world, there are many kinds of motions. Many objects, such as rotating objects, roller coasters that rotate in different directions, fireworks, and some stunts in movies, are not translational motions. If these moving objects, especially those in user generated content (UGC) scenarios, are coded by using block motion compensation techniques based on the translational motion model in current coding standards, the coding efficiency will be greatly affected. Therefore, in order to improve coding effect, prediction based on a non-translational motion model is provided.

[0254] In prediction based on a non-translational motion model, the same motion model is used at the encoder side and the decoder side to derive the motion information of each sub-motion compensation unit in the current coding block, and then motion compensation is performed based on the motion information of the sub-motion compensation unit to obtain the prediction sub-block of each sub-block to improve prediction efficiency. Frequently used non-translational motion models include a four-parameter affine motion model and a six-parameter affine motion model.

[0255] In addition, skip mode is a special model of merge mode. The difference is that in skip mode, there is no residual during transmission, and only merge candidate index is transferred. The merge index is used to indicate the best or target candidate motion information in the merge candidate motion information list.

[0256] When a picture is predicted, different modes can be used. The following describes these common modes in detail.

[0257] Merge mode using motion vector difference

[0258] In the merge with motion vector difference (MMVD) mode, one or more candidate motion vectors are selected from the merge candidate motion vector list, and then motion vector (MV) extension representation is performed based on the candidate motion vectors. The MV extension representation includes the starting point, motion step, and motion direction of the MV.

[0259] Generally, the type of the candidate motion vector selected in the MMVD mode is the default merge type (e.g., MRG_TYPE_DEFAULT_N). The selected candidate motion vector is the starting point of the MV. In other words, the selected candidate motion vector is used to determine the initial position of the MV.

[0260] As shown in Table 1, the base candidate index (Base candidate IDX) indicates which candidate motion vector is selected as the best candidate motion vector from the candidate motion vector list. If the merge candidate motion vector list contains one candidate motion vector for selection, the Base candidate IDX does not need to be determined.

[0261] [Table 1]

[0262] For example, if the base candidate index is 1, the selected candidate motion vector is the second motion vector in the merge candidate motion vector list.

[0263] The distance index (Distance IDX) represents the offset distance information of the motion vector. The value of the distance index represents the distance offset from the initial position (e.g., a preset distance). The distance here is represented by pixel distance. The pixel distance may be more simply called pel. The correspondence between the distance index and the pixel distance may be represented in Table 2.

[0264] [Table 2]

[0265] The direction index (Direction IDX) is used to represent the direction of the motion vector differential (MVD) based on the initial position. The direction index may include a total of four cases. A specific definition may be shown in Table 3.

[0266] [Table 3]

[0267] The process of determining the predicted pixel values ​​of the current picture block based on the MMVD scheme includes the following steps.

[0268] Step 1: Determine the start point of the MV based on the base candidate IDX.

[0269] FIG. 8 is a schematic diagram of an MMVD search point according to one embodiment of the present application, and FIG. 9 is a schematic diagram of an MMVD search process according to this embodiment of the present application.

[0270] For example, the starting point of the MV is the central hollow point in FIG. 8, or the position corresponding to the solid line in FIG.

[0271] Step 2: Based on the Direction IDX, determine the ab offset direction based on the start point of the MV.

[0272] Step 3: Based on the Distance IDX, determine the quantity of pixel elements that are offset in the direction indicated by the Direction IDX.

[0273] For example, Direction IDX==00 and Distance IDX=2 indicates that a motion vector offset by one pixel element in the positive X direction is used as the motion vector of the current picture block to predict or obtain the predicted pixel value of the current picture block. Combined Intra and Inter modes.

[0274] In a coding block / CU coded in merge mode, an indicator (e.g., ciip_flag) may be sent to indicate whether a combined inter and intra prediction (CIIP) mode is used for the current picture. When the CIIP mode is used, an intra prediction block may be generated based on an intra prediction mode selected from an intra candidate mode list according to a related syntax element, and an inter prediction block is generated by using a conventional inter prediction method. Finally, an adaptive weighting form is used to combine the intra coding prediction block and the inter coding prediction block to generate a final prediction block.

[0275] For a luma block, the intra candidate mode list may be selected from four modes: DC mode, planar mode, horizontal mode, and vertical mode. The size of the intra candidate mode list is selected based on the shape of the current coding block, and there may be three or four modes in the intra candidate mode list. When the width of the current coding block / CU is greater than twice its height, the intra candidate mode list does not include a horizontal mode. When the height of the current coding block / CU is greater than twice its width, the intra candidate mode list does not include a vertical mode.

[0276] In the combined intra-coding and inter-coding weighting method, different weighting factors are used for different intra-prediction modes. Specifically, when DC or planar modes are used for intra-coding, or when the length or width of the current coding block is 4 or less, the same weight value / weight factor is used for the predictor obtained through intra-prediction and the predictor obtained through inter-prediction. Otherwise, the weight value / weight factor may be determined based on the intra-prediction mode used by the current block and / or the position of the prediction sample within the current block. For example, when horizontal and vertical modes are used for intra-coding, variable weight factors are used.

[0277] Triangle prediction unit mode:

[0278] The triangle prediction unit mode (triangle PU) may also be referred to as a triangle partition mode (TPM) or a merged triangle mode. For ease of description, in this application, the triangle prediction unit mode or the triangle partition mode is simply referred to as TPM, which is also applicable hereinafter.

[0279] As shown in FIG. 11, the current picture block is divided into two triangular prediction units, and a motion vector and a reference index are selected from the uni-prediction candidate list for each triangular prediction unit. Then, a predictor is obtained for each of the two triangular prediction units, and a predictor is obtained by performing adaptive weighting on the pixels included in each hypotenuse region. Then, a transform and quantization process is performed on the entire current picture block. It should be noted that the triangular prediction unit method is usually only applied in skip mode or merge mode. The left side of FIG. 10 represents the upper left to lower right division mode (in other words, the division from the upper left to the lower right), and the right side of FIG. 10 represents the upper right to lower left division mode (in other words, the division from the upper right to the lower left).

[0280] The uni-prediction candidate list in the triangular prediction unit mode may usually include five candidate predictive motion vectors. These candidate predictive motion vectors are obtained, for example, by using the seven surrounding neighboring blocks (five spatially adjacent blocks and two temporally co-located blocks) in FIG. 5. The motion information of the seven neighboring blocks is searched, and the seven neighboring blocks are sequentially placed in the uni-prediction candidate list. For example, the sequence may be a bi-predictive motion vector in L0, a bi-predictive motion vector in L1, and an average of the motion vectors in L0 and L1. If there are less than five candidates, a zero motion vector 0 is added to the uni-prediction candidate list. During encoding, the uni-prediction candidate list is obtained in the aforementioned manner. For example, in the uni-prediction candidate list, forward predictive motion information is used to predict pixel predictors of one triangle PU, and backward predictive motion information is used to predict pixel predictors of other triangle PUs. The encoder side selects the optimal motion vector through traversal. For example, the following manner {m,i,j}, i.e., {0,1,0}, {1,0,1}, {1,0,2}, {0,0,1}, {0,2,0} {1,0,3}, {1,0,4}, {1,1,0}, {0,3,0}, {0,4,0} {0,0,2}, {0,1,2}, {1,1,2}, {0,0,4}, {0,0,3} {0,1,3}, {0,1,4}, {1,1,4}, {1,1,3}, {1,2,1} {1,2,0}, {0,2,1}, {0,4,3}, {1,3,0}, {1,3,2} {1,3,4}, {1,4,0}, {1,3,1}, {1,2,3}, {1,4,1} {0,4,1}, {0,2,3}, {1,4,2}, {0,3,2}, {1,4,3} {0,3,1}, {0,2,4}, {1,2,4}, {0,4,2}, {0,3,4} may be used, where {m,i,j}, m in the first position represents a partitioning mode from upper left to lower right or from upper right to lower left, the second position represents forward motion information of the i-th candidate predicted motion vector used for the first triangle PU, and the third position represents backward motion information of the j-th candidate predicted motion vector used for the second triangle PU.

[0281] For the adaptive weighting process performed based on the predictor of the pixel included in the hypotenuse region, see Figure 11. After the prediction for the triangular prediction units P1 and P2 is completed, the adaptive weighting process is performed on the pixel included in the hypotenuse region to obtain the final predictor of the current picture block.

[0282] For example, in the picture on the left of FIG. 11, the predictor for the pixel at position 2 is

[0283]

number

[0284] P1 represents the predictor for pixels in the top right region of FIG. 11, and P2 represents the predictor for pixels in the bottom left region of FIG.

[0285] The two sets of weighted parameters are:

[0286] A first set of weighted parameters {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for the luma and chroma points, respectively.

[0287] A second set of weighted parameters {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for the luma and chroma points, respectively.

[0288] One set of weighted parameters is used to encode and decode the current picture block. When the reference pictures of the two prediction units are different or the difference between the motion vectors of the two prediction units is greater than 16 pixels, the second set of weighted parameters is selected; otherwise, the first set of weighted parameters is used.

[0289] FIG. 12 is a schematic block diagram of a video communication system according to one embodiment of the present application.

[0290] The video communication system 500 depicted in Figure 12 includes a source device 600 and a destination device 700. The source device 600 can encode a captured video and transmit the encoded video bitstream to the receiving device 700. The destination device 700 can parse the received video bitstream to obtain video pictures and display the video by using a display device.

[0291] The picture prediction method in the embodiment of this application may be performed by the source device 600 or the destination device 700. Specifically, the picture prediction method in the embodiment of this application may be performed by the video encoder 603 or the video decoder 702.

[0292] The video communication system 500 may also be referred to as a video coding system. The source device 600 may also be referred to as a video encoder or video encoding device. The destination device 700 may also be referred to as a video decoder or video decoding device.

[0293] In FIG. 12, source device 600 includes video capture device 601, video memory 602, video encoder 603, and transmitter 604. Video memory 602 may store video captured by video capture device 601. Video encoder 603 may encode video data from video memory 602 and video capture device 601. In some examples, source device 600 transmits the encoded video data directly to destination device 700 through transmitter 604. The encoded video data may be further stored in a storage medium or a file server such that destination device 700 later extracts the encoded video data for decoding and / or playback.

[0294] 12, destination device 700 includes a receiver 701, a video decoder 702, and a display device 703. In some examples, receiver 701 may receive encoded video data over a channel 800. Display device 703 may be integrated with destination device 700 or may be external to destination device 700. Typically, display device 700 displays the decoded video data. Display device 700 may include multiple types of display devices, such as a liquid crystal display, a plasma display, an organic light emitting diode display, or another type of display device.

[0295] A specific implementation of source device 600 and destination device 700 may be any one of the following devices: a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a smartphone, a handset, a television, a camera, a display device, a digital media player, a video game console, an in-vehicle computer, or another similar device.

[0296] The destination device 700 may receive the encoded video data from the source device 600 through the channel 800. The channel 800 may include one or more media and / or devices capable of moving the encoded video data from the source device 600 to the destination device 700. In one example, the channel 800 may include one or more communication media that may enable the source device 600 to transmit the encoded video data directly to the destination device 700 in real time. In this example, the source device 600 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the destination device 700. The one or more communication media may include wireless and / or wired communication media, for example, a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network (e.g., the Internet)). The one or more communication media may include a router, a switch, a base station, or another device that enables communication between the source device 600 and the destination device 700.

[0297] In another example, the channel 800 may include a storage medium that stores the encoded video data generated by the source device 600. In this example, the destination device 700 may access the storage medium through disk access or card access. The storage medium may include a plurality of locally accessible data storage media, such as Blu-ray, high density digital video disc (DVD), compact disc read-only memory (CD-ROM), flash memory, or another suitable digital storage medium configured to store the encoded video data.

[0298] In another example, the channel 800 may include a file server or another intermediate storage device that stores the encoded video data generated by the source device 600. In this example, the destination device 700 may access the encoded video data stored in the file server or another intermediate storage device through streaming transmission or download. The file server may be of a server type that stores the encoded video data and can transmit the encoded video data to the destination device 700. For example, the file server may include a world wide web (Web) server (e.g., used for websites), a file transfer protocol (FTP) server, a network attached storage (NAS) device, and a local disk drive.

[0299] The destination device 700 may access the encoded video data through a standard data connection (e.g., an Internet connection). Example types of data connections include wireless channels, wired connections (e.g., cable modems), or combinations thereof suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the file server may be a streaming transmission, a download transmission, or a combination thereof.

[0300] The following describes in detail the picture prediction method in the embodiments of this application with reference to specific accompanying drawings.

[0301] Figure 13 is a schematic flow chart of a picture prediction method according to an embodiment of this application. The picture prediction method shown in Figure 13 can be executed by a picture prediction device (the picture prediction device can be located in a picture encoding device (system) or a picture decoding device (system)). Specifically, the method shown in Figure 13 can be executed by a picture encoding device or a picture decoding device. The method shown in Figure 13 may be executed on the encoder side or on the decoder side. The method shown in Figure 13 includes steps 1001 to 1008. The following will describe these steps in detail separately.

[0302] 1001:Start.

[0303] Step 1001 indicates that picture prediction begins.

[0304] 1002: Determine whether merge mode is used for the current picture block.

[0305] Optionally, the method depicted in FIG. 13 further comprises a step of obtaining a current picture block before step 1002.

[0306] The current picture block may be a picture block within the current picture to be encoded or decoded.

[0307] It should be understood that in this application, the current picture block may be obtained in the process of determining the target merging mode of the current picture block, or after the target merging mode of the current picture block has been determined.

[0308] On the decoder side, in step 1002, whether the merge mode is used for the current picture block may be determined specifically based on the CU-level syntax element merge_flag[x0][y0].

[0309] If merge_flag[x0][y0]=1, it is determined that the merge mode is used to predict the current picture block. If merge_flag[x0][y0]=1, it is determined that the merge mode is not used to predict the current picture block. x0 and y0 represent the coordinate position of the luma pixel element in the upper left corner of the current picture block relative to the luma pixel element in the upper left corner of the current picture.

[0310] After it is determined based on the CU-level syntax element merge_flag[x0][y0] that a merge mode is to be used for the current picture block, the target merge mode to be finally used may be determined by analyzing specific information in the CU-level syntax element merge_flag[x0][y0].

[0311] When it is determined in step 1002 that the merge mode is not to be used for the current picture block, another mode other than the merge mode may be used to predict the current picture block. For example, when it is determined that the merge mode is not to be used for the current picture block, the AMVP mode may be used to predict the current picture block.

[0312] After it is determined in step 1002 that a merge mode is to be used for the current picture block, step 1003 is subsequently performed to determine a target merge mode applicable to the current picture block.

[0313] 1003: Determine whether to use level 1 merge mode.

[0314] Specifically, whether the level 1 merge mode is available may be determined based on higher layer syntax elements corresponding to the level 1 merge mode and / or available status information corresponding to the level 1 merge mode.

[0315] In a possible implementation of the present invention, it is assumed that the level 1 merge modes include a total of two merge modes, namely, merge mode A and merge mode B. In this case, it is determined one by one whether there is an available merge mode in the level 1 merge modes. If there is an available merge mode, step 1005 is executed. If no merge mode is available in the level 1 merge modes, it is determined that the level 1 merge modes are not available. In this case, the target merge mode needs to be determined from the level 2 merge modes. In other words, step 1004 is executed.

[0316] 1004: Determine whether an upper layer syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used.

[0317] The first merge mode belongs to a level 2 merge mode, which further includes a second merge mode.

[0318] When it is determined in step 1004 that the higher layer syntax element corresponding to the first merge mode indicates that the first merge mode is prohibited from being used, step 1006 is performed to determine the second merge mode as the target merge mode.

[0319] In this application, when the upper layer syntax element of the first merge mode indicates that the first merge mode is prohibited from being used, there is no need to analyze the available status information of the remaining second merge modes, and the second merge mode can be directly determined as the final target merge mode, which can reduce the redundancy caused by determining the target merge mode in the picture prediction process as much as possible.

[0320] When it is determined in step 1004 that the higher layer syntax element corresponding to the first merge mode indicates that the first merge mode is permitted to be used, step 1007 is performed to further determine a target merge mode.

[0321] 1005: Predict the current picture block based on level 1 merging mode.

[0322] It should be understood that in step 1005, if merge mode A among the level 1 merge modes is available, the current picture block is predicted based on merge mode A.

[0323] 1006: Determine a second merging mode as a target merging mode applicable to the current picture block.

[0324] When the higher layer syntax elements corresponding to the first merge mode indicate that the first merge mode is prohibited from being used, the second merge mode may be directly determined as the target merge mode without the need to parse the higher layer syntax elements corresponding to the second merge mode and / or available status information.

[0325] 1007: Determine a target merge mode based on an upper layer syntax element corresponding to the second merge mode and / or available status information of the second merge mode.

[0326] The availability status information of the second merge mode is used to indicate whether the second merge mode is used when the current picture block is predicted.

[0327] For example, the second merge mode is a CIIP mode, and the available status information of the second merge mode is the value of ciip_flag. When ciip_flag is 0, the CIIP mode is not used for the current picture block. When ciip_flag is 1, the CIIP mode is used to predict the current picture block.

[0328] For the CIIP mode, it should be understood that if the CIIP mode is to be selected as the target merge mode, the upper layer syntax element corresponding to CIIP needs to indicate that the CIIP mode is allowed to be used, and the available status information indicating the available status of the CIIP mode needs to indicate that CIIP is available.

[0329] For example, when sps_ciip_enabled_flag=1 and ciip_flag=1, the CIIP mode may be determined as the target merge mode of the current picture block.

[0330] In step 1007, both the first and second merge modes may be used as target merge modes for the current picture block because the first merge mode is allowed to be used. Thus, the target merge mode may be determined from the level 2 merge modes based on the higher layer syntax corresponding to one of the merge modes and the available status information.

[0331] Optionally, the first merge mode is a TPM mode and the second merge mode is a CIIP mode.

[0332] The following describes in detail how to determine the target merge mode when the first merge mode is the TPM mode and the second merge mode is the CIIP mode.

[0333] Optionally, the CIIP mode is determined as the target merge mode when the higher layer syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used.

[0334] For example, when the sps_triangle_enabled_flag corresponding to the TPM mode is 0, the TPM mode is prohibited from being used. In this case, there is no need to analyze a specific value of the ciip_flag. Instead, the CIIP mode can be directly determined as the target merge mode. In this way, unnecessary analysis processes can be reduced, and redundancy of the solution can be reduced.

[0335] In this application, when the upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, it is not necessary to determine whether the CIIP mode is available by analyzing the upper layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode. Instead, the CIIP mode can be directly determined as the target merge mode. This can reduce redundancy in the process of determining the target merge mode.

[0336] Optionally, when an upper layer syntax element corresponding to the TPM mode indicates that the TPM mode is permitted to be used, the target merge mode is determined based on an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating the available status of the CIIP mode.

[0337] For example, when the sps_triangle_enabled_flag corresponding to the TPM mode is 1, the TPM mode is allowed to be used. In this case, both the TPM mode and the CIIP mode can be used as the target merge mode. Therefore, it is necessary to further determine whether to select the TPM mode or the CIIP mode as the target merge mode.

[0338] Optionally, the TPM mode is determined as the target merge mode when the higher layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used.

[0339] Cases 1 to 3 include cases where the higher layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used.

[0340] Case 1: The higher layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is prohibited from being used, and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is not available.

[0341] Case 2: The higher layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used, and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is not available.

[0342] Case 3: The available status information indicating the available status of the CIIP mode indicates that the CIIP mode is not available.

[0343] Optionally, the CIIP mode is determined as the target merge mode when the higher layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is available.

[0344] 1008: Predict the current picture block based on the target merging mode.

[0345] In the method shown in FIG. 13, before step 1007 is executed, the method shown in FIG. 13 further includes a step of determining that the type of the slice or slice group in which the current picture block is located is B, and a step of determining that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is greater than or equal to 2.

[0346] Optionally, before the target merge mode is determined based on the upper layer syntax element corresponding to the CIIP mode and / or available status information indicating the available status of the CIIP mode, the method further includes a step of determining that the type of the slice or slice group in which the current picture block is located is B, and a step of determining that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is greater than or equal to 2.

[0347] Optionally, in one embodiment, the method depicted in FIG. 13 further includes determining the CIIP mode as the target merge mode when the level 1 merge mode is not available and the higher layer syntax element corresponding to the TPM mode indicates that the TPM mode is allowed to be used, but the current picture block does not satisfy at least one of condition A and condition B.

[0348] Conditions A and B are as follows.

[0349] Condition A: The slice in which the current picture block is located is of type B.

[0350] Condition B: The maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is 2 or greater.

[0351] Condition A and condition B may be expressed in some specific forms. For example, condition A may be specifically expressed by slice_type==B, and condition B may be specifically expressed by MaxNumTriangleMergeCand≧2. MaxNumTriangleMergeCand indicates the maximum quantity of candidate TPM modes supported by the slice or slice group in which the current picture block is located.

[0352] The TPM mode can be selected as the target merging mode that is ultimately used to predict the current picture block only when both condition A and condition B are satisfied.

[0353] If either condition A or condition B is not met, the CIIP mode is determined as the target merge mode.

[0354] When the higher layer syntax element corresponding to the TPM mode indicates that the TPM mode is prohibited from being used, if either condition A or condition B is not met, the CIIP mode is determined as the target merge mode.

[0355] When the higher layer syntax element corresponding to the TPM mode indicates that the TPM mode is permitted to be used, if either condition A or condition B is not met, then the CIIP mode is determined as the target merge mode.

[0356] In other words, the CIIP mode may be determined as the target merge mode, provided that sps_trangle_enabled_flag=1, condition A, and one of condition B are not satisfied.

[0357] Conversely, if sps_trangle_enabled_flag=1, condition A, and condition B are all satisfied, the target merge mode should be determined based on ciip_flag according to some conditions in the prior art.

[0358] Optionally, the higher layer syntax element is a syntax element at at least one of a sequence level, a picture level, a slice level, and a slice group level.

[0359] Optionally, level 1 merge modes include normal merge mode, MMVD mode, and sub-block merge mode.

[0360] When determining whether level 1 merge modes are available, whether these modes are available may be determined sequentially in the sequence of normal merge mode, MMVD mode, and sub-block merge mode.

[0361] For example, it may first be determined whether a normal merge mode is available. When the normal merge mode is not available, it is followed by determining whether an MMVD mode is available (if the normal merge mode is available, it may be directly used as the final target merge mode). When the MMVD mode is not available, it is followed by determining whether a sub-block merge mode is available.

[0362] Optionally, the method depicted in FIG. 13 may be applied on the encoder side to encode the current picture block.

[0363] Optionally, the method depicted in FIG. 13 may be applied on the decoder side to decode the current picture block.

[0364] In order to better understand the specific process of the picture prediction method in the embodiment of this application, the following will describe in detail the process of determining a picture merge mode in the picture prediction method in the embodiment of this application with reference to a specific example.

[0365] The following describes in detail the picture prediction method in the embodiment of this application with reference to FIG.

[0366] Figure 14 illustrates a process of determining a merge mode according to an embodiment of this application. The process illustrated in Figure 14 includes steps 3001 to 3007. The following describes these steps in detail.

[0367] 3001:Start.

[0368] Step 3001 indicates that picture prediction begins.

[0369] 3002: Determine whether merge mode is used for the current picture block.

[0370] Specifically, when step 3002 is executed by the decoder side, in step 3002, whether the merge mode is used for the current picture block may be determined based on the value of the CU-level syntax element merge_flag[x0][y0] corresponding to the current picture block.

[0371] For example, as shown in Table 4, when merge_flag[x0][y0]=0, merge mode is not used for the current picture block. In this case, the current picture block may be predicted in another manner. For example, the current picture block may be predicted in AMVP mode.

[0372] The merge mode is used for the current picture block when merge_flag[x0][y0]=1. Next, it may be further determined which merge mode is used to predict the current picture block.

[0373] It should be understood that when merge_flag[x0][y0] is not present in the bitstream, merge_flag[x0][y0] defaults to 0.

[0374] (x0,y0) represents the coordinate position of the luma pixel element in the upper left corner of the current picture block relative to the luma pixel element in the upper left corner of the current picture. The meaning of (x0,y0) in the following syntax elements is the same, and the details will not be described here.

[0375] Optionally, the method depicted in FIG. 14 further comprises a step of obtaining a current picture block before step 3002.

[0376] The current picture block may be a picture block within the current picture to be encoded or decoded.

[0377] It should be understood that in this application, the current picture block may be obtained in the process of determining the target merging mode of the current picture block, or after the target merging mode of the current picture block has been determined.

[0378] 3003: Determine whether the normal merge mode is used for the current picture block.

[0379] Specifically, in step 3003, whether the regular merge mode is used for the current picture block may be determined by analyzing the value of the syntax element regular_merge_flag[x0][y0].

[0380] When regular_merge_flag[x0][y0]=1, it is determined that the regular merge mode is used for the current picture block. In this case, step 3007 may be performed. Specifically, the current picture block is predicted based on the regular merge mode.

[0381] When regular_merge_flag[x0][y0]=0, it is determined that the regular merge mode is not used for the current picture block. In this case, it is necessary to continue to execute step 3004 to further determine the merge mode to be used for the current picture block.

[0382] It should be understood that when regular_merge_flag[x0][y0] is not present in the bitstream, regular_merge_flag[x0][y0] defaults to 0.

[0383] 3004: Determine whether the MMVD mode is used for the current picture block.

[0384] Specifically, in step 3004, when the upper layer syntax element corresponding to the MMVD mode indicates that the MMVD may be allowed to be used, and the area of ​​the current picture block is not equal to 32, whether the MMVD is used for the current picture block may be determined by analyzing the value of the syntax element mmvd_flag[x0][y0].

[0385] When mmvd_flag[x0][y0]=1, it is determined that the MMVD mode is used for the current picture block. In this case, step 3007 may be executed. Specifically, the current picture block is predicted based on the MMVD mode.

[0386] When mmvd_flag[x0][y0]=0, it is determined that the MMVD mode is not used for the current picture block. In this case, it is necessary to continue to perform step 3005 to further determine the merge mode to be used for the current picture block.

[0387] It should be understood that when mmvd_flag[x0][y0] is not present in the bitstream, mmvd_flag[x0][y0] defaults to 0.

[0388] 3005: Determine whether the sub-block merging mode is used for the current picture block.

[0389] In step 3004, whether the sub-block merging mode is used for the current picture block may be determined based on the value of the syntax element merge_subblock_flag[x0][y0] obtained by parsing the bitstream.

[0390] When merge_subblock_flag[x0][y0]=1, it is determined that the subblock merging mode is used for the current picture block. In this case, step 3007 may be performed. Specifically, the current picture block is predicted based on the subblock merging mode.

[0391] When merge_subblock_flag[x0][y0]=0, it is determined that the sub-block merging mode is not used for the current picture block. In this case, it is necessary to continue to perform step 3006 to further determine the merging mode to be used for the current picture block.

[0392] It should be understood that when merge_subblock_flag[x0][y0] is not present in the bitstream, merge_subblock_flag[x0][y0] defaults to 0.

[0393] Further, in step 3004, the value of the syntax element merge_subblock_flag[x0][y0] may be parsed only when the maximum length of the subblock merge candidate list is greater than 0 and both the width and height of the current picture block are greater than or equal to 8, followed by execution of step 3007 when the obtained value of merge_subblock_flag[x0][y0] is 0.

[0394] 3006: Determine the merge mode to be used for the current picture block from the CIIP mode and the TPM mode.

[0395] Specifically, in step 3006, if all of the following six conditions (1) to (6) are satisfied, ciip_flag[x0][y0] is parsed from the bitstream, and the merge mode used for the current picture block is determined based on the value of ciip_flag[x0][y0]: When ciip_flag[x0][y0]=1, the CIIP mode is used to predict the current picture block.

[0396] In addition, when the following condition (1) is satisfied, if any one of the following conditions (2) to (6) is not satisfied, the CIIP mode is used to predict the current picture block. (1)sps_ciip_enabled_flag=1, (2) sps_triangle_enabled_flag=1, (3)cu_skip_flag[x0][y0]==0, (4) (cbWidth*cbHeight)≧64, (5) cbWidth<128, and (6) cbHeight<128.

[0397] cbWidth and cbHeight are the width and height of the current picture block, respectively.

[0398] Optionally, when the merge mode to be used for the current block is determined in step 3006, more decision conditions may be further added.

[0399] Based on the above conditions (1) to (6), conditions (7) and (8) may be further added. (7) slice_type==B, and (8) MaxNumTriangleMergeCand≧2.

[0400] Optionally, in step 3006, if all the eight conditions in the above conditions (1) to (8) are satisfied, ciip_flag[x0][y0] is parsed from the bitstream, and the merge mode used for the current picture block is determined based on the value of ciip_flag[x0][y0]. When ciip_flag[x0][y0]=1, the CIIP mode is used to predict the current picture block.

[0401] In addition, when the above condition (1) is satisfied, if any one of the above conditions (2) to (8) is not satisfied, the CIIP mode is used to predict the current picture block.

[0402] 3007: Predict the current picture block based on the merging mode used for the current picture block.

[0403] When it is determined in step 3003 that the normal merging mode is used for the current picture block, the current picture block is predicted based on the normal merging mode in step 3007. When it is determined in step 3004 that the MMVD mode is used for the current picture block, the current picture block is predicted based on the MMVD mode in step 3007. When it is determined in step 3005 that the sub-block merging mode is used for the current picture block, the current picture block is predicted based on the sub-block merging mode in step 3007.

[0404] Table 4 shows how to determine the merge mode to be used for the current picture block when the merge mode is used based on the corresponding syntax element. The following describes in detail the determination of the merge mode of the current picture block with reference to Table 4.

[0405] [Table 4]

[0406] When regular_merge_flag[x0][y0] shown in Table 4 is 1, it is determined that the regular merge mode is used for the current picture block. In this case, the parameters of the regular merge mode can be obtained by parsing the syntax element merge_idx[x0][y0]. When regular_merge_flag[x0][y0] shown in Table 4 is 0, it is determined that the regular merge mode is not used for the current picture block, and the merge mode to be used for the current picture block needs to be further determined.

[0407] When sps_mmvd_enabled_flag and cbWidth*cbHeight! shown in Table 4 are 1 and 32, respectively, it indicates that the MMVD mode may be used for the current picture block. In this case, the merge mode of the current picture block may be determined based on the value of mmvd_flag[x0][y0]. If mmvd_flag[x0][y0]=1, it is determined that the MMVD mode is used for the current picture block, and the parameters of the MMVD mode may be determined by analyzing the syntax elements mmvd_merge_flag[x0][y0], mmvd_distance_idx[x0][y0], and mmvd_direction_idx[x0][y0]. If mmvd_flag[x0][y0]=0, the merge mode used for the current picture block needs to be further determined.

[0408] When merge_subblock_flag[x0][y0] shown in Table 4 is 1, it is determined that the subblock merge mode is used for the current picture block. When merge_subblock_flag[x0][y0] shown in Table 4 is 0, it is determined that the subblock merge mode is not used for the current picture block, and the merge mode to be used for the current picture block needs to be further determined.

[0409] When the sps_ciip_enabled_flag represented in Table 4 is 0, it may be directly determined that the TPM mode is used for the current picture block. However, when the sps_ciip_enabled_flag and sps_ciip_enabled_flag represented in Table 4 are 1 and 0, respectively, it may be directly determined that the CIIP mode is used for the current picture block.

[0410] As shown in Table 4, when all six conditions in the following conditions (1) to (6) are satisfied, the indicator of the available status information of the Ciip mode, i.e., the value of ciip_flag[x0][y0], should be obtained from the bitstream. If ciip_flag[x0][y0]=1, it is determined that the CIIP mode is used for the current picture block. If ciip_flag[x0][y0]=0, it is determined that the TPM mode is used for the current picture block. (1)sps_ciip_enabled_flag=1, (2) sps_triangle_enabled_flag=1, (3)cu_skip_flag[x0][y0]==0, (4) (cbWidth*cbHeight)≧64, (5) cbWidth<128, (6) cbHeight<128.

[0411] Optionally, if(sps_ciip_enabled_flag && sps_triangle_enabled_flag && cu_skip_flag[x0][y0]==0 && (cbWidth*cbHeight)≧64 && cbWidth<128 && cbHeight<128) in Table 4 may alternatively be replaced with if(sps_triangle_enabled_flag && sps_ciip_enabled_flag && cu_skip_flag[x0][y0]==0 && (cbWidth*cbHeight)≧64 && cbWidth<128 && cbHeight<128). In other words, the sequence of sps_ciip_enabled_flag and sps_triangle_enabled_flag may be adjusted. Specific results may be shown in Table 5.

[0412] [Table 5]

[0413] It should be noted that in Tables 4 and 5, the time sequence for determining the CIIP is earlier than the time sequence for determining the TPM. Specifically, the CIIP is determined first, and the prediction mode finally used for the current block is determined based on the status of the CIIP. If the CIIP is true, there is no need to further determine information about the TPM. If the CIIP is false, it means that only the TPM is available. In this case, the final prediction mode of the current block may be set to the TPM mode. The priority settings or logic for determining the time sequence are merely examples, and may alternatively be adjusted as needed. For example, the time sequence of the TPM is made earlier than the time sequence of the CIIP. In this case, the condition for determining whether the TPM mode is applicable also needs to be adjusted as needed.

[0414] [Table 6]

[0415] As shown in Table 6, when all the following eight conditions (1) to (8) are satisfied, the indicator of available status information of the CIIP mode, i.e., the value of ciip_flag[x0][y0], should be obtained from the bitstream. If ciip_flag[x0][y0]=1, it is determined that the CIIP mode is used for the current picture block. If ciip_flag[x0][y0]=0, it is determined that the TPM mode is used for the current picture block. (1)sps_ciip_enabled_flag=1, (2) sps_triangle_enabled_flag=1, (3)cu_skip_flag[x0][y0]==0, (4) (cbWidth*cbHeight)≧64, (5) cbWidth<128, (6) cbHeight<128.

[0416] Optionally, if(sps_ciip_enabled_flag && sps_triangle_enabled_flag && slice_type==B && MaxNumTriangleMergeCand≧2 && cu_skip_flag[x0][y0]==0 && (cbWidth*cbHeight)≧64 && cbWidth<128 && cbHeight<128) in Table 6 may alternatively be replaced with if(sps_triangle_enabled_flag && sps_ciip_enabled_flag && slice_type==B && MaxNumTriangleMergeCand≧2 && cu_skip_flag[x0][y0]==0 && (cbWidth*cbHeight)≧64 && cbWidth<128 && cbHeight<128). In other words, the sequences of sps_ciip_enabled_flag and sps_triangle_enabled_flag can be adjusted. Specific results can be shown in Table 7.

[0417] [Table 7]

[0418] The above describes in detail the picture prediction method in the embodiment of this application with reference to Figure 13 and Figure 14. The following describes the picture prediction method in the embodiment of this application with reference to Figure 15.

[0419] Figure 15 is a schematic flow chart of a picture prediction method according to an embodiment of this application. The picture prediction method shown in Figure 15 can be performed by a picture prediction device (the picture prediction device can be located in a picture decoding device (system) or a picture encoding device (system)). Specifically, the method shown in Figure 15 can be performed by a picture encoding device or a picture decoding device. The method shown in Figure 15 can be performed on the encoder side or on the decoder side. The method shown in Figure 15 includes steps 4001 to 4007. The following will describe these steps in detail separately.

[0420] 4001:Start.

[0421] Step 4001 indicates that picture prediction begins.

[0422] 4002: Determine whether merge mode is used for the current picture block.

[0423] On the decoder side, in step 4002, whether the merge mode is used for the current picture block may be determined based on the CU-level syntax element merge_flag[x0][y0]. For the specific decision process, please refer to the related description under step 1003.

[0424] When it is determined in step 4002 that the merge mode is not used for the current picture block, another mode other than the merge mode may be used to predict the current picture block. For example, when it is determined that the merge mode is not used for the current picture block, the AMVP mode may be used to predict the current picture block.

[0425] Following that, in step 4002, after it is determined that a merge mode is to be used for the current picture block, step 4003 is performed to determine a target merge mode applicable to the current picture block.

[0426] Optionally, the method depicted in FIG. 15 further comprises, before step 4002, a step of obtaining a current picture block.

[0427] The current picture block may be a picture block within the current picture to be encoded or decoded.

[0428] It should be understood that in this application, the current picture block may be obtained in the process of determining the target merging mode of the current picture block, or after the target merging mode of the current picture block has been determined.

[0429] 4003: Determine whether to use level 1 merge mode.

[0430] Specifically, whether the level 1 merge mode is available may be determined based on higher layer syntax elements corresponding to the level 1 merge mode and / or available status information corresponding to the level 1 merge mode.

[0431] Optionally, the level 1 merge modes in step 4003 include a normal merge mode, an MMVD mode, and a sub-block merge mode.

[0432] When it is determined whether the level 1 merge modes are available, the availability of these modes may be determined sequentially in the sequence of the normal merge mode, the MMVD mode, and the sub-block merge mode. When all the modes are not available, it is determined that the level 1 merge mode is not available.

[0433] When it is determined in step 4003 that the level 1 merge mode is not available, step 4004 may be performed subsequently to determine a target merge mode from the level 2 merge modes.

[0434] For the picture prediction method shown in FIG. 15, the level 1 merge mode and the level 2 merge mode may include all optional merge modes of the current picture block, and a final target merge mode for the current picture block needs to be determined from the level 1 merge mode and the level 2 merge mode.

[0435] Optionally, the priority of a level 1 merge mode is higher than the priority of a level 2 merge mode.

[0436] The priority of the merge modes in level 1 is higher than the priority of the merge modes in level 2, which means that in the process of determining the target merge mode for the current picture block, the target merge mode is preferentially determined from the merge modes in level 1. If there is no available merge mode in the merge modes in level 1, the target merge mode is then determined from the merge modes in level 2.

[0437] 4004: Determine whether conditions 1 to 5 are satisfied.

[0438] Conditions 1 to 5 are as follows:

[0439] Condition 1: TPM mode is allowed to be used.

[0440] Condition 2: The slice or slice group in which the current picture block is located is of type B.

[0441] Condition 3: It is determined that the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located is two or greater.

[0442] Condition 4: The size of the current picture block meets a preset condition.

[0443] Condition 5: Skip mode is not used to predict the current picture block.

[0444] Condition 1 may be specifically expressed by sps_triangle_enabled_flag=1, condition 2 may be specifically expressed by slice_type==B, and condition 3 may be specifically expressed by MaxNumTriangleMergeCand≧2. MaxNumTriangleMergeCand indicates the maximum number of candidate TPM modes supported by the slice or slice group in which the current picture block is located.

[0445] In step 4004, when it is determined that any one of the conditions 1 to 5 is not satisfied, the CIIP mode can be directly determined as the target merge mode. In other words, step 4005 is executed. In step 4004, when it is determined that the five conditions, i.e., the conditions 1 to 5, are satisfied, the target merge mode needs to be further determined based on the related information of the CIIP mode. In other words, step 4006 is executed.

[0446] 4005: When the CIIP mode is permitted to be used, the CIIP mode is determined as the target merge mode.

[0447] In other words, in step 4005, when the CIIP mode is permitted to be used and any one of conditions 1 to 5 is not satisfied, the CIIP mode is determined as the target merge mode.

[0448] Optionally, when any one of conditions 1 to 5 is not satisfied, a value of the available status information indicating the available status of the CIIP mode is set to a first value. When the value of the available status information indicating the available status of the CIIP mode is the first value, the CIIP mode is used to perform picture prediction on the current picture block.

[0449] It should be understood that setting the value of the available status information indicating the available status of the CIIP mode to a first value is equivalent to determining CIIP as the target merge mode.

[0450] Optionally, available status information indicating the available status of the CIIP mode is ciip_flag.

[0451] Setting the value of the available status information indicating the available status of the CIIP mode to a first value may specifically be setting ciip_flag to 1.

[0452] In addition, when the value of the available status information indicating the available status of the CIIP mode is set to a second value, it may mean that the CIIP mode is not used to perform picture prediction on the current picture block. For example, when the available status information indicating the available status of the CIIP mode is ciip_flag, and ciip_flag=0, the CIIP mode is not used to perform picture prediction on the current picture block.

[0453] 4006: Determine a target merge mode based on an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating an available status of the CIIP mode.

[0454] The available status information indicating the available status of the CIIP mode is used to indicate whether the CIIP mode is used when the current picture block is predicted.

[0455] In other words, in step 4006, when all conditions from condition 1 to condition 5 are satisfied, the target merge mode needs to be further determined based on the upper layer syntax elements corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode.

[0456] The available status information indicating the available status of the CIIP mode may be the value of ciip_flag. When ciip_flag is 0, the CIIP mode is not available for the current picture block. When ciip_flag is 1, the CIIP mode is available for the current picture block.

[0457] Optionally, determining the target merge mode based on an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating an available status of the CIIP mode includes determining the TPM mode as the target merge mode when the upper layer syntax element corresponding to the CIIP mode and / or the available status information indicating an available status of the CIIP mode indicates that the CIIP mode is prohibited from being used.

[0458] Cases 1 and 3 include cases where the higher layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used.

[0459] Case 1: The higher layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is prohibited from being used, and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is not available.

[0460] Case 2: The higher layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used, and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is not available.

[0461] Case 3: The available status information indicating the available status of the CIIP mode indicates that the CIIP mode is not available.

[0462] It should be understood that the CIIP mode is determined as the target merge mode when the upper layer syntax element corresponding to the CIIP mode indicates that the CIIP mode is allowed to be used and the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is available.

[0463] Optionally, when the upper layer syntax element corresponding to the CIIP mode and / or the available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, determining the TPM mode as the target merge mode includes:

[0464] When an upper layer syntax element corresponding to the CIIP mode and / or available status information indicating the available status of the CIIP mode indicates that the CIIP mode is prohibited from being used, a value of the available status information indicating the available status of the TPM mode is set to a first value, and when the value of the available status information indicating the available status of the TPM mode is the first value, the TPM mode is used to perform picture prediction on the current picture block.

[0465] It should be understood that setting the value of the available status information indicating the available status of the TPM mode to a first value is equivalent to the TPM being determined as the target merge mode.

[0466] Optionally, the available status information indicating the available status of the TPM mode is MergeTriangleFlag.

[0467] Setting the value of the available status information indicating the available status of the TPM mode to the first value may specifically be setting MergeTriangleFlag to 1.

[0468] In this application, the target merge mode can be determined based on the upper layer syntax elements of the CIIP mode and / or the available status information indicating the available status of the CIIP mode only when five preset conditions are met. Compared with the conventional solution, more conditions need to be met before the target merge mode is further determined based on the upper layer syntax elements of the CIIP mode and the available status information. Otherwise, the CIIP mode can be directly determined as the target merge mode. This can reduce some redundant processes in the process of determining the target merge mode.

[0469] From another perspective, when the level 1 merge mode is not available, it may be determined whether to select the CIIP mode as the final merge mode based on some preset conditions, and the CIIP mode may be directly determined as the target merge mode on the condition that any one of the preset conditions is not met, which reduces redundancy generated in the process of determining the target merge mode.

[0470] 4007: Predict the current picture block based on the target merging mode.

[0471] Optionally, before the target merge mode is determined based on higher layer syntax elements corresponding to the CIIP mode and / or available status information indicating an available status of the CIIP mode, the method depicted in FIG. 15 further comprises: The method further includes determining that at least one of the following conditions is met:

[0472] The size of the current picture block meets a preset condition, and The skip mode is not used to predict the current picture block.

[0473] Optionally, the size of the current picture block satisfies the preset condition if the current picture block satisfies the following three conditions: (cdWidth*cbHeight)≧64, cbWidth<128, and cbHeight<128 This includes satisfying the following:

[0474] cdWidth is the width of the current picture block, and cbHeight is the height of the current picture block.

[0475] The above describes in detail the picture prediction method in an embodiment of this application with reference to the accompanying drawings. The following describes the picture prediction device in an embodiment of this application with reference to Fig. 16. It should be understood that the picture prediction device shown in Fig. 16 can perform steps in the picture prediction method in an embodiment of this application. In order to avoid unnecessary repetition, the following appropriately omits repeated descriptions when describing the picture prediction device in this embodiment of this application.

[0476] FIG. 16 is a schematic block diagram of a picture prediction device according to an embodiment of this application.

[0477] The picture prediction device 5000 shown in FIG. 16 includes a decision unit 5001 and a prediction unit 5002 .

[0478] The picture prediction device 5000 shown in Figure 16 is configured to perform the picture prediction method in the embodiment of this application. Specifically, the decision unit 5001 in the picture prediction device 5000 may be configured to perform a process of determining a target merging mode in the picture prediction method shown in Figures 13 to 15. The prediction unit 5002 in the picture prediction device 5000 is configured to perform a process of performing picture prediction on a current picture block based on the target merging mode in the picture prediction method shown in Figures 13 to 15.

[0479] Figure 17 is a schematic diagram of the hardware structure of a picture prediction device according to an embodiment of this application. The picture prediction device 6000 (the device 6000 may specifically be a computer device) shown in Figure 17 includes a memory 6001, a processor 6002, a communication interface 6003, and a bus 6004. The communication connection between the memory 6001, the processor 6002, and the communication interface 6003 is realized through the bus 6004.

[0480] The memory 6001 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 6001 may store a program. When the program stored in the memory 6001 is executed by the processor 6002, the processor 6002 is configured to execute the steps of the picture prediction method in the embodiment of this application.

[0481] The processor 6002 may be a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, and is configured to execute associated programs to implement the picture detection method in the method embodiments of this application.

[0482] The processor 6002 may be an integrated circuit chip and has signal processing capabilities. In the implementation process, the steps of the picture prediction method in this application can be completed by using a hardware integrated logic circuit or an instruction in the form of software in the processor 6002.

[0483] The processor 6002 may alternatively be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor 602 may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or similar. The steps of the methods disclosed with reference to the embodiments of this application may be directly executed and completed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a mature storage medium in this technical field, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory 6001. The processor 6002 reads the information in the memory 6001 and, in combination with the hardware of the processor 6002, completes the functions that need to be performed by the units included in the picture prediction device or performs the picture prediction method in the method embodiment of this application.

[0484] The communication interface 6003 uses a transceiver device, such as, but not limited to, a transceiver, to facilitate communication between the apparatus 6000 and another device or communication network. For example, information regarding the neural network to be constructed and training data required in the neural network construction process may be obtained through the communication interface 6003.

[0485] The bus 6004 may include a path for transmitting information between components of the device 6000 (eg, the memory 6001, the processor 6002, and the communication interface 6003).

[0486] The decision unit 5001 and the prediction unit 5002 in the picture prediction device 5000 are equivalent to the processor 6002 in the picture prediction device 6000 .

[0487] Figure 18 is a schematic diagram of a hardware structure of a picture encoding / decoding device according to an embodiment of this application. The picture encoding / decoding device 7000 shown in Figure 18 (the device 7000 may specifically be a computer device) includes a memory 7001, a processor 7002, a communication interface 7003, and a bus 7004. The communication connection between the memory 7001, the processor 7002, and the communication interface 7003 is realized through the bus 7004.

[0488] The above limitations and descriptions of the modules in the picture prediction device 6000 are also applicable to the picture encoding / decoding device 7000, and the details will not be described again here.

[0489] The memory 7001 can be configured to store a program. The processor 7002 is configured to execute the program stored in the memory 7001. When the program stored in the memory 7001 is executed, the processor 7002 is configured to execute the steps of the picture prediction method in the embodiment of this application.

[0490] In addition, when encoding a video picture, the picture encoding / decoding apparatus 7000 may obtain a video picture through the communication interface and then encode the obtained video picture to obtain encoded video data. The encoded video data may be transmitted to a video decoding device through the communication interface 7003.

[0491] When decoding a video picture, the picture encoding / decoding device 7000 may obtain a video picture through a communication interface, and then decode the obtained video picture to obtain a video picture to be displayed.

[0492] Those skilled in the art may recognize that the units and algorithm steps may be realized by electronic hardware, or a combination of computer software and electronic hardware, in combination with the examples described in the embodiments disclosed in this specification. Whether a function is performed by hardware or software depends on a specific application and the design constraints of the technical solution. Those skilled in the art may use different methods to realize the described functions for each specific application, but the implementation should not be considered to go beyond the scope of this application.

[0493] For convenient and simple description, for detailed operation processes of the aforementioned systems, devices, and units, please refer to the corresponding processes in the aforementioned method embodiments, and the details will not be described again here, which can be clearly understood by those skilled in the art.

[0494] In some embodiments provided in this application, it should be understood that the disclosed system, device, and method can be realized in other ways. For example, the described device embodiment is merely an example. For example, the unit division is merely a logical functional division, and there may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the mutual couplings or direct couplings or communication connections shown or discussed may be realized through some interfaces. Indirect couplings or communication connections between devices or units may be realized in electronic, mechanical, or other forms.

[0495] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units, and may be located in one location or distributed in multiple network units. Some or all of the units may be selected based on the actual requirements to achieve the objectives of the solutions of the embodiments.

[0496] In addition, the functional units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist physically alone, or two or more units may be integrated into one unit.

[0497] When the function is realized in the form of a software functional unit and sold or used as an independent product, the function may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of this application essentially, or the part contributing to the prior art, or some of the technical solutions may be realized in the form of a software product. The software product is stored in a storage medium and includes some instructions for instructing a computer device (which may be a personal computer, a server, or a network device) to execute all or some of the steps of the method described in the embodiments of this application. The aforementioned storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0498] The above description is merely a specific implementation of this application, but is not intended to limit the protection scope of this application. Any variation or replacement that is easily understood by a person skilled in the art within the technical scope disclosed in this application should fall within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims. [Explanation of symbols]

[0499] 10. Video Encoding and Decoding Systems 12 Source device, source equipment 13 Links 14 Destination device, destination equipment 16 Picture Source 17 Raw Picture Data 18 Picture Preprocessor 19 Preprocessed Picture, Preprocessed Picture Data 20 Encoder 21 Encoded Picture Data 22 Communication Interface 28 Communication Interface 30 Decoder 31 Decoded Picture, Decoded Picture Data 32 Picture Post Processor 33 Post-Processed Picture Data 34 Display Devices 40 Video Coding System 41 Imaging Devices 42 Antenna 43 Processors 44 Memory 45 Display Devices 46 Processing Unit 47 Logic Circuits 201 Pictures 202 Input 203 Picture Block, Picture Image Block 204 Residual Calculation Unit 205 Residual Blocks 206 Conversion Processing Unit 207 Conversion Factors 208 Quantization Units 209 Quantized transform coefficients, quantized residual coefficients 210 Inverse Quantization Unit 211 Dequantized Coefficients, Dequantized Residual Coefficients 212 Inverse Transformation Processing Unit 213 Inverse Transform Block, Inverse Transform Dequantized Block, Inverse Transform Residual Block, Reconstructed Residual Block 214 Reconstruction Unit 215 reconstructed blocks 216 Buffer, Buffer Unit, Line Buffer 220 Loop filter unit, loop filter, filter 221 Filtered Block, Filtered Reconstructed Block 230 Decoded Picture Buffer, DBP 231 Decoded Pictures 244 Inter Prediction Units 245 Inter prediction block, prediction block 254 intra prediction units 255 intra prediction blocks, prediction blocks 260 Prediction processing unit, block prediction processing unit 262 Mode Selection Unit 265 predicted blocks 270 Entropy Coding Unit 272 Output 304 Entropy Decoding Unit 309 Quantized Coefficients 310 Inverse Quantization Unit 312 Inverse Transformation Processing Unit 313 Reconstructed residual block, inverse transform block 314 Reconstruction Unit, Adder 315 Reconstructed Blocks 316 Buffer 320 Loop filter, loop filter unit, filter 321 Filtered Blocks, Decoded Video Blocks 330 Decoded Picture Buffer, DPB 332 Output 344 Inter Prediction Units 354 intra prediction units 360 Prediction Processing Unit 362 Mode Selection Unit 365 predicted blocks 400 Video coding device, video encoding device, video decoding device 410 Input Port 420 Receiving unit (Rx), receiver unit 430 Central Processing Unit (CPU), Processor 440 Transmitter unit (Tx), Transmitter unit 450 output ports 460 Memory 470 Coding module, Encoding module, Decoding module, Encoding / Decoding module 500 Apparatus, coding device, video communication system 510 Processor 530 Memory 533 Operating Systems 535 Application Program 550 Bus System, Bus 570 Display 600 Source Device 601 Video capture device 602 Video memory, processor 603 Video Encoder 604 Transmitter 700 destination device 701 Receiver 702 Video Decoder 703 Display device 800 Channels 5000 Picture Predictor 5001 Decision Unit 5002 prediction units 6000 Picture prediction device, device 6001 Memory 6002 Processor 6003 Communication Interface 6004 Bus 7000 Destination device, picture encoding / decoding device, device 7001 Memory 7002 Processor 7003 Communication Interface 7004 Bus

Claims

1. determining whether a merge mode is used for the current picture block; determining whether a level 1 merge mode is available when the merge mode is to be used for the current picture block, the level 1 merge mode comprising a normal merge mode; determining a second merge mode in the level 2 merge modes as a target merge mode applicable to the current picture block when the level 1 merge mode is unavailable and an upper layer syntax element corresponding to a first merge mode in the level 2 merge modes indicates that the first merge mode is prohibited from being used, wherein the level 2 merge modes include the first merge mode and the second merge mode, the first merge mode being a triangular partition mode (TPM) and the second merge mode being a combined intra and inter prediction (CIIP) mode; predicting the current picture block based on the target merging mode; 1. A picture prediction method comprising:

2. The method further comprising:

2. The method of claim 1 , further comprising: when the level 1 merge mode is not available and the higher layer syntax element corresponding to the first merge mode in the level 2 merge modes indicates that the first merge mode is allowed to be used, determining the target merge mode based on a higher layer syntax element corresponding to the second merge mode in the level 2 merge modes and / or availability status information of the second merge mode, wherein the availability status information of the second merge mode is used to indicate whether the second merge mode is used when the current picture block is predicted.

3. determining the target merge mode based on higher layer syntax elements corresponding to the second merge mode and / or available status information of the second merge mode, 3. The method of claim 2, comprising determining the first merge mode as the target merge mode when the higher layer syntax element corresponding to the second merge mode and / or the available status information of the second merge mode indicates that the second merge mode is prohibited from being used.

4. Prior to the step of determining the target merge mode based on higher layer syntax elements corresponding to the second merge mode and / or available status information of the second merge mode, the method further comprises: The current picture block satisfies the following conditions: the size of the current picture block satisfies a preset condition; A skip mode is not used to predict the current picture block. The method of claim 2 or 3, further comprising determining that at least one of the following is satisfied:

5. The size of the current picture block satisfies a preset condition. The current picture block satisfies the following three conditions: (cdWidth*cbHeight)≧64, cbWidth<128, and cbHeight<128 Including satisfying the following: The method of claim 4 , wherein cdWidth is the width of the current picture block and cbHeight is the height of the current picture block.

6. The method according to claim 1 , wherein the higher layer syntax elements are syntax elements at at least one of a sequence level, a picture level, a slice level, and a slice group level.

7. The method of claim 1 , wherein the level 1 merging modes further include a merge using motion vector differential (MMVD) mode and a sub-block merging mode.

8. The method according to claim 1 , wherein the prediction method is applied on an encoder side for encoding the current picture block.

9. The method according to claim 1 , wherein the prediction method is applied on a decoder side for decoding the current picture block.

10. A picture prediction device, said picture prediction device comprising modules adapted to perform the method according to any one of claims 1 to 9.

11. a memory configured to store a program; A processor configured to execute the program stored in the memory, the processor executing the method of any one of claims 1 to 9 when the program stored in the memory is executed by the processor; A picture prediction device comprising:

12. An encoding device, the encoding device comprising a picture prediction device according to claim 10 or 11.

13. A decoding device, the decoding device comprising the picture prediction device according to claim 10 or 11.

14. An electronic device, said electronic device comprising an encoding device according to claim 12 and / or a decoding device according to claim 13.

15. 10. A computer-readable storage medium having stored thereon a computer program executable by a processor, the computer-readable storage medium causing the processor to perform the method of any one of claims 1 to 9 when the computer program is executed by the processor.

16. 10. A program comprising a program code for performing the method according to any one of claims 1 to 9, when the program is executed on a computer or processor.

17. A coder comprising processing circuitry for carrying out the method according to any one of claims 1 to 9.

Images