VIDEO ENCODER, VIDEO DECODER, AND CORRESPONDING METHODS - Patent application

Abstract

To provide a video encoder, a video decoder, and a corresponding method balancing complexity of encoding / decoding and prediction accuracy to some extent.SOLUTION: A method acquires a prediction sample value of a current picture block by performing bi-directional optical flow BDOF processing of the current picture block when a plurality of preset conditions are satisfied. The plurality of preset conditions at least include a condition that a size of the current picture block satisfies a first preset size. The size of the current picture block satisfying the first preset size makes height H of the current picture block 8 or larger, makes width W unequal to the height H when a product of the width w and height H of the current picture block is equal to 64, reconfigures the current picture block on the basis of the prediction sample value, and encodes a syntax element. The syntax element is at least used to indicate that bidirectional prediction is applied.SELECTED DRAWING: Figure 6A

Description

[Technical Field]

[0001] This application relates to the field of video coding techniques, and in particular to video encoders, video decoders, and corresponding methods. [Background technology]

[0002] Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radiotelephones (so-called "smartphones"), video conferencing devices, video streaming devices, and the like. Digital video devices implement video compression techniques, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, and ITU-T H.264 / MPEG-4 Part 10: Advanced Video Coding (AVC), video coding standard H.265 / High Efficiency Video Coding (HEVC) standards, and extensions to such standards. Implementing such video compression techniques allows video devices to transmit, receive, encode, decode, and / or store digital video information more efficiently.

[0003] Video compression techniques are used to perform spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. In block-based video coding, a video slice (i.e., a video frame or a portion of a video frame) is divided into multiple picture blocks, which may also be referred to as tree blocks, coding units (CUs), and / or coding nodes. Picture blocks in an intra-coded (I) slice of a picture are coded through spatial prediction based on reference samples in neighboring blocks in the same picture. Picture blocks in an inter-coded (P or B) slice of a picture may use spatial prediction based on reference samples in neighboring blocks in the same picture or temporal prediction based on reference samples in another reference picture. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0004] Various video coding standards, including the High Efficiency Video Coding (HEVC) standard, propose predictive coding modes to be used for picture blocks. Specifically, a currently coded block is predicted based on coded video data blocks. In intra-prediction modes, the current block is predicted based on one or more previously decoded neighboring blocks in the same picture as the current block. In inter-prediction modes, the current block is predicted based on decoded blocks in different pictures.

[0005] Prediction includes forward prediction, backward prediction, bidirectional prediction, and the like. In bidirectional prediction, a forward prediction block (e.g., corresponding to the predicted sample value in the first list) and a backward prediction block (e.g., corresponding to the predicted sample value in the second list) need to be obtained based on the motion information in the forward prediction motion compensation process and the backward prediction motion compensation process, respectively, and then a predicted block of a current block is determined based on the forward prediction block and the backward prediction block. How to reduce coding complexity while improving the prediction accuracy of the predicted block of a current block is a technical problem that has been studied by those skilled in the art. Summary of the Invention

[0006] SUMMARY OF THE INVENTION Embodiments of the present invention disclose a video encoder, a video decoder, and corresponding methods for achieving some balance between encoding / decoding complexity and prediction accuracy.

[0007] According to a first aspect, an embodiment of the present application provides a video encoding method, the method comprising: When multiple preset conditions are satisfied, a bidirectional optical flow (BDOF) process is performed on the current picture block to obtain a predicted sample value of the current picture block, and the multiple preset conditions include at least a condition that the size of the current picture block satisfies a first preset size. The size of the current picture block is usually represented by its width and height (abbreviated as W×H), where the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0008] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0009] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0010] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0011] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0012] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0013] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0014] The method further includes reconstructing the current picture block based on at least the predicted sample values ​​of the current picture block; encoding a syntax element into a transmitted bitstream, the syntax element being used to indicate at least that bidirectional prediction is applied; This includes:

[0015] It should be understood that in the process of performing BDOF on a current picture block, prediction is performed by sub-block. Specifically, performing bidirectional optical flow BDOF processing on a current picture block may specifically be performing bidirectional optical flow BDOF processing on a current sub-block of the current picture block to obtain predicted sample values ​​of the current sub-block of the current picture block. The predicted sample values ​​of the current picture block include predicted sample values ​​of one or more sub-blocks, and the one or more sub-blocks include the current sub-block. If the current picture block includes only one sub-block, the size of the sub-block may be equal to the size of the current picture block.

[0016] It should be noted that, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block according to the BDOF technology, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0017] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0018] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0019] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0020] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0021] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0022] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0023] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0024] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0025] Condition I: The size of the current picture block is the first preset size.

[0026] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0027] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0028] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in an encoding process, the current picture block is a picture block being encoded (encoding block).

[0029] In this embodiment of the present application, a status determination operation may further be included, for example, determining whether the plurality of preset conditions are satisfied.

[0030] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0031] It should be understood that the method in this embodiment of the application can be performed by a video encoder or an electronic device with video encoding functionality.

[0032] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0033] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0034] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0035] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0036] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to a first list (e.g., list0) and a second motion vector predictor corresponding to a second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (optionally, when the MMVD technique is used, only one MVD may be transmitted, and an MVD of another direction that needs to be used by the decoder side may be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0037] A first motion vector (for example, Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (for example, Mvp0_L0) corresponding to the first list and a first MVD (for example, MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0038] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on the second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and the second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0039] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0040] In an optional solution, the method further includes determining a sample residual based on the sample values ​​of the current picture block and the predicted sample values ​​of the current picture block, and encoding the syntax element into the bitstream to be transmitted includes encoding the syntax element and the sample residual into the bitstream to be transmitted. In other words, in addition to the syntax element, the sample residual can also be encoded into the bitstream to be transmitted (the sample residual can be obtained through calculation by using a residual calculation unit, for example, the sample residual is zero or not zero). This solution can be applied to AMVP mode and merge mode. Also, there are cases where the sample residual does not need to be encoded into the bitstream to be transmitted. For example, in skip mode, the sample residual does not need to be encoded into the bitstream to be transmitted.

[0041] According to a second aspect, an embodiment of the present application provides an encoding method, the method comprising: When the size of the current picture block is the second preset size, a predicted sample value (e.g., a predicted sample of a current coding unit) of the current picture block is determined (or obtained through prediction) using a first processing manner based on the reference sample value corresponding to the first list (i.e., list0) and the reference sample value corresponding to the second list (i.e., list1), and the first processing manner is not BDOF (also referred to as bidirectional optical flow (BIO)). In other words, when the size of the current picture block is the second preset size, it is not allowed to use BDOF to determine the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list. The size of the current picture block is usually represented by width and height (shortened as W×H), and the width and height are measured in samples (or pixels). The second preset sizes include 8x8 (the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples), 4xN (4 indicates that the width of the current picture block is 4 samples, and N indicates that the height of the current picture block is N samples; other sizes can be inferred by analogy), 8x16, or 16x8, where N is a power of 2 and is greater than or equal to 8. For example, the value of N can be 16, 32, 64, etc.

[0042] In this embodiment of the present application, the reference sample value corresponding to the first list and the reference sample value corresponding to the second list are obtained through prediction based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1), respectively. For example, the reference sample value corresponding to the first list (i.e., list0) can be determined based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0), and the reference sample value corresponding to the second list (i.e., list1) can be determined based on the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1).

[0043] The method then includes reconstructing the current picture block based on at least the predicted sample values ​​of the current picture block, where "at least" indicates that the predicted sample values ​​of the current picture block must be used to reconstruct the current picture block, but other information may also be used.

[0044] The method then includes encoding a syntax element into the transmitted bitstream, the syntax element being used to indicate at least that bidirectional prediction is to be applied.

[0045] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in an encoding process, the current picture block is a picture block being encoded (encoding block).

[0046] In addition, the above-mentioned cases in which the second preset size includes 8×8, 4×N, 8×16, or 16×8 can also be limited to the second preset size being 8×8, 4×N, 8×16, or 16×8. In other words, BDOF may be prohibited from being used for one of the above-listed sizes 8×8, 4×N, 8×16, and 16×8, or may be prohibited from being used for multiple sizes among these. An example is used below for explanation.

[0047] Example 1: The second preset size includes (or is) 8x8. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 8x8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is 8x8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0048] Example 2: The second preset size includes (or is) 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or more, when the size of the current picture block is 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0049] Example 3: The second preset size includes (or is) 8×8 or 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8 and 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or greater, when the size of the current picture block is either 8×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0050] Example 4: The second preset size includes (or is) 8×8, 4×N, 8×16, or 16×8, where N is greater than or equal to 8. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8, 4×N, 8×16, and 16×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is any one of 8×8, 4×N, 8×16, and 16×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0051] Example 5: The second preset size includes (or is) N×8, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0052] Example 6: The second preset size includes (or is) N×8 or 4×N, where N is 4 or greater. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes N×8 or 4×N. In this case, when the size of the current picture block is the second preset size, obtaining a predicted sample value of the current picture block through prediction in a first processing manner based on reference sample values ​​corresponding to the first list and reference sample values ​​corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8 or 4×N, obtaining a predicted sample value of the current picture block through prediction in a first processing manner based on reference sample values ​​corresponding to the first list and reference sample values ​​corresponding to the second list. Furthermore, in some scenarios, BDOF is prohibited from being used even for a size M×4. Therefore, in this application, when BDOF is prohibited from being used for the second preset size, BDOF may also be prohibited from being used for a size M×4, where M is an integer greater than or equal to 4.

[0053] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0054] It should be understood that the method in this embodiment of the application can be performed by a video encoder or an electronic device with video encoding functionality.

[0055] According to the above method, when the BDOF technique is used, the characteristics of current picture blocks of various sizes are fully taken into consideration. When the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to a technique other than the BDOF technique. This significantly reduces coding complexity and improves coding efficiency.

[0056] In an optional solution, determining the predicted sample value of the current picture block in a first processing manner according to the reference sample value corresponding to the first list and the reference sample value corresponding to the second list includes: performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain a predicted sample value of the current picture block; or Obtaining a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a joint inter / intra prediction CIIP technique; Includes:

[0057] In this embodiment of the present application, whether the weighting calculation method, the MMVD technique, the CIIP technique, or other prediction techniques are specifically used may be determined by using corresponding conditions, or may be indicated by using corresponding indication information.

[0058] It should be understood that the BDOF technique is equivalent to a patch optimization technique in the prior art for predicting predicted sample values ​​of a current picture block. According to the BDOF technique, predicted sample values ​​that are closer to the actual sample values ​​of the current picture block can be obtained in many scenarios. Therefore, the sample residuals obtained based on the predicted sample values ​​are smaller, and therefore coding efficiency can be improved. The above-mentioned weighting calculation scheme may be regarded as a prior art technique for obtaining predicted sample values ​​of a current picture block, and the above-mentioned MMVD technique and CIIP technique may each be regarded as a patch optimization technique in the prior art. In this embodiment of the present application, when the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to the prior art (e.g., weighting calculation scheme) or other patch optimization techniques (e.g., MMVD or CIIP), rather than the BDOF technique. This can significantly reduce coding complexity and improve coding efficiency.

[0059] In an optional solution, the method further comprises: The method includes performing bidirectional optical flow BDOF processing on a current picture block to obtain a predicted sample value of the current picture block when multiple preset conditions are satisfied, wherein the multiple preset conditions include at least a condition that the size of the current picture block satisfies a first preset size.

[0060] The size of the current picture block satisfying the first preset size includes the following: the height H of the current picture block is greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H of the current picture block is equal to 64.

[0061] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0062] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater, the width W of the current picture block being 8 or greater, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0063] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0064] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0065] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0066] The method further includes reconstructing the current picture block based at least on the predicted sample values ​​of the current picture block.

[0067] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0068] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0069] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0070] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0071] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0072] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0073] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0074] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0075] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0076] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0077] Condition I: The size of the current picture block is the first preset size.

[0078] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0079] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0080] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0081] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0082] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0083] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to a first list (e.g., list0) and a second motion vector predictor corresponding to a second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (optionally, when the MMVD technique is used, only one MVD may be transmitted, and an MVD of another direction that needs to be used by the decoder side may be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0084] A first motion vector (e.g., Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (e.g., Mvp0_L0) corresponding to the first list and a first MVD (e.g., MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0085] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on a second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and a second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0086] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0087] In one optional solution, the method further includes determining a sample residual based on the sample values ​​of the current picture block and the predicted sample values ​​of the current picture block, and encoding the syntax elements into the bitstream to be transmitted includes encoding the syntax elements and the sample residual into the bitstream to be transmitted.

[0088] According to a third aspect, an embodiment of the present application provides a video decoding method, the method comprising: Parsing the bitstream to obtain syntax elements, the syntax elements being used to indicate at least that bidirectional prediction is applied; When a plurality of preset conditions are satisfied, a bidirectional optical flow (BDOF) process is performed on the current picture block to obtain a predicted sample value of the current picture block, and the plurality of preset conditions at least include a condition that the size of the current picture block satisfies a first preset size; The size of the current picture block is usually represented by its width and height (abbreviated as W×H), where the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0089] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0090] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0091] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0092] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0093] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0094] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0095] The method further includes determining sample values ​​of the current picture block based on at least predicted sample values ​​of the current picture block, where the predicted sample values ​​of the current picture block include predicted sample values ​​of one or more sub-blocks.

[0096] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0097] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0098] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0099] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0100] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0101] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0102] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0103] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0104] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0105] Condition I: The size of the current picture block is the first preset size.

[0106] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0107] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0108] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in a decoding process, the current picture block is a picture block being decoded (decoding block).

[0109] In this embodiment of the present application, a status determination operation may further be included, for example, determining whether the plurality of preset conditions are satisfied.

[0110] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0111] It should be understood that the method in this embodiment of the present application can be performed by a video decoder or an electronic device with video decoding capabilities.

[0112] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0113] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0114] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0115] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0116] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to the first list (e.g., list0) and a second motion vector predictor corresponding to the second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (e.g., when only one MVD is included, the MVD that is not transmitted can be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0117] A first motion vector (for example, Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (for example, Mvp0_L0) corresponding to the first list and a first MVD (for example, MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0118] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on the second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and the second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0119] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0120] In one optional solution, parsing the bitstream to obtain syntax elements includes parsing the bitstream to obtain sample residuals and syntax elements, and determining sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block includes determining sample values ​​of the current picture block based on the sample residuals and the predicted sample values ​​of the current picture block.

[0121] According to a fourth aspect, an embodiment of the present application provides a decoding method, the method comprising: Parsing the bitstream to obtain syntax elements, the syntax elements being used to indicate at least that bidirectional prediction is applied; When the size of the current picture block is the second preset size, a predicted sample value (e.g., a predicted sample of a current coding unit) of the current picture block is determined (or obtained through prediction) using a first processing manner based on the reference sample value corresponding to the first list (i.e., list0) and the reference sample value corresponding to the second list (i.e., list1), and the first processing manner is not BDOF (also referred to as bidirectional optical flow (BIO)). In other words, when the size of the current picture block is the second preset size, it is not allowed to use BDOF to determine the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list. The size of the current picture block is usually represented by width and height (shortened as W×H), and the width and height are measured in samples (or pixels). The second preset sizes include 8x8 (the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples), 4xN (4 indicates that the width of the current picture block is 4 samples, and N indicates that the height of the current picture block is N samples; other sizes can be inferred by analogy), 8x16, or 16x8, where N is a power of 2 and is greater than or equal to 8. For example, the value of N can be 16, 32, 64, etc.

[0122] In this embodiment of the present application, the reference sample value corresponding to the first list and the reference sample value corresponding to the second list are obtained through prediction based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1), respectively. For example, the reference sample value corresponding to the first list (i.e., list0) can be determined based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0), and the reference sample value corresponding to the second list (i.e., list1) can be determined based on the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1).

[0123] The method further includes determining sample values ​​of the current picture block based at least on the predicted sample values ​​of the current picture block.

[0124] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in a decoding process, the current picture block is a picture block being decoded (decoding block).

[0125] In addition, the above-mentioned cases in which the second preset size includes 8×8, 4×N, 8×16, or 16×8 can also be limited to the second preset size being 8×8, 4×N, 8×16, or 16×8. In other words, BDOF may be prohibited from being used for one of the above-listed sizes 8×8, 4×N, 8×16, and 16×8, or may be prohibited from being used for multiple sizes among these. An example is used below for explanation.

[0126] Example 1: The second preset size includes (or is) 8x8. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 8x8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is 8x8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0127] Example 2: The second preset size includes (or is) 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or more, when the size of the current picture block is 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0128] Example 3: The second preset size includes (or is) 8×8 or 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8 and 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or greater, when the size of the current picture block is either 8×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0129] Example 4: The second preset size includes (or is) 8×8, 4×N, 8×16, or 16×8, where N is equal to or greater than 8. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8, 4×N, 8×16, and 16×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is any one of 8×8, 4×N, 8×16, and 16×8, where N is equal to or greater than 8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0130] Example 5: The second preset size includes (or is) N×8, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0131] Example 6: The second preset size includes (or is) N×8 or 4×N, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8 or 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0132] Furthermore, in some scenarios, BDOF is prohibited from being used even for size M×4, so in this application, when BDOF is prohibited from being used for the second preset size, BDOF may further be prohibited from being used for size M×4, where M is an integer greater than or equal to 4.

[0133] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0134] It should be understood that the method in this embodiment of the present application can be performed by a video decoder or an electronic device with video decoding capabilities.

[0135] According to the above method, when the BDOF technique is used, the characteristics of current picture blocks of various sizes are fully taken into consideration. When the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to a technique other than the BDOF technique. This significantly reduces coding complexity and improves coding efficiency.

[0136] In an optional solution, obtaining a predicted sample value of a current picture block through prediction in a first processing manner according to a reference sample value corresponding to a first list and a reference sample value corresponding to a second list includes: performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain a predicted sample value of the current picture block; or Obtaining a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining predicted sample values ​​of the current picture block through prediction based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list according to a joint inter / intra prediction CIIP technique; or Obtaining a predicted sample value of the current picture block through prediction based on the reference samples corresponding to the first list and the reference samples corresponding to the second list according to a decoder-side motion vector refinement DMVR technique; Includes:

[0137] In this embodiment of this application, whether the weighting calculation method, the MMVD technique, or the CIIP technique is specifically used may be determined by using corresponding conditions, or may be indicated by using corresponding indication information.

[0138] It should be understood that the BDOF technique is equivalent to a patch optimization technique in the prior art for predicting predicted sample values ​​of a current picture block. According to the BDOF technique, in many scenarios, predicted sample values ​​closer to the actual sample values ​​of the current picture block can be obtained. Therefore, the sample residuals obtained based on the predicted sample values ​​are smaller, thereby improving coding efficiency. Therefore, the BDOF technique is usually used on the encoder side. To more accurately restore predicted sample values, the BDOF technique is correspondingly used on the decoder side. The aforementioned weighting calculation scheme may be considered a prior art technique for obtaining predicted sample values ​​of a current picture block, and the aforementioned MMVD technique, CIIP technique, and DMVR technique may each be considered a patch optimization technique in the prior art. In this embodiment of the present application, when the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to the prior art (e.g., the weighting calculation scheme) or other patch optimization techniques (e.g., MMVD, CIIP, or DMVR), rather than the BDOF technique. This can significantly reduce the decoding complexity and improve the decoding efficiency.

[0139] In an optional solution, the method further comprises: The method includes performing bidirectional optical flow BDOF processing on a current picture block to obtain a predicted sample value of the current picture block when multiple preset conditions are satisfied, wherein the multiple preset conditions include at least a condition that the size of the current picture block satisfies a first preset size.

[0140] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H when the product of the width W and height H of the current picture block is equal to 64.

[0141] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0142] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater, the width W of the current picture block being 8 or greater, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0143] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0144] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0145] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0146] The method further includes reconstructing the current picture block based at least on the predicted sample values ​​of the current picture block.

[0147] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0148] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0149] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0150] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0151] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0152] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0153] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0154] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0155] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0156] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0157] Condition I: The size of the current picture block is the first preset size.

[0158] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0159] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0160] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0161] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0162] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0163] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to a first list (e.g., list0) and a second motion vector predictor corresponding to a second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (optionally, when the MMVD technique is used, only one MVD may be transmitted, and an MVD of another direction that needs to be used by the decoder side may be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0164] A first motion vector (e.g., Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (e.g., Mvp0_L0) corresponding to the first list and a first MVD (e.g., MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0165] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on a second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and a second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0166] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0167] In one optional solution, parsing the bitstream to obtain syntax elements includes parsing the bitstream to obtain sample residuals and syntax elements, and determining sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block includes determining sample values ​​of the current picture block based on the sample residuals and the predicted sample values ​​of the current picture block.

[0168] According to a fifth aspect, an embodiment of the present application provides an inter prediction method, the method comprising: Determining whether the current picture block satisfies a plurality of preset conditions; When the plurality of preset conditions are satisfied, a bidirectional optical flow (BDOF) process is performed on the current picture block to obtain predicted sample values ​​of the current picture block, and the plurality of preset conditions include at least a condition that the size of the current picture block satisfies a first preset size. The size of the current picture block is usually represented by width and height (abbreviated as W×H), and the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0169] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0170] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0171] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0172] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0173] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0174] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0175] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0176] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0177] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0178] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0179] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0180] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0181] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0182] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0183] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0184] Condition I: The size of the current picture block is the first preset size.

[0185] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0186] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0187] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed. For example, in an encoding process, the current picture block is a picture block being encoded (encoding block). Certainly, the current picture block here can also be understood as a picture block being processed. For example, in a decoding process, the current picture block is a picture block being decoded (decoding block).

[0188] In an optional solution, the method further includes: skipping performing bidirectional optical flow (BDOF) processing on the current picture block when at least one of the plurality of preset conditions is not satisfied. Specifically, the predicted sample values ​​of the current picture block may be obtained through prediction in a first processing manner based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list. The first processing manner is not BDOF. For details, please refer to the relevant parts of this specification. The details will not be described again here.

[0189] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0190] It should be understood that the method in this embodiment of the present application can be performed by a video encoder or an electronic device having a video encoding function, or can be performed by a video decoder or an electronic device having a video decoding function, for example, the method can be specifically performed by an inter-prediction unit within these devices.

[0191] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0192] According to a sixth aspect, an embodiment of the present application provides a video encoding device, the encoding device including an inter prediction unit, a reconstruction unit, and an entropy encoding unit.

[0193] The inter-prediction unit is configured to perform a bidirectional optical flow (BIO or BDOF) process on a current picture block to obtain predicted sample values ​​of the current picture block when multiple preset conditions are satisfied, the multiple preset conditions including at least a condition that the size of the current picture block satisfies a first preset size. The size of the current picture block is usually represented by width and height (abbreviated as W×H), and the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0194] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0195] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0196] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0197] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0198] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0199] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0200] The reconstruction unit is configured to reconstruct the current picture block based on at least the predicted sample values ​​of the current picture block.

[0201] The entropy coding unit is configured to code a syntax element into the bitstream to be transmitted, the syntax element being used to indicate at least that bidirectional prediction is applied.

[0202] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0203] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0204] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0205] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0206] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0207] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0208] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0209] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0210] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0211] Condition I: The size of the current picture block is the first preset size.

[0212] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0213] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0214] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in an encoding process, the current picture block is a picture block being encoded (encoding block).

[0215] In this embodiment of the application, a state determining operation may further be included, for example, the inter prediction unit is further configured to determine whether the plurality of preset conditions are satisfied.

[0216] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0217] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0218] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0219] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0220] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0221] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to a first list (e.g., list0) and a second motion vector predictor corresponding to a second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (optionally, when the MMVD technique is used, only one MVD may be transmitted, and an MVD of another direction that needs to be used by the decoder side may be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0222] A first motion vector (for example, Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (for example, Mvp0_L0) corresponding to the first list and a first MVD (for example, MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0223] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on the second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and the second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0224] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0225] In an optional solution, the video coding apparatus further includes a residual calculation unit configured to determine a sample residual based on the sample values ​​of the current picture block and the predicted sample values ​​of the current picture block, and the entropy coding unit is specifically configured to encode the syntax element and the sample residual into the bitstream to be transmitted. In other words, in addition to the syntax element, the sample residual can also be encoded into the bitstream to be transmitted (the sample residual can be obtained through calculation by using the residual calculation unit). This solution can be applied to AMVP mode and merge mode. Also, there are cases where the sample residual does not need to be encoded into the bitstream to be transmitted. For example, in skip mode, the sample residual does not need to be encoded into the bitstream to be transmitted.

[0226] According to a seventh aspect, an embodiment of the present application provides a video encoding device, the video encoding device including an inter prediction unit, a reconstruction unit, and an entropy encoding unit.

[0227] The inter prediction unit is configured to determine (or obtain through prediction) predicted sample values ​​of the current picture block (e.g., predicted samples of the current coding unit) in a first processing manner based on reference sample values ​​corresponding to the first list (i.e., list0) and reference sample values ​​corresponding to the second list (i.e., list1) when the size of the current picture block is a second preset size, and the first processing manner is not BDOF (also referred to as bidirectional optical flow (BIO)). In other words, when the size of the current picture block is the second preset size, it is not permitted to use BDOF to determine predicted sample values ​​of the current picture block based on reference sample values ​​corresponding to the first list and reference sample values ​​corresponding to the second list. The size of the current picture block is usually represented by width and height (shortened as W×H), and the width and height are measured in samples (or pixels). The second preset sizes include 8x8 (the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples), 4xN (4 indicates that the width of the current picture block is 4 samples, and N indicates that the height of the current picture block is N samples; other sizes can be inferred by analogy), 8x16, or 16x8, where N is a power of 2 and is greater than or equal to 8. For example, the value of N can be 16, 32, 64, etc.

[0228] In this embodiment of the present application, the reference sample value corresponding to the first list and the reference sample value corresponding to the second list are obtained through prediction based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1), respectively. For example, the reference sample value corresponding to the first list (i.e., list0) can be determined based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0), and the reference sample value corresponding to the second list (i.e., list1) can be determined based on the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1).

[0229] The reconstruction unit is configured to reconstruct the current picture block based on at least the predicted sample values ​​of the current picture block, where "at least" indicates that the predicted sample values ​​of the current picture block must be used to reconstruct the current picture block, but other information may also be used.

[0230] The entropy coding unit is configured to code a syntax element into the bitstream to be transmitted, the syntax element being used to indicate at least that bidirectional prediction is applied.

[0231] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in an encoding process, the current picture block is a picture block being encoded (encoding block).

[0232] In addition, the above-mentioned cases in which the second preset size includes 8×8, 4×N, 8×16, or 16×8 can also be limited to the second preset size being 8×8, 4×N, 8×16, or 16×8. In other words, BDOF may be prohibited from being used for one of the above-listed sizes 8×8, 4×N, 8×16, and 16×8, or may be prohibited from being used for multiple sizes among these. An example is used below for explanation.

[0233] Example 1: The second preset size includes (or is) 8x8. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 8x8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is 8x8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0234] Example 2: The second preset size includes (or is) 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or more, when the size of the current picture block is 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0235] Example 3: The second preset size includes (or is) 8×8 or 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8 and 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or greater, when the size of the current picture block is either 8×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0236] Example 4: The second preset size includes (or is) 8×8, 4×N, 8×16, or 16×8, where N is equal to or greater than 8. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8, 4×N, 8×16, and 16×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is any one of 8×8, 4×N, 8×16, and 16×8, where N is equal to or greater than 8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0237] Example 5: The second preset size includes (or is) N×8, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0238] Example 6: The second preset size includes (or is) N×8 or 4×N, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8 or 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0239] Furthermore, in some scenarios, BDOF is prohibited from being used even for size M×4, so in this application, when BDOF is prohibited from being used for the second preset size, BDOF may further be prohibited from being used for size M×4, where M is an integer greater than or equal to 4.

[0240] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0241] According to the above method, when the BDOF technique is used, the characteristics of current picture blocks of various sizes are fully taken into consideration. When the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to a technique other than the BDOF technique. This significantly reduces coding complexity and improves coding efficiency.

[0242] In an optional solution, in the process of determining the predicted sample value of the current picture block in a first processing manner according to the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, the inter prediction unit specifically comprises: Performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain a predicted sample value of the current picture block; or Obtaining a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or According to a combined inter / intra prediction CIIP technique, obtain a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list; It is configured as follows.

[0243] In this embodiment of this application, whether the weighting calculation method, the MMVD technique, or the CIIP technique is specifically used may be determined by using corresponding conditions, or may be indicated by using corresponding indication information.

[0244] It should be understood that the BDOF technique is equivalent to a patch optimization technique in the prior art for predicting predicted sample values ​​of a current picture block. According to the BDOF technique, predicted sample values ​​that are closer to the actual sample values ​​of the current picture block can be obtained in many scenarios. Therefore, the sample residuals obtained based on the predicted sample values ​​are smaller, and therefore coding efficiency can be improved. The above-mentioned weighting calculation scheme may be regarded as a prior art technique for obtaining predicted sample values ​​of a current picture block, and the above-mentioned MMVD technique and CIIP technique may each be regarded as a patch optimization technique in the prior art. In this embodiment of the present application, when the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to the prior art (e.g., weighting calculation scheme) or other patch optimization techniques (e.g., MMVD or CIIP), rather than the BDOF technique. This can significantly reduce coding complexity and improve coding efficiency.

[0245] In one optional solution, The inter prediction unit is further configured to perform bidirectional optical flow BDOF processing on the current picture block to obtain predicted sample values ​​of the current picture block when multiple preset conditions are satisfied, where the multiple preset conditions include at least a condition that the size of the current picture block satisfies a first preset size.

[0246] The size of the current picture block satisfying the first preset size includes the following: the height H of the current picture block is greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H of the current picture block is equal to 64.

[0247] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0248] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater, the width W of the current picture block being 8 or greater, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0249] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0250] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0251] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0252] The reconstruction unit is further configured to reconstruct the current picture block based on at least the predicted sample values ​​of the current picture block.

[0253] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0254] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0255] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0256] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0257] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0258] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0259] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0260] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0261] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0262] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0263] Condition I: The size of the current picture block is the first preset size.

[0264] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0265] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0266] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0267] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0268] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0269] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to a first list (e.g., list0) and a second motion vector predictor corresponding to a second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (optionally, when the MMVD technique is used, only one MVD may be transmitted, and an MVD of another direction that needs to be used by the decoder side may be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0270] A first motion vector (e.g., Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (e.g., Mvp0_L0) corresponding to the first list and a first MVD (e.g., MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0271] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on a second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and a second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0272] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0273] In one optional solution, the video encoding device further includes a residual calculation unit configured to determine sample residuals based on sample values ​​of the current picture block and predicted sample values ​​of the current picture block, and the entropy encoding unit is specifically configured to encode the syntax elements and the sample residuals into a bitstream to be transmitted.

[0274] According to an eighth aspect, an embodiment of the present application provides a video decoding device, the video decoding device including an entropy decoding unit, an inter prediction unit, and a reconstruction unit.

[0275] The entropy decoding unit is configured to parse the bitstream to obtain a syntax element, which is used to indicate at least that bidirectional prediction is applied.

[0276] The inter-prediction unit is configured to perform a bidirectional optical flow (BIO or BDOF) process on a current picture block to obtain predicted sample values ​​of the current picture block when multiple preset conditions are satisfied, the multiple preset conditions including at least a condition that the size of the current picture block satisfies a first preset size. The size of the current picture block is usually represented by width and height (abbreviated as W×H), and the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0277] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0278] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0279] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0280] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0281] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0282] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0283] The reconstruction unit is configured to determine sample values ​​of the current picture block based on at least predicted sample values ​​of the current picture block.

[0284] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0285] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0286] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0287] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0288] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0289] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0290] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0291] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0292] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0293] Condition I: The size of the current picture block is the first preset size.

[0294] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0295] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0296] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in a decoding process, the current picture block is a picture block being decoded (decoding block).

[0297] In this embodiment of the application, a state determining operation may further be included, for example, the inter prediction unit is further configured to determine whether the plurality of preset conditions are satisfied.

[0298] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0299] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0300] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0301] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0302] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0303] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to the first list (e.g., list0) and a second motion vector predictor corresponding to the second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (e.g., when only one MVD is included, the MVD that is not transmitted can be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0304] A first motion vector (for example, Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (for example, Mvp0_L0) corresponding to the first list and a first MVD (for example, MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0305] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on the second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and the second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0306] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0307] In one optional solution, the entropy decoding unit is specifically configured to parse the bitstream to obtain sample residuals and syntax elements, and the reconstruction unit is specifically configured to determine sample values ​​of the current picture block based on the sample residuals and predicted sample values ​​of the current picture block.

[0308] According to a ninth aspect, an embodiment of the present application provides a video decoding device, the video decoding device including an entropy decoding unit, an inter prediction unit, and a reconstruction unit.

[0309] The entropy decoding unit is configured to parse the bitstream to obtain a syntax element, which is used to indicate at least that bidirectional prediction is applied.

[0310] The inter prediction unit is configured to determine (or obtain through prediction) predicted sample values ​​of the current picture block (e.g., predicted samples of the current coding unit) in a first processing manner based on reference sample values ​​corresponding to the first list (i.e., list0) and reference sample values ​​corresponding to the second list (i.e., list1) when the size of the current picture block is a second preset size, and the first processing manner is not BDOF (also referred to as bidirectional optical flow (BIO)). In other words, when the size of the current picture block is the second preset size, it is not permitted to use BDOF to determine predicted sample values ​​of the current picture block based on reference sample values ​​corresponding to the first list and reference sample values ​​corresponding to the second list. The size of the current picture block is usually represented by width and height (shortened as W×H), and the width and height are measured in samples (or pixels). The second preset sizes include 8x8 (the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples), 4xN (4 indicates that the width of the current picture block is 4 samples, and N indicates that the height of the current picture block is N samples; other sizes can be inferred by analogy), 8x16, or 16x8, where N is a power of 2 and is greater than or equal to 8. For example, the value of N can be 16, 32, 64, etc.

[0311] In this embodiment of the present application, the reference sample value corresponding to the first list and the reference sample value corresponding to the second list are obtained through prediction based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1), respectively. For example, the reference sample value corresponding to the first list (i.e., list0) can be determined based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0), and the reference sample value corresponding to the second list (i.e., list1) can be determined based on the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1).

[0312] The reconstruction unit is configured to determine sample values ​​of the current picture block based on at least predicted sample values ​​of the current picture block.

[0313] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in a decoding process, the current picture block is a picture block being decoded (decoding block).

[0314] In addition, the above-mentioned cases in which the second preset size includes 8×8, 4×N, 8×16, or 16×8 can also be limited to the second preset size being 8×8, 4×N, 8×16, or 16×8. In other words, BDOF may be prohibited from being used for one of the above-listed sizes 8×8, 4×N, 8×16, and 16×8, or may be prohibited from being used for multiple sizes among these. An example is used below for explanation.

[0315] Example 1: The second preset size includes (or is) 8x8. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 8x8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is 8x8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0316] Example 2: The second preset size includes (or is) 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or more, when the size of the current picture block is 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0317] Example 3: The second preset size includes (or is) 8×8 or 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8 and 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or greater, when the size of the current picture block is either 8×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0318] Example 4: The second preset size includes (or is) 8×8, 4×N, 8×16, or 16×8, where N is greater than or equal to 8. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8, 4×N, 8×16, and 16×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is any one of 8×8, 4×N, 8×16, and 16×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0319] Example 5: The second preset size includes (or is) N×8, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0320] Example 6: The second preset size includes (or is) N×8 or 4×N, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8 or 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0321] Furthermore, in some scenarios, BDOF is prohibited from being used even for size M×4, so in this application, when BDOF is prohibited from being used for the second preset size, BDOF may further be prohibited from being used for size M×4, where M is an integer greater than or equal to 4.

[0322] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0323] According to the above method, when the BDOF technique is used, the characteristics of current picture blocks of various sizes are fully taken into consideration. When the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to a technique other than the BDOF technique. This significantly reduces coding complexity and improves coding efficiency.

[0324] In an optional solution, in the process of obtaining the predicted sample value of the current picture block through prediction in a first processing manner according to the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, the inter prediction unit specifically comprises: Performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain a predicted sample value of the current picture block; or Obtaining a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining a predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a joint inter / intra prediction CIIP technique; or According to a decoder-side motion vector refinement DMVR technique, obtain a predicted sample value of the current picture block through prediction based on the reference samples corresponding to the first list and the reference samples corresponding to the second list; It is configured as follows.

[0325] In this embodiment of this application, whether the weighting calculation method, the MMVD technique, or the CIIP technique is specifically used may be determined by using corresponding conditions, or may be indicated by using corresponding indication information.

[0326] It should be understood that the BDOF technique is equivalent to a patch optimization technique in the prior art for predicting predicted sample values ​​of a current picture block. According to the BDOF technique, in many scenarios, predicted sample values ​​closer to the actual sample values ​​of the current picture block can be obtained. Therefore, the sample residuals obtained based on the predicted sample values ​​are smaller, thereby improving coding efficiency. Therefore, the BDOF technique is usually used on the encoder side. To more accurately restore predicted sample values, the BDOF technique is correspondingly used on the decoder side. The aforementioned weighting calculation scheme may be considered a prior art technique for obtaining predicted sample values ​​of a current picture block, and the aforementioned MMVD technique, CIIP technique, and DMVR technique may each be considered a patch optimization technique in the prior art. In this embodiment of the present application, when the size of the current picture block is the second preset size, the predicted sample values ​​of the current picture block are determined according to the prior art (e.g., the weighting calculation scheme) or other patch optimization techniques (e.g., MMVD, CIIP, or DMVR), rather than the BDOF technique. This can significantly reduce the decoding complexity and improve the decoding efficiency.

[0327] In one optional solution, the inter prediction unit is further configured to perform bidirectional optical flow BDOF processing on the current picture block to obtain predicted sample values ​​of the current picture block when multiple preset conditions are satisfied, and the multiple preset conditions include at least a condition that the size of the current picture block satisfies a first preset size.

[0328] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H when the product of the width W and height H of the current picture block is equal to 64.

[0329] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0330] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater, the width W of the current picture block being 8 or greater, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0331] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0332] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than 8.

[0333] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than 8 and the width W of the current picture block being equal to or greater than 8.

[0334] The reconstruction unit is further configured to reconstruct the current picture block based on at least the predicted sample values ​​of the current picture block.

[0335] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0336] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0337] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0338] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0339] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0340] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0341] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0342] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0343] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0344] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0345] Condition I: The size of the current picture block is the first preset size.

[0346] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0347] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0348] In one optional solution, the syntax element includes an index of target candidate motion information (e.g., merge_idx[xCb][yCb]), where the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1), where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list (i.e., list0) and a second motion vector corresponding to the second list (i.e., list1), and the reference frame index includes an index of the first reference frame corresponding to the first list (e.g., refIdxL0) and an index of the second reference frame corresponding to the second list (e.g., refIdxL1).

[0349] In this embodiment of this application, the reference frame index is used to identify the reference picture corresponding to the used motion vector (e.g., the first motion vector or the second motion vector) in the specified reference picture list (the first list list0 or the second list list1). A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

[0350] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0351] In one optional solution, the syntax elements include a first index, a second index, prediction direction indication information, and a motion vector difference MVD, where the first index (e.g., mvp_lX_flag[xCb][yCb]) is used to indicate a target candidate motion vector predictor, where the target candidate motion vector predictor includes a first motion vector predictor corresponding to a first list (e.g., list0) and a second motion vector predictor corresponding to a second list (e.g., list1), and the motion vector difference MVD includes the first MVD corresponding to the first list and / or the second MVD corresponding to the second list (optionally, when the MMVD technique is used, only one MVD may be transmitted, and an MVD of another direction that needs to be used by the decoder side may be derived based on the transmitted MVD). The second index (also referred to as a reference frame index) is used to indicate a reference frame of the current picture block, where the reference frame includes a first reference frame corresponding to the first list and a second reference frame corresponding to the second list. For example, the second indexes are refIdxL0 and refIdxL1, where refIdxL0 is the index of the first reference frame corresponding to the first list, and refIdxL1 is the index of the second reference frame corresponding to the second list. Prediction direction indication information (e.g., predFlagL0=1 and predFlagL1=1) is used to indicate that bidirectional prediction is applied.

[0352] A first motion vector (e.g., Mv0_L0) corresponding to the first list is obtained based on a first motion vector predictor (e.g., Mvp0_L0) corresponding to the first list and a first MVD (e.g., MVD0) corresponding to the first list. For example, Mv0_L0=Mvp0_L0+MVD0.

[0353] A second motion vector (for example, Mv1_L1) corresponding to the second list is obtained based on a second motion vector predictor (for example, Mvp1_L1) corresponding to the second list and a second MVD (for example, MVD1) corresponding to the second list, for example, Mv1_L1=Mvp1_L1+MVD1.

[0354] In the field of video coding, inter-prediction modes include forward prediction, backward prediction, and bidirectional prediction (including forward prediction and backward prediction), and the specific prediction mode used at the encoder side is usually indicated by prediction direction indication information. For example, the prediction direction indication information may include syntax elements predFlagL0 and predFlagL1. When predFlagL0=1 and predFlagL1=1, the prediction direction indication information indicates that bidirectional prediction is applied. When predFlagL0=1 and predFlagL1=0, the prediction direction indication information indicates that forward prediction is used. When predFlagL0=0 and predFlagL1=1, the prediction direction indication information indicates that backward prediction is used. It may be understood that "forward" and "backward" correspond to reference picture list 0 (list0, i.e., the first list mentioned above) and reference picture list 1 (list1, i.e., the second list mentioned above) of the current picture, respectively.

[0355] In one optional solution, the entropy decoding unit is specifically configured to parse the bitstream to obtain sample residuals and syntax elements, and the reconstruction unit is specifically configured to determine sample values ​​of the current picture block based on the sample residuals and predicted sample values ​​of the current picture block.

[0356] According to a tenth aspect, an embodiment of the present application provides an inter prediction device (for example, an inter prediction unit), the inter prediction device comprising: a determining unit configured to determine whether a current picture block satisfies a plurality of preset conditions; and an inter-prediction processing unit configured to perform bidirectional optical flow (BDOF) processing on a current picture block to obtain predicted sample values ​​of the current picture block when the plurality of preset conditions are satisfied, the plurality of preset conditions including at least a condition that the size of the current picture block satisfies a first preset size. The size of the current picture block is usually represented by width and height (abbreviated as W×H), and the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0357] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0358] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0359] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0360] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0361] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0362] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0363] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0364] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0365] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0366] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0367] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0368] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0369] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0370] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0371] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0372] Condition I: The size of the current picture block is the first preset size.

[0373] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0374] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0375] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed. For example, in an encoding process, the current picture block is a picture block being encoded (encoding block). Certainly, the current picture block here can also be understood as a picture block being processed. For example, in a decoding process, the current picture block is a picture block being decoded (decoding block).

[0376] In one optional solution, the inter-prediction processing unit is further configured to skip performing bi-directional optical flow (BDOF, abbreviated as BIO or BDOF) processing on the current picture block when at least one of the plurality of preset conditions is not satisfied.

[0377] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0378] It should be understood that the method in this embodiment of the present application can be performed by a video encoder or an electronic device having a video encoding function, or can be performed by a video decoder or an electronic device having a video decoding function, for example, the method can be specifically performed by an inter-prediction unit within these devices.

[0379] Specifically, the conditions for enabling or using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology on the current picture block with a first preset size, predicted sample values ​​that are closer to the original sample values ​​can be obtained, and the coding complexity can be appropriately controlled. This balances the encoding / decoding complexity and prediction accuracy to a certain extent, thereby improving coding efficiency.

[0380] According to an eleventh aspect, an embodiment of the present application provides a video data decoding device, the device comprising: a memory configured to store video data in the form of a bitstream; a video decoder configured to: decode a bitstream to obtain a syntax element, the syntax element being at least used to indicate that bidirectional prediction is applied; and perform a bidirectional optical flow BDOF processing on a current picture block to obtain a predicted sample value of the current picture block when a plurality of preset conditions are satisfied, the plurality of preset conditions including at least a condition that the size of the current picture block satisfies a first preset size; Includes:

[0381] The size of the current picture block satisfying the first preset size includes the following: the height H of the current picture block is greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H of the current picture block is equal to 64.

[0382] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is 8 or more and the width W of the current picture block is 8 or more.

[0383] Alternatively, the size of the current picture block satisfying the first preset size includes the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0384] Alternatively, the size of the current picture block satisfying the first preset size includes the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0385] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0386] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0387] According to a twelfth aspect, an embodiment of the present application provides a video data encoding device, the device comprising: a memory configured to store video data, the video data including one or more picture blocks; A video encoder configured to perform a bidirectional optical flow BDOF process on a current picture block to obtain predicted sample values ​​of the current picture block when a plurality of preset conditions are satisfied, the plurality of preset conditions including at least a condition that the size of the current picture block satisfies a first preset size; Includes:

[0388] The size of the current picture block satisfying the first preset size includes the following: the height H of the current picture block is greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H of the current picture block is equal to 64.

[0389] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is 8 or more and the width W of the current picture block is 8 or more.

[0390] Alternatively, the size of the current picture block satisfying the first preset size includes the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0391] Alternatively, the size of the current picture block satisfying the first preset size includes the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0392] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0393] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0394] The video encoder is further configured to encode a syntax element into the bitstream to be transmitted, the syntax element being used to indicate at least that bidirectional prediction is applied.

[0395] According to a thirteenth aspect, an embodiment of the present application provides an encoding device including a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform some or all of the steps of any of the methods in the first aspect.

[0396] According to a fourteenth aspect, an embodiment of the present application provides a decoding device including a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform some or all of the steps of any of the methods in the third aspect.

[0397] According to a fifteenth aspect, an embodiment of the present application provides a computer-readable storage medium storing program code, the program code including instructions used to perform some or all of the steps of any of the methods in the first aspect.

[0398] According to a sixteenth aspect, an embodiment of the present application provides a computer program product which, when run on a computer, enables the computer to perform some or all of the steps of any of the methods in the first aspect.

[0399] It should be understood that the technical solutions in the second to sixteenth aspects of this application are consistent with the technical solution in the first aspect of this application, and the beneficial effects achieved by these aspects and corresponding feasible implementations are similar, and the details will not be described again. [Brief explanation of the drawings]

[0400] To more clearly describe the technical solutions in the embodiments or background of this application, the following describes the accompanying drawings for describing the embodiments or background of this application. [Figure 1A] 1 is a block diagram of an example of a video encoding and decoding system 10 that implements an embodiment of the present application. [Figure 1B] 1 is a block diagram of an example of a video coding system 40 implementing an embodiment of the present application. [Figure 2] 1 is a block diagram of an example configuration of an encoder 20 for implementing an embodiment of the present application. [Figure 3] FIG. 3 is a block diagram of an example configuration of a decoder 30 for implementing an embodiment of the present application. [Figure 4] 4 is a block diagram of an example of a video coding device 400 for implementing an embodiment of the present application. [Figure 5] 1 is a block diagram of another example of an encoding or decoding device 10 for implementing an embodiment of the present application. [Figure 5A] 2 is a schematic diagram of motion information of a current picture block and a reference block according to an embodiment of the present application; [Figure 5B] FIG. 1 is a schematic diagram of an inter-prediction scenario according to an embodiment of the present application; [Figure 5C] FIG. 1 is a schematic diagram of an inter-adjacent block scenario according to an embodiment of the present application; [Figure 6A] 1 is a schematic flowchart of an encoding method according to an embodiment of the present application; [Figure 6B] FIG. 2 is a schematic diagram of forward and backward reference pictures implementing one embodiment of the present application; [Figure 6C] FIG. 2 is a schematic diagram of a motion vector refinement process implementing an embodiment of the present application; [Figure 6D] FIG. 1 is a schematic diagram of a scenario of a padding-based expansion technique according to an embodiment of the present application. [Figure 7] 1 is a schematic flowchart of a decoding method according to an embodiment of the present application; [Figure 8] FIG. 1 is a schematic configuration diagram of an inter prediction device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0401] The following description of the present application will be made with reference to the accompanying drawings in the embodiments of the present application. In the following description, reference is made to the accompanying drawings, which form a part of this disclosure and which illustrate, by way of example, certain aspects of the embodiments of the present application or in which the embodiments of the present application may be used. It should be understood that the embodiments of the present application may be used in other ways and may include structural or logical changes not depicted in the accompanying drawings. Therefore, the following detailed description is not to be construed in a limiting sense, and the scope of the present application is defined by the appended claims. For example, it should be understood that content disclosed in connection with a described method may also apply to a corresponding apparatus or system configured to perform the method, and vice versa. For example, when one or more particular method steps are described, the corresponding apparatus may include one or more units, e.g., functional units, for performing one or more of the described method steps (e.g., one unit that performs the one or more steps, or multiple units that each perform one or more of the steps), even if such one or more units are not explicitly described or shown in the accompanying drawings. Also, for example, if a particular apparatus is described based on one or more units, e.g., functional units, a corresponding method may include a step used to perform the function of the one or more units (e.g., a step used to perform the function of the one or more units, or multiple steps each used to perform the function of one or more of the units), even if such step or steps are not explicitly described or shown in the accompanying drawings. Furthermore, it should be understood that features of various example embodiments and / or aspects described in this specification may be combined with each other unless specifically stated otherwise.

[0402] The technical solutions in the embodiments of this application can be applied not only to existing video coding standards (e.g., standards such as H.264 and HEVC), but also to future video coding standards (e.g., the H.266 standard). The terms used in the implementation of this application are intended to merely describe specific embodiments of this application and are not intended to limit this application. Below, we first briefly describe the concepts related to the embodiments of this application.

[0403] Video coding typically refers to processing a series of pictures that make up a video or video sequence. In the field of video coding, the terms "picture," "frame," and "image" may be used synonymously. As used herein, video coding includes video encoding and video decoding. Video encoding is performed at the source side and typically involves processing original video pictures (e.g., by compressing) to reduce the amount of data required to represent the video picture for more efficient storage and / or transmission. Video decoding is performed at the destination side and typically involves the reverse process relative to the encoder to reconstruct the video picture. In the embodiments, "coding" a video picture should be understood as "encoding" or "decoding" a video sequence. The combination of the encoding and decoding parts is also referred to as coding (encoding and decoding).

[0404] A video sequence includes a series of pictures, which are further divided into slices, and slices are further divided into blocks. Video coding is performed by blocks. Some new video coding standards further extend the concept of a "block." For example, the H.264 standard introduces the macroblock (MB). A macroblock can be further partitioned into multiple prediction blocks that can be used for predictive coding. The high efficiency video coding (HEVC) standard uses basic concepts such as a "coding unit" (CU), a "prediction unit" (PU), and a "transform unit" (TU). Multiple block units are obtained through functional partitioning and described by using a new tree-based structure. For example, a CU can be partitioned into smaller CUs based on a quadtree, and the smaller CUs can be further partitioned to generate a quadtree structure. A CU is the basic unit for partitioning and encoding a coded picture. PUs and TUs also have a similar tree structure. A PU may correspond to a prediction block and is the basic unit for predictive coding. A CU is further divided into multiple PUs in a certain partitioning mode. A TU may correspond to a transform block and is the basic unit used to transform prediction residuals. However, in practice, CUs, PUs, and TUs are all conceptually blocks (or picture blocks).

[0405] For example, in HEVC, a CTU is divided into multiple CUs by using a quadtree structure, represented as a coding tree. The decision of whether to encode a picture region through inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs based on the PU partition type. The same prediction process is applied within a PU, and related information is transmitted to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying the prediction process based on the PU partition type, the CU can be divided into transform units (TUs) based on another quadtree structure similar to the coding tree used for the CU. In recent developments in video compression technology, quadtree plus binary tree (QTBT) partition frames are used to divide coding blocks. In the QTBT block structure, CUs can be square or rectangular.

[0406] In this specification, for ease of explanation and understanding, a picture block to be processed in a current picture may be referred to as a current block or a current picture block. For example, in encoding, a picture block is a block being coded, and in decoding, a picture block is a block being decoded. A decoded picture block in a reference picture used to predict a current block is referred to as a reference block. Specifically, a reference block is a block that provides a reference signal for the current block, and the reference signal represents sample values ​​in the picture block. A block in a reference picture that provides a prediction signal for the current block may be referred to as a prediction block. The prediction signal represents pixel values, sample values, or sampling signals in the prediction block. For example, after multiple reference blocks are considered, an optimal reference block is found. The optimal reference block provides a prediction for the current block, and the block is referred to as a prediction block.

[0407] In lossless video coding, the original video picture can be reconstructed. Specifically, the reconstructed video picture has the same quality as the original video picture (assuming no transmission or other data loss occurs during storage or transmission). In lossy video coding, further compression is performed, for example through quantization, to reduce the amount of data required to represent the video picture, and the video picture cannot be perfectly reconstructed at the decoder side. Specifically, the quality of the reconstructed video picture is lower or inferior to the quality of the original video picture.

[0408] Some H.261 video coding standards are concerned with "lossy hybrid video coding" (specifically, spatial and temporal prediction in the sample domain is combined with 2D transform coding to apply quantization in the transform domain). Each picture in a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. Specifically, at the encoder side, video is typically processed, i.e., encoded, at the block (video block) level. For example, a predictive block is generated through spatial (intra-picture) prediction and temporal (inter-picture) prediction, and the predictive block is subtracted from a current block (the block being processed or being processed) to obtain a residual block, which is then transformed and quantized in the transform domain to reduce the amount of data to be transmitted (compressed). At the decoder side, the coded or compressed block is subjected to processing parts inverse to those of the encoder to reconstruct the current block for representation. Additionally, the encoder replicates the decoder processing loop so that the encoder and decoder generate the same predictions (eg, intra- and inter-predictions) and / or reconstructions for processing or encoding subsequent blocks.

[0409] A system architecture to which an embodiment of this application is applied will now be described. FIG. 1A is a schematic block diagram of an example of a video encoding and decoding system 10 to which an embodiment of this application is applied. As shown in FIG. 1A, the video encoding and decoding system 10 may include a source device 12 and a destination device 14. The source device 12 generates coded video data, and therefore, the source device 12 may be referred to as a video coding device. The destination device 14 can decode the coded video data generated by the source device 12, and therefore, the destination device 14 may be referred to as a video decoding device. In various implementation solutions, the source device 12, the destination device 14, or both the source device 12 and the destination device 14 may include one or more processors and memory coupled to the one or more processors. The memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium capable of being used to store desired program code in the form of computer-accessible instructions or data structures as described herein. The source device 12 and the destination device 14 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, or the like.

[0410] 1A depicts source device 12 and destination device 14 as separate devices, a device embodiment may instead include both source device 12 and destination device 14, or the functionality of both source device 12 and destination device 14, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software, separate hardware and / or software, or any combination thereof.

[0411] The communication connection between source device 12 and destination device 14 may be implemented via link 13. Destination device 14 may receive encoded video data from source device 12 via link 13. Link 13 may include one or more media or devices capable of moving encoded video data from source device 12 to destination device 14. In one example, link 13 may include one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real time. In this example, source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to destination device 14. The one or more communication media may include wireless communication media and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission cables. The one or more communication media may be part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from source device 12 to destination device 14 .

[0412] Source device 12 includes an encoder 20. Optionally, source device 12 may further include a picture source 16, a picture preprocessor 18, and a communication interface 22. In one particular implementation, encoder 20, picture source 16, picture preprocessor 18, and communication interface 22 may be hardware components within source device 12 or may be software programs within source device 12. Separate descriptions are provided below.

[0413] Picture source 16 may include or be, for example, any type of picture capture device configured to capture real-world pictures and / or any type of device that generates pictures or comments (for screen content coding, some text on the screen is also considered part of the picture or image to be coded), such as, for example, a computer graphics processor configured to generate computer-animated pictures, or any type of device configured to obtain and / or provide real-world pictures or computer-animated pictures (e.g., screen content or virtual reality (VR) pictures) and / or any combination thereof (e.g., augmented reality (AR) pictures). Picture source 16 may be a camera configured to capture pictures or a memory configured to store pictures. Picture source 16 may further include any type of interface (internal or external) through which previously captured or generated pictures are stored and / or through which pictures are obtained or received. If picture source 16 is a camera, picture source 16 may be, for example, a local camera or an integrated camera integrated in the source device. If the picture source 16 is a memory, it may be a local memory or an integrated memory, for example, integrated in the source device. If the picture source 16 includes an interface, it may be, for example, an external interface for receiving pictures from an external video source. The external video source may be, for example, an external picture capturing device such as a camera, an external memory, or an external picture generating device. The external picture generating device may be, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface, for example, a wired or wireless interface or an optical interface according to some proprietary or standardized interface protocol.

[0414] A picture can be considered as a two-dimensional array or matrix of picture elements. A picture element in an array can also be referred to as a sample. The number of samples in the horizontal and vertical directions (or axes) of the array or picture determines the size and / or resolution of the picture. For color representation, three color components are usually employed, and specifically, a picture can be represented as or contain three sample arrays. For example, in an RBG format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each sample is usually represented in a luminance / chrominance format or color space. For example, a picture in YUV format contains a luminance component denoted by Y (sometimes alternatively denoted by L) and two chrominance components denoted by U and V. The luminance (luma) component Y represents brightness or gray-level intensity (e.g., both are the same in a grayscale picture), and the two chrominance (chroma) components U and V represent chromaticity or color information components. Correspondingly, a picture in YUV format includes a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(U and V). A picture in RGB format may be transformed or converted to YUV format, and vice versa. This process is also called color conversion or transformation. If a picture is monochrome, it may include only a luminance sample array. In this embodiment of the application, the picture sent by the picture source 16 to the picture processor may be referred to as raw picture data 17.

[0415] The picture preprocessor 18 is configured to receive the raw picture data 17 and perform preprocessing on the raw picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. For example, the preprocessing performed by the picture preprocessor 18 may include cropping, color format conversion (e.g., from RGB format to YUV format), color correction, or noise removal.

[0416] Encoder 20 (also referred to as video encoder 20) is configured to receive pre-processed picture data 19 and process the pre-processed picture data 19 in an associated prediction mode (e.g., a prediction mode in an embodiment of this specification) to provide encoded picture data 21. (Details of the configuration of encoder 20 are described further below with reference to FIG. 2, FIG. 4, or FIG. 5.) In some embodiments, encoder 20 may be configured to perform various embodiments described below to implement encoder-side applications of the chroma block prediction methods described in this application.

[0417] The communication interface 22 may be configured to receive the coded picture data 21 and transmit the coded picture data 21 via the link 13 to the destination device 14 or some other device (e.g., memory) for storage or direct reconstruction. The other device may be any device used for decoding or storage. The communication interface 22 may be configured, for example, to encapsulate the coded picture data 21 into a format suitable for transmission over the link 13, such as, for example, data packets.

[0418] The destination device 14 includes a decoder 30. Optionally, the destination device 14 may further include a communication interface 28, a picture post-processor 32, and a display device 34. The following description is provided separately.

[0419] The communication interface 28 may be configured to receive the coded picture data 21 from the source device 12 or some other source. The some other source may be, for example, a storage device. The storage device may be, for example, a coded picture data storage device. The communication interface 28 may be configured to transmit or receive the coded picture data 21 via a link 13 between the source device 12 and the destination device 14 or via any type of network. The link 13 may be, for example, a direct wired or wireless connection. The any type of network may be, for example, a wired or wireless network or any combination thereof, or any type of private or public network or any combination thereof. The communication interface 28 may be configured, for example, to decapsulate data packets transmitted via the communication interface 22 to obtain the coded picture data 21.

[0420] Both communication interface 28 and communication interface 22 may be configured as one-way or two-way communication interfaces, for example, to send and receive messages to establish a connection, as well as to verify and exchange any other information related to the communication link and / or data transmission, such as, for example, coded picture data transmission.

[0421] Decoder 30 is configured to receive encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (details of the configuration of decoder 30 are described further below with reference to Figure 3, Figure 4 or Figure 5). In some embodiments, decoder 30 may be configured to perform various embodiments described below to implement decoder-side applications of the chroma block prediction methods described in this application.

[0422] The picture post-processor 32 is configured to post-process the decoded picture data 31 (also referred to as reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 may include color format conversion (e.g., from YUV format to RGB format), color correction, cropping, resampling, or some other processing. The picture post-processor 32 may further be configured to send the post-processed picture data 33 to a display device 34.

[0423] The display device 34 is configured to receive the post-processed picture data 33 and display the picture, for example, to a user or viewer. The display device 34 can be or include any type of display for presenting the reconstructed picture, such as an integrated or external display or monitor. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0424] FIG. 1A depicts the source device 12 and the destination device 14 as separate devices, but the device embodiments may instead include both the source device 12 and the destination device 14, or the functions of both the source device 12 and the destination device 14, i.e., the source device 12 or the corresponding function and the destination device 14 or the corresponding function. In such an embodiment, the source device 12 or the corresponding function, and the destination device 14 or the corresponding function may be implemented by using the same hardware and / or software, separate hardware and / or software, or some combination thereof.

[0425] As will be apparent to those skilled in the art based on these descriptions, the presence and (exact) functional partitioning of the different units or functions of the source device 12 and / or the destination device 14 shown in FIG. 1A may vary depending on the actual device and application. The source device 12 and the destination device 14 may include, for example, any type of handheld device or fixed device such as a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, an in-vehicle device, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content delivery server, etc.), a broadcast receiver device, or a broadcast transmitter device, and may use or not use any kind of operating system.

[0426] Encoder 20 and decoder 30 may each be implemented as any of a variety of suitable circuitry, such as, for example, one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. Where techniques are implemented in part using software, an apparatus may store software instructions on a suitable non-transitory computer-readable storage medium and execute the instructions using hardware, such as one or more processors, to perform the techniques of this disclosure. Any of the above (including hardware, software, combinations of hardware and software, and the like) may be considered as one or more processors.

[0427] In some cases, the video encoding and decoding system 10 shown in FIG. 1A is merely an example, and the technology of this application may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoder and decoder devices. In another example, data may be retrieved from local memory, streamed over a network, etc. A video encoder may encode data and store the data in memory, and / or a video decoder may retrieve data from memory and decode the data. In some examples, encoding and / or decoding are performed by devices that only encode data and store the data in memory and / or retrieve data from memory and decode the data, but do not communicate with each other.

[0428] 1B is an illustration of an example video coding system 40 including the encoder 20 of FIG. 2 and / or the decoder 30 of FIG. 3, according to an example embodiment. The video coding system 40 may implement a combination of various techniques in the embodiments of this application. In the illustrated implementation, the video coding system 40 may include an image capture device 41, the encoder 20, the decoder 30 (and / or a video encoder / decoder implemented by logic 47 of processing circuitry 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0429] 1B, imaging device 41, antenna 42, processing circuitry 46, logic circuitry 47, encoder 20, decoder 30, processor 43, memory 44, and / or display device 45 may be in communication with one another. As illustrated, although video coding system 40 is shown with encoder 20 and decoder 30, video coding system 40 may include only encoder 20 or only decoder 30 in different examples.

[0430] In some examples, antenna 42 may be configured to transmit or receive an encoded bitstream of video data. Additionally, in some examples, display device 45 may be configured to present the video data. In some examples, logic circuitry 47 may be implemented by processing circuitry 46. Processing circuitry 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. Video coding system 40 may also include optional processor 43. Optional processor 43 may also include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. In some examples, logic circuitry 47 may be implemented by hardware, such as dedicated hardware for video coding. Processor 43 may be implemented by general-purpose software, an operating system, or the like. Furthermore, memory 44 may be any type of memory, such as, for example, volatile memory (e.g., Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM)) or non-volatile memory (e.g., flash memory). In one non-limiting example, memory 44 may be implemented by cache memory. In some examples, logic circuitry 47 may access memory 44 (e.g., to implement a picture buffer). In other examples, logic circuitry 47 and / or processing circuitry 46 may include memory (e.g., a cache) for implementing a picture buffer.

[0431] In some examples, video encoder 20 implemented using logic circuitry may include a picture buffer (which may be implemented, for example, by processing circuitry 46 or memory 44) and a graphics processing unit (which may be implemented, for example, by processing circuitry 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include encoder 20 implemented by logic circuitry 47 to implement various modules described with reference to FIG. 2 and / or any other encoder system or subsystem described herein. The logic circuitry may be configured to perform various operations described herein.

[0432] In some examples, decoder 30 may be similarly implemented by logic circuitry 47 to implement the various modules described with reference to decoder 30 of FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, decoder 30 implemented using logic circuitry may include a picture buffer (which may be implemented with processing circuitry 46 or memory 44) and a graphics processing unit (which may be implemented, for example, by processing circuitry 46). The graphics processing unit may be communicatively coupled to the picture buffer. The graphics processing unit may include decoder 30 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 3 and / or any other decoder system or subsystem described herein.

[0433] In some examples, antenna 42 may be configured to receive an encoded bitstream of video data. As described, the encoded bitstream may include data related to video frame coding and described herein, such as data related to a coding partition (e.g., transform coefficients or quantized transform coefficients, optional indicators (to be described), and / or data defining the coding partition), indicators, index values, mode selection data, or the like. Video coding system 40 may further include a decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0434] It should be understood that in this embodiment of the application, with respect to the examples described with reference to encoder 20, decoder 30 may be configured to perform an inverse process. With respect to signaling syntax elements, decoder 30 may be configured to receive and parse such syntax elements and correspondingly decode the associated video data. In some examples, encoder 20 may entropy encode the syntax elements into the coded video bitstream. In such examples, decoder 30 may parse the syntax elements and correspondingly decode the associated video data.

[0435] It should be noted that the video picture encoding method described in the embodiment of this application is executed on the encoder 20, and the video picture decoding method described in the embodiment of this application is executed on the decoder 30. The encoder 20 and the decoder 30 in the embodiment of this application may be encoders / decoders compatible with video standard protocols such as H.263, H.264, HEVC, MPEG-2, MPEG-4, VP8, or VP9, ​​or next-generation video standard protocols (such as H.266).

[0436] 2 is a schematic / conceptual block diagram of an example of an encoder 20 configured to implement an embodiment of this application. In the example of FIG. 2, the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder based on a hybrid video codec.

[0437] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 20, whereas, for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a reverse signal path of the encoder 20. The reverse signal path of the encoder corresponds to the signal path of the decoder (see decoder 30 in FIG. 3).

[0438] Encoder 20 receives, for example via input 202, a picture 201, or a picture block 203 of picture 201, e.g., a picture in a series of pictures forming a video or a video sequence. Picture block 203 may also be called a current picture block or a picture block to be coded, and picture 201 may also be called a current picture or a picture to be coded (particularly in video coding, to distinguish the current picture from other pictures, e.g., previously coded and / or decoded pictures in the same video sequence, i.e., the video sequence that also includes the current picture).

[0439] An embodiment of encoder 20 may include a division unit (not shown in FIG. 2) configured to divide picture 201 into multiple blocks, such as picture blocks 203. Picture 201 is typically divided into non-overlapping blocks. The division unit may be configured to use the same block size and a corresponding grid defining the block sizes for all pictures in a video sequence, or may vary block sizes among pictures, or subsets, or groups of pictures, to divide each picture into corresponding blocks.

[0440] In one example, prediction processing unit 260 of encoder 20 may be configured to perform any combination of the above-mentioned partitioning techniques.

[0441] Although the size of the picture block 203 is smaller than the size of the picture 201, like the picture 201, the picture block 203 is also or can be considered as a two-dimensional array or matrix of samples having sample values. In other words, the picture block 203 may include, for example, one sample array (e.g., a luma array in the case of a monochrome picture 201), three sample arrays (e.g., one luma array and two chroma arrays in the case of a color picture), or some other number and / or type of arrays depending on the applied color format. The number of samples in the horizontal and vertical directions (or axes) of the picture block 203 determine the size of the picture block 203.

[0442] The encoder 20 shown in FIG. 2 is arranged to encode a picture 201 block by block, for example to perform encoding and prediction for each picture block 203.

[0443] The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture block 203 and the prediction block 265 (further details about the prediction block 265 will be provided later), for example, by subtracting the sample values ​​of the prediction block 265 from the sample values ​​of the picture block 203 on a sample-by-sample (pixel-by-pixel) basis to obtain the residual block 205 in the sample domain.

[0444] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in a transform domain. The transform coefficients 207, sometimes referred to as transform residual coefficients, represent the residual block 205 in the transform domain.

[0445] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as the transform specified in HEVC / H.265. Compared to an orthogonal DCT transform, such an integer approximation is typically scaled based on a factor. To preserve the norm of the residual block processed by using the forward and inverse transforms, an additional scale factor is applied as part of the transform process. The scale factor is typically selected based on several constraints, such as the scale factor being a power of two due to shift operations, the bit depth of the transform coefficients, or a trade-off between accuracy and implementation cost. For example, by using [mathematical formula - see original document], a particular scale factor may be specified for the inverse transform, e.g., by the inverse transform processing unit 212 at the decoder 30 side (and the corresponding inverse transform, e.g., by the inverse transform processing unit 212 at the encoder 20 side), and a corresponding scale factor may be specified for the forward transform, e.g., by the transform processing unit 206 at the encoder 20 side.

[0446] The quantization unit 208 is configured to quantize the transform coefficients 207, for example, by applying scalar quantization or vector quantization, to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth for some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be changed by adjusting a quantization parameter (QP). For example, in scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, and a larger quantization step size corresponds to coarser quantization. The quantization parameter (QP) may indicate an appropriate quantization step size. For example, the quantization parameter may be an index to a predetermined set of appropriate quantization step sizes. For example, a smaller quantization parameter may correspond to finer quantization (smaller quantization step size), while a larger quantization parameter may correspond to coarser quantization (larger quantization step size), and vice versa. Quantization may include division by the quantization step size and corresponding quantization or inverse quantization, such as performed by the inverse quantization unit 210, or may include multiplication by the quantization step size. In embodiments according to some standards, such as HEVC, the quantization parameter may be used to determine the quantization step size. In general, the quantization step size may be calculated based on the quantization parameter by using a fixed-point approximation of an equation involving division. An additional scale factor may be introduced into quantization and dequantization to restore the norm of the residual block, which may be modified due to the scale used in the fixed-point approximation of the equation for the quantization step size and the quantization parameter. In one implementation, the scale of the inverse transform may be combined with the scale of the dequantization.Alternatively, a customized quantization table may be used and signaled from the encoder to the decoder, for example, in the bitstream. Quantization is a lossy operation, with larger quantization steps indicating larger losses.

[0447] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients to obtain dequantized coefficients 211, e.g., to apply the inverse of the quantization scheme applied by the quantization unit 208, based on or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211, which are sometimes referred to as dequantized residual coefficients 211, may correspond to the transform coefficients 207, although the dequantized coefficients 211 typically differ from the transform coefficients due to losses caused by quantization.

[0448] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform dequantization block 213 or an inverse transform residual block 213.

[0449] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265, for example by adding the sample values ​​of the reconstructed residual block 213 and the sample values ​​of the prediction block 265, to obtain a reconstructed block 215 in the sample domain.

[0450] Optionally, a buffer unit 216 (or "buffer" 216 for short), e.g., a line buffer 216, is configured to buffer or store the reconstructed block 215 and corresponding sample values, e.g., for intra prediction. In other embodiments, the encoder may be configured to use the unfiltered reconstructed block and / or corresponding sample values ​​stored in the buffer unit 216 for some type of estimation and / or prediction, e.g., intra prediction.

[0451] For example, in one embodiment, encoder 20 may be configured such that buffer unit 216 stores reconstructed blocks 215 that are used not only for intra prediction 254 but also for units (not shown in FIG. 2) within loop filter unit 220, and / or such that buffer unit 216 and decoded picture buffer 230 form one buffer, for example. In another embodiment, filtered blocks 221 and / or blocks or samples from decoded picture buffer 230 (not shown in FIG. 2) are used as input or basis for intra prediction unit 254.

[0452] Loop filter unit 220 (or "loop filter" 220 for short) is configured to filter reconstructed block 215 to smooth sample transitions or improve video quality, resulting in filtered block 221. Loop filter unit 220 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although loop filter unit 220 is shown in FIG. 2 as an in-loop filter, in another implementation, loop filter unit 220 may be implemented as a post-loop filter. Filtered block 221 may also be referred to as filtered reconstructed block 221. After loop filter unit 220 performs the filtering process on the reconstructed coded block, decoded picture buffer 230 may store the reconstructed coded block.

[0453] In one embodiment, the encoder 20 (and correspondingly, the loop filter unit 220) may be configured to output loop filter parameters (e.g., sample adaptive offset information), e.g., directly or after entropy coding performed by the entropy coding unit 270 or some other entropy coding unit, so that the decoder 30 can receive and apply the same loop filter parameters for decoding.

[0454] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding video data by the encoder 20. The DPB 230 may include any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), or resistive RAM (RRAM)), or other types of memory devices. The DPB 230 and the buffer 216 may be provided by the same memory device or by separate memory devices. In one example, the decoded picture buffer (DPB) 230 is configured to store the filtered blocks 221. The decoded picture buffer 230 may further be configured to store other previously filtered blocks, such as the previously reconstructed filtered block 221, of the same current picture or of a different picture, such as a previously reconstructed picture, and may also provide a complete previously reconstructed or decoded picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), e.g., for inter-prediction. In one example, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215 when the reconstructed block 215 is reconstructed without in-loop filtering.

[0455] The prediction processing unit 260, also referred to as block prediction processing unit 260, is configured to receive or obtain a picture block 203 (current picture block 203 of current picture 201) and reconstructed picture data, such as reference samples of the same (current) picture from buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from decoded picture buffer 230, and to process such data for prediction, in particular to provide a prediction block 265, which may be an inter prediction block 245 or an intra prediction block 255.

[0456] The mode selection unit 262 may be configured to select a prediction mode (e.g., an intra-prediction mode or an inter-prediction mode) and / or a corresponding prediction block 245 or 255 to be used as the prediction block 265 for calculation of the residual block 205 and for reconstruction of the reconstruction block 215.

[0457] In one embodiment, mode selection unit 262 may be configured to select a prediction mode (e.g., from prediction modes supported by prediction processing unit 260) that provides the best match or, in other words, the smallest residual (smallest residual means better compression for transmission or storage), or that provides the smallest signaling overhead (smallest signaling overhead means better compression for transmission or storage), or that considers or balances both. Mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), specifically, to select a prediction mode that provides the smallest rate distortion optimization, or to select a prediction mode whose associated rate distortion at least satisfies a prediction mode selection criterion.

[0458] The prediction process (eg, performed by prediction processing unit 260) and mode selection (eg, performed by mode selection unit 262) performed by an example encoder 20 are described in detail below.

[0459] As mentioned above, the encoder 20 is configured to determine or select an optimal or best prediction mode from a (predetermined) set of prediction modes, which may include, for example, intra-prediction modes and / or inter-prediction modes.

[0460] The set of intra prediction modes may include 35 different intra prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those specified in H.265, or may include 67 different intra prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those specified in the currently developing H.266.

[0461] In one possible implementation, the set of inter prediction modes depends on available reference pictures (i.e., at least some decoded pictures stored in DBP 230, e.g., as described above) and other inter prediction parameters, such as whether the entire reference picture is used or only a portion of the reference picture, e.g., a search window region surrounding the region of the current block, is used, and / or whether sample interpolation, e.g., half-sample and / or quarter-sample interpolation, is applied. The set of inter prediction modes may include, for example, a skip mode and a merge mode. In a particular implementation, the set of inter prediction modes may include, in an embodiment of this application, a skip-based merge with motion vector difference (MMVD) mode or a merge-based MMVD mode. In one example, intra prediction unit 254 may be configured to perform some combination of the inter prediction techniques described below.

[0462] In addition to the above prediction modes, skip mode and / or direct mode may also be applied in embodiments of this application.

[0463] Prediction processing unit 260 may further be configured to divide picture block 203 into smaller block partitions or sub-blocks, e.g., by repeatedly using quad-tree (QT) partitioning, binary-tree (BT) partitioning, ternary-tree (TT) partitioning, or some combination thereof, and then perform prediction on each of the block partitions or sub-blocks, e.g., Mode selection includes selecting the tree structure of picture block 203 to be divided and selecting the prediction mode to be used for each of the block partitions or sub-blocks.

[0464] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain a picture block 203 (current picture block 203 of current picture 201) and a decoded picture 231 or at least one or more previous reconstructed blocks, such as one or more reconstructed blocks of another / different previous decoded picture 231, for motion estimation. For example, a video sequence may include the current picture and the previous decoded picture 231, or in other words, the current picture and the previous decoded picture 231 may be part of or form part of a series of pictures that form a video sequence.

[0465] For example, the encoder 20 may be configured to select a reference block from multiple reference blocks in the same picture or multiple different pictures among multiple other pictures, and provide the reference picture to a motion estimation unit (not shown in FIG. 2 ) and / or provide an offset (spatial offset) between the position of the reference block (coordinates X and Y) and the position of the current block as an inter-prediction parameter. This offset is also called a motion vector (MV).

[0466] The motion compensation unit is configured to obtain inter prediction parameters and perform inter prediction based on or by using the inter prediction parameters to obtain inter prediction block 245. Motion compensation performed by the motion compensation unit (not shown in FIG. 2 ) may include fetching or generating a predictive block based on motion / block vectors determined through motion estimation (possibly performing interpolation with sub-pixel accuracy). Interpolation filtering can generate additional pixel samples from known samples, potentially increasing the number of candidate predictive blocks that can be used to code the picture block. Upon receiving a motion vector for the PU of the current picture block, motion compensation unit 246 may locate the predictive block pointed to by the motion vector within one of the reference picture lists. Motion compensation unit 246 may also generate syntax elements related to blocks and video slices, which decoder 30 may use to decode picture blocks within the video slice.

[0467] Specifically, the inter prediction unit 244 may send a syntax element to the entropy coding unit 270, where the syntax element includes inter prediction parameters (e.g., an indication of a selection of an inter prediction mode to be used for predicting a current block after considering multiple inter prediction modes). In a possible application scenario, if there is only one inter prediction mode, instead, the inter prediction parameters may not be carried in the syntax element. In this case, the decoder side 30 may directly perform decoding by using a default prediction mode. It may be understood that the inter prediction unit 244 may be configured to perform some combination of multiple inter prediction techniques.

[0468] The intra prediction unit 254 is configured to obtain, e.g., receive, the picture block 203 (current picture block) and one or more previous reconstructed blocks, e.g., reconstructed neighboring blocks of the same picture, for intra estimation. The encoder 20 may be configured, for example, to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[0469] In one embodiment, the encoder 20 is configured to select an intra prediction mode according to an optimization criterion, such as based on minimum residual (e.g., the intra prediction mode that provides the predicted block 255 that is most similar to the current picture block 203) or minimum rate distortion.

[0470] The intra prediction unit 254 is further configured to determine the intra-predicted block 255 based on, for example, the intra prediction parameters of the selected intra prediction mode. In any case, after selecting the intra prediction mode for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters, i.e., information indicating the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform some combination of multiple intra prediction techniques.

[0471] Specifically, the intra prediction unit 254 can send a syntax element to the entropy encoding unit 270, where the syntax element includes intra prediction parameters (e.g., an indication of the selection of the intra prediction mode used to predict the current block after considering multiple intra prediction modes). In one possible application scenario, if there is only one intra prediction mode, instead, the intra prediction parameters may not be carried in the syntax element. In this case, the decoder side 30 may directly perform decoding in the default prediction mode.

[0472] The entropy coding unit 270 is configured to apply (or not apply) an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding method or technique) to one or all of the quantized residual coefficients 209, the inter-prediction parameters, the intra-prediction parameters, and / or the loop filter parameters to obtain coded picture data 21, e.g., in the form of a coded bitstream 21, which may be output via an output 272. The coded bitstream may be transmitted to the video decoder 30 or may be archived for later transmission or retrieval by the video decoder 30. Entropy encoding unit 270 may also be configured to entropy encode other syntax elements for the current video slice being encoded.

[0473] Other video encoders 20 with slightly different structures may be used to encode the video stream. For example, a non-transform-based encoder 20 may quantize the residual signal directly for some blocks or frames without using the transform processing unit 206. In another implementation, the encoder 20 may have the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0474] Specifically, in an embodiment of this application, the encoder 20 may be configured to implement the video picture encoding methods described in the following embodiments.

[0475] It should be understood that another video encoder 20 with a slightly different structure may be used to encode the video stream. For example, for some picture blocks or picture frames, the video encoder 20 may directly quantize the residual signal, so that processing by the transform processing unit 206 and, correspondingly, processing by the inverse transform processing unit 212 is not required. Alternatively, for some picture blocks or picture frames, the video encoder 20 does not generate residual data, so that processing by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 is not required. Alternatively, the video encoder 20 may directly store the reconstructed picture block as a reference block, so that processing by the filter 220 is not required. Alternatively, the quantization unit 208 and the inverse quantization unit 210 in the video encoder 20 may be combined. The loop filter 220 is optional. Also, in the case of lossless compression encoding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are optional. It should be understood that in different application scenarios, the inter prediction unit 244 and the intra prediction unit 254 can be selectively used.

[0476] 3 is a schematic / conceptual block diagram of an example of a decoder 30 configured to implement an embodiment of this application. Video decoder 30 is configured to receive coded picture data (e.g., coded bitstream) 21, e.g., coded by encoder 20, to obtain decoded pictures 231. In the decoding process, video decoder 30 receives video data from video encoder 20, e.g., a coded video bitstream representing picture blocks of coded video slices and associated syntax elements.

[0477] 3, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., adder 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, video decoder 30 may perform a decoding process that is generally inverse to the encoding process described with respect to video encoder 20 of FIG.

[0478] Entropy decoding unit 304 is configured to perform entropy decoding on coded picture data 21 to obtain, e.g., quantized coefficients 309 and / or decoded coding parameters (not shown in FIG. 3 ), such as, e.g., any one or all of (decoded) inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements. Entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to prediction processing unit 360. Video decoder 30 may receive video slice-level and / or video block-level syntax elements.

[0479] The inverse quantization unit 310 may have the same functionality as the inverse quantization unit 210. The inverse transform processing unit 312 may have the same functionality as the inverse transform processing unit 212. The reconstruction unit 314 may have the same functionality as the reconstruction unit 214. The buffer 316 may have the same functionality as the buffer 216. The loop filter 320 may have the same functionality as the loop filter 220. The decoded picture buffer 330 may have the same functionality as the decoded picture buffer 230.

[0480] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. Inter prediction unit 344 may be similar in function to inter prediction unit 244, and intra prediction unit 354 may be similar in function to intra prediction unit 254. Prediction processing unit 360 is typically configured to perform block prediction and / or obtain prediction blocks 365 from encoded data 21, as well as to receive or obtain (explicitly or implicitly) information about prediction-related parameters and / or a selected prediction mode, e.g., from entropy decoding unit 304.

[0481] Also, when a video slice is coded into an intra-coded (I) slice, intra prediction unit 354 of prediction processing unit 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra prediction mode and data of a previously decoded block of the current frame or picture. When a video frame is coded into an inter-coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) in prediction processing unit 360 is configured to generate a prediction block 365 for a video block in the current video slice based on the motion vector and other syntax elements received from entropy decoding unit 304. In inter prediction, the prediction block may be generated from one of the reference pictures in a reference picture list. Video decoder 30 may construct reference frame lists, List 0 and List 1, by using a default construction technique and based on the reference pictures stored in DPB 330.

[0482] Prediction processing unit 360 is configured to determine prediction information for video blocks in the current video slice by analyzing the motion vectors and other syntax elements, and to use the prediction information to generate a prediction block for the current video block being decoded. In one example of this application, prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra-prediction or inter-prediction) for encoding video blocks in the video slice, an inter-prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information for one or more of a reference picture list for the slice, a motion vector for each inter-coded video block in the slice, an inter-prediction status for each inter-coded video block in the slice, and other information, and decodes the video blocks in the current video slice. In another example of this disclosure, the syntax elements received from the bitstream by video decoder 30 include syntax elements within one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

[0483] Inverse quantization unit 310 may be configured to perform inverse quantization (i.e., dequantization) on the quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by video encoder 20 for each video block in a video slice to determine the degree of quantization to be applied and the degree of inverse quantization to be applied.

[0484] The inverse transform processing unit 312 is configured to apply an inverse transform (eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to produce residual blocks in the sample domain.

[0485] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365, for example by adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the prediction block 365, to obtain a reconstructed block 315 in the sample domain.

[0486] The loop filter unit 320 (in the coding loop or after the coding loop) is configured to filter the reconstruction block 315 to obtain a filtered block 321 to smooth sample transitions or improve video quality. In one example, the loop filter unit 320 may be configured to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is shown in FIG. 3 as an in-loop filter, in another implementation, the loop filter unit 320 may be implemented as a post-loop filter.

[0487] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0488] Decoder 30 is arranged, for example, to output decoded pictures 331 via output 332 for presentation or viewing to a user.

[0489] A slightly different video decoder 30 may be used to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without using the loop filter unit 320. For example, the non-transform-based decoder 30 may inverse quantize the residual signal directly for some blocks or frames without using the inverse transform processing unit 312. In another implementation, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0490] Specifically, in the embodiments of this application, the decoder 30 is configured to implement the video picture decoding methods described in the following embodiments.

[0491] It should be understood that another video decoder 30 with a slightly different structure can be used to decode the coded video bitstream. For example, the video decoder 30 may generate an output video stream without processing by the filter 320. Alternatively, for some picture blocks or picture frames, the entropy decoding unit 304 of the video decoder 30 does not obtain quantized coefficients through decoding, and correspondingly, the inverse quantization unit 310 and the inverse transform processing unit 312 are not required to perform the processing. The loop filter 320 is optional. Also, in the case of lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are optional. It should be understood that in different application scenarios, the inter prediction unit and the intra prediction unit can be selectively used.

[0492] It should be understood that in the encoder 20 and the decoder 30 in this application, the processing result of a certain procedure may be further processed before being output to the next procedure, for example, after a procedure such as interpolation filtering, motion vector derivation, or loop filtering, an operation such as clipping or shifting may be further performed on the processing result of the corresponding procedure.

[0493] For example, the motion vectors of the control points of the current picture block or the motion vectors of the sub-blocks of the current picture block derived from the motion vectors of neighboring affine coding blocks may be further processed. This is not a limitation of this application. For example, the value of the motion vector may be constrained to be within a specific bit width range. Assuming that the allowed bit width of the motion vector is bitDepth, the value of the motion vector ranges from -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents exponentiation. If bitDepth is 16, the value ranges from -32768 to 32767. If bitDepth is 18, the value ranges from -131072 to 131071. In another example, the values ​​of the motion vectors (e.g., the motion vectors MV of four 4x4 sub-blocks in one 8x8 picture block) are constrained so that the maximum difference between the integer parts of the MVs of the four 4x4 sub-blocks does not exceed N samples, for example, not more than 1 sample.

[0494] 4 is a schematic block diagram of a video coding device 400 (e.g., video encoder 400 or video decoder 400) according to one embodiment of the present application. The video coding device 400 is suitable for implementing the embodiments described herein. In one embodiment, the video coding device 400 may be a video decoder (e.g., decoder 30 of FIG. 1A) or a video encoder (e.g., encoder 20 of FIG. 1A). In another embodiment, the video coding device 400 may be one or more components of decoder 30 of FIG. 1A or encoder 20 of FIG. 1A.

[0495] The video coding device 400 includes an ingress port 410 and a receiving unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing the data, a transmitter unit (Tx) 440 (or simply referred to as Tx 440) and an egress port 450 for transmitting the data, and a memory 460 (e.g., memory 460) configured to store the data. The video coding device 400 may further include optical-to-electrical and electro-optical (EO) components coupled to the ingress port 410, the receiver unit 420 (or simply referred to as Tx 420), the transmitter unit 440, and the egress port 450 for inputting and outputting optical or electrical signals.

[0496] The processor 430 is implemented in hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470 (e.g., an encoding module 470 or a decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed herein to implement the chrominance block prediction method provided in the embodiments of this application. For example, the encoding / decoding module 470 performs, processes, or provides various coding operations. Thus, the encoding / decoding module 470 substantially enhances the functionality of the video coding device 400 and affects the transformation of the video coding device 400 into multiple different states. Alternatively, the encoding / decoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0497] Memory 460 may include one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store programs when they are selected for execution, and to store instructions and data read during program execution. Memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0498] FIG. 5 is a simplified block diagram of an apparatus 500 that can be used as either or both of the source device 12 and the destination device 14 of FIG. 1A according to an example embodiment. The apparatus 500 can implement the techniques of this application. In other words, FIG. 5 is a schematic block diagram of an implementation of an encoding or decoding apparatus (coding apparatus 500 for short) according to an embodiment of this application. The coding apparatus 500 may include a processor 510, a memory 530, and a bus system 550. The processor and the memory are connected via the bus system. The memory is configured to store instructions. The processor is configured to execute the instructions stored in the memory. The memory of the coding apparatus may store program code, and the processor may invoke the program code stored in the memory to perform various video picture encoding or decoding methods described in this application, particularly video encoding or decoding methods in various inter-prediction or intra-prediction modes. To avoid repetition, the details will not be described again here.

[0499] In this embodiment of the application, processor 510 may be a Central Processing Unit ("CPU") for short. Alternatively, processor 510 may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.

[0500] Memory 530 may include a read-only memory (ROM) device or a random-access memory (RAM) device. Alternatively, any other suitable type of storage device may be used as memory 530. Memory 530 may include code and data 531 accessed by processor 510 via bus 550. Memory 530 may also include an operating system 533 and application programs 535. Application programs 535 include at least one program that enables processor 510 to perform the video encoding or decoding methods described herein (particularly the video picture prediction or inter-prediction methods described herein). For example, application programs 535 may include Applications 1 through N, and may further include a video encoding or decoding application (referred to simply as a video coding application) that performs the video encoding or decoding methods described herein.

[0501] In addition to a data bus, the bus system 550 may further include a power bus, a control bus, a status signal bus, and the like. However, for clarity of explanation, various types of buses in the figures are labeled as the bus system 550.

[0502] Optionally, coding apparatus 500 may further include one or more output devices, such as a display 570. In one example, display 570 may be a touch display that combines a display with a touch unit that operatively detects touch input. Display 570 may be connected to processor 510 via bus 550.

[0503] In the embodiment of this application, inter-prediction is an important step. From the encoder's perspective, inter-prediction refers to searching a reconstructed picture for a reference block that matches a current coding block in a current picture. As shown in FIG. 5A, sample values ​​of samples in a reference block (e.g., reference block Fr1 or reference block Fr2, where sample values ​​in the reference block can be understood as reference sample values ​​in this application) are used as prediction information or predicted values ​​(hereinafter, "information" and "value" are not distinguished) of sample values ​​in a current coding block (i.e., current picture block) Fc. This process is referred to as motion estimation ME. Furthermore, the motion information of the current coding block is transmitted. From the decoder's perspective, motion information needs to be obtained through analysis during inter-prediction, and a reference block is determined in a reconstructed picture based on the obtained motion information, and sample values ​​of samples in the reference block (i.e., reference sample values) are used as prediction information. This process is referred to as motion compensation MC. Reconstructed information can be obtained by combining prediction information and residual information and performing a filtering operation on the combined information. The idea in this application may be used in HEVC or other video coding standards. In the following, the possible cases of inter prediction mode at the encoder side are explained by using an example of using this idea in HEVC. The possible cases of inter prediction mode at the decoder side are similar, and therefore no additional explanation is provided.

[0504] In HEVC, there are two modes: an Advanced Motion Vector Prediction (AMVP) mode and a Merge mode. These two modes will be described separately below.

[0505] In AMVP mode, an affine candidate motion vector list is first constructed by using motion information of coding blocks spatially or temporally adjacent to the current coding block. Then, an optimal motion vector is determined from the affine candidate motion vector list as the motion vector predictor (MVP) of the current coding block. The rate-distortion cost is calculated using Equation (0-1), where J is the rate-distortion cost (RD cost), SAD is the sum of absolute differences (SAD) between predicted sample values ​​and original sample values ​​obtained through motion estimation using the candidate motion vector predictor, R is the bit rate, and λ is the Lagrange multiplier. The encoder transmits the index value of the selected motion vector predictor in the affine candidate motion vector list and the reference frame index value to the decoder. Furthermore, a motion search is performed within a neighboring region centered on the MVP to obtain the actual motion vector of the current coding block. The encoder transmits the difference between the MVP and the actual motion vector (motion vector difference) to the decoder. J=SAD+λR (0-1)

[0506] In merge mode, a candidate motion information list is first constructed by using motion information from coding blocks spatially or temporally neighboring the current coding block. Then, the best motion information is determined from the candidate motion information list based on the rate-distortion cost and used as the motion information for the current coding block. The index value (hereinafter referred to as the merge index) of the best motion information's position in the candidate motion information list is then transmitted to the decoder. Spatial and temporal candidate motion information for the current coding block is shown in FIG. 5B. The spatial candidate motion information is from five spatially neighboring blocks (A0, A1, B0, B1, and B2). If a neighboring block is unavailable or in intra-coding mode, the neighboring block is not added to the candidate motion information list. The temporal candidate motion information for the current coding block is obtained after scaling the motion vectors of co-located blocks in the reference frame based on the picture order counts (POC) of the reference frame and the current frame. First, it is determined whether the block at position T in the reference frame is available. If the block is unavailable, the block at position C is selected.

[0507] In HEVC inter-prediction, all samples in a current coding block use the same motion information, and motion compensation is performed based on the motion information to obtain predicted sample values ​​of the current coding block. The motion information mainly includes: (1) the prediction direction of the coding block, including forward prediction, backward prediction, and bidirectional prediction, where forward prediction indicates that the coding block is obtained through prediction based on a previous coding frame, backward prediction indicates that the coding block is obtained through prediction based on a subsequent coding frame, and bidirectional prediction indicates that the coding block is obtained through prediction referring to a previous coding frame and a subsequent coding frame; (2) the reference frame index of the coding block, indicating the frame in which the reference block of the current coding block is located; and (3) the motion vector MV of the coding block, indicating the motion displacement of the coding block relative to the reference block, where MV is the horizontal component (MV x ) and vertical component (MV y and ), which respectively represent the motion displacement of the coding block relative to the reference block in the horizontal direction and the motion displacement of the coding block relative to the reference block in the vertical direction. When forward prediction or backward prediction is performed on the coding block, only one MV exists. When bidirectional (i.e., forward and backward) prediction is performed on the coding block, two MVs exist. Figure 5C explains the aforementioned motion information. In Figure 5C and the following description of motion information and prediction information, 0 represents forward prediction and 1 represents backward prediction. For example, Ref0 represents a forward reference frame, Ref1 represents a backward reference frame, MV0 represents a forward motion vector, and MV1 represents a backward motion vector. A, B, and C represent the forward reference block, the current coding block, and the backward reference block, respectively. Cur represents the current coding frame, and the dashed line represents the movement trajectory of B.

[0508] The basic motion compensation process of forward prediction is as follows: As shown in Figure 5C, the current coding block is block B in the figure, and the height and width of B are H and W, respectively. In this case, based on the motion information, the forward reference frame of the current coding block B is frame Ref0, and the forward motion vector MV0 of the current coding block B is (MV0 x ,MV0 y ) can be seen. When coding block B in frame Cur is coded, first, the same coordinate point is found in frame Ref0 based on the coordinates (i, j) of the upper left corner point of B in frame Cur, and block B' in Ref0 can be obtained based on the length and width of block B. Then, B' is moved to block A based on the MV0 of block B'. ​​Finally, interpolation is performed on block A to obtain the predicted block of the current coding block B. The sample value of each sample in the predicted block of the current coding block B is referenced as the predicted value of the corresponding sample in block B. The motion compensation process of backward prediction is the same as that of forward prediction, but the reference direction is different. Note that the predicted blocks obtained through motion compensation in backward prediction and forward prediction are called forward predicted block and backward predicted block, respectively. When bidirectional prediction is not performed on a coding block, the obtained forward predicted block and backward predicted block are the predicted block of the current coding block.

[0509] In bidirectional prediction, first, in the forward prediction motion compensation process and the backward prediction motion compensation process, a forward prediction block and a backward prediction block are obtained based on the motion information, and then weighted prediction or BDOF (also called bi-directional optical flow, BIO) is performed on the sample values ​​at the same position in the forward prediction block and the backward prediction block to obtain the prediction block of coding block B.

[0510] In the BDOF technology, after bidirectional motion compensation is performed on the current coding block, a forward prediction block and a backward prediction block are obtained, and then a modified motion vector of each 4x4 sub-block in the current coding block is derived based on the sample values ​​of the forward prediction block and the backward prediction block. Finally, compensation is performed again on each sample in the current coding block based on the modified motion vector to obtain a predicted value of the current coding block, and a final predicted block of the current coding block is obtained. The embodiments of this application focus on specific conditions for performing motion compensation using BDOF in order to reduce coding complexity and improve coding performance.

[0511] FIG. 6A is a schematic flowchart of an encoding method according to an embodiment of the present invention. The encoding procedure may be performed by a video encoding device. The video encoding device may be the aforementioned video encoder 20 or a device including the video encoder 20. Specifically, the procedure may be performed by an inter-prediction unit (also referred to as an inter-predictor) 244, an entropy encoding unit (also referred to as an entropy encoder) 270, or the like, within the video encoder 20. The aforementioned procedure is described as a series of steps or operations. It should be understood that the steps of the procedure may be performed in various orders and / or simultaneously, and are not limited to the execution order shown in FIG. 6A. The relevant description of the procedure corresponding to FIG. 6A is as follows.

[0512] The encoding device first determines whether multiple preset conditions are satisfied, and if the multiple preset conditions are satisfied, executes S600; if the multiple preset conditions are not satisfied, executes S601. If the multiple preset conditions are not satisfied, only the case of S601 is used as a representative for detailed description herein. Other cases that are the same as those in the prior art will not be described in this application.

[0513] Step S600: When multiple preset conditions are satisfied, the encoding apparatus performs bidirectional optical flow BDOF processing on the current picture block to obtain predicted sample values ​​of the current picture block.

[0514] Bi-directional optical flow (BDOF) may be abbreviated as BIO or BDOF.

[0515] The plurality of preset conditions at least includes a condition that the size of the current picture block satisfies a first preset size. The size of the current picture block is usually represented by width and height (abbreviated as W×H), and the width and height are measured in samples (or pixels). For example, if the size of the current picture block is 8×8, the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples. In another example, if the size of the current picture block is 8×16, the 8 indicates that the width of the current picture block is 8 samples, and the 16 indicates that the height of the current picture block is 16 samples.

[0516] The size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, and the width W is not equal to the height H of the current picture block when the product of the width W and height H is equal to 64.

[0517] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being 8 or greater and the width W of the current picture block being 8 or greater.

[0518] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 64.

[0519] Alternatively, the size of the current picture block satisfying the first preset size includes (or is) the height H of the current picture block being greater than or equal to 8, the width W of the current picture block being greater than or equal to 8, and the product of the width W of the current picture block and the height H of the current picture block being greater than 128.

[0520] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8.

[0521] Alternatively, the size of the current picture block satisfying the first preset size includes that the height H of the current picture block is greater than 8 and the width W of the current picture block is equal to or greater than 8.

[0522] It should be noted that, according to the BDOF technology, before determining whether to determine (or obtain through prediction) the predicted sample value of the current picture block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list, there may be other reference conditions in addition to the size of the current picture block. The other reference conditions are not limited here. For ease of understanding, the following uses an example for explanation.

[0523] Condition A: The high-level syntax identifier sps_bdof_enabled_flag is present, indicating that the BDOF technique is allowed to be used for the current video sequence.

[0524] Condition B: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1.

[0525] Condition C: The picture order count POC_L0 of the reference frame corresponding to the first list (list0), the picture order count POC_L1 of the reference frame corresponding to the second list (list1), and the picture order count POC_Cur of the picture in which the current picture block is located satisfy the relationship (POC_L0-POC_Cur) x (POC_L1-POC_Cur) < 0. In other words, the picture containing the current picture block is between the two reference pictures.

[0526] Condition D: MotionModelIdc[xCb][yCb] is equal to 0. MotionModelIdc is a motion model index for motion compensation. MotionModelIdc[xCb][yCb] equal to 0 indicates that the motion model for motion compensation of the current block is translational motion.

[0527] Condition E: merge_subblock_flag[x0][y0] is equal to 0, where merge_subblock_flag[x0][y0] equal to 0 indicates that the subblock merge mode is not applied to the current picture block.

[0528] Condition F: sym_mvd_flag[x0][y0] is equal to 0. sym_mvd_flag[x0][y0] equal to 0 indicates that the mvd_coding(x0,y0,refList,cpIdx) syntax structure is present for the current block.

[0529] Condition G: bcwIdx[xCb][yCb] is equal to 0. bcwIdx indicates the bidirectional prediction weight index for the current picture block.

[0530] Condition H: cIdx is equal to 0. cIdx represents the color component index of the current picture block.

[0531] Condition I: The size of the current picture block is the first preset size.

[0532] Condition J: luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] are both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction.

[0533] For example, when all of the conditions A to J are satisfied, it is determined that the BDOF technique can be used to predict the predicted sample value of the current picture block. For example, bdofFlag is set to true. Note that the above conditions are merely examples, and other conditions may be added, or one or more of the above conditions may be replaced, or one or more of the above conditions may be removed.

[0534] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in an encoding process, the current picture block is a picture block being encoded (encoding block).

[0535] It can be understood that with reference to the above cases, other cases can be further obtained, and the other cases also fall within the protection scope of this application.

[0536] It should be understood that the method in this embodiment of the present application can be performed by a picture prediction device, such as a video encoder or an electronic device with video encoding functionality, and specifically, can be performed by an inter prediction unit within a video encoder.

[0537] Specifically, the conditions for using the BDOF technology in this solution are different from those in the prior art, especially the requirements for the size of the current picture block. By using the BDOF technology for the current picture block having a first preset size, predicted sample values ​​that are closer to actual sample values ​​can be obtained, the coding complexity can be well controlled, and the coding efficiency can be improved.

[0538] In this embodiment of the application, the encoding device performing bidirectional optical flow BDOF processing on the current picture block when multiple preset conditions are satisfied may specifically be as follows: when multiple preset conditions are satisfied, the encoding device performs bidirectional optical flow BDOF processing on the current picture block based on the reference sample values ​​corresponding to the first list and the reference samples corresponding to the second list.

[0539] In this embodiment of the present application, the reference sample value corresponding to the first list and the reference sample value corresponding to the second list are obtained through prediction based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1), respectively. For example, the reference sample value corresponding to the first list (i.e., list0) can be determined based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0), and the reference sample value corresponding to the second list (i.e., list1) can be determined based on the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1).

[0540] It can be understood that the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1) are obtained in different manners in different inter prediction modes. The specific inter prediction mode to be used can be selected by using a mode selection unit. Hereinafter, how to determine the first motion vector and the second motion vector in the AMVP mode, merge mode, and skip mode will be described separately.

[0541] Possible Case 1: The inter prediction mode is the AMVP mode.

[0542] The encoder first builds a list of candidate motion vector predictors MVP.

[0543] Specifically, the video encoding apparatus uses an inter prediction unit (also referred to as an inter prediction module) to construct a candidate motion vector predictor MVP list (also referred to as a candidate motion vector list). The constructed candidate motion vector predictor MVP list may be a triplet candidate motion vector predictor MVP list or a 2-tuple candidate motion vector predictor MVP list. In this embodiment of the present application, the video encoding apparatus performs inter prediction in a bidirectional prediction manner.

[0544] The video encoder then determines the target candidate motion information in the MVP list according to a rate-distortion cost criterion.

[0545] Specifically, the target candidate motion information may be referred to as the best candidate motion vector, or may have another name. In conclusion, the target candidate motion information is the best candidate determined in the MVP list according to a rate-distortion cost criterion (e.g., minimizes the usage cost). The target candidate motion information corresponds to a unique index number in the candidate motion vector predictor MVP list.

[0546] Subsequently, a first motion vector and a second motion vector are determined based on the target candidate motion information. The target candidate motion information includes a first motion vector predictor (e.g., Mvp0_L0) corresponding to the first list (i.e., list0) and a second motion vector predictor (e.g., Mvp1_L1) corresponding to the second list (i.e., list1). For example, the first motion vector Mv0_L0 (forward) corresponding to the first list (list0) may be obtained based on the sum of the first motion vector predictor Mvp0_L0 and the motion vector difference MVD0. Similarly, the second motion vector Mv1_L1 may be obtained based on the sum of the second motion vector predictor Mvp1_L1 and the motion vector difference MVD1. Optionally, the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) here may be expressed as a first motion vector corresponding to a first direction (e.g., forward direction), and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1) may be expressed as a second motion vector corresponding to a second direction (e.g., backward direction).

[0547] Possible Case 2: The inter prediction mode is the merge mode.

[0548] The encoding device first builds a list of candidate motion information.

[0549] Specifically, the video encoding apparatus uses an inter prediction unit (also referred to as an inter prediction module) to construct a candidate motion information list (also referred to as a candidate motion vector list).

[0550] The video encoder then determines a target candidate motion information in the candidate motion information list according to a rate-distortion cost criterion.

[0551] Specifically, the target candidate motion information includes a target candidate motion vector, a reference frame index, and prediction direction indication information, where the prediction direction indication information is used to indicate that bidirectional prediction is applied, the target candidate motion vector includes a first motion vector corresponding to the first list and a second motion vector corresponding to the second list, and the reference frame index includes an index of the first reference frame corresponding to the first list and an index of the second reference frame corresponding to the second list.

[0552] The target candidate motion information may be referred to as the best candidate motion information or may have another name. In conclusion, the target candidate motion information is the best candidate determined in the candidate motion information list according to a rate-distortion cost criterion (e.g., the usage cost is minimized).

[0553] Possible Case 3: The inter prediction mode is skip mode. For related explanations, please refer to the description of merge mode in Case 2 above. The difference from merge mode is that skip mode does not require the transmission of residual samples.

[0554] 6B, in this embodiment of the present application, the current picture to which the current picture block belongs has two reference pictures, a previous reference picture and a subsequent reference picture, which are respectively a first reference picture (e.g., a forward reference picture or a forward reference frame, where "forward" may also be referred to as a first direction) and a second reference picture (e.g., a backward reference picture or a backward reference frame, where "backward" may also be referred to as a second direction). The reference samples corresponding to the first list are reference samples on the first reference picture, and the reference samples corresponding to the second list are reference samples on the second reference picture.

[0555] Optionally, obtaining a predicted sample value of the current block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a BDOF technique may include: deriving a modified motion vector of each sub-block of the current block based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a BDOF technique, so as to obtain a predicted sample value of the current block, and then performing motion compensation on each sub-block based on the modified motion vector to obtain a predicted sample value of each sub-block. For ease of understanding, an example is provided below for explanation.

[0556] The sample value of the reference sample value corresponding to the first list (list0) is I (0) (i,j), and the sample values ​​of the reference sample values ​​corresponding to the second list (list1) are I (1) (i,j). These two values ​​can be expressed as the forward and backward predicted values ​​I (k) (i,j), where i=0..cuW-1, j=0..cuH-1, and k can be 0 and 1. Here, I (k) (i, j) can be obtained through interpolation using an 8-tap interpolation filter, and sample values ​​at other positions (extended by one row and one column) can be obtained through interpolation using a bilinear interpolation filter. Note that the sample values ​​of the extended region may instead be obtained by using another method, for example, by using an 8-tap interpolation filter, or may be reference sample values ​​at integer sample positions. This is not limited here. Here, cuW and cuH are the width and height of the current picture block, respectively. According to the BDOF technology, obtaining predicted sample values ​​of the current picture block through prediction based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list may specifically include the following steps:

[0557] Step 1: Calculate the sum of absolute differences SAD between the sample values ​​of the reference sample values ​​corresponding to the first list and the sample values ​​of the reference sample values ​​corresponding to the second list, determine whether the SAD is less than the threshold value TH_CU, skip performing BDOF if the SAD is less than the threshold value TH_CU, and perform BDOF if the SAD is equal to or greater than the threshold value TH_CU. This step is optional.

[0558] The formula for calculating SAD is as follows:

number

[0559] The threshold TH_CU can be set to (1<<(BD-8+shift))*cuW*cuH, where shift is Max(2,14-BD).

[0560] Alternatively, TH_CU may be set to 1<<(BD-3+shift).

[0561] Optionally, whether to perform BDOF may instead be determined in other ways, and those other ways will not be listed one by one here.

[0562] When it is decided to perform BDOF, proceed to perform step 4.

[0563] Step 2: Calculate the horizontal and vertical gradient values ​​of the forward and backward predicted values ​​of the current CU.

[0564] Horizontal gradient value ∂I of forward and backward predicted values ​​of the current CU (k) / ∂x(i,j) and the vertical gradient value ∂I (k) / ∂y(i,j) is the horizontal gradient value ∂I at the sample position (i,j) of the reference sample value corresponding to the first list. (0) / ∂x(i,j) and the vertical gradient value ∂I (0) / ∂y(i,j) and the horizontal gradient value ∂I at sample position (i,j) of the reference sample value corresponding to the second list (1) / ∂x(i,j) and the vertical gradient value ∂I (1) / ∂y(i,j), where k is set to 0 and 1 sequentially. (k) / ∂x(i,j) and the vertical gradient value ∂I (k) ∂y(i,j) can be obtained using the following formula:

number

[0565] where i=0..cuW-1 and j=0..cuH-1.

[0566] In this step, a cuW×cuH prediction matrix for the current picture block can be obtained.

[0567] Step 4: Perform expansion on the forward and backward predicted values, horizontal gradient values, and vertical gradient values ​​of the current CU through padding.

[0568] Calculating the modified motion vector for a 4x4 sub-block involves calculating the forward and backward predicted values ​​I of the 6x6 region in which the 4x4 sub-block is located. (k) (x,y) and the horizontal gradient value ∂I of the forward and backward predicted values (k) / ∂x(i,j) and the vertical gradient value ∂I (k) / ∂y(i,j). Calculating the gradient value of a 6x6 region requires the forward and backward predicted values ​​of an 8x8 region. Therefore, when using an interpolation filter to obtain the forward and backward predicted values, in order to calculate the gradient value of (W+2)x(H+2), it is necessary to perform two row and two column extensions around to obtain a predicted sample block whose size is (W+4)x(H+4), where W is the width of the current CU and H is the height of the current CU. In order to reduce the complexity of BDOF, special processing can be performed on the boundary of the CU. The details are as follows:

[0569] First, a prediction value for the W × H region is obtained using an 8-tap filter, the region is extended by one row and one column around it, and a prediction value for the extended region is obtained using a bilinear filter, resulting in a prediction value for the (W + 2) × (H + 2) region.

[0570] Then, the gradient value of the W×H region can be obtained through calculation based on the predicted value of the (W+2)×(H+2) region according to the formula:

[0571] Finally, the gradient values ​​in the W×H region are expanded to the surroundings according to the padding method to obtain horizontal and vertical gradient values ​​in the (W+2)×(H+2) region, and the predicted values ​​in the W×H region are expanded to the surroundings to obtain predicted values ​​in the (W+2)×(H+2) region.

[0572] The padding is shown in Figure 6D. I for i=-1..cuW and j=-1..cuH (k) (i,j), ∂I (k) / ∂x(i,j) and ∂I (k) To obtain ∂y(i,j), padding is performed according to the method in Figure 6D. Specifically, a prediction matrix, horizontal gradient matrix, and vertical gradient matrix of (cuW+2) × (cuH+2) are obtained.

[0573] Step 5: Derive the modified motion vectors for each 4x4 sub-block, and then perform weighting.

[0574] For each 4x4 sub-block, a modified motion vector (v x ,v y ) is obtained. Specifically, the modified motion vector (v x ,v y ) is obtained by applying BDOF to a 6×6 window Ω around the sub-block to minimize the L0 and L1 predictions. x ,v y ) is derived according to the following formula:

number

[0575] where:

number

[0576] S1, S2, S3, S5, and S6 are calculated according to the following formulas:

number

[0577] where:

number

[0578] where I (0) (i, j) is the forward predicted value of the sample position (i, j) in the current CU, that is, the sample value of the sample position (i, j) in the reference sample value of the current CU corresponding to the first list, and I (1) (i,j) is the backward predicted value of the sample position (i,j) in the current CU, i.e., the sample value of the sample position (i,j) in the reference sample values ​​of the current CU that correspond to the second list.

[0579] After the modified motion vector is obtained according to equation (2-3), the final predicted value of each sample in the current picture block, i.e., the sample value in the predicted block of the current picture block, can be determined according to the following equation:

number

[0580] where shift and offset are 15-BD and 1<<(14-BD)+2·(1<<13). rnd(.) is the rounding function (rounding off). pred BIO (i,j) is the predicted sample value at sample position (i,j) of the current picture block.

[0581] Step S601: When the size of the current picture block is a second preset size, the encoding device determines predicted sample values ​​of the current picture block using a first processing manner based on reference sample values ​​corresponding to the first list and reference sample values ​​corresponding to the second list.

[0582] Specifically, the first processing manner includes not using BDOF (also referred to as bi-directional optical flow (BIO)). In other words, when the size of the current picture block is the second preset size, it is not allowed to use BDOF to determine predicted sample values ​​of the current picture block based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list. The size of the current picture block is usually represented by width and height (shortened as W×H), and the width and height are measured in samples (samples or pixels). The second preset sizes include 8×8 (the first 8 indicates that the width of the current picture block is 8 samples, and the second 8 indicates that the height of the current picture block is 8 samples), 4×N (4 indicates that the width of the current picture block is 4 samples, and N indicates that the height of the current picture block is N samples; other sizes can be deduced by analogy), 8×16, or 16×8. N is a power of 2 and is greater than or equal to 8. For example, the value of N can be 16, 32, 64, etc.

[0583] In this embodiment of the present application, the reference sample value corresponding to the first list and the reference sample value corresponding to the second list are obtained through prediction based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0) and the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1), respectively. For example, the reference sample value corresponding to the first list (i.e., list0) can be determined based on the first motion vector (e.g., Mv0_L0) corresponding to the first list (i.e., list0), and the reference sample value corresponding to the second list (i.e., list1) can be determined based on the second motion vector (e.g., Mv1_L1) corresponding to the second list (i.e., list1).

[0584] It should be noted that the first and second motion vectors are determined in different manners in different inter prediction modes. For details on how the first and second motion vectors are determined in AMVP mode, merge mode, and skip mode, please refer to the description in step S600. The details will not be described again here.

[0585] It should be understood that the current picture block (also referred to as the current block) here can be understood as a picture block being processed, for example, in an encoding process, the current picture block is a picture block being encoded (encoding block).

[0586] In addition, the above-mentioned cases in which the second preset size includes 8×8, 4×N, 8×16, or 16×8 can also be limited to the second preset size being 8×8, 4×N, 8×16, or 16×8. In other words, BDOF may be prohibited from being used for one of the above-listed sizes 8×8, 4×N, 8×16, and 16×8, or may be prohibited from being used for multiple sizes among these. An example is used below for explanation.

[0587] Example 1: The second preset size includes (or is) 8x8. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 8x8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is 8x8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0588] Example 2: The second preset size includes (or is) 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or more, when the size of the current picture block is 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0589] Example 3: The second preset size includes (or is) 8×8 or 4×N. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8 and 4×N. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: where N is 8 or greater, when the size of the current picture block is either 8×8 or 4×N, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0590] Example 4: The second preset size includes (or is) 8×8, 4×N, 8×16, or 16×8, where N is greater than or equal to 8. In other words, in this embodiment of the application, BDOF is prohibited from being used for sizes 8×8, 4×N, 8×16, and 16×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is any one of 8×8, 4×N, 8×16, and 16×8, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list.

[0591] Example 5: The second preset size includes (or is) N×8, where N is greater than or equal to 4. In other words, in this embodiment of the application, BDOF is prohibited from being used for the size N×8. In this case, when the size of the current picture block is the second preset size, obtaining the predicted sample value of the current picture block through prediction in a first processing manner based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list can be specifically expressed as follows: when the size of the current picture block is N×8, obtaining the predicted sample value of the current picture block through prediction in a ...

Claims

1. 1. An encoding method comprising: When a plurality of preset conditions are satisfied, a bidirectional optical flow BDOF process is performed on a current picture block to obtain predicted sample values ​​of the current picture block; when the size of the current picture block is a second preset size, the predicted sample values ​​of the current picture block are determined in a first processing manner based on reference sample values ​​corresponding to a first list and reference sample values ​​corresponding to a second list (S600, S601); The plurality of preset conditions include: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1, where predFlagL0 and predFlagL1 both equal to 1 indicate that bidirectional prediction is applied to the current picture block; MotionModelIdc is equal to 0, where MotionModelIdc equal to 0 indicates that the motion model for motion compensation of the current picture block is translational motion; a merge_subblock_flag included in a syntax element related to the current picture block is equal to 0, where a merge_subblock_flag equal to 0 indicates that a sub-block merge mode is not applied to the current picture block; The syntax element includes a sym_mvd_flag equal to 0, where a sym_mvd_flag equal to 0 indicates that an mvd_coding syntax structure exists for the current picture block; The bcwIdx included in the syntax element is equal to 0, where bcwIdx indicates a bidirectional prediction weight index for the current picture block; cIdx is equal to 0, where cIdx represents the color component index of the current picture block; The height H of the current picture block is equal to or greater than 8, the width W of the current picture block is equal to or greater than 8, and the product of the width W of the current picture block and the height H of the current picture block is greater than 128, where W and H are each 2. ｎ where n is an integer, and the syntax element includes luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction; the first processing method is not a bidirectional optical flow (BDOF) method, and the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list are obtained through prediction based on a first motion vector corresponding to the first list and a second motion vector corresponding to the second list, respectively; the second preset size is 8x8; or the second preset size has 4×N, where N is 8 or greater; or the second preset size has 8x8 or 4xN, where N is 8 or greater; or the second preset size has 4×N, where N is 4 or greater; obtaining reconstructed sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block (S602); encoding a syntax element into a bitstream to be transmitted (S603), the syntax element being used to indicate at least that bidirectional prediction is applied to the current picture block; A method comprising:

2. determining predicted sample values ​​of the current picture block in a first processing manner based on reference sample values ​​corresponding to a first list and reference sample values ​​corresponding to a second list, performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain the predicted sample values ​​of the current picture block; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a joint inter / intra prediction CIIP technique; 2. The method of claim 1, comprising:

3. 3. The method of claim 1, wherein the syntax element includes an index of target candidate motion information, the target candidate motion information including a target candidate motion vector, a reference frame index, and prediction direction indication information, the prediction direction indication information being used to indicate that bidirectional prediction is applied to the current picture block, the target candidate motion vector including the first motion vector corresponding to the first list and the second motion vector corresponding to the second list, and the reference frame index including an index of a first reference frame corresponding to the first list and an index of a second reference frame corresponding to the second list.

4. the syntax element includes a first index, a second index, prediction direction indication information, and a motion vector difference MVD, the first index is used to indicate a target candidate motion vector predictor, the target candidate motion vector predictor includes a first motion vector predictor corresponding to the first list and a second motion vector predictor corresponding to the second list, and the motion vector difference MVD includes a first MVD corresponding to the first list and / or a second MVD corresponding to the second list; the second index is used to indicate a reference frame of the current picture block, the reference frame having a first reference frame corresponding to the first list and a second reference frame corresponding to the second list; the prediction direction indication information is used to indicate that bidirectional prediction is applied to the current picture block; the first motion vector corresponding to the first list is obtained based on the first motion vector predictor corresponding to the first list and the first MVD corresponding to the first list; the second motion vector corresponding to the second list is obtained based on the second motion vector predictor corresponding to the second list and the second MVD corresponding to the second list; 3. The method according to claim 1 or 2.

5. The method further comprises: determining sample residuals based on sample values ​​of the current picture block and the predicted sample values ​​of the current picture block; and the step of encoding syntax elements into a transmitted bitstream comprises encoding the syntax elements and the sample residuals into the transmitted bitstream.

3. The method according to claim 1 or 2.

6. 1. A decoding method comprising: Parsing a bitstream to obtain a syntax element (S700), the syntax element being used to indicate that bidirectional prediction is applied to a current picture block; When a plurality of preset conditions are satisfied, performing a bidirectional optical flow BDOF process on the current picture block to obtain predicted sample values ​​of the current picture block, and when the size of the current picture block is a second preset size, determining the predicted sample values ​​of the current picture block in a first processing manner based on reference sample values ​​corresponding to a first list and reference sample values ​​corresponding to a second list (S701, S702); The plurality of preset conditions include: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1, where predFlagL0 and predFlagL1 both equal to 1 indicate that bidirectional prediction is applied to the current picture block; MotionModelIdc is equal to 0, where MotionModelIdc equal to 0 indicates that the motion model for motion compensation of the current picture block is translational motion; a merge_subblock_flag included in a syntax element related to the current picture block is equal to 0, where a merge_subblock_flag equal to 0 indicates that a sub-block merge mode is not applied to the current picture block; The syntax element includes a sym_mvd_flag equal to 0, where a sym_mvd_flag equal to 0 indicates that an mvd_coding syntax structure exists for the current picture block; The bcwIdx included in the syntax element is equal to 0, where bcwIdx indicates a bidirectional prediction weight index for the current picture block; cIdx is equal to 0, where cIdx represents the color component index of the current picture block; The height H of the current picture block is equal to or greater than 8, the width W of the current picture block is equal to or greater than 8, and the product of the width W of the current picture block and the height H of the current picture block is greater than 128, where W and H are each 2. ｎ where n is an integer, and the syntax element includes luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction; the first processing method is not a bidirectional optical flow (BDOF) method, and the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list are obtained through prediction based on a first motion vector corresponding to the first list and a second motion vector corresponding to the second list, respectively; the second preset size is 8x8; or the second preset size has 4×N, where N is 8 or greater; or the second preset size has 8x8 or 4xN, where N is 8 or greater; or the second preset size has 4×N, where N is 4 or greater; determining reconstructed sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block (S703); A method comprising:

7. determining predicted sample values ​​of the current picture block through prediction in a first processing manner based on reference sample values ​​corresponding to a first list and reference sample values ​​corresponding to a second list, performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain the predicted sample values ​​of the current picture block; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining the predicted sample values ​​of the current picture block through prediction based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list according to a joint inter / intra prediction CIIP technique; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a decoder-side motion vector refinement DMVR technique; 7. The method of claim 6, comprising:

8. 8. The method of claim 6 or 7, wherein the syntax element includes an index of target candidate motion information, the target candidate motion information including a target candidate motion vector, a reference frame index, and prediction direction indication information, the prediction direction indication information being used to indicate that bidirectional prediction is applied to the current picture block, the target candidate motion vector including the first motion vector corresponding to the first list and the second motion vector corresponding to the second list, and the reference frame index including a first reference frame index corresponding to the first list and a second reference frame index corresponding to the second list.

9. the syntax element includes a first index, a second index, prediction direction indication information, and a motion vector difference MVD, the first index is used to indicate a target candidate motion vector predictor, the target candidate motion vector predictor includes a first motion vector predictor corresponding to the first list and a second motion vector predictor corresponding to the second list, and the motion vector difference MVD includes a first MVD corresponding to the first list and / or a second MVD corresponding to the second list; the second index is used to indicate a reference frame of the current picture block, the reference frame having a first reference frame corresponding to the first list and a second reference frame corresponding to the second list; the prediction direction indication information is used to indicate that bidirectional prediction is applied to the current picture block; the first motion vector corresponding to the first list is obtained based on the first motion vector predictor corresponding to the first list and the first MVD corresponding to the first list; the second motion vector corresponding to the second list is obtained based on the second motion vector predictor corresponding to the second list and the second MVD corresponding to the second list; 8. The method according to claim 6 or 7.

10. the step of parsing the bitstream to obtain syntax elements comprises parsing the bitstream to obtain sample residuals and the syntax elements; determining reconstructed sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block comprises determining the reconstructed sample values ​​of the current picture block based on the sample residuals and the predicted sample values ​​of the current picture block.

8. The method according to claim 6 or 7.

11. 1. An encoding device, comprising: an inter prediction unit (244) configured to perform a bidirectional optical flow BDOF process on a current picture block to obtain predicted sample values ​​of the current picture block when a plurality of preset conditions are satisfied, and to determine the predicted sample values ​​of the current picture block in a first processing manner based on reference sample values ​​corresponding to a first list and reference sample values ​​corresponding to a second list when a size of the current picture block is a second preset size; The plurality of preset conditions include: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1, where predFlagL0 and predFlagL1 both equal to 1 indicate that bidirectional prediction is applied to the current picture block; MotionModelIdc is equal to 0, where MotionModelIdc equal to 0 indicates that the motion model for motion compensation of the current picture block is translational motion; a merge_subblock_flag included in a syntax element related to the current picture block is equal to 0, where a merge_subblock_flag equal to 0 indicates that a sub-block merge mode is not applied to the current picture block; The syntax element includes a sym_mvd_flag equal to 0, where a sym_mvd_flag equal to 0 indicates that an mvd_coding syntax structure exists for the current picture block; The bcwIdx included in the syntax element is equal to 0, where bcwIdx indicates a bidirectional prediction weight index for the current picture block; cIdx is equal to 0, where cIdx represents the color component index of the current picture block; The height H of the current picture block is equal to or greater than 8, the width W of the current picture block is equal to or greater than 8, and the product of the width W of the current picture block and the height H of the current picture block is greater than 128, where W and H are each 2. ｎ where n is an integer, and the syntax element includes luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction; the first processing method is not a bidirectional optical flow (BDOF) method, and the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list are obtained through prediction based on a first motion vector corresponding to the first list and a second motion vector corresponding to the second list, respectively; the second preset size is 8x8; or the second preset size has 4×N, where N is 8 or greater; or the second preset size has 8x8 or 4xN, where N is 8 or greater; or an inter prediction unit having the second preset size of 4×N, where N is 4 or greater; a reconstruction unit (214) configured to obtain reconstructed sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block; an entropy encoding unit (270) configured to encode a syntax element into a bitstream to be transmitted, the syntax element being used to indicate at least that bidirectional prediction is applied to the current picture block; An encoding device comprising:

12. In the process of determining the predicted sample values ​​of the current picture block in the first processing manner based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list, the inter prediction unit (244) specifically: performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain the predicted sample values ​​of the current picture block; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a joint inter / intra prediction CIIP technique; 12. The encoding device according to claim 11, configured to:

13. 13. The encoding device of claim 11 or 12, wherein the syntax element includes an index of target candidate motion information, the target candidate motion information including a target candidate motion vector, a reference frame index, and prediction direction indication information, the prediction direction indication information being used to indicate that bidirectional prediction is applied to the current picture block, the target candidate motion vector including the first motion vector corresponding to the first list and the second motion vector corresponding to the second list, and the reference frame index including a first reference frame index corresponding to the first list and a second reference frame index corresponding to the second list.

14. the syntax element includes a first index, a second index, prediction direction indication information, and a motion vector difference MVD, the first index is used to indicate a target candidate motion vector predictor, the target candidate motion vector predictor includes a first motion vector predictor corresponding to the first list and a second motion vector predictor corresponding to the second list, and the motion vector difference MVD includes a first MVD corresponding to the first list and / or a second MVD corresponding to the second list; the second index is used to indicate a reference frame of the current picture block, the reference frame having a first reference frame corresponding to the first list and a second reference frame corresponding to the second list; the prediction direction indication information is used to indicate that bidirectional prediction is applied to the current picture block; the first motion vector corresponding to the first list is obtained based on the first motion vector predictor corresponding to the first list and the first MVD corresponding to the first list; the second motion vector corresponding to the second list is obtained based on the second motion vector predictor corresponding to the second list and the second MVD corresponding to the second list; 13. The encoding device according to claim 11 or 12.

15. The encoding device further comprises: a residual calculation unit (204) configured to determine sample residuals based on sample values ​​of the current picture block and the predicted sample values ​​of the current picture block; and the entropy coding unit is specifically configured to encode the syntax elements and the sample residuals into the bitstream to be transmitted; 13. The encoding device according to claim 11 or 12.

16. A decoding device, comprising: an entropy decoding unit (304) configured to parse the bitstream to obtain a syntax element, the syntax element being used to indicate at least that bidirectional prediction is applied to a current picture block; and an inter prediction unit (344) configured to perform a bidirectional optical flow BDOF process on the current picture block to obtain predicted sample values ​​of the current picture block when a plurality of preset conditions are satisfied, and to determine the predicted sample values ​​of the current picture block in a first processing manner based on reference sample values ​​corresponding to a first list and reference sample values ​​corresponding to a second list when a size of the current picture block is a second preset size; The plurality of preset conditions include: The prediction direction indication information predFlagL0 corresponding to the first list (List1) is equal to 1, and the prediction direction indication information predFlagL1 corresponding to the second list (list0) is equal to 1, where predFlagL0 and predFlagL1 both equal to 1 indicate that bidirectional prediction is applied to the current picture block; MotionModelIdc is equal to 0, where MotionModelIdc equal to 0 indicates that the motion model for motion compensation of the current picture block is translational motion; a merge_subblock_flag included in a syntax element related to the current picture block is equal to 0, where a merge_subblock_flag equal to 0 indicates that a sub-block merge mode is not applied to the current picture block; The syntax element includes a sym_mvd_flag equal to 0, where a sym_mvd_flag equal to 0 indicates that an mvd_coding syntax structure exists for the current picture block; The bcwIdx included in the syntax element is equal to 0, where bcwIdx indicates a bidirectional prediction weight index for the current picture block; cIdx is equal to 0, where cIdx represents the color component index of the current picture block; The height H of the current picture block is equal to or greater than 8, the width W of the current picture block is equal to or greater than 8, and the product of the width W of the current picture block and the height H of the current picture block is greater than 128, where W and H are each 2. ｎ where n is an integer, and the syntax element includes luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] both equal to 0, where luma_weight_l0_flag equal to 0 indicates that there is no weighting factor for the luma component of list0 prediction, and luma_weight_l1_flag equal to 0 indicates that there is no weighting factor for the luma component of list1 prediction; the first processing method is not a bidirectional optical flow (BDOF) method, and the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list are obtained through prediction based on a first motion vector corresponding to the first list and a second motion vector corresponding to the second list, respectively; the second preset size is 8x8; or the second preset size has 4×N, where N is 8 or greater; or the second preset size has 8x8 or 4xN, where N is 8 or greater; or an inter prediction unit having the second preset size of 4×N, where N is 4 or greater; a reconstruction unit (314) configured to determine reconstructed sample values ​​of the current picture block based on at least the predicted sample values ​​of the current picture block; A decoding device comprising:

17. In the process of determining the predicted sample values ​​of the current picture block through prediction in the first processing manner based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list, the inter prediction unit (344) specifically: performing a weighting calculation on sample values ​​having the same position in the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list to obtain the predicted sample values ​​of the current picture block; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a merge-with-motion-vector-difference (MMVD) technique; or Obtaining the predicted sample values ​​of the current picture block through prediction based on the reference sample values ​​corresponding to the first list and the reference sample values ​​corresponding to the second list according to a joint inter / intra prediction CIIP technique; or Obtaining the predicted sample value of the current picture block through prediction based on the reference sample value corresponding to the first list and the reference sample value corresponding to the second list according to a decoder-side motion vector refinement DMVR technique; 17. The decoding device according to claim 16, configured to:

18. 18. The decoding device of claim 16 or 17, wherein the syntax element includes an index of target candidate motion information, the target candidate motion information including a target candidate motion vector, a reference frame index, and prediction direction indication information, the prediction direction indication information being used to indicate that bidirectional prediction is applied to the current picture block, the target candidate motion vector including the first motion vector corresponding to the first list and the second motion vector corresponding to the second list, and the reference frame index including an index of a first reference frame corresponding to the first list and an index of a second reference frame corresponding to the second list.

19. the syntax element includes a first index, a second index, prediction direction indication information, and a motion vector difference MVD, the first index is used to indicate a target candidate motion vector predictor, the target candidate motion vector predictor includes a first motion vector predictor corresponding to the first list and a second motion vector predictor corresponding to the second list, and the motion vector difference MVD includes a first MVD corresponding to the first list and / or a second MVD corresponding to the second list; the second index is used to indicate a reference frame of the current picture block, the reference frame having a first reference frame corresponding to the first list and a second reference frame corresponding to the second list; the prediction direction indication information is used to indicate that bidirectional prediction is applied to the current picture block; the first motion vector corresponding to the first list is obtained based on the first motion vector predictor corresponding to the first list and the first MVD corresponding to the first list; the second motion vector corresponding to the second list is obtained based on the second motion vector predictor corresponding to the second list and the second MVD corresponding to the second list; 18. A decoding device according to claim 16 or 17.

20. The entropy decoding unit (304) is specifically configured to parse the bitstream to obtain sample residuals and the syntax elements; the reconstruction unit (314) is specifically configured to determine the reconstructed sample values ​​of the current picture block based on the sample residuals and the predicted sample values ​​of the current picture block; 18. A decoding device according to claim 16 or 17.

21. 6. A computer-readable storage medium storing a computer program which, when executed on a processor, performs the method of any one of claims 1 to 5.

22. 11. A computer-readable storage medium storing a computer program which, when executed on a processor, performs the method of any one of claims 6 to 10.

23. An encoder comprising a processing circuit for carrying out the method according to any one of claims 1 to 5.

24. A decoder comprising processing circuitry for carrying out the method according to any one of claims 6 to 10.

Images