A method, computing device and medium for video encoding

By performing mode information classification and weighted prediction processing on video codec blocks, inter-frame mode tool is selected to solve the delay and complexity problems when the inter-frame mode tool is enabled in VVC design, and better encoding and decoding efficiency and Delay balance.

CN116886925BActive Publication Date: 2024-11-08BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310971968.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-02-08
Filing Date
2020-02-08
Publication Date
2024-11-08
Estimated Expiration
2040-02-08

AI Technical Summary

Technical Problem

The existing video codec standard VVC design has problems of increased latency and complexity when implementing the balance between inter-mode codec tools BDOF and DMVR, resulting in the failure to achieve the optimal codec efficiency in some scenarios. and delay balance.

Method used

Decide whether to disable DMVR applications by classifying based on the schema information of the current block, dividing it into DMVR classes or BDOF classes, and applying only one of the tools, or enabling weighted prediction and averaging of different weights when specific conditions are met, Thus, bidirectional optical flow and decoder-side motion vector refinement are selectively applied on the inter-mode codec block.

Benefits of technology

A better balance between maintaining codec efficiency and delay is achieved, unnecessary complexity and delay increase is avoided, while improving codec gain in appropriate scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116886925B_ABST
    Figure CN116886925B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method, computing device and medium for video encoding. The method includes: obtaining multiple blocks segmented from a video picture, wherein each of the multiple blocks is encoded by an intra-frame prediction mode or an inter-frame prediction mode; for a current block in the multiple blocks: in response to the current block being eligible for both decoder-side motion vector refinement (DMVR) application and bidirectional optical flow (BDOF) application, determining whether a predefined criterion including the sum of absolute differences (SAD) or the sum of squared differences (SSD) between list 0 prediction samples and list 1 prediction samples of the current block is less than a predefined threshold number is satisfied; in response to the predefined criterion being satisfied, avoiding applying BDOF to sub-blocks in the current block; and outputting the prediction mode information of the current block through a video bitstream.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of Chinese patent application No. 202080013232.4, which is a Chinese national phase application of international patent application PCT / US2020 / 017382 filed on February 8, 2020, and which claims priority to U.S. patent application No. 62 / 803,417 filed on February 8, 2019. Technical Field

[0002] The present disclosure relates generally to video coding and compression. More specifically, the present disclosure relates to systems and methods for performing video coding by selectively applying bidirectional optical flow and decoder-side motion vector refinement to inter-mode codec blocks. Background Art

[0003] This section provides background information related to the present disclosure. The information contained in this section should not necessarily be construed as prior art.

[0004] Any of a variety of video codec techniques may be used to compress video data. Video coding may be performed according to one or more video codec standards. Some illustrative video codec standards include Versatile Video Codec (VVC), Joint Exploration Test Model (JEM) codec, High Efficiency Video Codec (H.265 / HEVC), Advanced Video Codec (H.264 / AVC), and Moving Picture Experts Group (MPEG) codec.

[0005] Video codecs typically use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that exploit the redundancy inherent in video images or sequences. One goal of video codec technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing video quality degradation.

[0006] Prediction methods used in video codecs typically include performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data, and these prediction methods are typically associated with block-based video codecs.

[0007] In block-based video coding, the input video signal is processed block by block. For each block (also referred to as a codec unit (CU)), spatial prediction and / or temporal prediction can be performed.

[0008] Spatial prediction (also called "intra prediction") uses pixels from samples of already coded neighboring blocks in the same video picture / slice (which are called reference samples) to predict the current block. Spatial prediction reduces the spatial redundancy inherent in video signals.

[0009] During the decoding process, the video bitstream is first entropy decoded at the entropy decoding unit. The codec mode and prediction information are sent to the spatial prediction unit (in the case of intra-frame coding) or the temporal prediction unit (in the case of inter-frame coding) to form a prediction block. The residual transform coefficients are sent to the inverse quantization unit and the inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then added. The reconstructed block may be further loop filtered before it is stored in the reference picture storage device. The reconstructed video in the reference picture storage device is then sent out to drive the display device and is used to predict future video blocks.

[0010] In newer video codec standards, such as the current VVC design, new inter-mode codec tools have been introduced, such as bidirectional optical flow (BDOF) and decoder-side motion vector refinement (DMVR). Such new inter-mode codec tools generally help improve the efficiency of motion compensated prediction and thus improve codec gain. However, such improvements may come at the cost of increased complexity and latency.

[0011] In order to achieve the right balance between the improvements and costs associated with new inter-mode tools, current VVC designs now already impose constraints on when new inter-mode codec tools (such as DMVR and BDOF) are enabled on inter-mode codec blocks.

[0012] However, the constraints that exist in current VVC designs do not necessarily achieve the best balance between improvement and cost. On the one hand, the constraints that exist in current VVC designs allow both DMVR and BDOF to be applied to the same inter-mode codec block, which increases latency because there is a dependency between the operations of the two tools. On the other hand, the constraints that exist in current VVC designs may be too loose for DMVR in some cases, resulting in unnecessary increase in complexity and latency, and not loose enough for BDOF in some cases, resulting in missed opportunities for more codec gains. Summary of the invention

[0013] This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.

[0014] According to a first aspect of the present disclosure, a video encoding and decoding method is performed at a computing device having one or more processors and a memory storing a plurality of programs to be executed by the one or more processors. The method includes: using a predefined criterion based on the mode information of the current block, classifying a current block that is eligible for both DMVR application and BDOF application based on a plurality of predefined conditions into one of two predefined classes, namely, a DMVR class and a BDOF class. The method further includes: using the classification of the current block to apply DMVR or BDOF to the current block, but not both.

[0015] According to a second aspect of the present disclosure, a video encoding and decoding method is performed at a computing device having one or more processors and a memory storing a plurality of programs to be executed by the one or more processors. The method includes: determining whether to enable weighted prediction for a current block that is eligible for DMVR encoding and decoding based on a plurality of predefined conditions. The method further includes: determining whether different weights are used when averaging list 0 prediction value samples and list 1 prediction value samples for the current block. The method further includes: determining whether to disable DMVR application to the current block based on the two determinations.

[0016] According to a third aspect of the present disclosure, a video encoding and decoding method is performed at a computing device, the computing device having one or more processors and a memory, the memory storing a plurality of programs to be executed by the one or more processors. The method includes: when the current block is encoded and decoded in a sub-block merge mode, enabling BDOF application to the current block.

[0017] According to a fourth aspect of the present application, a computing device includes one or more processors, a memory, and a plurality of programs stored in the memory. When the programs are executed by the one or more processors, the computing device performs the operations described in the first three aspects of the present application.

[0018] According to a fifth aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by a computing device having one or more processors. When these programs are executed by the one or more processors, the computing device performs the operations described above in the first three aspects of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In the following, multiple groups of illustrative, non-limiting embodiments of the present disclosure will be described in conjunction with the accompanying drawings. A person of ordinary skill in the relevant art may implement variations of structure, method, or function based on the examples presented herein, and such variations are all included within the scope of the present disclosure. In the absence of conflicts, the teachings of different embodiments may, but need not, be combined with each other.

[0020] Figure 1 is a block diagram illustrating an illustrative block-based hybrid video encoder that can be used in conjunction with many video codec standards.

[0021] Figure 2 is a block diagram illustrating an illustrative video decoder that can be used in conjunction with many video codec standards.

[0022] Figure 3 is an illustration of block partitioning in multiple types of tree structures that can be used in conjunction with many video codec standards.

[0023] Figure 4 is a diagram of the bidirectional optical flow (BDOF) process.

[0024] Figure 5 is an illustration of two-sided matching used in decoder-side motion vector refinement (DMVR).

[0025] Figure 6 is a flow chart illustrating the operation of the first aspect of the present disclosure.

[0026] Figure 7 is a flow chart illustrating the operation of the second aspect of the present disclosure.

[0027] Figure 8 is a flow chart illustrating the operation of the third aspect of the present disclosure. DETAILED DESCRIPTION

[0028] The terms used in this disclosure are intended to illustrate specific examples rather than to limit the disclosure. The singular forms "a", "an", and "the" used in this disclosure and the appended claims also refer to the plural forms unless the context clearly implies otherwise. It should be understood that the term "and / or" as used herein refers to any or all possible combinations of one or more associated enumerated items.

[0029] It should be understood that although the terms "first", "second", "third", etc. may be used herein to describe various information, such information should not be limited by these terms. These terms are merely used to distinguish one type of information from another type of information. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information; and similarly, the second information may also be referred to as the first information. As used herein, the term "if" may be understood to mean "when..." or "upon..." or "in response to...", depending on the context.

[0030] Throughout this specification, references to "one embodiment," "another embodiment," etc. in the singular or plural mean that one or more specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present disclosure. Thus, the phrases "in one embodiment" or "in one embodiment," "in another embodiment," etc. appearing in various places throughout this specification in the singular or plural are not necessarily all referring to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable manner.

[0031] In concept, many video codec standards are similar, including those mentioned previously in the background section. For example, almost all video codec standards use block-based processing and share similar video codec block diagrams to achieve video compression.

[0032] Figure 1 A block diagram of an illustrative block-based hybrid video encoder 100 that can be used in conjunction with many video codec standards is shown. In the encoder 100, a video frame is segmented into multiple video blocks for processing. For each given video block, a prediction is formed based on an inter-frame prediction method or an intra-frame prediction method. In inter-frame prediction, one or more prediction values ​​are formed through motion estimation and motion compensation based on pixels from a previously reconstructed frame. In intra-frame prediction, multiple prediction values ​​are formed based on reconstructed pixels in the current frame. Through mode decision, the best prediction value can be selected to predict the current block.

[0033] The prediction residual representing the difference between the current video block and its predicted value is sent to the transform circuitry 102. The transform coefficients are then sent from the transform circuitry 102 to the quantization circuitry 104 for entropy reduction. The quantized coefficients are then fed to the entropy encoding and decoding circuitry 106 to generate a compressed video bitstream. Figure 1 As shown, prediction-related information 110 (such as video block segmentation information, motion vectors, reference picture indexes, and intra-frame prediction modes) from the inter-frame prediction circuitry and / or the intra-frame prediction circuitry 112 is also fed through the entropy encoding and decoding circuitry 106 and saved into a compressed video bitstream 114.

[0034] In encoder 100, decoder related circuitry is also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by inverse quantization circuitry 116 and inverse transform circuitry 118. The reconstructed prediction residual is combined with block prediction value 120 to generate unfiltered reconstructed pixels for the current video block.

[0035] Temporal prediction (also known as "inter-frame prediction" or "motion compensated prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs) that indicate the amount and direction of motion between the current CU and its temporal reference. In addition, if multiple reference pictures are supported, a reference picture index is additionally sent that identifies which reference picture in the reference picture storage device the temporal prediction signal comes from.

[0036] After performing spatial prediction and / or temporal prediction, the intra / inter mode decision circuit system 121 in the encoder 100 selects the best prediction mode, for example based on a rate-distortion optimization method. The block prediction value 120 is then subtracted from the current video block; and the resulting prediction residual is decorrelated using the transform circuit system 102 and the quantization circuit system 104. The resulting quantized residual coefficients are inversely quantized by the inverse quantization circuit system 116 and inversely transformed by the inverse transform circuit system 118 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Further loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO) and / or adaptive loop filter (ALF), may be applied to the reconstructed CU before it is placed in a reference picture storage device of a picture buffer 117 and used to encode and decode future video blocks. To form the output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are sent to the entropy coding unit 106 for further compression and packing to form a bitstream.

[0037] For example, a deblocking filter is available in the current version of VVC as well as AVC, HEVC. In HEVC, an additional loop filter called SAO (Sample Adaptive Offset) is defined to further improve coding efficiency. In the current version of the VVC standard, yet another loop filter called ALF (Adaptive Loop Filter) is being actively studied and is likely to be included in the final standard.

[0038] These loop filter operations are optional. Performing these operations helps improve coding efficiency and visual quality. These operations can also be turned off according to the decision made by the encoder 100 to save computational complexity.

[0039] It should be noted that if these filter options are turned on by the encoder 100, intra prediction is typically based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels.

[0040] Figure 2 is a block diagram illustrating an illustrative video decoder 200 that can be used in conjunction with many video codec standards. The decoder 200 is located at Figure 1 The reconstruction of the relevant part in the encoder 100 is similar. In the decoder 200 ( Figure 2 ), the incoming video bitstream 201 is first decoded by entropy decoding 202 to obtain quantization coefficient levels and prediction related information. The quantization coefficient levels are then processed by inverse quantization 204 and inverse transform 206 to obtain reconstructed prediction residuals. The block prediction value mechanism implemented in the intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residuals from the inverse transform 206 with the prediction output generated by the block prediction value mechanism using an adder 214.

[0041] The reconstructed block may further pass through a loop filter 209 before it is stored in a picture buffer 213 used as a reference picture storage device. The reconstructed video in the picture buffer 213 may then be sent out to drive a display device and used to predict future video blocks. When the loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to obtain a final reconstructed video output 222.

[0042] In video codec standards such as HEVC, blocks can be partitioned based on quadtrees. In newer video codec standards such as the current VVC, more partitioning methods are used, and a codec tree unit (CTU) can be divided into multiple CUs based on quadtrees, binary trees, or ternary trees to adapt to different local characteristics. In most codec modes in the current VVC, the separation of CU, prediction unit (PU), and transform unit (TU) does not exist, and each CU has been used as a basic unit for both prediction and transformation without further partitioning. However, in some specific codec modes such as intra-frame sub-partitioning codec mode, each CU may still contain multiple TUs. In a multi-type tree structure, a CTU is first partitioned by a quadtree structure. Then, each quadtree leaf node can be further partitioned by a binary tree structure and a ternary tree structure.

[0043] Figure 3 Five types of partitioning used in the current VVC are shown, namely, quadruple partitioning 301, horizontal binary partitioning 302, vertical binary partitioning 303, horizontal ternary partitioning 304, and vertical ternary partitioning 305. In the case of using a multi-type tree structure, a CTU is first partitioned by a quadtree structure. Then, each quadtree leaf node can be further partitioned by a binary tree structure and a ternary tree structure.

[0044] use Figure 3 One or more of the exemplary block partitions 301, 302, 303, 304, or 305 may be used Figure 1 . The configuration shown in is used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra-frame prediction") uses pixels from samples of already coded neighboring blocks in the same video picture / slice (which are called reference samples) to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal.

[0045] In newer video codec standards, such as the now current VVC, new inter-mode codec tools have been introduced, and two examples of these new inter-mode codec tools are: Bidirectional Optical Flow (BDOF) and Decoder-side Motion Vector Refinement (DMVR).

[0046] Conventional bidirectional prediction in video codecs is a simple combination of two temporal prediction blocks obtained from an already reconstructed reference picture. However, due to the limitations of block-based motion compensation, there may still be residual small motion that can be observed between the samples of the two prediction blocks, thus reducing the efficiency of motion compensated prediction. To address this problem, BDOF is applied in the current VVC design to reduce the impact of this motion for each sample inside a block.

[0047] Figure 4 is an illustration of the BDOF process. BDOF is a sample-wise motion refinement performed on top of block-based motion compensated prediction when bidirectional prediction is used. The motion refinement for each 4×4 sub-block 401 is calculated by minimizing the difference between reference picture list 0 (L0) and reference picture list 1 (L1) prediction samples 402 and 403 after applying BDOF inside a 6×6 window around the sub-block. Based on the motion refinement thus derived, the final bidirectional prediction samples of the CU are calculated by interpolating the L0 / L1 prediction samples along the motion trajectory based on the optical flow model.

[0048] DMVR is a bi-directional prediction technique for merging blocks with two initially signaled MVs that can be further refined by using bilateral matching prediction.

[0049] Figure 5is an illustration of bilateral matching used in DMVR. Bilateral matching is used to obtain the motion information of the current CU 501 in the following manner: in two different reference pictures 505 and 506, the closest match between two blocks 503 and 504 is found along the motion trajectory 502 of the current CU 501. The cost function used in the matching process is the row-subsampled sum of absolute differences (SAD). After the matching process is completed, the refined MVs 507 and 508 are used for motion compensation in the prediction stage, boundary strength calculation in the deblocking filter, temporal motion vector prediction for subsequent pictures, and cross-CTU spatial motion vector prediction for subsequent CUs. Under the assumption of continuous motion trajectories, the motion vectors MV0 507 and MV1 508 pointing to the two reference blocks 503 and 504 should be proportional to the temporal distance (i.e., TD0 509 and TD1510) between the current picture 511 and the two reference pictures 505 and 506. As a special case, when the current picture 511 is temporally between the two reference pictures 505 and 506 and the temporal distances from the current picture 511 to the two reference pictures 505 and 506 are the same, the bilateral matching becomes a mirror-based bidirectional MV.

[0050] In order to strike the right balance between the increased codec efficiency that newer inter-mode codec tools (such as BDOF and DMVR) may bring on the one hand, and the increased complexity and latency associated with the newer inter-mode tools on the other hand, the current VVC design now imposes constraints on when BDOF or DMVR can be enabled for the current block.

[0051] In the current VVC design, BDOF is enabled only when all of the following predefined BDOF conditions listed in the box immediately following this paragraph are true:

[0052]

[0053] In the current VVC design, DMVR is enabled only when all of the following predefined DMVR conditions listed in the box immediately following this paragraph are true:

[0054]

[0055] Although the above constraints existing in the current VVC design have made great progress in achieving the desired balance between encoding and decoding efficiency on the one hand and complexity and delay on the other hand, they have not completely solved the problem.

[0056] A legacy issue of the current VVC design is that, although several constraints have been imposed on the enabling of BDOF and DMVR, in some cases, both decoder-side inter-frame prediction refinement tools BDOF and DMVR may be enabled when encoding and decoding a block. In the current VVC design, when these two decoder-side inter-frame prediction refinement tools are enabled, BDOF relies on the final motion compensation samples of DMVR, which creates a delay problem for hardware design.

[0057] Now the second legacy issue of the current VVC design is that, although several constraints have been imposed on the enabling of DMVR, these constraints as a whole are still too relaxed for the enabling of DMVR, because in some scenarios, disabling DMVR and the subsequent reduction in complexity and latency will achieve a better balance between codec efficiency on the one hand and complexity and latency on the other hand, but now the current VVC design will enable DMVR in these scenarios.

[0058] Now the third legacy issue of the current VVC design is that the constraints that have been imposed on the enabling of BDOF are not loose enough on the enabling of BDOF as a whole, because in some scenarios, enabling BDOF and the subsequent increase in codec gain will achieve a better balance between codec efficiency on the one hand and complexity and latency on the other hand, but now the current VVC design will not enable BDOF in these scenarios.

[0059] According to a first aspect of the present disclosure, when a current block is eligible for both DMVR application and BDOF application based on a plurality of predefined conditions, a predefined criterion based on the mode information of the current block is used to classify the current block into one of two predefined classes, namely, a DMVR class and a BDOF class. Subsequently, the classification of the current block is used to apply DMVR or BDOF to the current block, but not both. The method can be combined with the current VVC on top of the above predefined conditions, or can be implemented independently.

[0060] Figure 6is a flowchart illustrating the operation of the first aspect of the present disclosure. When processing a current block (601), the operation of this aspect of the present disclosure may apply multiple predefined conditions to the current block (602), and determine whether the current block is eligible for both DMVR application and BDOF application based on the multiple predefined conditions (603). If it is determined based on the multiple predefined conditions that the current block is not eligible for both DMVR application and BDOF application, the operation of this aspect of the present disclosure may continue with the process that exists in the current VVC design (604). On the other hand, if it is determined based on the multiple predefined conditions that the current block is eligible for both DMVR application and BDOF application, the operation of this aspect of the present disclosure may use a predefined criterion based on the mode information of the current block to classify the current block into one of two predefined classes, namely, a DMVR class and a BDOF class (605). Subsequently, the operation of this aspect of the present disclosure may use the classification result of the current block to apply DMVR or BDOF to the current block, but not both (606). Applying DMVR or BDOF but not both to the current block using the classification result of the current block (606) may include: if the current block is classified into the DMVR class, continuing in the DMVR class (applying DMVR without applying BDOF) (607), and if the current block is classified into the BDOF class, continuing in the BDOF class (applying BDOF without applying DMVR) (608).

[0061] The plurality of predefined conditions based on which the current block is eligible for both DMVR application and BDOF application may be, but need not be, the plurality of predefined BDOF conditions and predefined DMVR conditions listed in the above box.

[0062] The mode information used as the basis for the predefined criteria includes, but is not limited to, prediction mode (such as whether merge mode is used), merge mode index, motion vector, block shape, block size, prediction sample value.

[0063] According to one or more embodiments of the present disclosure, applying DMVR or BDOF to the current block instead of both using the classification of the current block includes: optionally sending a flag with a signal to indicate the classification of the current block, that is, in some examples of this embodiment, a flag is sent with a signal to specify whether BDOF or DMVR is applied to the block, while in some other examples of this embodiment, no such flag is sent with a signal.

[0064] According to one or more embodiments of the present disclosure, applying DMVR or BDOF to the current block instead of both using the classification of the current block further includes: when the current block is classified as a DMVR class, applying DMVR to the current block without applying BDOF using an existing merge candidate list or using a separately generated merge candidate list. That is, in some examples of this embodiment, when the current block is classified as a DMVR class and DMVR is applied to the current block, a separate merge candidate list is specifically generated and used. Semantics are signaled to indicate the DMVR merge mode, and if the size of the DMVR merge candidate list is greater than 1, a merge index is also signaled. In some other examples of this embodiment, when the current block is classified as a DMVR class and DMVR is applied to the current block, such a separate merge candidate is not generated, and applying DMVR to the current block uses an existing merge candidate list, and no further semantics or signaling is involved.

[0065] According to one or more embodiments of the present disclosure, applying DMVR or BDOF to the current block instead of both using the classification of the current block further includes: when the current block is classified as the BDOF class, applying BDOF to the current block without applying DMVR using an existing merge candidate list or using a separately generated merge candidate list. That is, in some examples of this embodiment, when the current block is classified as the BDOF class and BDOF is applied to the current block, a separate merge candidate list is specifically generated and used. Semantics are signaled to indicate the BDOF merge mode, and if the size of the BDOF merge candidate list is greater than 1, a merge index is also signaled. In some other examples of this embodiment, when the current block is classified as the BDOF class and BDOF is applied to the current block, such a separate merge candidate is not generated, and applying BDOF to the current block uses an existing merge candidate list, and no further semantics or signaling is involved.

[0066] According to another embodiment of the first aspect of the present disclosure, a current block is classified into one of two predefined classes, namely, a DMVR class and a BDOF class, using a predefined criterion based on pattern information of the current block, including: when the predefined criterion is met, the current block is classified into the DMVR class, and when the predefined criterion is not met, the current block is classified into the BDOF class.

[0067] According to another embodiment of the first aspect of the present disclosure, a current block is classified into one of two predefined classes, namely, a DMVR class and a BDOF class, using a predefined criterion based on pattern information of the current block, including: when the predefined criterion is met, the current block is classified into the BDOF class, and when the predefined criterion is not met, the current block is classified into the DMVR class.

[0068] According to another embodiment of the first aspect of the present disclosure, the predefined criterion includes whether to select a normal mode for the current block.

[0069] According to another embodiment of the first aspect of the present disclosure, the predefined criterion includes whether the codec merge index of the current block has a predefined mathematical property.

[0070] In another example, the predefined mathematical property includes a property that is equal to or greater than a predefined threshold number.

[0071] According to another embodiment of the first aspect of the present disclosure, the predefined criterion includes whether the motion vector of the current block satisfies a predefined test.

[0072] In one example, the predefined test includes whether the sum of the magnitudes of all motion vector components is greater than a predefined threshold number.

[0073] According to another embodiment of the first aspect of the present disclosure, the predefined criterion includes whether the current block has a predefined shape.

[0074] In one example, the predefined shape is a square shape.

[0075] According to another embodiment of the first aspect of the present disclosure, the predefined criterion includes whether a block size of the current block has a predefined mathematical property.

[0076] In one example, the predefined mathematical property includes a property that is equal to or greater than a predefined threshold number.

[0077] According to another embodiment of the first aspect of the present disclosure, the predefined criterion includes whether the sum of absolute differences or the sum of squared differences (SAD or SSD) between the list 0 prediction value samples and the list 1 prediction value samples of the current block has a predefined mathematical property.

[0078] In one example, the predefined mathematical property includes a property that is greater than a predefined threshold number.

[0079] According to a second aspect of the present disclosure, when a current block is eligible for DMVR encoding and decoding based on multiple predefined conditions, a first determination is made as to whether weighted prediction is enabled for the current block, and a second determination is made as to whether different weights are used when averaging list 0 prediction value samples and list 1 prediction value samples for the current block, and whether to disable DMVR application to the current block can be determined based on the first determination and the second determination. The method can be combined with the current VVC on top of the above-mentioned predefined conditions, or can be implemented independently.

[0080] Figure 7is a flowchart illustrating an example method of the second aspect of the present disclosure. When processing a current block (701), the operation of this aspect of the present disclosure may apply multiple predefined conditions to the current block (702), and determine whether the current block is eligible for DMVR encoding and decoding based on the multiple predefined conditions (703). If it is determined that the current block is not eligible for DMVR encoding and decoding based on the multiple predefined conditions, the operation of this aspect of the present disclosure may continue with the process that exists in the current VVC design (704). On the other hand, if it is determined that the current block is eligible for DMVR encoding and decoding based on the multiple predefined conditions, the operation of this aspect of the present disclosure may determine whether weighted prediction is enabled for the current block (705), and also determine whether different weights are used when averaging list 0 prediction value samples and list 1 prediction value samples for the current block (706), and then determine whether to disable DMVR application to the current block based on these two determinations (707).

[0081] The plurality of predefined conditions based on which the current block is eligible for DMVR encoding and decoding may be, but need not be, the plurality of predefined DMVR conditions listed in the above box.

[0082] According to an embodiment of the second aspect of the present disclosure, determining whether to disable DMVR application to the current block based on the two determinations includes: disabling DMVR application to the current block when it is determined that weighted prediction is enabled for the current block.

[0083] According to another embodiment of the second aspect of the present disclosure, determining whether to disable DMVR application to the current block based on these two determinations includes: disabling DMVR application to the current block when it is determined that different weights are used when averaging list 0 prediction value samples and list 1 prediction value samples for the current block.

[0084] According to another embodiment of the second aspect of the present disclosure, determining whether to disable DMVR application to the current block based on these two determinations includes: when it is determined that weighted prediction is enabled for the current block, and at the same time it is determined that different weights are used when averaging list 0 prediction value samples and list 1 prediction value samples for the current block, disabling DMVR application to the current block.

[0085] According to a third aspect of the present disclosure, when the current block is encoded and decoded in sub-block merging mode, BDOF application to the current block can be enabled. The method can be combined with the current VVC on the basis of the above predefined conditions, or can be implemented independently.

[0086] Figure 8is a flowchart illustrating the operation of the third aspect of the present disclosure. When processing a current block (801), the operation of this aspect of the present disclosure may determine whether the current block is encoded and decoded in a sub-block merge mode (802). If it is determined that the current block is not encoded and decoded in a sub-block merge mode, the operation of this aspect of the present disclosure may continue with the process that exists in the current VVC design (803). On the other hand, if it is determined that the current block is encoded and decoded in a sub-block merge mode, the operation of this aspect of the present disclosure may enable the application of BDOF to the current block (804).

[0087] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media or communication media, where computer-readable storage media corresponds to tangible media such as data storage media, and communication media includes any media that facilitates, for example, transferring a computer program from one place to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) a non-transient tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, codes, and / or data structures for implementing the embodiments described in the present application. A computer program product may include a computer-readable medium.

[0088] Further, the above method can be implemented using a device including one or more circuit systems, which include an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components. The device can use these circuit systems in combination with other hardware or software components to perform the method described above. Each module, submodule, unit, or subunit disclosed above can be implemented at least in part using these one or more circuit systems.

[0089] Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any changes, uses or adaptations made to the invention that follow its general principles and include such deviations from the disclosure within known or customary practices in the art. The specification and examples are intended to be considered exemplary only, with the true scope and spirit of the invention being indicated by the appended claims.

[0090] It will be understood that the present invention is not limited to the exact examples described above and shown in the accompanying drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is intended to be limited only by the appended claims.

Claims

1. A method for video encoding, comprising: Acquire a plurality of blocks segmented from a video picture, wherein each of the plurality of blocks is encoded by an intra-frame prediction mode or an inter-frame prediction mode; For a current block in the plurality of blocks: In response to the current block being eligible for both decoder-side motion vector refinement (DMVR) application and bidirectional optical flow (BDOF) application, determining whether a predefined criterion including a sum of absolute differences (SAD) or a sum of squared differences (SSD) between list 0 prediction samples and list 1 prediction samples of the current block being less than a predefined threshold number is satisfied; In response to the predefined criterion being satisfied, avoiding applying BDOF to a sub-block in the current block; as well as Outputting the prediction mode information of the current block through a video bitstream, The current block eligible for DMVR application satisfies at least the following conditions: The distance between the current picture and the forward reference picture is the same as the distance between the current picture and the backward reference picture; The height of the current block is equal to or greater than 8; The current block is not encoded in affine mode; The current block is not encoded in sub-block merging mode; and The current block is not encoded in merge mode with motion vector difference (MMVD).

2. The method of claim 1, wherein: Block size information is used for the predefined criteria.

3. The method of claim 1, wherein: The current block eligible for BDOF application satisfies conditions including at least the following: The current block uses bidirectional prediction using a forward reference picture and a backward reference picture; Weighted forecasting is not enabled; The size of the current block is not equal to 4×8; The current block is not encoded in a symmetric motion vector difference (MVD) mode; The current block is not encoded in a sub-block merging mode; as well as The current block does not use different weights when averaging the prediction samples from list 0 and list 1.

4. A computing device comprising: one or more processors; a non-transitory storage device coupled to the one or more processors; as well as a plurality of programs stored in the non-transitory storage device, the plurality of programs, when executed by the one or more processors, causing the computing device to perform actions including: Acquire a plurality of blocks segmented from a video picture, wherein each of the plurality of blocks is encoded by an intra-frame prediction mode or an inter-frame prediction mode; For a current block in the plurality of blocks: In response to the current block being eligible for both decoder-side motion vector refinement (DMVR) application and bidirectional optical flow (BDOF) application, determining whether a predefined criterion including a sum of absolute differences (SAD) or a sum of squared differences (SSD) between list 0 prediction samples and list 1 prediction samples of the current block being less than a predefined threshold number is satisfied; In response to the predefined criterion being satisfied, avoiding applying BDOF to a sub-block in the current block; as well as The prediction mode information of the current block is output through the video bit stream, The current block eligible for DMVR application satisfies at least the following conditions: The distance between the current picture and the forward reference picture is the same as the distance between the current picture and the backward reference picture; The height of the current block is equal to or greater than 8; The current block is not encoded in affine mode; The current block is not encoded in sub-block merging mode; and The current block is not encoded in merge mode with motion vector difference (MMVD).

5. The computing device of claim 4, wherein: Block size information is used for the predefined criteria.

6. The computing device of claim 4, wherein: The current block eligible for BDOF application satisfies conditions including at least the following: The current block uses bidirectional prediction using a forward reference picture and a backward reference picture; Weighted forecasting is not enabled; The size of the current block is not equal to 4×8; The current block is not encoded in a symmetric motion vector difference (MVD) mode; The current block is not encoded in a sub-block merging mode; as well as The current block does not use different weights when averaging the prediction samples from list 0 and list 1.

7. A non-transitory computer-readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein: When the plurality of programs are executed by the one or more processors, the computing device executes the method of any one of claims 1 to 3 to generate a video bitstream and stores the generated video bitstream in the non-transitory computer-readable storage medium.

8. A computer program product, comprising instructions for being executed by a computing device having one or more processors, wherein when the instructions are executed by the one or more processors, the computing device implements the method according to any one of claims 1 to 3 to generate a video bitstream.

9. A computer-readable storage medium for storing a bit stream, wherein the bit stream is generated by encoding by a computing device using the following method: Acquire a plurality of blocks segmented from a video picture, wherein each of the plurality of blocks is encoded by an intra-frame prediction mode or an inter-frame prediction mode; For a current block in the plurality of blocks: In response to the current block being eligible for both decoder-side motion vector refinement (DMVR) application and bidirectional optical flow (BDOF) application, determining whether a predefined criterion including a sum of absolute differences (SAD) or a sum of squared differences (SSD) between list 0 prediction samples and list 1 prediction samples of the current block being less than a predefined threshold number is satisfied; In response to the predefined criterion being satisfied, avoiding applying BDOF to a sub-block in the current block; as well as Outputting the prediction mode information of the current block through a video bitstream, The current block eligible for DMVR application satisfies at least the following conditions: The distance between the current picture and the forward reference picture is the same as the distance between the current picture and the backward reference picture; The height of the current block is equal to or greater than 8; The current block is not encoded in affine mode; The current block is not encoded in sub-block merging mode; and The current block is not encoded in merge mode with motion vector difference (MMVD).