Video decoding method, apparatus and computer program product

Abstract

A decoding method includes: calculating a first cost value associated with a first code value for differential weighted bi-prediction and a second cost value associated with a second code value; determining a code value selected from the first code value and the second code value according to the first cost value and the second cost value; and determining a first weight value and a second weight value for a current pixel in a current block based on a difference between a first reference pixel of a first reference block in a first reference picture and a second reference pixel of a second reference block in a second reference picture. The method includes reconstructing the current pixel according to the first weight value, the second weight value, and the determined code value according to a weighted sum of the first reference pixel and the second reference pixel.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims benefit of priority to U.S. Provisional Application No. 63 / 444,891, entitled "Improvement Of Bi-Prediction With CU Level Weight (BCW)," filed February 10, 2023, which claims benefit of priority to U.S. Patent Application No. 18 / 420,947, entitled "IMPROVEMENT OF BI-PREDICTION WITH CU LEVEL WEIGHT (BCW)," filed January 24, 2024. The disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] This disclosure describes embodiments generally related to video coding. [Background technology]

[0003] The background art discussion provided herein is intended to provide a comprehensive context for the present disclosure, and the inventors' work, along with aspects of the description that may not otherwise be admitted as prior art at the time of filing, is not expressly or implicitly admitted as prior art to the present disclosure, to the extent that that work is described in this background art section. Image / video compression can help transmit image / video data over different devices, storage, and networks with minimal quality loss. In some examples, video codec technology can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-prediction, which can compress images based on spatial redundancy. For example, intra-prediction can use reference data from the current picture being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-prediction, which can compress images based on temporal redundancy. For example, inter-prediction can predict samples in a current picture from a previously reconstructed picture using motion compensation. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention [Means for solving the problem]

[0004] Aspects of the present disclosure include methods and apparatuses for video encoding / decoding. In some examples, an apparatus for video decoding includes a processing circuit for performing video encoding and decoding. The video decoding method includes receiving a coded video bitstream including coded information of a current block in a current picture. The coded information indicates bi-prediction with weights selected from a plurality of weight candidates, the plurality of weight candidates including at least a first weight candidate and a second weight candidate. The first weight candidate corresponds to differential weighted bi-prediction with a first code value, and the second weight candidate corresponds to differential weighted bi-prediction with a second code value. The differential weighted bi-prediction uses pixel differences to derive bi-prediction weights at the pixel level, and the code value indicates a direction of a larger weight value for the differential weighted bi-prediction. The method also includes calculating a first cost value for differential weighted bi-prediction using a first code value and a second cost value for differential weighted bi-prediction using a second code value, determining a code value selected from the first code value and the second code value according to the first cost value and the second cost value, and determining a first weight value and a second weight value based on a difference between a first reference pixel and a second reference pixel for a current pixel in a current block. The first reference pixel is in a first reference block from the first reference picture, and the second reference pixel is in a second reference block from the second reference picture. The method further includes reconstructing the current pixel based on a weighted sum of the first reference pixel and the second reference pixel according to the first weight value, the second weight value, and the selected code value.

[0005] In some examples, to calculate the first cost value and the second cost value, the method includes: generating a first reference template for a current template of a current block according to a first weight candidate (e.g., a first code value); calculating a first template matching cost between the first reference template and the current template as the first cost value; generating a second reference template for the current template of the current block according to a second weight candidate (e.g., a second code value); and calculating a second template matching cost between the second reference template and the current template as the second cost value.

[0006] In some examples, to determine the selected code value, the method includes determining the selected code value having the lowest cost value.

[0007] In some examples, to determine the selected code value, the method includes sorting the first reference block and the second reference block in the list according to a first cost value associated with the first code value and a second cost value associated with the second code value, where the first code value indicates a first reference block having a higher weight value and the second code value indicates a second reference block having a higher weight value. The method includes determining a flag indicating a most likely reference block from the list according to the context model.

[0008] In some examples, to calculate the first cost value and the second cost value, the method includes: generating a first reconstructed block of the current block according to a first code value; calculating a first boundary cost between a boundary of the first reconstructed block and a reconstructed adjacent sample of the current block as the first cost value; generating a second reconstructed block of the current block according to a second code value; and calculating a second boundary cost between a boundary of the second reconstructed block and a reconstructed adjacent sample of the current block as the second cost value.

[0009] In one example, to generate a first reconstructed block, the method includes combining a residual block of the current block with a predicted block predicted by differential weighted bi-prediction using a first code value to generate the first reconstructed block.

[0010] In some examples, to determine the selected code value, the method includes determining that a neighboring block of the current block is coded using differential weighted bi-prediction, and determining the selected code value according to a reference index used by the neighboring block.

[0011] In some examples, the multiple weight candidates include a first weight candidate and a second weight candidate associated with different code values ​​for differential weighted bi-prediction and multiple weight candidates for bi-prediction with coding unit level weights (BCW) having different weights. In one example, the method further includes decoding a flag from the coding information of the current block indicating one of differential weighted bi-prediction and BCW. In response to the flag indicating differential weighted bi-prediction, the method includes decoding a first syntax indicating a weight candidate from the first weight candidate and the second weight candidate. In response to the flag indicating BCW, the method includes decoding a second syntax indicating a particular weight candidate for BCW.

[0012] In some examples, the method includes calculating a template matching cost value for each of a plurality of weight candidates and selecting the weight candidate having the lowest template matching value.

[0013] In some examples, the method includes calculating template matching cost values ​​for a plurality of weight candidates, respectively. In response to the lowest template matching cost value being associated with differential weighted bi-prediction, the method includes decoding, from coded information of the current block, a first syntax indicating a weight candidate from the first weight candidate and the second weight candidate. In response to the lowest template matching cost value being associated with the BCW, the method includes decoding, from coded information of the current block, a second syntax indicating a particular weight candidate of the BCW.

[0014] In some examples, the method includes calculating a template matching cost value for each of a plurality of weight candidates, sorting the plurality of weight candidates according to the template matching cost values, and decoding, from coded information of the current block, syntax indicating a weight candidate from the sorted plurality of weight candidates.

[0015] According to another aspect of the present disclosure, an apparatus is provided, the apparatus including a processing circuit, the processing circuit being configured to perform any of the described methods for video decoding / encoding.

[0016] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform any of the described methods for video decoding / encoding. [Brief explanation of the drawings]

[0017] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0018] [Figure 1] FIG. 1 is a schematic diagram of an exemplary block diagram of a communication system.

[0019] [Figure 2]FIG. 2 is a schematic diagram of an exemplary block diagram of a decoder.

[0020] [Figure 3] FIG. 3 is a schematic diagram of an exemplary block diagram of an encoder.

[0021] [Figure 4] FIG. 4 shows the location of spatial merge candidates in some examples.

[0022] [Figure 5] FIG. 5 shows candidate pairs considered for redundancy check of spatial merge candidates in some examples.

[0023] [Figure 6] FIG. 6 illustrates exemplary motion vector scaling for temporal merge candidates.

[0024] [Figure 7] FIG. 7 shows exemplary candidate positions for temporal merge candidates for the current coding block.

[0025] [Figure 8] FIG. 8 shows an example of template matching.

[0026] [Figure 9] FIG. 9 shows an example of template matching in affine merge mode.

[0027] [Figure 10] FIG. 10 shows a diagram illustrating boundary cost calculations in some examples.

[0028] [Figure 11] FIG. 11 shows a flowchart outlining a decoding process according to some embodiments of the present disclosure.

[0029] [Figure 12]FIG. 12 shows a flowchart outlining an encoding process according to some embodiments of the present disclosure.

[0030] [Figure 13] FIG. 13 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0031] 1 shows a block diagram of a video processing system (100) in some examples. The video processing system (100) is an example of an application for the subject matter of this disclosure, a video encoder, and a video decoder, in a streaming environment. The subject matter of this disclosure may be equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, and storage of compressed video on digital media, including CDs, DVDs, memory sticks, etc.

[0032] The video processing system (100) includes a capture subsystem (113), which may include a video source (101), such as a digital camera, that generates a stream of uncompressed video pictures (102). In one example, the stream of video pictures (102) includes samples captured by the digital camera. The stream of video pictures (102), depicted as a thick line to emphasize its high data volume compared to the encoded video data (104) (or coded video bitstream), can be processed by an electronic device (120) that includes a video encoder (103) coupled to the video source (101). The video encoder (103), as described in more detail below, may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter. The encoded video data (104) (or coded video bitstream), depicted as a thin line to emphasize its low data volume compared to the stream of video pictures (102), can be stored on a streaming server (105) for future use. One or more streaming client subsystems, such as the client subsystems (106) and (108) of Figure 1, can access the streaming server (105) to retrieve copies (107) and (109) of the encoded video data (104). The client subsystem (106) can include, for example, a video decoder (110) within an electronic device (130). The video decoder (110) decodes an input copy (107) of the encoded video data and creates an output stream of video pictures (111) that can be rendered on a display (112) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (104), (107), and (109) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. An example of such a standard is ITU-T Recommendation H.265.In one example, a developing video coding standard is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0033] It should be noted that the electronic devices (120) and (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0034] 2 shows an example block diagram of a video decoder (210). The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., a receiving circuit). The video decoder (210) may be used in place of the video decoder (110) in the example of FIG. 1.

[0035] The receiver (231) may receive one or more coded video sequences, e.g., included in a bitstream, to be decoded by the video decoder (210). In one embodiment, one coded video sequence is received at a time, and the decoding of each coded video sequence is independent of the decoding of the other coded video sequences. The coded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (231) may receive other data, e.g., coded audio data and / or ancillary data streams, that may be transferred to each other using entities (not shown). The receiver (231) may separate the coded video sequences from other data. To combat network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter, "parser (220)"). In certain applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory may be external to the video decoder (210) (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder (210), for example, to combat network jitter, plus another buffer memory (215) internal to the video decoder (210), for example, to handle playback timing. When the receiver (231) is receiving data from a store / forward device with sufficient bandwidth and controllability, or from an asynchronous network, the buffer memory (215) may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (215) may be needed, may be relatively large, may advantageously be adaptively sized, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (210).

[0036] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (210) and potential information for controlling a rendering device, such as a rendering device (212) (e.g., a display screen) that is not an integral part of the electronic device (230) but may be coupled to the electronic device (230), as shown in FIG. 2. The control information for the rendering device(s) may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (220) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) can extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups can include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (220) can also extract coded video sequence information such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0037] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to create symbols (221).

[0038] The reconstruction of the symbols (221) may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block), as well as other factors. Which units are involved and how can be controlled by subgroup control information parsed by the parser (220) from the coded video sequence. The flow of such subgroup control information between the parser (220) and the following units is not shown for clarity.

[0039] Beyond the functional blocks already mentioned, the video decoder (210) may be conceptually subdivided into several functional units, as described below. In actual implementations operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate:

[0040] The first unit is a scalar / inverse transform unit (251), which receives quantized transform coefficients and control information from the parser (220), including the transform to use, block size, quantization coefficients, quantization scaling matrices, etc., as symbol(s) (221). The scalar / inverse transform unit (251) may output blocks containing sample values ​​that may be input to an aggregator (255).

[0041] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks are blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information can be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates a block of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from the current picture buffer (258). The current picture buffer (258), for example, buffers a partially reconstructed and / or fully reconstructed current picture. The aggregator (255) optionally appends the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a sample-by-sample basis.

[0042] In other cases, the output samples of the scalar / inverse transform unit (251) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (253) can access the reference picture memory (257) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (221) associated with the block, these samples can be appended by the aggregator (255) to the output of the scalar / inverse transform unit (251) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (257) from which the motion-compensated prediction unit (253) fetches prediction samples can be controlled by motion vectors available to the motion-compensated prediction unit (253), for example, in the form of symbols (221) that can have X, Y, and reference picture components. Motion compensation can also include interpolation of sample values ​​fetched from the reference picture memory (257) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, and the like.

[0043] The output samples of the aggregator (255) may be subjected to various loop filtering techniques in a loop filter unit (256). Video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called a coded video bitstream) and made available to the loop filter unit (256) as symbols (221) from the parser (220). Video compression may also be responsive to previously reconstructed and loop filtered sample values, as well as meta information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence.

[0044] The output of the loop filter unit (256) can be a sample stream that can be output to a rendering device (212) and stored in a reference picture memory (257) for use in future inter-picture prediction.

[0045] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257), and a new current picture buffer can be reallocated before beginning reconstruction of a subsequent coded picture.

[0046] The video decoder (210) can perform decoding operations according to a given video compression technology or standard, such as ITU-T Recommendation H.265. A coded video sequence can conform to the syntax specified by the video compression technology or standard being used, in the sense that the coded video sequence adheres to both the syntax of the video compression technology or standard and the profile documented in the video compression technology or standard. Specifically, a profile can select specific tools from all tools available in the video compression technology or standard as the only tools usable under that profile. Compliance may also require that the complexity of the coded video sequence be within the boundaries defined by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained by a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0047] In one embodiment, the receiver (231) can receive additional (redundant) data with the encoded video. The additional data may be included as part of the coded video sequence(s). The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0048] 3 shows an example block diagram of a video encoder (303). The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmitting circuit). The video encoder (303) can be used in place of the video encoder (103) in the example of FIG. 1.

[0049] The video encoder (303) can receive video samples from a video source (301) (which in the example of FIG. 3 is not part of the electronic device (320)) that can capture the video image(s) to be coded by the video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0050] The video source (301) may provide a source video sequence to be coded by the video encoder (303) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601, YCrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (301) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, impart motion. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples depending on the sampling structure, color space, etc., in use. The following discussion focuses on samples.

[0051] According to one embodiment, the video encoder (303) can code and compress pictures of a source video sequence into a coded video sequence (343) in real time, or under any other time constraints as needed. Achieving the appropriate coding rate is one function of the controller (350). In some embodiments, the controller (350) controls and is operatively coupled to other functional units, as described below, which couplings are not shown for clarity. Parameters set by the controller (350) can include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) can be configured with other suitable functions for the video encoder (303), optimized for a particular system design.

[0052] In some embodiments, the video encoder (303) is configured to operate in a coding loop. As a very simplified explanation, in one example, the coding loop can include a source coder (330) (e.g., responsible for creating a symbol-like symbol stream based on an input picture to be coded and reference picture(s)) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols and creates sample data in a manner similar to that of a (remote) decoder. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Because decoding of the symbol stream produces bit-exact results regardless of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" the exact sample values ​​the decoder would "see" when using prediction during decoding. This basic principle of reference picture synchrony (and the drift that occurs when synchrony cannot be maintained, for example due to channel errors) is also used in several related fields.

[0053] The operation of the "local" decoder (333) may be the same as a "remote" decoder, such as the video decoder (210) already described in detail above in connection with Figure 2. However, briefly referring also to Figure 2, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (345) and parser (220) may be lossless, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and parser (220), may not be fully implemented in the local decoder (333).

[0054] In one embodiment, decoder techniques present in a decoder, excluding analysis / entropy decoding, are present in the corresponding encoder in the same or substantially the same functional form. Therefore, the subject matter of the disclosure focuses on decoder operation. A description of the encoder techniques can be omitted, as they are the opposite of the decoder techniques, which are described generically. In certain areas, more detailed descriptions are provided below.

[0055] In operation, in some examples, the source coder (330) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (332) codes differences between pixel blocks of the input picture and pixel blocks of reference picture(s) that may be selected as predictive reference(s) for the input picture.

[0056] The local video decoder (333) may decode coded video data of pictures that may be designated as reference pictures based on symbols created by the source coder (330). The operation of the coding engine (332) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 3), the reconstructed video sequence may typically be a replica of the source video sequence, with some error. The local video decoder (333) may replicate the decoding process that may be performed on the reference pictures by the video decoder and store the reconstructed reference pictures in a reference picture memory (334). In this way, the video encoder (303) may locally store copies of reconstructed reference pictures that have common content as reconstructed reference pictures that would be retrieved by the far-end video decoder (without transmission error).

[0057] The predictor (335) can perform the prediction search for the coding engine (332). That is, for a new picture to be coded, the predictor (335) can search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that can serve as suitable prediction references for the new picture. The predictor (335) can operate on a sample block / pixel block basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (335), the input picture can have prediction references taken from multiple reference pictures stored in the reference picture memory (334).

[0058] The controller (350) can manage the coding operations of the source coder (330), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0059] The output of all of the aforementioned functional units may undergo entropy coding in an entropy coder (345), which converts the symbols produced by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0060] The transmitter (340) may buffer the coded video sequence(s) created by the entropy coder (345) and prepare them for transmission over a communication channel (360), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (340) may merge the coded video data from the video encoder (303) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0061] The controller (350) can manage the operation of the video encoder (303). During coding, the controller (350) can assign a particular type of coded picture to each coded picture, which can affect the coding technique that can be applied to the respective picture. For example, pictures can often be assigned as one of the following picture types:

[0062] Intra-pictures (I-pictures) may be coded and decoded without using any other picture in the sequence as a prediction source. Some video codecs also allow for different types of intra-pictures, including, for example, independent decoder refresh ("IDR") pictures.

[0063] Predictive pictures (P pictures) may be coded and decoded using intra prediction or inter prediction, which uses motion vectors and reference indices to predict the sample values ​​of each block.

[0064] Bidirectionally predictive pictures (B pictures) can be coded and decoded using intra prediction or inter prediction, which uses two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multi-predictive pictures can use three or more reference pictures and associated metadata for the reconstruction of a single block.

[0065] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction with reference to one previously coded reference picture or via temporal prediction. Blocks of a B-picture may be predictively coded via spatial prediction with reference to one or two previously coded reference pictures or via temporal prediction.

[0066] The video encoder (303) may perform coding operations in accordance with a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. In doing so, the video encoder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy within the input video sequence. Thus, the coded video data may conform to a syntax specified by the video coding technique or standard being used.

[0067] In one embodiment, the transmitter (340) may transmit additional data along with the encoded video. The source coder (330) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0068] Video may be captured in time sequence as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is divided into multiple blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. A motion vector points to a reference block within the reference picture and may have a third dimension that identifies the reference picture when multiple reference pictures are used.

[0069] In some embodiments, a bi-prediction technique can be used for inter-picture prediction. According to the bi-prediction technique, two reference pictures, such as a first reference picture and a second reference picture, are used, both of which precede the current picture in the video in decoding order (but may be past and future in display order, respectively). A block in the current picture can be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. A block can be predicted by a combination of the first reference block and the second reference block.

[0070] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0071] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed block-by-block. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one CTB for luma and two CTBs for chroma. Each CTU may be recursively quadtree-divided into one or more coding units (CUs). For example, a 64x64 pixel CTU may be divided into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter-prediction type or an intra-prediction type. The CU is divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Generally, each PU includes one prediction block (PB) of luma and two PBs of chroma. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0072] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using any suitable technique. In one embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more processors executing software instructions.

[0073] Aspects of this disclosure provide techniques for bi-prediction with CU-level weights (BCW).

[0074] In video coding, various inter-prediction modes can be used. For example, in VVC, for an inter-predicted CU, motion parameters can include MV(s), one or more reference picture indices, a reference picture list usage index, and additional information on specific coding features used to generate the inter-predicted sample. Motion parameters can be signaled explicitly or implicitly. If a CU is coded in skip mode, the CU can be associated with a PU and cannot have significant residual coefficients, coded motion vector deltas or MV differentials (e.g., MVDs), or reference picture indices. A merge mode can be specified in which motion parameters for the current CU are obtained from neighboring CU(s), and the motion parameters of the current CU include spatial and / or temporal candidates, and optionally additional information such as those introduced in VVC. The merge mode can be applied not only to skip mode but also to inter-predicted CUs. In one example, an alternative to the merge mode is explicit transmission of motion parameters, where MV(s), the corresponding reference picture index of each reference picture list, and a reference picture list usage flag and other information are explicitly signaled for each CU.

[0075] In embodiments such as VVC, the VVC Test Model (VTM) reference software includes one or more improved inter-prediction coding tools, including enhanced merge prediction, merge motion vector differential (MMVD) mode, adaptive motion vector prediction with symmetric MVD signaling (AMVP) mode, affine motion compensation prediction, sub-block-based temporal motion vector prediction (SbTMVP), adaptive motion vector resolution (AMVR), motion field storage (1 / 16 luma sample MV storage and 8x8 motion field compression), bi-prediction with CU-level weights (BCW), bidirectional optical flow (BDOF), prediction refinement using optical flow (PROF), decoder-side motion vector refinement (DMVR), combined inter and intra prediction (CIIP), geometric partitioning mode (GPM), etc. Inter prediction and related methods are described in detail below.

[0076] In some examples, enhanced merge prediction can be used. In examples such as in VTM4, a merge candidate list is constructed by including five types of candidates in order: spatial motion vector predictor(s) (MVP) from spatially adjacent CU(s), temporal MVP(s) from co-located CU(s), history-based MVP(s) (HMVP) from a first-in-first-out (FIFO) table, pairwise average MVP(s), and zero MV(s).

[0077] The size of the merge candidate list can be signaled in the slice header. In one example, the maximum allowed size of the merge candidate list is 6 in VTM4. For each CU coded in merge mode, the index of the best merge candidate (e.g., merge index) can be coded using truncated unary binarization (TU). The first bin of the merge index can be coded with context (e.g., context-adaptive binary arithmetic coding (CABAC)), and bypass coding can be used for the other bins.

[0078] Some examples of the generation process for each category of merge candidates are provided below: In one embodiment, spatial candidate(s) are derived as follows: The derivation of spatial merge candidates in VVC can be the same as that in HEVC. In one example, up to four merge candidates are selected from the candidates in the positions shown in Figure 4.

[0079] 4 illustrates the positions of spatial merge candidates according to one embodiment of the present disclosure. Referring to FIG. 4, the derivation order is B1, A1, B0, A0, and B2. Position B2 is considered only if the CUs at positions A0, B0, B1, and A1 are all unavailable (e.g., because the CUs belong to another slice or another tile) or are intra-coded. After the candidate at position A1 is added, the addition of the remaining candidates undergoes a redundancy check that ensures that candidates with the same motion information are removed from the candidate list, improving coding efficiency.

[0080] To reduce computational complexity, not all possible candidate pairs are considered in the above redundancy check. Instead, only pairs connected by arrows in Figure 5 are considered, and a candidate is added to the candidate list only if the corresponding candidates used in the redundancy check do not have the same motion information.

[0081] 5 illustrates candidate pairs considered for spatial merge candidate redundancy checking according to one embodiment of the present disclosure. Referring to FIG. 5, each arrow-connected pair includes A1 and B1, A1 and A0, A1 and B2, B1 and B0, and B1 and B2. Thus, candidates at positions B1, A0, and / or B2 can be compared with candidates at position A1, and candidates at positions B0 and / or B2 can be compared with candidates at position B1.

[0082] In one embodiment, the temporal candidate(s) are derived as follows: In one example, only one temporal merge candidate is added to the candidate list. Figure 6 shows exemplary motion vector scaling for a temporal merge candidate. To derive a temporal merge candidate for a current CU (611) in a current picture (601), a scaled MV (621) (e.g., shown by the dotted line in Figure 6) can be derived based on a co-located CU (612) belonging to a co-located reference picture (604). The reference picture list used to derive the co-located CU (612) can be explicitly signaled in the slice header. The scaled MV (621) for the temporal merge candidate can be obtained as shown by the dotted line in Figure 6. The scaled MV (621) can be scaled from the MV of the co-located CU (612) using picture order count (POC) distances tb and td. The POC distance tb may be defined to be the POC difference between the current reference picture (602) of the current picture (601) and the current picture (601). The POC distance td may be defined to be the POC difference between the co-located reference picture (604) of the co-located picture (603) and the co-located picture (603). The reference picture index of a temporal merge candidate may be set to zero. A co-located picture is a reference picture used as a source picture for temporal motion information derivation. A co-located picture may be identified in one of two lists, called list 0 or list 1. In some examples, an encoder may determine the co-located pictures and signal the co-located pictures using suitable syntax techniques.

[0083] FIG. 7 shows exemplary candidate positions (e.g., C0 and C1) of temporal merge candidates for the current CU. The position of the temporal merge candidate can be selected from candidate positions C0 and C1. Candidate position C0 is located at the bottom right corner of the current CU's co-located CU (710). Candidate position C1 is located at the center of the current CU's co-located CU (710). If the CU at candidate position C0 is unavailable, intra-coded, or outside the current row of the CTU, candidate position C1 is used to derive the temporal merge candidate. Otherwise, for example, the CU at candidate position C0 is available, inter-coded, and located in the current row of the CTU, candidate position C0 is used to derive the temporal merge candidate.

[0084] In some examples, CU-level weighted bi-prediction (BCW) can be used to weight predictions from different reference pictures differently. The BCW technique is designed to predict blocks by weighted averaging of two motion-compensated prediction blocks. BCW differs from a technique called weighted prediction (WP), which indicates weights at the slice level. BCW can signal weight information at the CU level using an index denoted as bcwIdx. The index can indicate a weight selected from a list of predefined candidate weights. In some examples, the list includes five predefined candidate weights, such as {-2, 3, 4, 5, 10} / 8, selected for reference pictures in Reference List 1 (also referred to as Reference Picture List 1), where -2 / 8 and 10 / 8 can be used to reduce negatively correlated noise between bi-predictive prediction blocks. If forward and backward reference pictures in both reference lists are used to achieve a better tradeoff between performance and complexity, the list of predefined candidate weights can be reduced to {3, 4, 5} / 8. In some examples, a unity gain constraint is applied, so once the weight, denoted as w, pointed to by bcwIdx, corresponding to reference list 1 is determined, the weights corresponding to other reference lists can be calculated by 1-w. In one example, each luma / chroma prediction sample of the BCW is calculated as equation (1):

number

[0085] In some examples, the use of bcwIdx can be buffered for subsequent CUs in the same frame to perform spatial motion merging for regular or affine merge modes. In one example, when a spatially adjacent merge candidate is bi-predicted and the current CU selects this candidate (a spatially adjacent merge candidate), all reference indices and motion vectors (or control point motion vectors (CPMVs) in the case of inherited affine merge mode) containing its bcwIdx are inherited by the current CU. In some examples, the only exception occurs when the current CU enables the CIIP flag: bcwIdx is inherited from the one associated with the top-left control point motion vector (or the top-right control point motion vector if the top-left control point motion vector is not used). Note that when the inferred bcwIdx points to a weight of non-0.5, both decoder-side motion vector refinement (DMVR) and bidirectional optical flow (BDOF) are turned off.

[0086] In some examples (e.g., ECM), the BCW index of a merge-coded CU is derived based on a template matching (TM) cost instead of being derived from neighboring blocks. In one example, for a selected merge candidate, TM cost values ​​are calculated using different bi-prediction weights, and then the bi-prediction weight with the smallest TM cost value is used to predict the merged CU.

[0087] In some examples, several rules are applied to calculate the TM cost of bi-predicted weights (also called bi-prediction weights). For example, three rules can be applied. According to the first rule, because the inherited bi-predicted weight is likely to have higher accuracy than other weights, only the inherited bi-prediction weight and its two neighboring weights (for example, ±1) are considered. For example, if the inherited bi-prediction weight is 4, three weights {3, 4, 5} are involved in the TM cost calculation. According to the second rule, the TM cost of the inherited BCW index is multiplied by 0.90625, that is, the cost is reduced by 3 / 32. According to the third rule, because bi-predicted samples are beneficial for BDOF and BDOF is only applied to CUs with equal weights, the TM cost of equal weights is multiplied by 0.90625.

[0088] In some examples, template matching-based BCW index derivation is applied to CUs coded in regular merge, template matching, adaptive decoder-side motion vector refinement, and MMVD modes. Additionally, in some examples, the bi-prediction weights for merge mode are expanded from {-2,3,4,5,10} to {1,2,3,4,5,6,7}. Furthermore, the negative bi-prediction weights {-2,10} for non-merge mode are replaced with positive weights {1,7}.

[0089] In some examples, decoder-side motion-refining template matching (TM) techniques can be used in video / image coding (e.g., VVC, ECM, etc.) to further improve compression efficiency. In TM mode, MV can be refined by constructing a template (e.g., current template) for a block (e.g., current block) in a current picture and determining the closest match between the template for the block in the current picture and multiple possible templates (e.g., multiple possible reference templates) in a reference picture. In one embodiment, the template for a block in the current picture may include a neighboring reconstructed sample to the left of the block and a neighboring reconstructed sample above the block.

[0090] 8 shows an example of template matching (800). The TM may be used to derive motion information for the current CU (801) (e.g., derive final motion information from initial motion information such as initial MV 802) by determining the closest match between a template (e.g., current template) (821) of a current CU (e.g., current block) (801) in a current picture (810) and a template (e.g., reference template) of multiple possible templates (e.g., one of the multiple possible templates is template (825)) in a reference picture (811). The template (821) of the current CU (801) may have any suitable shape and any suitable size.

[0091] In one embodiment, the template (821) of the current CU (801) includes a top template (822) and a left template (823). Each of the top template (822) and the left template (823) can have any suitable shape and any suitable size.

[0092] The top template (822) can include samples in one or more top adjacent blocks of the current CU (801). In one example, the top template (822) includes one or more rows of samples above the current CU (801). The left template (823) can include samples in one or more left adjacent blocks of the current CU (801). In one example, the left template (823) includes one or more columns of samples to the left of the current CU (801).

[0093] Each of multiple possible templates (e.g., template (825)) in the reference picture (811) corresponds to a template (821) in the current picture (810). In one embodiment, the initial MV (802) points from the current CU (801) to a reference block (803) in the reference picture (811). Each of the multiple possible templates (e.g., template (825)) in the reference picture (811) and the template (821) in the current picture (810) may have the same shape and size. For example, the template (825) for the reference block (803) includes a top template (826) in the reference picture (811) and a left template (827) in the reference picture (811). The top template (826) may include a sample above the reference block (803). The left template (827) may include a sample to the left of the reference block (803).

[0094] The TM cost may be determined based on a pair of templates, such as a template (e.g., a current template) (821) and a template (e.g., a reference template) (825). The TM cost may indicate a match between the template (821) and the template (825). An optimized MV (or final MV) may be determined based on a search around the initial MV (802) of the current CU (801) within a search range (815). The search range (815) may have any suitable shape and any suitable number of reference samples. In one example, the search range (815) within the reference picture (811) includes a [-L, L]pel range, where L is a positive integer such as 8 (e.g., 8 samples). For example, a difference (e.g., [0,1]) is determined based on the search range (815), and an intermediate MV is determined by adding the initial MV (802) and the difference (e.g., [0,1]). Based on the intermediate MV, an intermediate reference block and a corresponding template within the reference picture (811) may be determined. The TM cost can be determined based on the template (821) and an intermediate template in the reference picture (811). The TM cost can correspond to a difference (e.g., [0,0], [0,1], etc., corresponding to the initial MV (802)) determined based on the search range (815). In one example, the difference corresponding to the minimum TM cost is selected, and the optimized MV is the sum of the difference corresponding to the minimum TM cost and the initial MV (802). As described above, the TM can derive final motion information (e.g., the optimized MV) from the initial motion information (e.g., the initial MV 802).

[0095] In the example of FIG. 8, a better MV may be searched for around the initial motion vector of the current CU within a search range such as [-8pel, +8pel]. In some examples (e.g., ECM), template matching is also employed with some modifications. In one example, the search step size is determined by the AMVR mode. In another example, TMs can be cascaded in the bilateral matching process. In another example, template matching is also used to sort the indices of candidates in the merge candidate list and the AMVP candidate list.

[0096] In some examples, the TM process may be applied in an affine mode, such as an affine AMVP mode, an affine merge mode, or the like, and may be referred to as affine TM.

[0097] Figure 9 shows an example of a TM (900) in affine merge mode, etc. The template (921) of the current block (e.g., current CU) (901) can be defined similarly to the template (821) of Figure 8 in a TM applied to a translational motion model. The reference template (925) of a reference block in a reference picture can include multiple sub-block templates (e.g., 4x4 sub-blocks) pointed to by control point MV (CPMV)-derived MVs of neighboring sub-blocks (e.g., A0-A3 and L0-L3 as shown in Figure 9) at block boundaries.

[0098] In some examples, the search process of a TM (also referred to as an affine TM) applied in affine mode (e.g., affine merge mode) can start from CPMV0 while holding the other CPMV(s) constant (e.g., (i) CPMV1 if a four-parameter model is used, or (ii) CPMV1 and CPMV2 if a six-parameter model is used). The search can be performed horizontally and vertically. In one example, subsequent searches in the diagonal direction(s) are performed only if the zero vector is not the best difference vector found from the horizontal search and / or the vertical search. The affine TM can repeat the same search process for CPMV1. The affine TM can repeat the same search process for CPMV2 if a six-parameter model is used. If the zero vector is not the best difference vector from the previous iteration and the search process has iterated less than three times, the entire search process can be restarted (iterated) from the improved CPMV0 based on the improved CPMV.

[0099] In some examples (e.g., AV1), a technique called differential weighted bi-prediction may be used. For example, in AV1, a differential weighted predictor may be used for bi-prediction. A weighting factor (e.g., weight) is calculated for each pixel based on the difference between two reference pixels. Generally, if the difference between a pair of reference pixels is large, a larger weight is assigned to one of the two reference pixels. In one example, a binary code may be signaled at the coded block level to indicate which reference block has a larger weight when the pixel difference exceeds a certain threshold. The binary code may correspond to a reference index with a higher weight value in some examples. For example, a binary code of 0 may correspond to assigning a higher weight value to Reference 0 (indicating a reference pixel or reference block in a reference picture from Reference Picture List 0), and a binary code of 1 may correspond to assigning a higher weight value to Reference 1 (indicating a reference pixel or reference block in a reference picture from Reference Picture List 1).

[0100] For example, m(x,y) indicates the weighting factor of the reference 0 picture at pixel position (x,y). The sum of the total weighting is 64, and if the reference pixel difference at (x,y) is 16 or more, m(x,y) is set to a value different from the default constant.

[0101] In one example, equation (2) is used to assign m(x,y) to reference block 0 (a reference block in a reference picture obtained from reference picture list 0) based on a binary code.

number

[0102] In some examples, a binary code (also called a flag) is signaled at the block level to indicate which reference block in bi-prediction has a higher (or lower) weight value. For example, if the binary code is 0, the reference block obtained based on reference picture list 0 (e.g., reference 0 pixels) has a higher weight. If the binary code is 1, the reference block obtained based on reference picture list 1 (e.g., reference 1 pixels) has a higher weight.

[0103] In some examples, weights (also referred to as weight values ​​or weight coefficients) are determined by pixel based on the difference between the current pixel's reference pixels (the current pixel in the current block and a pixel in the co-located reference block). In one example, if the absolute value of the difference between the reference pixels is less than a threshold, default weights are assigned to the reference pixels according to a binary code. For example, if the total weight is 64 and the absolute value of the difference between the reference pixels is less than 16, a larger weight of 38 is assigned to the Reference 0 pixel and a smaller weight of 26 is assigned to the Reference 1 pixel in response to the binary code being 0. Similarly, if the total weight is 64 and the absolute value of the difference between the reference pixels is less than 16, a larger weight of 38 is assigned to the Reference 1 pixel and a smaller weight of 26 is assigned to the Reference 0 pixel in response to the binary code being 1.

[0104] Furthermore, if the absolute value of the difference between the reference pixels is 16 or more, the weight is determined according to equation (2).

[0105] Differential weighted bi-prediction utilizes pixel differences to derive bi-prediction weights at the pixel level. However, a flag (also referred to as a binary code) is signaled at the block level to indicate which reference block (e.g., from reference picture list 0 or from reference picture list 1) has a higher weight and is dominant (e.g., using an equation associated with code=0 in equation (2)). Some aspects of the present disclosure provide techniques for determining which reference block is dominant using reduced signaling overhead. For example, multiple weight candidates for bi-prediction include at least a first weight candidate for differential weighted bi-prediction with a first code value and a second weight candidate for differential weighted bi-prediction with a second code value. The encoder / decoder can calculate a first cost value for the first weight candidate and a second cost value for the second weight candidate, and select a weight candidate from the first weight candidate and the second weight candidate according to the first cost value and the second cost value. In differential weighted bi-prediction, the encoder / decoder can determine first and second weight values ​​based on a difference between a first reference pixel and a second reference pixel for a current pixel in a current block. The first reference pixel is in a first reference block from a first reference picture, and the second reference pixel is in a second reference block from a second reference picture. The encoder / decoder assigns first and second weight values ​​to the first reference pixel in the first reference block and the second reference pixel in the second reference block based on a code value associated with the selected weight candidate. The code value indicates which of the first and second reference blocks should be assigned a larger weight value. The encoder / decoder then reconstructs the current pixel based on a weighted sum of the first and second reference pixels according to the first and second weight values.

[0106] According to one aspect of the present disclosure, which reference block has a larger weight (e.g., using the first equation associated with sign=0 in Equation (2)) can be determined by using template matching (TM) when differential weighted bi-prediction is selected. In some examples, when a current template (also referred to as a current block) of a block to be coded is available, differential weighted bi-prediction is applied to the templates of reference block 0 (e.g., a reference block obtained based on reference picture list 0) and reference block 1 (e.g., a reference block obtained based on reference picture list 1) to obtain a reference template. The template matching (TM) cost can be calculated using any suitable calculation, such as sum of absolute differences (SAD), sum of absolute difference transforms (SATD), standard error of estimation (SEE), etc., between the current template and the reference template. Two differential weighted bi-prediction template matching processes are tested: one for reference block 0 with sign=0 in Equation (2) and another for reference block 1 with sign=0 in Equation (2).

[0107] For example, to perform a first test of the TM process for differential weighted bi-prediction, a binary code of 0 is used. The first equation in Equation (2) is used to determine the per-pixel weights of the template for reference block 0, and the second equation in Equation (2) is used to determine the per-pixel weights of the template for reference block 1. The template for reference block 0 and the template for reference block 1 are weighted accordingly to generate a reconstructed template referred to as the first reference template. Next, a first TM cost is calculated between the first reference template and the current template for the current block. Similarly, to perform a second test of the TM process for differential weighted bi-prediction, a binary code of 1 is used. The first equation in Equation (2) is used to determine the per-pixel weights of the template for reference block 1, and the second equation in Equation (2) is used to determine the per-pixel weights of the template for reference block 0. The template for reference block 0 and the template for reference block 1 are weighted accordingly to generate a reconstructed template referred to as the second reference template. Next, a second TM cost is calculated between the second reference template and the current template of the current block.

[0108] In some embodiments, the reference block i assigned the first equation (associated with code=0 in equation (2)) is derived from the minimum TM cost of the two tests of differential weighted bi-prediction. The reference block i with the minimum TM cost is selected by the first equation (associated with code=0 in equation (2)), where i is 0 or 1. There is no flag indicating which reference block has a higher weight. For example, if the first TM cost is lower than the second TM cost, reference block 0 is assigned a higher weight, for example, using the first equation in equation (2), and reference block 1 is assigned a lower weight, for example, using the second equation in equation (2). If the second TM cost is lower than the first TM cost, reference block 1 is assigned a higher weight, for example, using the first equation in equation (2), and reference block 0 is assigned a lower weight, for example, using the second equation in equation (2). Because no flag is signaled for the binary code, signaling costs are reduced.

[0109] In some embodiments, a flag is signaled to indicate the most likely reference block. In some examples, a list called a most likely reference block list is constructed using TM costs in ascending order. In some examples, this flag is context coded. For example, a most likely reference block list is constructed according to the TM costs using TM costs in ascending order. If the first TM cost is lower than the second TM cost, the most likely reference block list may be (reference block 0, reference block 1), where a flag of 0 indicates assigning a higher weight to reference block 0 and a flag of 1 indicates assigning a higher weight to reference block 1. If the first TM cost is higher than the second TM cost, the most likely reference block list may be (reference block 1, reference block 0), where a flag of 0 indicates assigning a higher weight to reference block 1 and a flag of 1 indicates assigning a higher weight to reference block 0. In one example, because the probability of the flag being 0 is much higher, signaling costs can be reduced by using context coding for the flag.

[0110] According to another aspect of the present disclosure, when differential weighted bi-prediction is selected, it can be determined which reference block has a larger weight (e.g., using the first equation associated with code=0 in equation (2)) by checking the boundary costs of two different reconstructions using binary codes of 0 and 1. In one example, the boundary of the current block is defined as the first row and the first column of the current block. The boundary smoothness can be determined by measuring the difference between the sample at the boundary of the current block and the adjacent reference sample.

[0111] FIG. 10 shows a diagram illustrating the calculation of boundary costs in some examples. In one example, the boundary of a current block (1001) is defined as the first row and first column of the current block, such as the sample indicated by "B" in FIG. 10. FIG. 10 also shows an upper reference sample (1010) (indicated by "A") located above the first row of the current block (1001) and a left reference sample (1020) (indicated by "L") located to the left of the first column of the current block (1001). Boundary smoothness (e.g., also referred to as boundary cost) can be measured as the difference between a sample (e.g., a reconstructed sample) at the boundary of the current block (1001) (e.g., indicated by "B") and an adjacent reference sample (e.g., an adjacent reconstructed sample, such as the upper reference sample indicated by "A" and the left reference sample indicated by "L"). Boundary smoothness can be calculated using any suitable difference measure, such as SAD, SATD, SEE, etc.

[0112] In some embodiments, two reconstructed blocks are generated by appending a predictive block using code=0 and a predictive block using code=1 to the residual block, and then the code value associated with the smallest boundary cost (a cost value that measures the boundary smoothness of the reconstructed block) is selected as the actual predicted block.

[0113] For example, to generate a first reconstructed block of a current block using differential weighted bi-prediction, a binary code of 0 is used. The first equation in Equation (2) is used to determine the per-pixel weights of reference block 0, and the second equation in Equation (2) is used to determine the per-pixel weights of reference block 1. Reference block 0 and reference block 1 are weighted accordingly to form a first predicted block. The first predicted block and a residual block (e.g., decoded from a coded video bitstream) are combined to generate a first reconstructed block of the current block. Then, a first boundary cost is calculated based on the difference between the boundary of the first reconstructed block and the adjacent reference sample. Similarly, to generate a second reconstructed block of the current block using differential weighted bi-prediction, a binary code of 1 is used. The first equation in Equation (2) is used to determine the per-pixel weights of reference block 1, and the second equation in Equation (2) is used to determine the per-pixel weights of reference block 0. Reference block 0 and reference block 1 are weighted accordingly to form a second predicted block. The second prediction block and the residual block (e.g., decoded from a video bitstream) are combined to generate a second reconstructed block of the current block. Then, a second boundary cost is calculated based on the difference between the boundary of the second reconstructed block and the adjacent reference sample. In some examples, if the first boundary cost is less than or equal to the second boundary cost, the binary code is determined to be 0, and if the first boundary cost is higher than the second boundary cost, the binary code is determined to be 1.

[0114] In some embodiments, a flag is signaled to indicate whether the most likely code value is applied, and the most likely code value is derived using a most likely reference block list. The most likely reference block list is constructed by using the boundary costs in ascending order. In some examples, this flag is context coded. For example, the most likely reference block list is constructed according to the boundary costs using the boundary costs in ascending order. If the first boundary cost is lower than the second boundary cost, the most likely reference block list may be (reference block 0, reference block 1), with a flag of 0 indicating a higher weight being assigned to reference block 0 and a flag of 1 indicating a higher weight being assigned to reference block 1. If the first boundary cost is higher than the second boundary cost, the most likely reference block list may be (reference block 1, reference block 0), with a flag of 0 indicating a higher weight being assigned to reference block 1 and a flag of 1 indicating a higher weight being assigned to reference block 0. In one example, because the flag is much more likely to be 0, signaling costs can be reduced by using context coding for the flag.

[0115] According to another aspect of the present disclosure, which reference block has a larger weight (e.g., using the first equation associated with code=0 in Equation (2)) can be determined when differential weighted bi-prediction is selected by the reference index used by the neighboring block. In one example, a block neighboring the current block may be coded using differential weighted bi-prediction with a reference block from reference picture list i (i is 0 or 1) with a larger weight, and then the current block may be coded using the reference block from reference picture list i with a larger weight when coding with differential weighted bi-prediction. Therefore, the binary code of the current block is not signaled.

[0116] According to another aspect of the present disclosure, a weighting method for a reference block is derived by applying different weighting methods to template samples of a current block. A predictor may include two template regions of reference block 0 and reference block 1 (e.g., a template region of reference block 0 and a template region of reference block 1). Because samples in all three template regions (e.g., a template region of reference block 0, a template region of reference block 1, and a template region of a current block) are fully available to both the encoder and the decoder, a weighting method with a smaller cost can be selected by comparing the costs of different weighting methods using BCW with different candidate weights or differential weighted bi-prediction with code=0 or code=1.

[0117] For example, possible weighting methods may include BCW with different candidate weights, differential weighted bi-prediction with a code of 0, and differential weighted bi-prediction with a code of 1. In one example, the BCW has five different candidate weights, and there are seven possible weighting methods, including BCW with five different candidate weights and differential weighted bi-prediction with two candidate code values. In some examples, each of the possible weighting methods may be applied to generate a reference template based on the template of reference block 0 and the template of reference block 1. The reference template and the current template are compared to calculate a template matching cost value.

[0118] In some embodiments, the weighting method with the smallest TM cost is selected.There is no flag indicating which weighting method is selected, and there is no syntax indicating which candidate weight of BCW or which code of differential weighted bi-prediction is selected.For example, from the TM cost values ​​associated with the possible weighting methods, the smallest TM cost value can be determined, and the weighting method associated with the smallest TM cost value can be selected to generate the final reconstruction of the current block.

[0119] In some embodiments, a flag is signaled to indicate the most likely weighting method. The most likely weighting method is BCW with multiple candidate weights or differential weighted bi-prediction with code=0 or code=1. Then, syntax is signaled to indicate which weight combination of BCW and differential weighted bi-prediction in the list is used. For example, the possible weighting methods may have two categories, such as a first category for BCW (with multiple candidate weights) and a second category for differential weighted bi-prediction (with code=0 or code=1). A flag is signaled to indicate which category of the possible weighting methods is used. If the first category is used, syntax is signaled to indicate which candidate weight is used. If the second category is used, syntax (e.g., a flag) is signaled to indicate the value of a binary code.

[0120] In one embodiment, a weight combination list is formed to include possible weighting methods, and the weight combination list is sorted using TM cost in ascending order. For example, the weight combination list includes possible weighting methods sorted based on TM cost in ascending order. In one example, if the first possible weighting method in the weight combination list is BCW, a flag of 0 indicates that the first category for BCW is used, and a flag of 1 indicates that the second category for differential weighted bi-prediction is used. If the first possible weighting method in the weight combination list is differential weighted bi-prediction, a flag of 0 indicates that the second category for differential weighted bi-prediction is used, and a flag of 1 indicates that the first category for BCW is used. It should be noted that the weighting methods of BCW with multiple candidate weights can be sorted based on TM cost values ​​in ascending order, and the weighting methods of differential weighted bi-prediction with differential code values ​​can be sorted based on TM cost values ​​in ascending order.

[0121] In some embodiments, the weighting method with the smallest TM cost is selected. There is no flag indicating which type of weighting method to select. If the BCW has the smallest TM cost, syntax is used to indicate which candidate weight of the BCW is selected. Otherwise, a flag is used to indicate which code of differential weighted bi-prediction is selected. For example, a flag indicating either the first category or the second category is not signaled. When the first possible weighting method in the sorted weight combination list is of the first category, the BCW is used, and syntax is used to indicate which candidate weight of the BCW is used. When the first possible weighting method in the sorted weight combination list is of the second category, differential weighted bi-prediction is used, and a flag can be signaled to indicate the value of the binary code.

[0122] In some embodiments, a weight combination list is constructed from all weighting combinations of BCW and differential weighted bi-prediction with a larger weight (e.g., the first equation associated with code=0 in equation (2)) for reference block 0 and reference block 1. TM is applied to all candidates in this list, and the list is sorted using an ascending TM cost. Syntax is signaled to indicate which candidate in the sorted list is selected. For example, a weight combination list is constructed to include BCW with multiple candidate weights (e.g., five different weights) and differential weighted bi-prediction with code values ​​of 0 and 1. The possible weighting methods in the weight combination list are sorted according to the ascending TM cost values. Then, syntax is signaled to indicate which candidate in the sorted weight combination list is used.

[0123] 11 shows a flowchart outlining a process (1100) according to one embodiment of the present disclosure. The process (1100) can be used in a video decoder. In various embodiments, the process (1100) is performed by a processing circuit, such as a processing circuit that performs the functions of the video decoder (110), a processing circuit that performs the functions of the video decoder (210), etc. In some embodiments, the process (1100) is implemented with software instructions, and thus, the processing circuit performs the process (1100) when it executes the software instructions. The process starts at (S1101) and proceeds to (S1110).

[0124] At (S1110), a coded video bitstream including coded information of a current block in a current picture is received, where the coded information indicates bi-prediction with weights selected from a plurality of weight candidates, the plurality of weight candidates including at least a first weight candidate for differential weighted bi-prediction with a first code value and a second weight candidate for differential weighted bi-prediction with a second code value.

[0125] In some examples, a coded video bitstream including coded information of a current block in a current picture is received. The coded information indicates bi-prediction with a weight selected from a plurality of weight candidates, the plurality of weight candidates including at least a first weight candidate and a second weight candidate. The first weight candidate is differential weighted bi-prediction with a first code value, and the second weight candidate is differential weighted bi-prediction with a second code value. The differential weighted bi-prediction with a code value refers to the use of pixel differences to derive weights for bi-prediction at a pixel level and the use of code values ​​to indicate the direction of larger weight values ​​for differential weighted bi-prediction.

[0126] At (S1120), a first cost value for the first weight candidate and a second cost value for the second weight candidate are calculated. For example, a first cost value for differential weighted bi-prediction using the first code value and a second cost value for differential weighted bi-prediction using the second code value are calculated.

[0127] In (S1130), a weight candidate is selected from the first weight candidate and the second weight candidate according to the first cost value and the second cost value. In one example, a selected code value is determined according to the first cost value and the second cost value. The selected weight candidate is associated with the selected code value.

[0128] At (S1140), a first weight value and a second weight value are calculated based on a difference between a first reference pixel and a second reference pixel for a current pixel in a current block, the first reference pixel being within a first reference block from a first reference picture, and the second reference pixel being within a second reference block from a second reference picture.

[0129] At (S1150), first and second weight values ​​are assigned to the first and second reference pixels in the first and second reference blocks based on a selected code value associated with the selected weight candidate, the selected code value indicating a particular reference block among the first and second reference blocks to which a larger weight value should be assigned.

[0130] In (S1160), the current pixel is reconstructed based on a weighted sum of the first reference pixel and the second reference pixel according to the first weight value and the second weight value.

[0131] In some examples, to calculate the first cost value and the second cost value, a first reference template for a current template of a current block is generated according to a first weight candidate, a first template matching cost between the first reference template and the current template is calculated as the first cost value, a second reference template is generated for the current template of a current block according to a second weight candidate, and a second template matching cost between the second reference template and the current template is calculated as the second cost value.

[0132] In some instances, the selected weight candidate is the one with the lowest cost value.

[0133] In some examples, to select a weight candidate, the first reference block and the second reference block are sorted in a list according to a first cost value for the first weight candidate associated with a first code value and a second cost value for the second weight candidate associated with a second code value, where the first code value indicates a first reference block with a higher weight value and the second code value indicates a second reference block with a higher weight value. A flag is determined according to the context model, and the flag indicates a weight candidate according to the most likely reference block from the list. For example, the list includes the reference blocks in order of most likely. Then, in one example, a flag of 0 indicates that the first most likely reference block in the list corresponds to the selected weight candidate, and a flag of 0 indicates that the first most likely reference block in the list does not correspond to the selected weight candidate.

[0134] In some examples, to calculate the first cost value and the second cost value, a first reconstructed block of the current block is generated according to a first weight candidate, and then a first boundary cost between a boundary of the first reconstructed block and a reconstructed adjacent sample of the current block is calculated as the first cost value.Furthermore, a second reconstructed block of the current block is generated according to a second weight candidate, and then a second boundary cost between a boundary of the second reconstructed block and a reconstructed adjacent sample of the current block is calculated as the second cost value.

[0135] In one example, to generate a first reconstructed block, the residual block of the current block is combined with a first predicted block predicted by differential weighted bi-prediction using a first weight candidate, and to generate a second reconstructed block, the residual block of the current block is combined with a second predicted block predicted by differential weighted bi-prediction using a second weight candidate.

[0136] In some examples, to select a weight candidate, neighboring blocks of a current block coded using differential weighting bi-prediction are identified, and the weight candidate is selected according to a reference index used by the neighboring blocks, where the reference index corresponds to the code value of the weight candidate. For example, if the code is 0, the reference index is 0, and if the code is 1, the reference index is 1.

[0137] In some examples, the multiple weight candidates include a first weight candidate and a second weight candidate associated with different code values ​​for differential weighted bi-prediction, and multiple weight candidates for bi-prediction with coding unit level weights (BCW) having different weights.

[0138] In one example, a flag is decoded from coded information of a current block, and the flag indicates one of differential weighted bi-prediction and BCW. In response to the flag indicating differential weighted bi-prediction, a first syntax (e.g., a flag) indicating a weight candidate from a first weight candidate and a second weight candidate is decoded from the coded information. In response to the flag indicating BCW, a second syntax indicating a particular weight candidate of the BCW is decoded from the coded information.

[0139] In some examples, a template matching cost value is calculated for each of a plurality of weight candidates, and the weight candidate having the lowest template matching value is selected.

[0140] In some examples, template matching cost values ​​are calculated for each of a plurality of weight candidates. In response to the lowest template matching cost value being associated with differential weighted bi-prediction, a first syntax is decoded from the coded information of the current block, the first syntax indicating a weight candidate from the first weight candidate and the second weight candidate. In response to the lowest template matching cost value being associated with the BCW, a second syntax is decoded from the coded information of the current block, the second syntax indicating a particular weight candidate of the BCW.

[0141] In some examples, a template matching cost value is calculated for each of the multiple weight candidates, the multiple weight candidates are sorted according to the template matching cost values, and a syntax is decoded from the coded information of the current block, where the syntax indicates a weight candidate from the sorted multiple weight candidates.

[0142] The process then proceeds to (S1199) and ends.

[0143] The process 1100 may be adapted as desired. Step(s) in the process 1100 may be modified and / or omitted. Additional step(s) may be added. Any suitable order of implementation may be used.

[0144] 12 shows a flowchart outlining a process (1200) according to one embodiment of the present disclosure. The process (1200) can be used in a video encoder. In various embodiments, the process (1200) is performed by a processing circuit, such as a processing circuit that performs the functions of the video encoder (103), a processing circuit that performs the functions of the video encoder (303), etc. In some embodiments, the process (1200) is implemented with software instructions, and thus, the processing circuit performs the process (1200) when it executes the software instructions. The process starts at (S1201) and proceeds to (S1210).

[0145] At (S1210), it is determined to use bi-prediction with weights selected from a plurality of weight candidates to code a current block in a current picture, where the plurality of weight candidates include at least a first weight candidate for differential weighted bi-prediction with a first code value and a second weight candidate for differential weighted bi-prediction with a second code value.

[0146] At (S1220), a first cost value for the first weight candidate and a second cost value for the second weight candidate are calculated.

[0147] In (S1230), a weight candidate is selected from the first weight candidate and the second weight candidate according to the first cost value and the second cost value.

[0148] At (S1240), according to the selected weight candidate, a first weight value and a second weight value are calculated based on a difference between a first reference pixel and a second reference pixel for a current pixel in a current block, the first reference pixel being within a first reference block from a first reference picture, and the second reference pixel being within a second reference block from a second reference picture.

[0149] In (S1250), a first weight value and a second weight value are assigned to a first reference pixel in a first reference block and a second reference pixel in a second reference block based on a code value associated with the selected weight candidate, and the code value indicates a particular reference block among the first reference block and the second reference block to which a larger weight value should be assigned.

[0150] In (S1260), the current pixel is reconstructed based on a weighted sum of the first reference pixel and the second reference pixel according to the first weight value and the second weight value.

[0151] In some examples, to calculate the first cost value and the second cost value, a first reference template for a current template of a current block is generated according to a first weight candidate, a first template matching cost between the first reference template and the current template is calculated as the first cost value, a second reference template is generated for the current template of a current block according to a second weight candidate, and a second template matching cost between the second reference template and the current template is calculated as the second cost value.

[0152] In some instances, the selected weight candidate is the one with the lowest cost value.

[0153] In some examples, to select a weight candidate, the first reference block and the second reference block are sorted in a list according to a first cost value for the first weight candidate associated with a first code value and a second cost value for the second weight candidate associated with a second code value, where the first code value indicates a first reference block with a higher weight value and the second code value indicates a second reference block with a higher weight value. A flag is coded according to the context model, and the flag indicates a weight candidate according to the most likely reference block from the list. For example, the list includes the reference blocks in order of most likely. Then, in one example, a flag of 0 indicates that the first most likely reference block in the list corresponds to the selected weight candidate, and a flag of 0 indicates that the first most likely reference block in the list does not correspond to the selected weight candidate.

[0154] In some examples, to calculate the first cost value and the second cost value, a first reconstructed block of the current block is generated according to a first weight candidate, and then a first boundary cost between a boundary of the first reconstructed block and a reconstructed adjacent sample of the current block is calculated as the first cost value.Furthermore, a second reconstructed block of the current block is generated according to a second weight candidate, and then a second boundary cost between a boundary of the second reconstructed block and a reconstructed adjacent sample of the current block is calculated as the second cost value.

[0155] In one example, to generate a first reconstructed block, the residual block of the current block is combined with a first predicted block predicted by differential weighted bi-prediction using a first weight candidate, and to generate a second reconstructed block, the residual block of the current block is combined with a second predicted block predicted by differential weighted bi-prediction using a second weight candidate.

[0156] In some examples, to select a weight candidate, neighboring blocks of a current block coded using differential weighting bi-prediction are identified, and the weight candidate is selected according to a reference index used by the neighboring blocks, where the reference index corresponds to the code value of the weight candidate. For example, if the code is 0, the reference index is 0, and if the code is 1, the reference index is 1.

[0157] In some examples, the multiple weight candidates include a first weight candidate and a second weight candidate associated with different code values ​​for differential weighted bi-prediction, and multiple weight candidates for bi-prediction with coding unit level weights (BCW) having different weights.

[0158] In one example, a flag is encoded in the coded information of the current block, and the flag indicates one of differential weighted bi-prediction and BCW for the selected weight candidate. In response to the flag indicating differential weighted bi-prediction, a first syntax (e.g., a flag) indicating a weight candidate from the first weight candidate and the second weight candidate is encoded in the coded information. In response to the flag indicating BCW, a second syntax indicating a particular weight candidate of BCW is encoded in the coded information.

[0159] In some examples, a template matching cost value is calculated for each of a plurality of weight candidates, and the weight candidate having the lowest template matching value is selected.

[0160] In some examples, template matching cost values ​​are calculated for each of a plurality of weight candidates. In response to the lowest template matching cost value being associated with differential weighted bi-prediction, a first syntax is encoded in the coded information of the current block, the first syntax indicating a weight candidate from the first weight candidate and the second weight candidate. In response to the lowest template matching cost value being associated with the BCW, a second syntax is encoded in the coded information of the current block, the second syntax indicating a particular weight candidate of the BCW.

[0161] In some examples, a template matching cost value is calculated for each of the multiple weight candidates, the multiple weight candidates are sorted according to the template matching cost value, and a syntax is encoded in the coded information of the current block, the syntax indicating a weight candidate from the sorted multiple weight candidates.

[0162] The process then proceeds to (S1299) and ends.

[0163] The process 1200 may be adapted as desired. Step(s) in the process 1200 may be modified and / or omitted. Additional step(s) may be added. Any suitable order of implementation may be used.

[0164] The techniques described above can be implemented using computer-readable instructions and as computer software physically stored on one or more computer-readable media. For example, Figure 13 illustrates a computer system (1300) suitable for implementing certain embodiments of the subject matter of this disclosure.

[0165] Computer software can be coded using any suitable machine code or computer language that can be subjected to mechanisms such as assembly, compilation, linking, etc. to create code containing instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc. directly, or via interpretation, microcode execution, etc.

[0166] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, internet of things devices, and the like.

[0167] 13 of the computer system (1300) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of the computer system (1300).

[0168] The computer system (1300) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0169] The input human interface devices may include one or more (only one of each is shown) of a keyboard (1301), a mouse (1302), a trackpad (1303), a touchscreen (1310), a data glove (not shown), a joystick (1305), a microphone (1306), a scanner (1307), and a camera (1308).

[0170] The computer system (1300) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1310), data gloves (not shown), or joystick (1305), although haptic feedback devices that do not function as input devices may also be present), audio output devices (e.g., speakers (1309), headphones (not shown)), visual output devices (e.g., screens (1310) including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output or three-dimensional hypervisor output via means such as stereo output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0171] The computer system (1300) may also include human-accessible storage devices and their associated media, such as optical media or similar media (1321) including CD / DVDROM / RW (1320) with CDs / DVDs, thumb drives (1322), removable hard drives or solid state drives (1323), legacy magnetic media such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown), etc.

[0172] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0173] The computer system 1300 may also include an interface 1354 to one or more communication networks 1355. The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant, or the like. Examples of networks include local area networks such as Ethernet; cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, and the like; TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and vehicular and industrial networks including CAN Bus. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port or peripheral bus 1349 (e.g., a USB port on the computer system 1300). Others are generally integrated into the core of the computer system 1300 by attachment to a system bus, as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 1300 can communicate with other entities. Such communications may be unidirectional, receive only (e.g., broadcast TV), transmit only (e.g., CANbus to a particular CANbus device), or bidirectional, for example, to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used with each of these networks and network interfaces, as described above.

[0174] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1340) of the computer system (1300).

[0175] The cores (1340) may include one or more central processing units (CPUs) (1341), graphics processing units (GPUs) (1342), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1343), task-specific hardware accelerators (1344), graphics adapters (1350), etc. These devices may be connected via a system bus (1348), along with read-only memory (ROM) (1345), random access memory (1346), internal mass storage such as an internal non-user-accessible hard drive, SSD, etc. (1347). In some computer systems, the system bus (1348) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripherals may be attached directly to the core's system bus (1348) or via a peripheral bus (1349). In one example, a screen (1310) may be connected to the graphics adapter (1350). Peripheral bus architectures include PCI, USB, etc.

[0176] The CPU (1341), GPU (1342), FPGA (1343), and accelerator (1344) can execute specific instructions that, in combination, can constitute the aforementioned computer code. That computer code can be stored in ROM (1345) or RAM (1346). Transient data can also be stored in RAM (1346), while permanent data can be stored, for example, in internal mass storage (1347). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory, which can be closely associated with one or more of the CPU (1341), GPU (1342), mass storage (1347), ROM (1345), RAM (1346), etc.

[0177] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0178] By way of example and not limitation, a computer system having the architecture (1300), and in particular the core (1340), can provide functionality as a result of processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage as introduced above, as well as media associated with specific storage of the core (1340) that is non-transitory in nature, such as core internal mass storage (1347) or ROM (1345). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1340). The computer-readable media can include one or more memory devices or chips, depending on particular needs. The software can cause the core (1340), and in particular the processor (including CPU, GPU, FPGA, etc.) therein, to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM (1346) and modifying such data structures in accordance with the software-defined processes. Additionally, or alternatively, a computer system may provide functionality as a result of logic embodied in hardwired or otherwise circuitry (e.g., accelerator (1344)), which may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software may encompass logic, and vice versa, where appropriate. References to computer-readable media may encompass, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both. The present disclosure encompasses any suitable combination of hardware and software.

[0179] The use of "at least one of" or "one of" in this disclosure is intended to include any one or combination of the listed elements. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; at least one of A through C is intended to include A only, B only, C only, or any combination thereof. Reference to one of A or B and one of A and B is intended to include A or B or (A and B). The use of "one of" does not exclude any combination of the listed elements, where applicable, such as when the elements are not mutually exclusive.

[0180] While this disclosure has described several exemplary embodiments, there are alterations, substitutions, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. 1. A method of video decoding, comprising: receiving a coded video bitstream including coded information of a current block in a current picture, the coded information indicating bi-prediction with a weight selected from a plurality of weight candidates, the plurality of weight candidates including at least a first weight candidate for differential weighted bi-prediction with a first code value and a second weight candidate for the differential weighted bi-prediction with a second code value, the differential weighted bi-prediction using a pixel difference to derive a bi-prediction weight at a pixel level with a code value indicating a direction of a larger weight value for the differential weighted bi-prediction; calculating a first cost value for the differential weighted bi-prediction using the first code value and a second cost value for the differential weighted bi-prediction using the second code value; determining a code value selected from the first code value and the second code value according to the first cost value and the second cost value; determining a first weight value and a second weight value based on a difference between a first reference pixel and a second reference pixel for a current pixel in the current block, the first reference pixel being in a first reference block from a first reference picture and the second reference pixel being in a second reference block from a second reference picture; reconstructing the current pixel based on a weighted sum of the first reference pixel and the second reference pixel according to the first weight value, the second weight value, and the selected code value; Including, method.

2. Calculating the first cost value and the second cost value includes: generating a first reference template for a current template of the current block according to the first code value; calculating a first template matching cost between the first reference template and the current template as the first cost value; generating a second reference template for the current template of the current block according to the second code value; calculating a second template matching cost between the second reference template and the current template as the second cost value; further comprising: The method of claim 1.

3. Determining the selected code value comprises: determining the selected code value having the lowest cost value; The method of claim 1.

4. Determining the selected code value comprises: sorting the first reference blocks and the second reference blocks in a list according to the first cost value associated with the first code value and the second cost value associated with the second code value, the first code value indicating the first reference block having a higher weight value and the second code value indicating the second reference block having a higher weight value; determining a flag indicating a most likely reference block from said list according to a context model; further comprising: The method of claim 1.

5. Calculating the first cost value and the second cost value includes: generating a first reconstructed block for the current block according to the first code value; calculating a first boundary cost between a boundary of the first reconstructed block and a reconstructed adjacent sample of the current block as the first cost value; generating a second reconstructed block for the current block according to the second code value; calculating a second boundary cost between a boundary of the second reconstructed block and the reconstructed adjacent sample of the current block as the second cost value; further comprising: The method of claim 1.

6. generating the first reconstruction block includes: combining a residual block of the current block with a prediction block predicted by the differential weighted bi-prediction using the first code value to generate the first reconstructed block. The method of claim 5.

7. Determining the selected code value comprises: determining that neighboring blocks of the current block are coded using the differential weighted bi-prediction coding method; determining the selected code value according to a reference index used by the neighboring block; further comprising: The method of claim 1.

8. the plurality of weight candidates include the first weight candidate and the second weight candidate associated with different code values ​​for the differential weighted bi-prediction, and a plurality of weight candidates for bi-prediction with coding unit level weights (BCW) having different weights; The method of claim 1.

9. decoding a flag from the coded information of the current block indicating one of the differential weighted bi-prediction and the BCW; decoding a first syntax indicating a weight candidate from the first weight candidate and the second weight candidate in response to the flag indicating the differential weighted bi-prediction; decoding a second syntax indicating a particular weight candidate for the BCW in response to the flag indicating the BCW; further comprising: The method of claim 8.

10. calculating a template matching cost value for each of the plurality of weight candidates; selecting the weight candidate with the lowest template matching value; further comprising: The method of claim 8.

11. calculating a template matching cost value for each of the plurality of weight candidates; In response to a lowest template matching cost value being associated with the differential weighted bi-prediction, decoding, from the coded information of the current block, a first syntax indicating a weight candidate from the first weight candidate and the second weight candidate; decoding, in response to the lowest template matching cost value being associated with the BCW, from the coded information of the current block, a second syntax indicating a particular candidate weight for the BCW; further comprising: The method of claim 8.

12. calculating a template matching cost value for each of the plurality of weight candidates; sorting the plurality of weight candidates according to the template matching cost values; decoding, from the coded information of the current block, syntax indicating weight candidates from the reordered plurality of weight candidates; further comprising: The method of claim 8.

13. 1. An apparatus for video decoding, comprising: receiving a coded video bitstream including coded information of a current block in a current picture, the coded information indicating bi-prediction with a weight selected from a plurality of weight candidates, the plurality of weight candidates including at least a first weight candidate for differential weighted bi-prediction with a first code value and a second weight candidate for the differential weighted bi-prediction with a second code value, the differential weighted bi-prediction using a pixel difference to derive bi-prediction weights at a pixel level with a code value indicating a direction of a larger weight value for the differential weighted bi-prediction; calculating a first cost value for the difference weighted bi-prediction using the first code value and a second cost value for the difference weighted bi-prediction using the second code value; determining a code value selected from the first code values ​​according to the first cost value and the second cost value; determining a first weight value and a second weight value based on a difference between a first reference pixel and a second reference pixel for a current pixel in the current block, the first reference pixel being in a first reference block from a first reference picture and the second reference pixel being in a second reference block from a second reference picture; reconstructing the current pixel based on a weighted sum of the first reference pixel and the second reference pixel according to the first weight value, the second weight value, and the selected code value; A processing circuit configured as follows: An apparatus comprising:

14. The processing circuitry generating a first reference template for a current template of the current block according to the first code value; calculating a first template matching cost between the first reference template and the current template as the first cost value; generating a second reference template for the current template of the current block according to the second code value; calculating a second template matching cost between the second reference template and the current template as the second cost value; It is configured as follows:

14. The apparatus of claim 13.

15. The processing circuitry configured to determine the selected code value having the lowest cost value.

14. The apparatus of claim 13.

16. The processing circuitry sorting the first reference blocks and the second reference blocks in a list according to the first cost value associated with the first code value and the second cost value associated with the second code value, the first code value indicating the first reference block having a higher weight value and the second code value indicating the second reference block having a higher weight value; determining a flag indicating the most likely reference block from said list according to a context model; It is configured as follows:

14. The apparatus of claim 13.

17. The processing circuitry generating a first reconstructed block for the current block according to the first code value; Calculating a first boundary cost between a boundary of the first reconstructed block and a reconstructed adjacent sample of the current block as the first cost value; generating a second reconstructed block for the current block according to the second code value; calculating a second boundary cost between a boundary of the second reconstructed block and the reconstructed adjacent sample of the current block as the second cost value; It is configured as follows:

14. The apparatus of claim 13.

18. The processing circuitry and combining a residual block of the current block with a prediction block predicted by the differential weighted bi-prediction using the first code value to generate the first reconstructed block.

18. The apparatus of claim 17.

19. The processing circuitry determining that a neighboring block of the current block is coded using the differential weighted bi-prediction coding method; determining the selected code value according to a reference index used by the neighboring block; It is configured as follows:

14. The apparatus of claim 13.

20. the plurality of weight candidates include the first weight candidate and the second weight candidate associated with different code values ​​for the differential weighted bi-prediction, and a plurality of weight candidates for bi-prediction with coding unit level weights (BCW) having different weights; 14. The apparatus of claim 13.

Images