Method and apparatus for video coding, and computer program

Abstract

To provide methods and apparatuses for video encoding / decoding.SOLUTION: A method of an apparatus for video decoding includes: decoding prediction information of a current block in a current picture from a coded video bitstream, the prediction information being indicative of an inter prediction mode having a probability that a refinement technique based on a first reference picture and a second reference picture is used; determining whether a first equal weighting condition of chroma components from the first reference picture and the second reference picture is satisfied; and disabling, in response to the fact that the first equal weighting condition of the chroma components from the first reference picture and the second reference picture is not satisfied, the refinement technique in a reconstruction of a sample in the current block.SELECTED DRAWING: Figure 12

Description

[Background technology]

[0001] Incorporation by Reference This application claims the benefit of priority to U.S. Patent Application No. 16 / 851,052, entitled "METHOD AND APPARATUS FOR VIDEO CODING," filed April 16, 2020, which claims the benefit of priority to U.S. Provisional Application No. 62 / 836,598, entitled "CONDITIONS FOR APPLYING DMVR / BDOF," filed April 19, 2019. The entire disclosures of the prior applications are hereby incorporated by reference in their entireties.

[0002] Technical Field This disclosure describes embodiments generally relating to video coding.

[0003] background The background discussion provided in this application is intended to generally present the context of the present disclosure. Work under the name of the current inventor is not admitted, expressly or impliedly, as prior art to the present disclosure to the extent that such work is described in this background section or in a descriptive manner that would not otherwise qualify as prior art at the time of filing.

[0004] Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luminance samples and associated chrominance samples. The sequence of pictures can have a fixed or variable picture rate (informally known as frame rate) of, for example, 60 pictures / second, i.e., 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luminance sample resolution at a frame rate of 60 Hz) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage space.

[0005] One of the goals of video coding and decoding can be said to be the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements, in some cases by more than one order of magnitude. Both lossless and non-lossless compression, as well as combinations thereof, can be used. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal. When using non-lossless compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application. For video, non-lossless compression is widely used. The amount of distortion that is tolerated depends on the application, e.g. users of certain consumer streaming applications may be able to tolerate higher distortion than users of television distribution applications. The achievable compression ratios can reflect that higher tolerable / bearable distortion may result in higher compression ratios.

[0006] Motion compensation may be a non-lossless compression technique, where blocks of sample data from a previously reconstructed picture or part of it (reference picture) are used to predict a newly reconstructed picture or part of a picture, after being spatially shifted in a direction indicated by a motion vector (hereafter MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two dimensions X and Y, or three dimensions, the third of which is an indication of the reference picture in use (the latter may indirectly be a temporal dimension).

[0007] In some video compression techniques, the MV applicable to a particular area of ​​sample data can be predicted from other MVs, e.g., those associated with other areas of sample data that are spatially adjacent to the area being reconstructed and that precede that MV in decoding order. In this way, the amount of data required to code the MV can be significantly reduced, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, when coding an input video signal derived from a camera (known as natural video), since there is a statistical likelihood that areas larger than the area to which a single MV is applicable move in similar directions and can therefore in some cases be predicted using similar motion vectors derived from MVs of neighboring areas. This results in an MV that, for a given area, is found to be similar or identical to the MV predicted from the surrounding MVs, which, after entropy coding, can be represented with fewer bits than would be used to code the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., an MV) derived from the original signal (i.e., a sample stream). In other cases, the MV prediction itself may be non-lossless due to rounding errors, for example when computing a predictor from some surrounding MVs.

[0008] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, “High Efficiency Video Coding”, December 2016). Among the many MV prediction mechanisms provided by H.265, the one described in this application is a technique hereafter referred to as “spatial merging”.

[0009] Referring to Figure 1, a current block (101) contains samples that have been found by the encoder during a motion search process to be predictable from a previous block of the same size but spatially shifted. Instead of coding its MV directly, the MV can be derived from metadata associated with one or more reference pictures, e.g., from the most recent reference picture (in decoding order), using MVs associated with any of five neighboring blocks (102 to 106, respectively), denoted A0, A1, and B0, B1, B2. In H.265, MV prediction can use predictors from the same reference picture as the neighboring blocks use. Summary of the Invention

[0010] Aspects of the disclosure provide a method and apparatus for video encoding / decoding. In some examples, the apparatus for video decoding includes a receiving circuit and a processing circuit. For example, the processing circuit decodes prediction information of a current block in a current picture from a coded video bitstream. The prediction information indicates an inter-prediction mode that may utilize a refinement technique based on a first reference picture and a second reference picture. The processing circuit determines whether a first equal weighting condition of chroma components from the first reference picture and the second reference picture is satisfied. In response to not satisfying the first equal weighting condition of chroma components from the first reference picture and the second reference picture, the processing circuit disables the refinement technique in the reconstruction of samples in the current block.

[0011] In some embodiments, the processing circuit disables refinement techniques in the reconstruction of luma samples in the current block in response to not satisfying a first equal-weighting condition for chroma components from the first reference picture and the second reference picture. In some examples, the processing circuit determines whether a second equal-weighting condition for luma components from the first reference picture and the second reference picture is satisfied. The processing circuit then disables refinement techniques in the reconstruction of luma samples in the current block in response to not satisfying at least one of the first equal-weighting condition for chroma components and the second equal-weighting condition for luma components.

[0012] In some embodiments, the processing circuitry disables refinement techniques in the reconstruction of chroma samples in the current block in response to not satisfying a first equal weighting condition for chroma components from the first reference picture and the second reference picture.

[0013] It should be noted that the refinement techniques may include at least one of bidirectional optical flow (BDOF) and decoder-side motion vector refinement (DMVR).

[0014] In some embodiments, one of the first reference picture and the second reference picture has a picture order count greater than the current picture, and the other of the first reference picture and the second reference picture has a picture order count less than the current picture.

[0015] In some embodiments, the processing circuit determines that an equal weighting condition for chroma components from the first reference picture and the second reference picture is not satisfied based on at least one of a first flag of a chroma weight of the first reference picture and a second flag of a chroma weight of the second reference picture not being equal to zero.

[0016] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a video decoding method for video decoding. [Brief description of the drawings]

[0017] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings.

[0018] [Figure 1] FIG. 2 is a schematic diagram of a current block and its surrounding spatial merging candidates in one example.

[0019] [Diagram 2] FIG. 2 is a schematic diagram of a simplified block diagram of a communication system (200) according to one embodiment.

[0020] [Diagram 3] FIG. 3 is a schematic diagram of a simplified block diagram of a communication system (300) according to one embodiment.

[0021] [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment.

[0022] [Diagram 5] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment.

[0023] [Figure 6] 4 shows a block diagram of an encoder according to another embodiment;

[0024] [Figure 7] 4 shows a block diagram of a decoder according to another embodiment;

[0025] [Figure 8]An example of an extended coding unit area in bi-directional optical flow (BDOF) is shown.

[0026] [Figure 9] This shows an example of decoder side vector refinement (DMVR).

[0027] [Figure 10A] The following is a list of conditions for applying the BDOF technique. [Figure 10B] The following is a list of conditions for applying the BDOF technique.

[0028] [Figure 11A] The following is a list of conditions for applying DMVR technology. [Figure 11B] The following is a list of conditions for applying DMVR technology.

[0029] [Figure 12] 1 shows a flowchart outlining an example process according to some embodiments of the present disclosure.

[0030] [Figure 13] 1 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] FIG. 2 illustrates a simplified block diagram of a communication system (200) according to one embodiment of the present disclosure. The communication system (200) includes a plurality of terminal devices that can communicate with each other, for example, via a network (250). For example, the communication system (200) includes a first pair of terminal devices (210) and (220) interconnected via the network (250). In the example of FIG. 2, the first pair of terminal devices (210) and (220) perform a unidirectional transmission of data. For example, the terminal device (210) can code video data (e.g., a stream of video pictures captured by the terminal device (210)) for transmission to the other terminal device (220) via the network (250). The encoded video data can be transmitted in the form of one or more coded video bitstreams. The terminal device (220) can receive the coded video data from the network (250), decode the coded video data to reconstruct the video picture, and display the video picture according to the reconstructed video data. The unidirectional data transmission may be common in media serving applications, etc.

[0032] In another example, the communication system (200) includes a second pair of terminal devices (230) and (240) for bidirectional transmission of coded video data, which may occur, for example, during a video conference. With respect to the bidirectional transmission of data, for example, each of the terminal devices (230) and (240) may code video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices (230) and (240) over the network (250). Each of the terminal devices (230) and (240) may also receive coded video data transmitted by the other of the terminal devices (230) and (240), may decode the coded video data to reconstruct the video pictures, and may display the video pictures on an accessible display device according to the reconstructed video data.

[0033] In the example of FIG. 2, the terminal devices (210), (220), (230), (240) are shown as servers, personal computers, and smartphones, although the principles of the present disclosure are not so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (250) may represent any number of networks that carry coded video data between the terminal devices (210), (220), (230), and (240), including, for example, wired (wired) and / or wireless communication networks. The communication network (250) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of the present disclosure, the architecture and topology of the network (250) may not be important to the operation of the present disclosure, unless otherwise described below.

[0034] Figure 3 illustrates the placement of a video encoder and video decoder in a streaming environment as an example application of the disclosed subject matter, which is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media (including CDs, DVDs, memory sticks, etc.), etc.

[0035] The streaming system may include a video source (301), e.g., a digital camera, and may include a capture subsystem (313) capable of generating a stream of, e.g., uncompressed video pictures (302). In one example, the stream of video pictures (302) includes samples captured by a digital camera. The stream of video pictures (302), depicted as a thick line to emphasize the larger amount of data compared to the encoded video data (304) (or coded video bitstream), may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (304) (or encoded video bitstream (304)), depicted as thin lines to emphasize the smaller amount of data compared to the stream of video pictures (302), can be stored in the streaming server (305) for future use. One or more streaming client subsystems, such as the client subsystems (306) and (308) of FIG. 3, can access the streaming server (305) to retrieve copies (307) and (309) of the encoded video data (304). The client subsystem (306) can include a video decoder (310), for example within the electronic device (330). The video decoder (310) decodes the incoming copy of the encoded video data (307) and generates an output stream of video pictures (311) that can be rendered on a display (312) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (304), (307), and (309) (e.g., a video bitstream) may be encoded according to a particular video coding / compression standard.Examples of these standards include ITU-T Recommendation H.265. In one example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0036] It should be noted that electronic devices (320) and (330) can include other components (not shown). For example, electronic device (320) can include a video decoder (not shown) and electronic device (330) can also include a video encoder (not shown).

[0037] 4 shows a block diagram of a video decoder (410) according to one embodiment of the present disclosure. The video decoder (410) may be included in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used in place of the video decoder (310) in the example of FIG. 3.

[0038] The receiver (431) may receive one or more coded video sequences to be decoded by the video decoder (410); in the same or another embodiment, it may receive one coded video sequence at a time, where the decoding of each coded video sequence is independent of the other coded video sequences. The coded video sequences may be received from a channel (401), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (431) may receive the coded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be transferred using respective entities (not shown). The receiver (431) may separate the coded video sequences from the other data. To address network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420), hereafter referred to as the "parser (420)". In certain applications, the buffer memory (415) is part of the video decoder (410). In other cases, it may be outside the video decoder (410) (not shown). In yet another example, there may be a buffer memory (not shown) outside the video decoder (410), for example to deal with network jitter, and even another buffer memory (415) inside the video decoder (410), for example to handle playback timing. If the receiver (431) is receiving data from a store-and-forward device with sufficient bandwidth and controllability, or from a synchronous network, the buffer memory (415) may not be necessary or may be small.For use in a best effort packet network such as the Internet, a buffer memory (415) may be required, which may be relatively large and may advantageously be adaptively sized and may be implemented at least in part in an operating system or similar element (not shown) outside the video decoder (410).

[0039] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the coded video sequence. These categories of symbols include information used to manage the operation of the video decoder (410) and potentially information for controlling a rendering device such as a rendering device (412) (e.g., a display screen) that is not an integral part of the electronic device (430) but may be coupled to the electronic device (430) as shown in FIG. 4. The control information for the rendering device may be in the form of supplemental enhancement information (SEI message) or video usability information (VUI) parameter set fragments (not shown). The parser (420) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context effects, etc. The parser (420) can extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on the at least one parameter corresponding to the group. The subgroups can include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (420) can also extract from the coded video sequence information such as transform coefficients, quantization parameter values, motion vectors, etc.

[0040] The parser (420) is capable of performing an entropy decoding / parsing process on the video sequence received from the buffer memory (415) to generate symbols (421).

[0041] The reconstruction of the symbols (421) may include different units depending on the type of coded video picture or part thereof (inter and intra pictures, inter and intra blocks) and other factors. Which units are included and how can be controlled by subgroup control information parsed by the parser (420) from the coded video sequence. The flow of such subgroup control information between the parser (420) and the following units is not depicted for clarity.

[0042] Beyond the functional blocks already mentioned, the video decoder (410) may be conceptually subdivided into a number of functional units as described below. In a practical implementation operating within commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is adequate.

[0043] The first unit is a scalar / inverse transform unit (451), which receives the quantized transform coefficients as well as control information (including the transform to use, block size, quantization factor, quantization scaling matrix, etc.) from the parser (420) as symbols (421). The scalar / inverse transform unit (451) can output blocks containing sample values ​​that can be input to an aggregator (455).

[0044] In some cases, the output samples of the scalar / inverse transform (451) may relate to intra-coded blocks: i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed part of the current picture. Such prediction information may be provided by an intra picture prediction unit (452). In some cases, the intra picture prediction unit (452) generates blocks of the same size and shape of the block being reconstructed using already reconstructed surrounding information retrieved from a current picture buffer (458). The current picture buffer (458) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (455) adds, possibly on a sample-by-sample basis, the prediction information generated by the intra prediction unit (452) to the output sample information as provided by the scalar / inverse transform unit (451).

[0045] In other cases, the output samples of the scalar / inverse transform unit (451) may relate to a block that may be inter-coded and motion compensated. In such a case, the motion compensated prediction unit (453) may access the reference picture memory (457) to retrieve samples used for prediction. After motion compensating the retrieved samples according to the symbols (421) related to the block, these samples are added by the aggregator (455) to the output of the scalar / inverse transform unit (451) (in this case called residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (457) from which the motion compensated prediction unit (453) fetches the prediction samples may be controlled by the motion vectors available to the motion compensated prediction unit (453), for example in the form of symbols (421) that may have X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​taken from a reference picture memory (457), motion vector prediction mechanisms, etc., where sub-sample accurate motion vectors are used.

[0046] The output samples of the aggregator (455) may be subjected to various loop filtering techniques in a loop filter unit (456). Video compression techniques are controlled by parameters contained in the coded video sequence (also called coded video bitstream) and made available to the loop filter unit (456) as symbols (421) from the parser (420), but may include in-loop filter techniques that may be responsive to meta-information obtained during decoding of previous parts of the coded picture or coded video sequence (in decoding order) as well as to previously reconstructed loop filtered sample values.

[0047] The output of the loop filter unit (456) may be a sample stream that can be output to a rendering device (412) or stored in a reference picture memory (457) for use in future inter-picture prediction.

[0048] Once a given coded picture has been fully reconstructed, it can be used as a reference picture for future predictions. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (420)), the current picture buffer (458) can become part of the reference picture memory (457) and a new current picture buffer can be reallocated before starting the reconstruction of a future coded picture.

[0049] The video decoder (410) may perform decoding operations according to a given video compression technique in a standard such as ITU-T Rec. H.265. A coded video sequence may comply with the syntax specified by the video compression technique or standard used in the sense that the coded video sequence conforms to both the syntax of the video compression technique or standard and the profile as documented in the video compression technique or standard. Specifically, a profile may select a particular tool from all tools available in the video compression technique or standard as the only tool that may be used under that profile. Also required for compliance is that the complexity of the coded video sequence is within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (e.g., measured in megasamples per second), the maximum reference picture size, etc. The limits set by the level may be further limited by a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management that may be signaled in the coded video sequence.

[0050] In one embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0051] 5 shows a block diagram of a video encoder (503) according to one embodiment of the present disclosure. The video encoder (503) is included in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used in place of the video encoder (303) in the example of FIG. 3.

[0052] The video encoder (503) may receive video samples from a video source (501) (which in the example of FIG. 5 is not part of the electronic device (520)) capable of capturing video images to be coded by the video encoder (503). In another example, the video source (501) is part of the electronic device (520).

[0053] The video source (501) may provide a source video sequence to be coded by the video encoder (503) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 YCrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (501) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (501) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that convey motion when viewed in sequence. The picture itself may be organized as a spatial array of pixels, each of which may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art will readily appreciate the relationship between pixels and samples. The following discussion focuses on examples.

[0054] According to one embodiment, the video encoder (503) is capable of coding and compressing pictures of a source video sequence into a coded video sequence (543) in real-time or under any other time constraint required by an application. Enforcing an appropriate coding rate is one function of the controller (550). In some embodiments, the controller (550) controls and is operatively coupled to other functional units as described below, the couplings of which are not depicted for clarity. Parameters set by the controller (550) may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) may be configured with other suitable functions associated with the video encoder (503) optimized for a particular system design.

[0055] In some embodiments, the video encoder (503) is configured to operate in a coding loop. As a simplified explanation, in one example, the coding loop may include a source coder (530) (e.g., responsible for generating symbols such as a symbol stream based on an input picture to be coded and reference pictures) and a (local) decoder (533) embedded in the video encoder (503). The decoder (533) reconstructs the symbols to generate sample data in a similar manner to that generated by the (remote) decoder (since any compression between the symbols and the coded video bitstream is lossless in the video compression techniques contemplated by the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (534). Since the decoding of the symbol stream produces bit-exact results independent of the location of the decoder (local or remote), the contents in the reference picture memory (534) are also bit-exact between the local and remote encoders. In other words, the prediction part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder would "see" if it used prediction during decoding. This basic principle of reference picture synchronization (and the resulting drift if synchronicity cannot be maintained, e.g. due to channel errors) is used in several related techniques as well.

[0056] The operation of the "local" decoder (533) may be the same as that of a "remote" decoder, such as the video decoder (410) already described in detail above in connection with Figure 4. However, with brief reference to Figure 4 as well, since symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder (545) and parser (420) may be lossless, the entropy decoding portion of the video decoder (410), including the buffer memory (415) and the parser (420), may not be fully implemented in the local decoder (533).

[0057] An observation that can be made at this point is that decoder techniques other than analysis / entropy decoding that are present in a decoder must necessarily be present in the corresponding encoder, in substantially the same functional form. For this reason, the disclosed subject matter focuses on the operation of the decoder. A description of the encoder techniques can be omitted, since they are the inverse of the decoder techniques that have been described generically. Only in certain areas is a more detailed description necessary, which is given below.

[0058] During operation, the source coder (530) may, in some examples, perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (532) codes differences between pixel blocks of the input picture and pixel blocks of the reference pictures that may be selected as predictive references for the input picture.

[0059] The local video decoder (533) can decode the coded video data of pictures that can be designated as reference pictures based on the symbols generated by the source coder (530). The operation of the coding engine (532) can advantageously be a non-lossless process. When the coded video data is decoded in a video decoder (not shown in FIG. 5), the reconstructed video sequence can typically be a replica of the source video sequence with some errors. The local video decoder (533) can repeat the decoding process that can be performed by the video decoder on the reference pictures, causing the reconstructed reference pictures to be stored in the reference picture cache (534). In this way, the video encoder (503) can locally store copies of reconstructed reference pictures that have common content as reconstructed reference pictures obtained by the far-end video decoder (assuming there are no transmission errors).

[0060] The predictor (535) can perform a prediction search for the coding engine (532). That is, for a new picture to be coded, the predictor (535) can search the reference picture memory (534) for sample data (as candidate reference pixel blocks) or predefined metadata (reference picture motion vectors, block shapes, etc.), which may serve as suitable prediction references for the new picture. The predictor (535) can operate on a sample block-pixel block basis to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (535), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (534).

[0061] The controller (550) may manage the coding operations of the source coder (530), including, for example, setting parameters and subgroup parameters used to encode the video data.

[0062] All outputs of the aforementioned functional units can be subjected to entropy coding in an entropy coder (545), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0063] The transmitter (540) can buffer the coded video sequence as produced by the entropy coder (545) and prepare it for transmission over a communication channel (560), which may be a hardware / software link to a storage device that stores the coded video data. The transmitter (540) can merge the coded video data from the video coder (503) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0064] The controller (550) can manage the operation of the video encoder (503). During coding, the controller (550) can assign a particular coded picture type to each of the coded pictures, which can affect the coding technique that can be applied to each picture. For example, a picture may be frequently assigned as one of the following picture types:

[0065] An intra picture (I-picture) is one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures, as well as their respective uses and characteristics.

[0066] A predictive picture (P-picture) is one that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0067] Bidirectionally predicted pictures (B-pictures) are those that can be coded and decoded using intra- or inter-prediction, which uses at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures can use more than two reference pictures and associated metadata for the reconstruction of a block.

[0068] A source picture is usually spatially subdivided into several sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples) and can be coded block by block. Blocks can be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the respective picture. For example, blocks of I pictures can be non-predictively coded or they can be predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of P pictures can be predictively coded with spatial or temporal prediction with reference to one previously coded reference picture. Blocks of B pictures can be predictively coded with spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0069] The video encoder (503) may perform coding operations according to a given video coding technique or standard, such as ITU-T Rec. H.265, in which the video encoder (503) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. The coded video data may thus conform to a syntax specified by the video coding technique or standard being used.

[0070] In one embodiment, the transmitter (540) can transmit additional data along with the encoded video. The source coder (530) can include such data as part of the coded video sequence. The additional data can include temporal / spatial / SNR enhancement layers, other forms of redundant data (redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.).

[0071] A video can be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra prediction) exploits spatial correlation in a given picture, and inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture under encoding / decoding, called the current picture, is divided into blocks. If a block in the current picture is similar to a reference block in a reference picture that was previously coded in the video and is still buffered, the block in the current picture can be coded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and can have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0072] In some embodiments, bi-prediction techniques may be used for inter-picture prediction. According to bi-prediction techniques, two reference pictures are used, such as a first reference picture and a second reference picture, both preceding in decoding order (but potentially past and future, respectively, in display order) a current picture. A block in the current picture may be coded with a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block may be predicted by a combination of the first and second reference blocks.

[0073] Furthermore, to improve coding efficiency, it is possible to use merge mode techniques for inter-picture prediction.

[0074] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a sequence of video pictures are partitioned into coding tree units for compression, and CTUs in a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can be recursively quad-tree partitioned into one or more coding units (CUs). For example, a CTU of 64×64 pixels can be partitioned into one CU of 64×64 pixels, four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine a prediction type of the CU, such as an inter prediction type or an intra prediction type. A CU is divided into one or more Prediction Units (PUs) depending on temporal and / or spatial predictability. In general, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels, such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0075] 6 shows a diagram of a video encoder (603) according to another embodiment of the present disclosure. The video encoder (603) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video picture in a sequence of video pictures and to encode the processed block into a coded picture that is part of a coded video sequence. In one example, the video encoder (603) is used in place of the video encoder (303) of the example of FIG. 3.

[0076] In an HEVC example, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as a predicted block of 8x8 samples. The video encoder (603) determines whether the processing block is best coded using intra mode, inter mode, or bi-predictive mode, e.g., using rate-distortion optimization. If the processing block is to be coded in intra mode, the video encoder (603) may use intra prediction techniques to code the processing block into a coded picture; if the processing block is to be coded in inter mode or bi-predictive mode, the video encoder (603) may use inter prediction techniques or bi-predictive techniques, respectively, to code the processing block into a coded picture. In certain video coding techniques, the merge mode may be an inter prediction picture sub-mode, in which case the motion vector is derived from one or more motion vector predictors without benefit of a coded motion vector component outside the predictor. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (603) includes other components, such as a mode decision module (not shown) to determine the mode of the processing block.

[0077] In the example of FIG. 6, the video encoder (603) includes an inter-encoder (630), an intra-encoder (622), a residual calculator (623), a switch (626), a residual encoder (624), a generalized controller (621), and an entropy encoder (625) coupled together as shown in FIG. 6.

[0078] The inter encoder (630) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks of a previous picture and blocks in a subsequent picture), generate inter prediction information (e.g., a description of redundant information due to the encoding technique, motion vectors, merge mode information), and calculate an inter prediction result (e.g., a prediction block) based on the inter prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that has been decoded based on the encoded video information.

[0079] The intra encoder (622) is configured to receive samples of a current block (e.g., a processing block), possibly compare the block to previously coded blocks in the same picture, generate transformed and quantized coefficients, and possibly also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In one example, the intra encoder (622) also calculates an intra prediction result (e.g., a prediction block) based on the intra prediction information and reference blocks in the same picture.

[0080] The general controller (621) is configured to determine general control data and control other components of the video encoder (603) based on the general control data. In one example, the general controller (621) determines the mode of the block and provides a control signal to the switch (626) based on the mode. For example, if the mode is intra mode, the general controller (621) controls the switch (626) to select intra mode results for use by the residual calculator (623) and controls the entropy encoder (625) to select intra prediction information and include the intra prediction information in the bitstream; if the mode is inter mode, the general controller (621) controls the switch (626) to select inter prediction results for use by the residual calculator (623) and controls the entropy encoder (625) to select inter prediction information and include the inter prediction information in the bitstream.

[0081] The residual calculator (623) is configured to calculate a difference (residual data) between the received block and a prediction result selected from the intra-encoder (622) or the inter-encoder (630). The residual encoder (624) is configured to operate on the residual data to encode the residual data and generate transform coefficients. In one example, the residual encoder (624) is configured to transform the residual data from a spatial domain to a frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be used by the intra-encoder (622) and the inter-encoder (630) as appropriate. For example, the inter encoder (630) may generate decoded blocks based on the decoded residual data and the inter prediction information, and the intra encoder (622) may generate decoded blocks based on the decoded residual data and the intra prediction information. The decoded blocks are suitably processed to generate decoded pictures, which may be buffered in a memory circuit (not shown) and, in some examples, used as reference pictures.

[0082] The entropy encoder (625) is configured to format a bitstream to include the encoded blocks. The entropy encoder (625) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (625) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that when coding blocks in a merged sub-mode of either the inter mode or the bi-prediction mode in accordance with the disclosed subject matter, the residual information is not present.

[0083] 7 shows a diagram of a video decoder (710) according to another embodiment of the present disclosure. The video decoder (710) is configured to receive coded pictures that are part of a coded video sequence and to decode the coded pictures to generate reconstructed pictures. In one embodiment, the video decoder (710) is used in place of the video decoder (310) in the example of FIG. 3.

[0084] In the example of FIG. 7, the video decoder (710) includes an entropy decoder (771), an inter decoder (780), a residual decoder (773), a reconstruction module (774), and an intra decoder (772), coupled together as shown in FIG. 7.

[0085] The entropy decoder (771) can be configured to reconstruct from the coded picture certain symbols that represent syntax elements that make up the coded picture. Such symbols can include, for example, prediction information (e.g., intra- or inter-prediction information), which can identify the mode in which the block is coded (e.g., intra- or inter-prediction mode, merged submode, or the latter two in another submode), certain samples or metadata used for prediction by the intra-decoder (772) or inter-decoder (780), respectively, residual information (e.g., in the form of quantized transform coefficients), etc. In one example, if the prediction mode is an inter- or bi-prediction mode, the inter-prediction information is provided to the inter-decoder (780); if the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra-decoder (772). The residual information can be inverse quantized and provided to the residual decoder (773).

[0086] The inter decoder (780) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0087] The intra decoder (772) is configured to receive intra prediction information and to generate a prediction result based on the intra prediction information.

[0088] The residual decoder (773) is configured to perform inverse quantization to extract un-quantized transform coefficients and to process the un-quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require certain control information (including a quantization parameter (QP)), which may be provided by the entropy decoder (771) (this may only be a small amount of control information, so a data path is not depicted).

[0089] The reconstruction module (774) is configured to combine, in the spatial domain, the residual as output by the residual decoder (773) and the prediction result (possibly output by an inter- or intra-prediction module) to form a reconstructed block, the reconstructed block being part of a reconstructed picture, which may be part of a reconstructed video. It should be noted that other suitable processes, such as deblocking processes, may be performed to improve visual quality.

[0090] It should be noted that the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using any suitable technology. In one embodiment, the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (303), (503), and (503) and the video decoders (310), (410), and (710) may be implemented using one or more processors executing software instructions.

[0091] Aspects of the present disclosure provide conditions for applying decoder-side motion vector refinement (DMVR) and / or bidirectional optical flow (BDOF).

[0092] Various coding standards, such as HEVC and VVC, are being developed to encompass new technologies.

[0093] In some examples of VVC, for each inter-predicted CU, the motion parameters include a motion vector, a reference picture index, and a reference picture list usage index, as well as additional information required for the new coding features of VVC used to generate the inter-predicted samples. The motion parameters can be signaled in an explicit or implicit manner. In one example, if a CU is coded in skip mode, the CU is associated with one PU, and the CU has no significant residual coefficients, coded motion vector deltas, or reference picture indexes. In another example, a merge mode is specified, whereby the motion parameters for the current CU are obtained from neighboring CUs, including spatial and temporal candidates, and an additional schedule introduced in VVC. The merge mode can be applied not only for skip mode, but also for any inter-predicted CU. An alternative to the merge mode is explicit transmission of the motion parameters, in which the motion vector, the corresponding reference picture index for each reference picture list, the reference picture list usage flag, and other necessary information are explicitly signaled for each CU.

[0094] Beyond the inter-coding features in HEVC, VVC Test Model 3 (VTM3) includes many new refined inter-predictive coding tools, such as enhanced merge prediction, merge mode with motion vector differential (MMVD), affine motion compensation prediction, sub-block based temporal motion vector prediction (SbTMVP), triangular partition prediction, combined inter and intra prediction (CIIP), etc. Some features of the above mentioned inter-predictive coding tools are described in this disclosure.

[0095] According to some aspects of the present disclosure, a motion refinement technique called bi-directional optical flow (BDOF) mode is used in inter prediction. BDOF is referred to as BIO in some examples. BDOF is used to refine the bi-predictive signal of a CU at the 4x4 sub-block level. BDOF is applied to a CU if the CU meets the following conditions: 1) the height of the CU is not 4 and the CU is not at a size of 4x8, 2) the CU is not coded using affine mode or ATMVP merge mode, and 3) the CU is coded using a "true" bi-predictive mode, i.e., one of the two reference pictures is before the current picture in display order and the other is after the current picture in display order. BDOF is only applied to the luma component in some examples.

[0096] In BDOF mode, motion refinement is based on the concept of optical flow, which assumes that object motion is smooth. For each 4 × 4 sub-block, the motion refinement (v x ,v y ) is calculated by minimizing the difference between the L0 and L1 predicted samples. Then, motion refinement is used to adjust the bi-predicted sample values ​​in the 4x4 sub-block. The following steps are applied in the BDOF process:

[0097] First, the horizontal and vertical gradients of the two prediction signals

number

number

[0098] Then the gradient auto- and cross-correlations S1, S2, S3, S4, S5, S6 are calculated:

number

[0099] Then, motion refinement (v x ,v y ) is derived using the auto- and cross-correlation terms as follows:

number

[0100] Based on the motion refinement and gradients, the following adjustments are computed for each sample in the 4x4 sub-block:

number

[0101] Finally, the BDOF samples for the CU are calculated by adjusting the bi-predictive samples as follows:

number

[0102] In the above case, n a ,n b ,n S2 The values ​​of are equal to 3, 6, and 12, respectively. These values ​​are selected so that the multiplier in the BDOF process does not exceed 15 bits, and the maximum bit width of the intermediate parameters in the BDOF process is kept within 32 bits.

[0103] To derive the gradient value, we select some prediction samples I in list k (k=0,1) outside the current CU boundary. (k) We can generate (i,j).

[0104] FIG. 8 shows an example of an extended CU region in BDOF. In the example of FIG. 8, a 4×4 CU (810) is shown as the shaded area. BDOF uses one row / column extended around the boundary of the CU, and the extended area is shown as a dashed 6×6 block (820). To limit the amount of computation to generate prediction samples outside the boundary, a bilinear filter is used to generate prediction samples in the extended area (white locations), and a regular 8-tap motion compensated interpolation filter is used to generate prediction samples inside the CU (gray locations). These extended sample values ​​are only used in the gradient calculation. In the remaining steps of the BDOF process, if any sample and gradient values ​​outside the CU boundary are needed, they are padded (i.e., repeated) from their nearest neighbors.

[0105] According to one aspect of the present disclosure, Decoder-Side Motion Vector Refinement (DMVR) is a Decoder-Side Motion Vector Derivation (DMVD) technique that is used to improve / refine MVs based on a starting point.

[0106] In some cases, decoder-side motion vector refinement based on bilateral matching can be applied to improve the accuracy of the motion vectors in merge mode. In bi-predictive operation, a refined MV is searched around the initial MV in reference picture list L0 and reference picture list L1. The bilateral matching method calculates the distortion between two candidate blocks in reference picture list L0 and list L1.

[0107] In one example, in the case of bi-predictive operation, two prediction blocks can be formed using MV0 from the first reference picture candidate list L0 and MV1 from the second reference picture candidate list L1 for prediction of one block region, respectively. In the DMVR method, the two bi-predictive motion vectors MV0 and MV1 are further refined by a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference pictures to obtain the refined MVs without transmitting additional motion information.

[0108] Figure 9 shows an example of DMVR based on bilateral template matching. As shown in Figure 9, A bilateral template (940) is generated as a weighted combination (i.e., average) of two prediction blocks (920) and (930) from the initial MV0 from the first reference picture candidate list L0 and MV1 from the second reference picture candidate list L1, respectively. The template matching process includes computing a cost measure between the sample region (around the initial prediction block) in the reference pictures Ref0 and Ref1 and the generated template (940). For each of the two reference pictures Ref0 and Ref1, the MV that results in the smallest template cost is considered as the updated MV of that list to replace the original MV. For example, MV0' replaces MV0 and MV1' replaces MV1. In some examples, nine MV candidates are searched for each list. The nine MV candidates include the original MV and eight surrounding MVs, the latter having one luma sample offset with respect to the original MV in the horizontal or vertical direction or both. Finally, two new MVs, namely MV0' and MV1', are used to generate the final bi-prediction result for the current block, as shown in Figure 9. The sum of absolute differences (SAD) can be used as a cost measure.

[0109] In some examples, DMVR is applied to a CU that is coded in a specific mode condition. For example, DMVR is applied to a CU in CU level merge mode with bi-predictive MV. Furthermore, one reference picture is in the past and the other reference picture is in the future with respect to the current picture. The distance from both reference pictures to the current picture (i.e., the Picture Order Count (POC) difference) is the same. The CU has more than 64 luma samples and the height of the CU is more than 8 luma samples.

[0110] The refined MVs derived by the DMVR process are used to generate inter prediction samples and are also used for temporal motion vector prediction for future picture coding, while the original MVs are used in the deblocking process and are also used for spatial motion vector prediction for future CU coding.

[0111] In some embodiments, based on signals in the received bitstream, a pair of merge candidates is determined and used as input to the DMVR process. For example, the pair of merge candidates is denoted as an initial motion vector (MV0, MV1). In some examples, the search points searched by the DMVR obey the motion vector difference mirroring condition. In other words, the points checked by the DMVR, i.e., the points denoted by a pair of candidate motion vectors (MV0', MV1'), obey (Eq.7) and (Eq.8):

number

[0112] In some embodiments, a technique called bi-prediction with weighted average (BWA) is used. The BWA technique is also called generalized bi-prediction (GBi). In one example, such as HEVC, the bi-prediction signal is generated by averaging two prediction signals obtained from two different reference pictures and / or using two different motion vectors. In another example using BWA, such as in the VVC Working Draft and VMT, the bi-prediction mode is extended beyond simple averaging to allow a weighted average of two prediction signals. In an example such as the VVC draft, GBi (generalized bi-prediction) is also called bi-prediction with CU level weighting (BCW). In the BWA / GBi / BCW mode, CU level weighted prediction is performed at the CU. For example, if the BWA / GBi / BCW mode is enabled for a CU, the weighting can be signaled for that CU by the BCW index. For example, the bi-prediction P bi-pred is generated using (Eq.9):

number

[0113] In an example implementation of GBi, five weights are allowed in weighted average bi-prediction, w∈{-2,3,4,5,10}. For each bi-predicted CU, the weight w is determined by one of the first and second methods. In the first method, for non-merged CUs, the weight index is signaled after the motion vector differential. In the second method, for merged CUs, the weight index is estimated from neighboring blocks based on the merge candidate index. In some examples, weighted average bi-prediction is applied only to CUs with 256 or more luma samples (i.e., CU width × CU height is 256 or more). For low latency pictures, all five weights can be used. For non-low latency pictures, in one example, only three weights are used (w∈{3,4,5}).

[0114] In some examples, such as AVC, HEVC, VVC, etc., weighted prediction (WP) is offered as a supported coding tool. In one example, WP can be used to improve the performance of inter prediction when the source material uses fading or cross fading, for example when subject to illumination variations.

[0115] In some examples, according to WP, the inter prediction signal P is replaced by a linear weighted prediction signal P′ (using weights w and offset o), for example according to (Eq. 10) for uni-prediction: Single-prediction: P'= w×P + o (Eq.10)

[0116] In bi-prediction, when the inter prediction signal P0 is for reference L0, with weights w0 and offset o0 for reference L0, and the inter prediction signal P1 is for reference L1, with weights w1 and offset o1 for reference L1, the linear weighted prediction signal P' can be calculated according to (Eq.11): Bi-prediction: P'= (w0 × P0 + o0 + w1 × P1 + o1) / 2 (Eq.11)

[0117] The applicable weights and offsets are selected by the encoder and conveyed in the bitstream from the encoder to the decoder. The L0 and L1 suffixes define the reference picture lists List0 and List1, respectively. Bit depth is maintained at 14-bit precision (HEVC version 1) before averaging the prediction signal, as is the case for the interpolation filters.

[0118] In some embodiments, the WP allows weighting parameters (weight and offset) to be signaled for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the weight and offset of the corresponding reference picture are applied. The WP and BWA are designed for different types of video content. To avoid the interaction between the WP and BWA complicating the VVC decoder design, if a CU uses the WP, the BWA weight index is not signaled and w is estimated to be 4 (i.e., equal weights are applied).

[0119] According to some aspects of the present disclosure, certain bi-prediction tools, such as BDOF and DMVR, rely on equal weighting for prediction from two directions.

[0120] In one example, whether to apply the BDOF method depends on conditions, including conditions on both the GBi and weighted flag of explicit weighted prediction for the luma component (also called the use flag of weighted prediction for the luma component).

[0121] FIG. 10A shows Table 1A summarizing a list of conditions for applying the BDOF method according to one embodiment. In the example of FIG. 10A, the condition (1010A) requires that the Gbi index is zero. The Gbi index can be signaled or inferred. The Gbi index is used to specify the weights used to weight the prediction signals from the two reference pictures in one example. If the Gbi index is zero, equal weights are used to weight the prediction signals from the two reference pictures.

[0122] Further, in the example of Figure 10A, condition (1020A) requires that the flags for using weighted prediction for the luma component in the reference picture lists L0 and L1 are zero. If the flags for using weighted prediction for the luma component in the reference picture lists L0 and L1 are zero, default weights can be used, and the default weights are equal for the two directions.

[0123] In an implementation example, if the Gbi index meets the condition (1010A) and the weight flag meets the condition (1020A), the BDOF is enabled, and whether to apply the BDOF can be further determined according to other conditions, such as other conditions in FIG.

[0124] In some cases, DMVR may search for non-matching blocks when the SAD of the reference blocks is weighted with non-uniform weights. Similar to the application of BDOF, whether to apply DMVR can be determined based on conditions. The conditions include conditions on both GBi and weighting flag of explicit weighted prediction for luma component (also called use weighted prediction for luma component flag).

[0125] FIG. 11A shows Table 2A summarizing a list of conditions for applying the DMVR method. In the example of FIG. 11A, the condition (1110A) requires that the Gbi index is zero. The Gbi index can be signaled or inferred. The Gbi index is used to specify the weights used to weight the prediction signals from the two reference pictures in one example. If the Gbi index is zero, equal weights are used to weight the prediction signals from the two reference pictures.

[0126] Further, in the example of Figure 11A, condition (1120A) requires that the flags for using weighted prediction for the luma component in reference picture lists L0 and L1 are zero. If the flags for using weighted prediction for the luma component in reference picture lists L0 and L1 are zero, default weights can be used, and the default weights are equal for the two directions.

[0127] In an implementation example, if the Gbi index meets the condition (1110A) and the weight flag meets the condition (1120A), the DMVR is enabled. And, whether to apply the DMVR can be further determined according to other conditions, such as other conditions in FIG. 11A.

[0128] According to some aspects of the disclosure, the conditions for applying BDOF and / or DMVR include checking a flag for use of weighted prediction for the luma component and also checking a flag for use of weighted prediction for the chroma components.

[0129] According to one aspect of the present disclosure, BDOF can be applied to only the luma component. In some embodiments, the chroma weighting of the current block of the weighted prediction can also be checked.

[0130] FIG. 10B shows Table 1B summarizing a list of conditions for applying the BDOF method according to some embodiments. In the example of FIG. 10B, the use of weighted prediction flags for chroma components as represented by chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] are checked, as indicated by (1030B). If the use of weighted prediction flags for chroma components are zero, equal weighting is used for the chroma components, and BDOF can be enabled. Furthermore, if other conditions in FIG. 10B are met, BDOF can be applied to the luma component. However, if at least one of chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] is not equal to 0, BDOF can be disabled and cannot be applied to the luma component.

[0131] According to another aspect of the present disclosure, BDOF may be applied separately to luma and chroma components, and the conditions for using the luma component BDOF may include a luma weighting of the current block of the weighted prediction, and the conditions for using the chroma component BDOF may include a chroma weighting of the current block of the weighted prediction.

[0132] In one embodiment, to determine the application of BDOF to the luma component, the use flag of weighted prediction for the luma component as represented by luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] is checked. If both use flags of weighted prediction for the luma component are zero, equal weighting is used and BDOF can be enabled. Furthermore, if other conditions in FIG. 10B are met, BDOF can be applied to the luma component. However, if at least one of luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] is not equal to 0, BDOF can be disabled and cannot be applied to the luma component.

[0133] In another embodiment, to determine the application of BDOF to a chroma component, the use flags of weighted prediction for the chroma components as represented by chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] are checked. If both use flags of weighted prediction for a chroma component are zero, equal weighting is used and BDOF can be enabled. Furthermore, if other conditions in FIG. 10B (except (1020B)) are satisfied, BDOF can be applied to the chroma components. However, if at least one of chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] is not equal to 0, BDOF can be disabled and cannot be applied to the chroma components.

[0134] According to one aspect of the present disclosure, the DMVR may be applied only to the luma component. In some embodiments, in addition to the condition, the chroma weighting of the current block of the weighted prediction may also be checked.

[0135] FIG. 11B shows Table 2B summarizing a list of conditions for applying the DMVR method according to some embodiments. In the example of FIG. 11B, the use of weighted prediction flag for the chroma components as represented by chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] is checked as shown by (1130B). If the use of weighted prediction flag for the chroma components is zero, equal weighting is used for the chroma components and DMVR can be enabled. Furthermore, if other conditions of FIG. 11B are satisfied, DMVR can be applied to the luma components. However, if at least one of chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] is not equal to 0, DMVR can be disabled and cannot be applied to the luma components.

[0136] According to another aspect of the present disclosure, the DMVR may be applied separately to the luma and chroma components, and the conditions for using the luma component DMVR may include a luma weighting of the current block of the weighted prediction, and the conditions for using the chroma component DMVR may include a chroma weighting of the current block of the weighted prediction.

[0137] In one embodiment, to determine the application of DMVR to the luma component, the use flag of weighted prediction for the luma component as represented by luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] is checked. If both use flags of weighted prediction for the luma component are zero, equal weighting is used and DMVR may be enabled. Furthermore, if other conditions in FIG. 11B are met, DMVR may be applied to the luma component. However, if at least one of luma_weight_l0_flag[refIdxL0] and luma_weight_l1_flag[refIdxL1] is not equal to 0, DMVR may be disabled and cannot be applied to the luma component.

[0138] In another embodiment, to determine the application of DMVR to a chroma component, the use flag of weighted prediction for the chroma component as represented by chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] is checked. If both use flags of weighted prediction for a chroma component are zero, equal weighting is used and DMVR can be enabled. Furthermore, if other conditions in FIG. 11B (except (1120B)) are satisfied, DMVR can be applied to the chroma component. However, if at least one of chroma_weight_l0_flag[refIdxL0] and chroma_weight_l1_flag[refIdxL1] is not equal to 0, DMVR can be disabled and cannot be applied to the chroma component.

[0139] FIG. 12 shows a flow chart that outlines the process (1200) according to an embodiment of the present disclosure. The process (1200) can be used to reconstruct a block and thus generate a prediction block for the block being reconstructed. In various embodiments, the process (1200) is performed by a processing circuit, such as a processing circuit in a terminal device (210), (220), (230), (240), a processing circuit performing the functions of a video encoder (303), a processing circuit performing the functions of a video decoder (310), a processing circuit performing the functions of a video decoder (410), a processing circuit performing the functions of a video encoder (503), etc. In some embodiments, the process (1200) is implemented by software instructions, such that the processing circuit performs the process (1200) when the processing circuit executes the software instructions. The process starts at (S1201) and proceeds to (S1210).

[0140] At (S1210), prediction information of a current block in a current picture is decoded from the coded video bitstream. The prediction information indicates an inter prediction mode that may use a refinement technique based on a first reference picture and a second reference picture. In some implementations, the refinement technique includes at least one of BDOF and DMVR. In some examples, the current picture has a larger picture order count (POC) than one of the first reference picture and the second reference picture and a smaller POC than the other of the first reference picture and the second reference picture.

[0141] In (S1220), a determination can be made as to whether a first equal-weighting condition of chroma components from a first reference picture and a second reference picture is satisfied. In some examples, a first flag for chroma weighting of the first reference picture (e.g., chroma_weight_l0_flag[refIdxL0]) and a second flag for chroma weighting of the second reference picture (e.g., chroma_weight_l1_flag[refIdsL1]) are checked. If both the first flag and the second flag are zero, the first equal-weighting condition of chroma components is satisfied. If at least one of the first flag and the second flag is not zero, it can be determined that the first equal-weighting condition is not satisfied.

[0142] At (S1230), in response to not satisfying the first equal-weighting condition, refinement techniques are disabled in the reconstruction of samples in the current block. In some embodiments, refinement techniques may be applied only to the luma component. Thus, in response to not satisfying the first equal-weighting condition, refinement techniques are disabled in the reconstruction of luma samples of the current block. In some embodiments, refinement techniques may be applied separately to the luma and chroma components. Thus, in response to not satisfying the first equal-weighting condition, refinement techniques are disabled in the reconstruction of chroma samples of the current block. The process then proceeds to (S1299) and ends.

[0143] It should be noted that once the first equal weighting condition is satisfied, other appropriate conditions, such as the conditions of FIG. 10B or FIG. 11B, are also checked to determine whether the refinement technique can be applied to the reconstruction of samples in the current block.

[0144] The techniques described above may be implemented as computer software using computer readable instructions and may be physically stored on one or more computer readable media. For example, Figure 13 illustrates a computer system (1300) suitable for implementing certain embodiments of the disclosed subject matter.

[0145] Computer software may be coded using any suitable machine code or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to produce code that contains instructions that may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or instructions that may be executed via interpretation, microcode execution, etc.

[0146] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0147] 13 for computer system (1300) are exemplary in nature and are not intended to suggest any limitations on the scope of use or functionality of the computer software implementing the embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system (1300).

[0148] The computer system (1300) may include certain human interface input devices that may be responsive to input by one or more human users, for example, via tactile input (e.g., keystrokes, swipes, data glove movements), auditory input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic pictures).

[0149] The input human interface devices may include one or more of (although only one of each is depicted) a keyboard (1301), a mouse (1302), a trackpad (1303), a touch screen (1310), a data glove (not shown), a joystick (1305), a microphone (1306), a scanner (1307), and a camera (1308).

[0150] The computer system (1300) may also include certain human interface output devices that may stimulate one or more of the senses of a human user, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen (1310), data gloves (not shown), joystick (1305), although there may be haptic feedback devices that do not serve as input), auditory output devices (e.g., speakers (1309), headphones (not shown)), visual output devices (e.g., screens (1310), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output, three or more dimensional output by means such as stereoscopic output; virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0151] The computer system (1300) may also include human accessible storage devices and their associated media, such as optical media (1321), including CD / DVD ROM / RW (1320), thumb drives (1322), removable hard drives or solid state drives (1323), legacy magnetic media (not shown), such as tapes and floppy disks (not shown), and specialized ROM / ASIC / PLD based devices such as security dongles (not shown).

[0152] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other transitional signals.

[0153] The computer system (1300) may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include Ethernet, wireless LAN, cellular networks (including GSM, 3G, 4G, 5G, LTE, etc.), TV wired or wireless wide area digital networks (including cable TV, satellite TV, and terrestrial TV), vehicular and industrial including CANBus, etc. Certain networks typically require an external network interface adapter that is attached to a specific general-purpose data port or peripheral bus (1349) (e.g., a USB port on the computer system (1300)); others are typically integrated into the core of the computer system (1300) by attaching to a system bus as described below (e.g., an Ethernet interface is integrated in a PC computer system, a cellular network interface is integrated in a smartphone computer system). Using any of these networks, the computer system (1300) may communicate with other entities. Such communications may be one-way receive-only (e.g., broadcast television), one-way transmit-only (e.g., CANbus to a particular CANbus device), or two-way, such as to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used with each of these networks and network interfaces, as described above.

[0154] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to a core (1340) of the computer system (1300).

[0155] The core (1340) may include one or more central processing units (CPUs) (1341), graphics processing devices (GPUs) (1342), specialized programmable processing devices in the form of field programmable gate areas (FPGAs) (1343), hardware accelerators for specific tasks (1344), etc. These devices may be connected via a system bus (1348) along with read only memory (ROM) (1345), random access memory (1346), and internal mass storage devices (e.g., internal non-user accessible hard drives, SSDs, etc.) (1347). In some computer systems, the system bus (1348) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1348) or via a peripheral bus (1349). Peripheral bus architectures include PCI, USB, and the like.

[0156] The CPU (1341), GPU (1342), FPGA (1343), and accelerator (1344) may combine to execute certain instructions that may constitute the aforementioned computer code. The computer code may be stored in a ROM (1345) or a RAM (1346). Temporary data may be stored in the RAM (1346), while persistent data may be stored in, for example, an internal mass storage (1347). Rapid storage and retrieval from any memory device may be made possible by utilizing a cache memory, which may be closely associated with one or more of the CPU (1341), GPU (1342), mass storage (1347), ROM (1345), RAM (1346), etc.

[0157] The computer-readable medium can have computer code thereon for performing various computer-implemented operations. The medium and computer code can be specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those having skill in the computer software arts.

[0158] By way of example, and not by way of limitation, a computer system having the architecture (1300), and in particular the core (1340), as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.), can provide the functionality of executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, as well as specific storage of the core (1340) of a non-transitory nature, such as mass storage (1347) within the core or ROM (1345). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1340). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (1340) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (1346) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1344)), which may execute in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry embodying logic for execution, or both, as appropriate. The present disclosure encompasses any appropriate combination of hardware and software.

[0159] While this disclosure has described several exemplary embodiments, there are alterations, substitutions, and various substitute equivalents that fall within the scope of this disclosure. Thus, it will be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

[0160] <Additional Notes> (Appendix 1) 1. A method for video decoding in a decoder, comprising: a processor decoding prediction information for a current block in a current picture from a coded video bitstream, the prediction information indicating an inter-prediction mode that may utilize a refinement technique based on a first reference picture and a second reference picture; determining whether a first equal weighting condition for chroma components from the first reference picture and the second reference picture is satisfied; in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture, the processor disabling the refinement techniques in reconstructing samples in the current block; The method includes: (Appendix 2) in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture, the processor disabling the refinement techniques in reconstructing luma samples in the current block. 2. The method of claim 1, further comprising: (Appendix 3) determining whether a second equal weighting condition for luma components from the first reference picture and the second reference picture is satisfied; in response to not satisfying at least one of the first equal-weighting condition for the chroma components and the second equal-weighting condition for the luma component, the processor disabling the refinement techniques in the reconstruction of the luma samples in the current block. 3. The method of claim 2, further comprising: (Appendix 4) in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture, the processor disabling the refinement techniques in reconstructing chroma samples in the current block. 4. The method of any one of claims 1 to 3, further comprising: (Appendix 5) 5. The method of any one of claims 1 to 4, wherein the refinement technique includes at least one of bidirectional optical flow (BDOF) and decoder-side motion vector refinement (DMVR). (Appendix 6) one of the first reference picture and the second reference picture has a picture order count greater than the current picture; 6. The method of any one of claims 1 to 5, wherein the other of the first reference picture and the second reference picture has a smaller picture order count than the current picture. (Appendix 7) determining, by the processor, that the equal weighting condition for the chroma components from the first reference picture and the second reference picture is not satisfied based on at least one of a first flag of a chroma weight of the first reference picture and a second flag of a chroma weight of the second reference picture not being equal to zero; 7. The method of any one of claims 1 to 6, further comprising: (Appendix 8) 1. An apparatus for video decoding comprising a processing circuit, the processing circuit comprising: decoding current block prediction information in a current picture from a coded video bitstream, the prediction information indicating an inter-prediction mode that may utilize a refinement technique based on a first reference picture and a second reference picture; determining whether a first equal weighting condition for chroma components from the first reference picture and the second reference picture is satisfied; disabling the refinement techniques in the reconstruction of samples in the current block in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture; The apparatus is configured to: (Appendix 9) The processing circuitry includes: disabling the refinement techniques in the reconstruction of luma samples in the current block in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture. 9. The apparatus of claim 8, further configured to: (Appendix 10) The processing circuitry includes: determining whether a second equal weighting condition for luma components from the first reference picture and the second reference picture is satisfied; disabling the refinement techniques in the reconstruction of the luma samples in the current block in response to not satisfying at least one of the first equal-weighting condition for the chroma components and the second equal-weighting condition for the luma component. 10. The apparatus of claim 9, configured to: (Appendix 11) The processing circuitry includes: disabling the refinement techniques in reconstructing chroma samples in the current block in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture. 11. The apparatus of any one of claims 8 to 10, configured to: (Appendix 12) 12. The apparatus of any one of claims 8 to 11, wherein the refinement techniques include at least one of bidirectional optical flow (BDOF) and decoder-side motion vector refinement (DMVR). (Appendix 13) one of the first reference picture and the second reference picture has a picture order count greater than the current picture; 13. The apparatus of any one of claims 8 to 12, wherein the other of the first reference picture and the second reference picture has a smaller picture order count than the current picture. (Appendix 14) The processing circuitry includes: determining that the equal weighting condition for the chroma components from the first reference picture and the second reference picture is not satisfied based on at least one of a first flag of a chroma weight of the first reference picture and a second flag of a chroma weight of the second reference picture not being equal to zero; 14. The apparatus of any one of claims 8 to 13, configured to: (Appendix 15) On the computer, decoding prediction information for a current block in a current picture from a coded video bitstream, the prediction information indicating an inter-prediction mode that may utilize a refinement technique based on a first reference picture and a second reference picture; determining whether a first equal weighting condition for chroma components from the first reference picture and the second reference picture is satisfied; disabling the refinement techniques in the reconstruction of samples in the current block in response to not satisfying the first equal-weighting condition for the chroma components from the first reference picture and the second reference picture; A computer program that runs a program. (Appendix 16) The computer includes: disabling the refinement techniques in the reconstruction of luma samples in the current block in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture. 16. The computer program of claim 15, further comprising: (Appendix 17) The computer includes: determining whether a second equal weighting condition for luma components from the first reference picture and the second reference picture is satisfied; disabling the refinement techniques in the reconstruction of the luma samples in the current block in response to not satisfying at least one of the first equal-weighting condition for the chroma components and the second equal-weighting condition for the luma component. 17. The computer program of claim 16, further comprising: (Appendix 18) The computer includes: disabling the refinement techniques in reconstructing chroma samples in the current block in response to not satisfying the first equal weighting condition for the chroma components from the first reference picture and the second reference picture. 18. The computer program of any one of claims 15 to 17, further comprising: (Appendix 19) 19. The computer program product of claim 15, wherein the refinement techniques include at least one of bidirectional optical flow (BDOF) and decoder-side motion vector refinement (DMVR). (Appendix 20) The computer includes: determining that the equal weighting condition of the chroma components from the first reference picture and the second reference picture is not satisfied when at least one of a first flag of a chroma weight of the first reference picture and a second flag of a chroma weight of the second reference picture is not equal to zero; 20. The computer program of any one of claims 15 to 19, further comprising: (Appendix 21) 1. A method for video encoding in an encoder, comprising: determining prediction information for a current block in a current picture and sending a coded video bitstream including the prediction information to an encoder; the prediction information indicating an inter-prediction mode that may utilize a refinement technique based on a first reference picture and a second reference picture; The method of claim 1, wherein the refinement techniques are disabled in reconstructing samples in the current block if a first equal weighting condition of chroma components from the first reference picture and the second reference picture is not satisfied.

[0161] Appendix A: Acronyms JEM: joint exploration model VVC: versatile video coding BMS: benchmark set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplementary Enhancement Information VUI: Video Usability Information GOPs: Groups of Pictures TUs: Transform Units PUs: Prediction Units CTUs: Coding Tree Units CTBs: Coding Tree Blocks PBs: Prediction Blocks HRD: Hypothetical Reference Decoder SNR: Signal Noise Ratio CPUs: Central Processing Units GPUs: Graphics Processing Units CRT: Cathode Ray Tube LCD: Liquid-Crystal Display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile communications LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Areas SSD: solid-state drive IC: Integrated Circuit CU: Coding Unit

Claims

1. 1. A method for video decoding in a decoder, comprising: extracting prediction information for a current block in a current picture from a signal received by the decoder, the prediction information indicating an inter-prediction mode that may utilize Bi-directional Optical Flow (BDOF), a refinement technique based on a first reference picture and a second reference picture; a processor of the decoder checking a first flag for a luma weight of the first reference picture and a second flag for a luma weight of the second reference picture based on the prediction information; in response to at least one of the first flag and the second flag being non-zero, the processor disabling the refinement techniques in reconstructing samples in the current block; The method includes:

2. The method of claim 1, The first flag is represented by luma_weight_l0_flag[refIdxL0], The second flag is represented by luma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

3. 1. A method for video decoding in a decoder, comprising: extracting prediction information for a current block in a current picture from a signal received by the decoder, the prediction information indicating an inter-prediction mode that may utilize Bi-directional Optical Flow (BDOF), a refinement technique based on a first reference picture and a second reference picture; a processor of the decoder checking a first flag for a chroma weight of the first reference picture and a second flag for a chroma weight of the second reference picture based on the prediction information; in response to at least one of the first flag and the second flag being non-zero, the processor disabling the refinement techniques in reconstructing samples in the current block; The method includes:

4. The method according to claim 3, The first flag is represented by chroma_weight_l0_flag[refIdxL0], The second flag is represented by chroma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

5. 1. A method for video decoding in a decoder, comprising: extracting prediction information for a current block in a current picture from a signal received by the decoder, the prediction information indicating an inter-prediction mode that may utilize a decoder-side motion vector refinement (DMVR) technique based on a first reference picture and a second reference picture; a processor of the decoder checking a first flag for a luma weight of the first reference picture and a second flag for a luma weight of the second reference picture based on the prediction information; in response to at least one of the first flag and the second flag being non-zero, the processor disabling the refinement techniques in reconstructing samples in the current block; The method includes:

6. The method according to claim 5, The first flag is represented by luma_weight_l0_flag[refIdxL0], The second flag is represented by luma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

7. 1. A method for video decoding in a decoder, comprising: extracting prediction information for a current block in a current picture from a signal received by the decoder, the prediction information indicating an inter-prediction mode that may utilize a decoder-side motion vector refinement (DMVR) technique based on a first reference picture and a second reference picture; a processor of the decoder checking a first flag for a chroma weight of the first reference picture and a second flag for a chroma weight of the second reference picture based on the prediction information; in response to at least one of the first flag and the second flag being non-zero, the processor disabling the refinement techniques in reconstructing samples in the current block; The method includes:

8. The method according to claim 7, The first flag is represented by chroma_weight_l0_flag[refIdxL0], The second flag is represented by chroma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

9. A computer program product configured to cause the processor to carry out the method according to any one of claims 1 to 8.

10. A storage medium for storing the computer program according to claim 9.

11. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and signaling the prediction information to an encoder. the prediction information indicating an inter-prediction mode that may utilize a bidirectional optical flow (BDOF) refinement technique based on a first reference picture and a second reference picture; A method according to claim 1, wherein, when the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a luma weight of the first reference picture and a second flag for a luma weight of the second reference picture in the prediction information is set to a non-zero value.

12. The method of claim 11, The first flag is represented by luma_weight_l0_flag[refIdxL0], The second flag is represented by luma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

13. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and signaling the prediction information to an encoder. the prediction information indicating an inter-prediction mode that may utilize a bidirectional optical flow (BDOF) refinement technique based on a first reference picture and a second reference picture; wherein, if the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a chroma weight of the first reference picture and a second flag for a chroma weight of the second reference picture in the prediction information is set to a non-zero value.

14. The method of claim 11, The first flag is represented by chroma_weight_l0_flag[refIdxL0], The second flag is represented by chroma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

15. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and signaling the prediction information to an encoder. the prediction information indicating an inter prediction mode that may utilize a decoder-side motion vector refinement (DMVR) that is a refinement technique based on a first reference picture and a second reference picture; A method according to claim 1, wherein, when the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a luma weight of the first reference picture and a second flag for a luma weight of the second reference picture in the prediction information is set to a non-zero value.

16. 16. The method of claim 15, The first flag is represented by luma_weight_l0_flag[refIdxL0], The second flag is represented by luma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

17. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and signaling the prediction information to an encoder. the prediction information indicating an inter prediction mode that may utilize a decoder-side motion vector refinement (DMVR) that is a refinement technique based on a first reference picture and a second reference picture; wherein, if the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a chroma weight of the first reference picture and a second flag for a chroma weight of the second reference picture in the prediction information is set to a non-zero value.

18. 18. The method of claim 17, The first flag is represented by chroma_weight_l0_flag[refIdxL0], The second flag is represented by chroma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

19. 19. A computer program product causing said processor to carry out a method according to any one of claims 11 to 18.

20. A storage medium storing the computer program according to claim 19.

21. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and transmitting a coded video bitstream including the prediction information to the encoder. the prediction information indicating an inter-prediction mode that may utilize a bidirectional optical flow (BDOF) refinement technique based on a first reference picture and a second reference picture; A method according to claim 1, wherein, when the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a luma weight of the first reference picture and a second flag for a luma weight of the second reference picture in the prediction information is set to a non-zero value.

22. 22. The method of claim 21, The first flag is represented by luma_weight_l0_flag[refIdxL0], The second flag is represented by luma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

23. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and transmitting a coded video bitstream including the prediction information to the encoder. the prediction information indicating an inter-prediction mode that may utilize a bidirectional optical flow (BDOF) refinement technique based on a first reference picture and a second reference picture; wherein, if the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a chroma weight of the first reference picture and a second flag for a chroma weight of the second reference picture in the prediction information is set to a non-zero value.

24. 24. The method of claim 23, The first flag is represented by chroma_weight_l0_flag[refIdxL0], The second flag is represented by chroma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

25. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and transmitting a coded video bitstream including the prediction information to the encoder. the prediction information indicating an inter prediction mode that may utilize a decoder-side motion vector refinement (DMVR) that is a refinement technique based on a first reference picture and a second reference picture; A method according to claim 1, wherein, when the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a luma weight of the first reference picture and a second flag for a luma weight of the second reference picture in the prediction information is set to a non-zero value.

26. 26. The method of claim 25, The first flag is represented by luma_weight_l0_flag[refIdxL0], The second flag is represented by luma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

27. 1. A method for video encoding in an encoder, comprising: A processor of the encoder determining prediction information for a current block in a current picture and transmitting a coded video bitstream including the prediction information to the encoder. the prediction information indicating an inter prediction mode that may utilize a decoder-side motion vector refinement (DMVR) that is a refinement technique based on a first reference picture and a second reference picture; wherein, if the refinement technique is disabled in the reconstruction of samples in the current block, at least one of a first flag for a chroma weight of the first reference picture and a second flag for a chroma weight of the second reference picture in the prediction information is set to a non-zero value.

28. 28. The method of claim 27, The first flag is represented by chroma_weight_l0_flag[refIdxL0], The second flag is represented by chroma_weight_l1_flag[refIdxL1], The method of claim 1, wherein the first flag and the second flag are both zero if the refinement technique is enabled.

29. 29. A computer program product causing said processor to carry out a method according to any one of claims 21 to 28.

30. A storage medium storing the computer program according to claim 29.

Images