Motion vector derivation in video encoding and decoding

Abstract

The invention relates to motion vector derivation in video encoding and decoding. Wherein video processing may involve encoding and / or decoding a picture based on determining activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and encoding and / or decoding a picture based on the modified motion vector refinement process and the second process.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application date of March 6, 2020, application number 202080018709.8, and invention name “Motion Vector Derivation in Video Coding and Decoding”. Technical Field

[0002] The present disclosure relates to video encoding and decoding. Background Art

[0003] To achieve high compression efficiency, image and video coding schemes usually employ prediction and transform to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original picture block and the predicted picture block, which is usually represented as a prediction error or prediction residual, is transformed, quantized, and entropy decoded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transform, quantization, and entropy decoding. Summary of the invention

[0004] At least one example of an embodiment is provided relating to a method for encoding picture information, comprising: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on an indicator; and encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0005] At least one example of an embodiment is provided relating to a method for decoding picture information, comprising: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on an indicator; and decoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0006] At least one example of an embodiment is provided that relates to an apparatus for encoding picture information, comprising: one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0007] At least one example of an embodiment is provided that relates to an apparatus for decoding picture information, comprising: one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0008] At least one example of an embodiment is provided involving a method for encoding picture information, comprising: determining activation of a processing mode involving a motion vector refinement process and a motion vector refinement function; modifying the motion vector refinement process based on the motion vector refinement function; and encoding at least a portion of the picture information based on the processing mode and the modified motion vector refinement process.

[0009] At least one example of an embodiment is provided involving a method for decoding picture information, comprising: determining activation of a processing mode involving a motion vector refinement process and a motion vector refinement function; modifying the motion vector refinement process based on the motion vector refinement function; and decoding at least a portion of the picture information based on the processing mode and the modified motion vector refinement process.

[0010] At least one example of an embodiment is provided involving an apparatus for encoding picture information, comprising: one or more processors. The one or more processors are configured to: determine activation of a processing mode involving a motion vector refinement process and a motion vector refinement function; modify the motion vector refinement process based on the motion vector refinement function; and encode at least a portion of the picture information based on the processing mode and the modified motion vector refinement process.

[0011] At least one example of an embodiment is provided involving an apparatus for decoding picture information, comprising: one or more processors. The one or more processors are configured to: determine activation of a processing mode involving a motion vector refinement process and a motion vector refinement function; modify the motion vector refinement process based on the motion vector refinement function; and decode a picture based on the modified motion vector refinement process.

[0012] At least one example of an embodiment is provided involving a method for encoding a picture, the method comprising: determining activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and encoding the picture based on the modified motion vector refinement process and the second process.

[0013] At least one other example of an embodiment is provided involving a method for decoding a picture, the method comprising: determining activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and decoding the picture based on the modified motion vector refinement process and the second process.

[0014] At least one other example of an embodiment is provided involving an apparatus for encoding a picture, the apparatus comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modify the motion vector refinement process based on the activation and the second process; and encode the picture based on the modified motion vector refinement process and the second process.

[0015] At least one other example of an embodiment is provided involving an apparatus for decoding a picture, the apparatus comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modify the motion vector refinement process based on the activation and the second process; and decode the picture based on the modified motion vector refinement process and the second process.

[0016] At least one other example of an embodiment is provided involving a method for encoding a picture, comprising: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation and the LIC process; and encoding the picture based on the modified DMVR process and the LIC process.

[0017] At least one other example of an embodiment is provided involving a method for decoding a picture, comprising: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation and the LIC process; and decoding the picture based on the modified DMVR process and the LIC process.

[0018] At least one other example of an embodiment is provided involving an apparatus for encoding a picture, the apparatus comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modify the DMVR process based on the activation and the LIC process; and encode the image based on the modified DMVR process and the LIC process.

[0019] At least one other example of an embodiment is provided involving an apparatus for decoding a picture, the apparatus comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modify the DMVR process based on the activation and the LIC process; and decode the picture based on the modified DMVR process and the LIC process.

[0020] A simplified overview of the subject matter is provided above in order to provide a basic understanding of some aspects of the present disclosure. This overview is not an exhaustive review of the subject matter. It is not intended to identify the key / important elements of each embodiment or to describe the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a preface to the more detailed description provided below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present disclosure may be better understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A block diagram depicting an example of an embodiment of a video encoder is provided;

[0023] Figure 2 A block diagram depicting an example of an embodiment of a video decoder is provided;

[0024] Figure 3 Aspects of the present disclosure relating to coding tree units (CTUs) are shown;

[0025] Figure 4 Aspects of the present disclosure relating to CTUs and coding units (CUs) are shown;

[0026] Figure 5 An example of decoding side motion vector refinement (DMVR) is shown;

[0027] Figure 6 A flow chart illustrating an example of a DMVR search process is provided;

[0028] Figure 7 An example of a DMVR integer brightness sample search mode is shown;

[0029] Figure 8 Examples of illumination compensation parameters, such as local illumination compensation (LIC) parameters, are shown, which are derived from neighboring reconstructed samples and corresponding reference samples transformed with motion vectors (MVs) for both a square CU (upper illustration) and a rectangular CU (lower illustration);

[0030] Fig. 9 A flow chart is provided showing an example of the derivation and application of LIC parameters for each prediction L0 and L1;

[0031] Fig.10 A flow chart showing an example of derivation of LIC parameters and application to combined predictions from L0 and L1 is provided;

[0032] Fig.11 An example of the positions of the left and top reference samples of a rectangular CU is shown;

[0033] Fig.12 A flow chart illustrating an example of bidirectional prediction with DMVR and LIC is provided;

[0034] Fig.13 A flowchart illustrating an example of derivation of LIC parameters based on decoded bi-directionally predicted motion vectors prior to DMVR refinement is provided;

[0035] Fig.14 Provides a block diagram illustrating an example of an embodiment of an apparatus according to various aspects described herein;

[0036] Fig.15 A flow chart illustrating an example of an embodiment involving bi-directional prediction with motion vector refinement, such as DMVR and weighted prediction (WP), is provided; and

[0037] Figures 16 to 19 Flowcharts illustrating various examples of embodiments according to the present disclosure are provided.

[0038] It should be understood that the drawings are for illustrating examples of various aspects, features and embodiments according to the present disclosure and are not necessarily the only possible configurations. In all drawings, the same reference numerals represent the same or similar features. DETAILED DESCRIPTION

[0039] Now turning to the attached figure, Figure 1 An example of a video encoder 100 is shown, such as a HEVC encoder. HEVC is a video Joint Collaboration Team on Coding and Decoding (JCT-VC) A compression standard developed by the ITU-T H.265 (see, for example, “ITU-T H.265 Telecommunication Standards Section (10 / 2014), Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services - Coding of Mobile Video, High Efficiency Video Coding, Recommendation ITU-T H.265”). Figure 1 Also shown may be encoders in which improvements have been made to the HEVC standard or encoders employing techniques similar to HEVC, such as encoders based on or improved upon the JEM (Joint Exploration Model) developed by the Joint Video Experts Group (JVET), such as the JEM associated with the development work of the specified Versatile Video Coding (VVC).

[0040] In this application, the terms "reconstruction" and "decoding" can be used interchangeably, the terms "pixel" and "sample" can be used interchangeably, and the terms "picture" and "frame" can be used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0041] The HEVC specification distinguishes between "blocks", which address a specific area in the sample array (e.g., luma, Y), and "units", which include a concatenated block of all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and prediction data associated with the block (e.g., motion vectors).

[0042] For decoding, the picture is partitioned into coding tree blocks (CTBs) of a square shape with a configurable size, and a continuous set of coding tree blocks is grouped into slices. A coding tree unit (CTU) contains the CTB of the encoded color component. The CTB is the root of a quadtree partitioned into coding blocks (CBs), and the coding block can be partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree partitioned into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, the coding unit (CU) includes a tree structure set of prediction units (PUs) and transform units (TUs), the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The sizes of the CB, PB, and TB of the luminance component are applicable to the corresponding CU, PU, ​​and TU. In the present application, the term "block" may be used to refer to any of CTU, CU, PU, ​​TU, CB, PB, and TB. Additionally, "block" may also be used to refer to macroblocks and partitions as specified in H.264 / AVC or other video coding standards, and more generally to data arrays of various sizes.

[0043] exist Figure 1 In the encoder 100, as described below, the picture is encoded by the encoder elements. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When the CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder decides (105) whether to encode the CU using one of intra mode or inter mode, and indicates the intra / inter decision through a prediction mode flag. The prediction residual is calculated by subtracting (110) the prediction block from the original picture block.

[0044] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients as well as the motion vectors and other syntax elements are entropy decoded (145) to output a bitstream. The encoder can also skip the transform and apply quantization directly to the untransformed residual signal on a 4×4TU basis. The encoder can also bypass the transform and quantization, i.e., directly decode the residual without applying the transform or quantization process. In direct PCM decoding, no prediction is applied and the coding unit samples are decoded directly into the bitstream.

[0045] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the picture block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0046] Figure 2 2 shows a block diagram of an example of a video decoder 200, such as a HEVC decoder. In the example decoder 200, a signal or bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described above. Figure 1 The encoding pass described in is reciprocal to the decoding pass, which performs video decoding as part of encoding the video data. Figure 2 It is also possible to show decoders in which improvements are made to the HEVC standard or decoders that employ techniques similar to HEVC, such as decoders based on or improved upon JEM.

[0047] Specifically, the decoder input includes a Figure 1 A video signal or bitstream is generated by a video encoder of the video encoder 100. The signal or bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors and other decoding information. The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode prediction residuals. The decoded prediction residuals are combined (255) with the prediction block to reconstruct the picture block. The prediction block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). Advanced motion vector prediction (AMVP) and merge mode techniques can be used to derive motion vectors for motion compensation, which can use interpolation filters to calculate interpolated values ​​of sub-integer samples of reference blocks. An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0048] In the HEVC video compression standard, motion compensated temporal prediction is used to exploit the redundancy that exists between consecutive pictures of a video. To this end, a motion vector is associated with each prediction unit (PU). Each coding tree unit (CTU) is represented by a coding tree (CT) in the compressed domain. This is a quadtree partitioning of the CTU, where each leaf is called a coding unit (CU), such as Figure 3 as shown in .

[0049] Each CU is then given some intra-frame or inter-frame prediction parameters (prediction information). To this end, it is spatially split into one or more prediction units (PUs), each of which is assigned some prediction information. Figure 4 As shown, intra or inter coding modes are allocated at the CU level, which shows an example of dividing the coding tree unit into coding units, prediction units, and transform units. To decode a CU, a prediction block or prediction unit (PU) is constructed from adjacent reconstructed samples (intra prediction) or from a previously reconstructed picture (inter prediction) stored in a decoded picture buffer (DPB). Next, the residual samples calculated as the difference between the original samples and the PU samples are transformed and quantized.

[0050] In inter prediction, motion compensated temporal prediction is employed to exploit the redundancy that exists between consecutive pictures of a video. To this end, a motion vector is associated with a PU and a reference index 0 (refIdx0) that indicates which of the multiple reference pictures listed in the list of the specified LIST_0 is used.

[0051] In HEVC, two modes are used to encode motion data. They are called AMVP (Adaptive Motion Vector Prediction) and Merge, respectively. AMVP involves signaling (one or more) reference pictures, motion vector predictor indices (obtained from lists of two predictors), and motion vector differences used to predict the current PU. The merge mode includes signaling and decoding indices of some motion data collected in a motion data predictor list. The list consists of five candidates and is constructed in the same way on the decoder and encoding sides. Therefore, the merge mode aims to derive some motion information obtained from the merge list. The merge list typically contains motion information associated with some spatial and temporal surrounding blocks, which are available in their decoding state when the current PU is being processed.

[0052] In other codecs, such as the one designated as the Joint Exploration Model (JEM) and the one developed by the JVC (Joint Video Exploration Team) group in the Universal Video Coding (VVC) reference software called the VVC Test Model (VTM), some modes used in inter prediction (e.g., bidirectional inter prediction or B-mode) can exploit transmitted parameters to compensate for illumination changes. In B-mode, the current block is associated with two motion vectors, specifying two reference blocks in two different images. The predictor block that allows the calculation of the residual block for the current block is the average of the two reference blocks. Several generalizations of bidirectional inter prediction have been proposed, in which the weights associated with each reference block are different. Weighted prediction (WP) can be considered a generalization of bidirectional inter prediction in some aspects. In WP, the residual block is calculated as the difference between the current block and a weighted version of the reference block in the case of unidirectional inter prediction or a weighted average of the two reference blocks in the case of bidirectional inter prediction. WP can be enabled at the sequence level in the sequence header (called Sequence Parameter Set (SPS) in VVC) or the picture level in the picture header (called Picture Parameter Set (PPS) in VVC). WP defines the weights and offsets for each group of CTUs (e.g., typically at the slice header level) associated with each component of each reference picture of each list (L0 and L1) of reference pictures.

[0053] While WP is enabled in the sequence header (SPS) and picture header (PPS), and the associated weights and offsets are specified in the slice header, a new mode called CU-level weighted bidirectional prediction (BCW) allows signaling of weights at the block level.

[0054] Some additional temporal prediction tools with associated parameters determined at the decoder side provide features such as motion vector refinement and compensation for issues such as illumination variations. Such additional tools may include, for example, motion vector refinement, such as decoder-side motion vector refinement (DMVR), and illumination compensation, such as local illumination compensation (LIC). One purpose of DMVR may be to further refine motion vectors by using prediction methods such as bilateral matching prediction. One purpose of LIC may be to compensate for illumination variations that may occur between a prediction block and its reference block employed by motion compensated temporal prediction. Both of these tools may involve, at least in part, a decoder-side process to derive parameters for prediction.

[0055] In more detail, the method of increasing the MV accuracy of the merge mode may involve applying decoder-side motion vector refinement based on bidirectional matching (BM). In a bidirectional prediction operation, the refined MV is searched around the initial MV in the reference picture list L0 and the reference picture list L1. The BM method is based on a method, such as the sum of absolute differences (SAD), to calculate the distortion between two candidate luminance blocks in the reference picture list L0 and the list L1. Figure 5 As shown in the example of , the SAD between blocks 500 and 501 (red blocks) is calculated based on each MV candidate around the initial MV. The MV candidate with the lowest SAD becomes the refined MV and is used to generate the bidirectional prediction signal.

[0056] A method of applying DMVR, including where DMVR can be applied to a CU decoded using the following mode:

[0057] CU-level merge mode with bi-directional prediction MV;

[0058] Relative to the current picture, one reference picture is in the past and the other reference picture is in the future;

[0059] The distances (e.g., picture order count (POC) differences) from two reference pictures to the current picture are the same;

[0060] • The CU has more than 64 luma samples and the CU height is more than 8 luma samples.

[0061] The refined MV derived by the DMVR process can be used to generate inter-frame prediction samples and can also be used for temporal motion vector prediction for future picture decoding, while the original MV can be used, for example, for deblocking processes and can also be used for spatial motion vector prediction for future CU decoding. Additional features of one or more methods of DMVR are explained below.

[0062] like Figure 5 As shown in the example shown in , the search points are around the initial MV, and the MV offsets follow the MV difference mirror rule. In other words, any point examined by DMVR in Figure 5 In the representation of candidate MV pairs (MV0, MV1), the following two equations apply:

[0063] MV0′=MV0+MV offset (1)

[0064] MV1′=MV1-MV offset (2)

[0065] MV offset represents the refinement offset between the initial MV and the refinement MV in one of the reference pictures. The refinement search range may be, for example, two integer luma samples from the initial MV.

[0066] Figure 6 Provides a diagram showing Figure 5 A flow chart of an example of a DMVR search process is shown in FIG. Figure 6 As shown in the example of , the search includes an integer sample offset search phase and a fractional sample refinement phase.

[0067] To reduce the complexity of the search, a fast search method with early termination mechanism can be applied in the integer sample offset search stage. Instead of a full search, for example 25 points, a double iterative search scheme can be applied to reduce the SAD checkpoints. Figure 7 As shown, a maximum of six SADs are checked in the first iteration. First, the SADs of five points (center and P1 to P4) are compared. If the SAD of the center position is the smallest, the integer sample stage of DMVR is terminated. Otherwise, another position P5 is checked (determined by the SAD distribution of P1 to P4). Then, the position with the smallest SAD (among P1 to P5) is selected as the center position for the second iteration search. The process of the second iteration search is the same as that of the first iteration search. The SAD calculated in the first iteration can be reused in the second iteration, so only the SAD of three additional points needs to be further calculated.

[0068] The integer sample search may be followed by fractional sample refinement. To reduce computational complexity, fractional sample refinement may be derived by using a parameter error surface equation instead of an additional search with SAD comparison. Fractional sample refinement may be conditionally called based on the output of the integer sample search phase. When the integer sample search phase ends with the center having the minimum SAD in the first or second iteration search, fractional sample refinement is further applied.

[0069] In the parametric error surface based sub-pixel offset estimation, the center location cost and the costs at four neighboring locations from the center are used to fit a 2D parabolic error surface equation of the following form:

[0070] E(x, y) = A(x min ) 2 +B(yy min ) 2 +C (3)

[0071] where (x_min, y_min) corresponds to the fractional position with the minimum cost, and C corresponds to the minimum cost value. By solving the above equation using the cost values ​​of the five search points, (x_min, y_min) is calculated as:

[0072]

[0073] y_min=(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0))) (5)

[0074] x min dy minThe value of e is automatically clamped between -8 and 8, since all cost values ​​are positive, and the minimum value is E(0,0). This corresponds to, for example, a half-pixel offset with 1 / 16th pixel MV accuracy. The calculated fraction (x_min, y_min) is added to the integer distance refinement MV to obtain a sub-pixel accurate refinement delta MV.

[0075] The codec can provide, for example, that the resolution of the MV can be 1 / 16 luminance samples. An 8-tap interpolation filter is used to interpolate samples at fractional positions. In DMVR, the search points are offset around the initial fractional pixel MV with integer samples, so it is necessary to interpolate samples at those fractional positions for the DMVR search process. In order to reduce computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect is that by using a bilinear filter, DVMR does not access more reference samples within a two-sample search range compared to the normal motion compensation process. After obtaining the refined MV using the DMVR search process, a normal 8-tap interpolation filter is applied to generate the final prediction. In order not to access more reference samples to the normal MC process, samples that are not required by the interpolation process based on the original MV but are required by the interpolation process based on the refined MV will be filled in from those available samples.

[0076] The maximum unit size of the DMVR search process may be limited to, for example, 16 x 16. When the width and / or height of a CU is greater than the maximum size, for example, greater than 16 luma samples, the CU may be further split into sub-blocks having a width and / or height equal to, for example, the maximum size (e.g., 16 luma samples).

[0077] Some modes used in inter prediction (or B-mode) can exploit the transmitted parameters to compensate for illumination changes. In B-mode, the current block is associated with two motion vectors, specifying two reference blocks in two different images. This allows the calculation of the predictor block of the residual block of the current block as the average of the two reference blocks. The predictor biPred of the current block is calculated as follows:

[0078]

[0079] where pred 0 is a motion compensated block predictor taken from a list L0 of reference pictures stored in a buffer of decoded pictures, generally referred to as a decoded picture buffer (DPB); pred 1 is the motion compensated block predictor taken from the list L1 of reference pictures stored in the DPB; 0 and w 1 is a weight equal to one in the case of classic bidirectional inter prediction.

[0080] Several generalizations of bidirectional inter prediction are proposed, where the weights w 0 and w 1 are different. Weighted prediction (WP) can be considered as a generalization of bidirectional inter prediction in some aspects. In WP, the residual block is calculated as the difference between the current block and a weighted version of the reference block in the case of unidirectional inter prediction or a weighted average of the two reference blocks in the case of bidirectional inter prediction. WP can be enabled at the sequence level in the sequence header (called Sequence Parameter Set (SPS) in VVC) or at the picture level in the picture header (called Picture Parameter Set (PPS) in VVC). WP defines a weight w for each group of CTUs (e.g., typically at the slice header level) associated with each component of each reference picture i of each list (L0 and L1) of reference pictures stored in the DPB. i and offset off i If the current block is decoded with bidirectional WP, the prediction sample pred(x, y) at the position (x, y) of the current block is calculated as follows:

[0081] pred(x,y)=((w 0 .pred 0 (x,y)+w 1 .pred 1 (x,y)+Off 01 )>>(shift+1))

[0082] Where Off 01 =(Off 0 +Off 1 +1)< <shift

[0083] where pred i (x, y) is the list L of reference pictures stored in the DPB i The motion compensated predictor samples obtained from and spatially corresponding to pred(x, y), w i is the weight, off i is the offset value.

[0084] In order to maintain increased numerical precision when weighting samples, intermediate sample values ​​with increased bit depth precision can be stored and manipulated. In this case, at the end of the prediction calculation process, the final right bit shift is used to obtain the final (desired) bit depth sample prediction precision (bitDepth). For example, the reference picture in the DPB is stored with precision bitDepth, but the intermediate motion compensated samples are stored in the intermediate buffer with increased precision (bitDepth+sp). One purpose of the shift value shift in the above two equations is to ensure this intermediate increased bit depth precision. Note that a similar intermediate bit depth precision increase process is typically used for all prediction tools that use the sample weighting process.

[0085] While WP is enabled in the sequence header (SPS) and picture header (PPS), and the associated weights and offsets are specified in the slice header, a new mode called CU-level weighted bidirectional prediction (BCW) allows the weights at the block level to be signaled. When BCW mode is applied to the current block, the predictor sample pred(x, y) of the current block is calculated as follows:

[0086] pred(x,y)=((8-w).pred 0 (x,y)+w.pred 1 (x,y)+4)>>3

[0087] where pred 0 (x, y) is a motion compensated predictor sample taken from list L0 of the reference picture stored in the DPB and spatially corresponding to pred(x, y), pred 1 (x, y) is a motion compensated predictor sample taken from the list L1 of the reference picture stored in the DPB and spatially corresponding to pred(x, y), and w is a weight taken from a set of five weights (w∈{-2,3,4,5,10}). The weight w is determined in one of two ways: 1) For a non-merged CU, the weight index bcw_idx is signaled after the motion vector difference; 2) For a merged CU, the weight index bcw_idx_ is inferred from neighboring blocks based on the merge candidate index.

[0088] The use of tools such as illumination compensation (e.g., local illumination compensation (LIC)) can be signaled at the CU level, for example, through a flag (LIC flag) associated with each coding unit (CU) coded in inter-frame mode. When the tool is activated, the decoder uses the following code based on the LIC flag: Figure 8The example shown in is used to calculate some prediction parameters based on: some reconstructed picture samples located on the left and / or top of the current block to be predicted; and reference picture samples located on the left and / or top of the motion compensation block. In the following, a set of samples consisting of samples located on rows above the current block and samples located on columns to the left of the current block will be referred to as an "L-shape" associated with the current block. For example, in Figure 8 An example of an L-shape includes small squares above and to the left of the current block.

[0089] LIC may be based on a linear model with parameters corresponding to, for example, weights and offsets and designated as (a, b). The parameters (a, b) may be determined, for example, based on minimization of the error or local distortion, distance "dist", between the current sample and a linearly modified reference sample, which may be defined as follows (Equation 6):

[0090]

[0091] Where: rec_cur(r) is the adjacent reconstructed sample in the current picture (given by Figure 8 The example on the right is shown);

[0092] rec_ref(s) is the reference sample constructed from the reference picture using motion compensation (MC) (by Figure 8 The example on the left shows), where s = r + mv;

[0093] rec_cur(r) and rec_ref(s) are the reconstructed samples and the co-located samples in the reference L-shape, respectively.

[0094] In some implementations, additional restrictions may be imposed when selecting the reconstruction and reference samples used in the L-shape. For example, it may be imposed that the reconstructed samples belonging to neighboring blocks are not coded in intra mode or are not coded using prediction constructed using intra mode (e.g., CIIP).

[0095] The value of (a, b) can be obtained using the following least squares minimization (Equation 7):

[0096]

[0097] Note that the value of N can be further adjusted (incrementally reduced) in order to sum the terms in Equation 7 to stay below the maximum integer storage value allowed (e.g., N<216). Also, for large blocks, the subsampling of the top and left sample sets can be incrementally increased. Once the encoder or decoder obtains the LIC parameters for the current CU, the prediction pred(current_block) of the current CU includes, for example, the example case of unidirectional prediction:

[0098] pred(current_block)=a×ref_block+b (8)

[0099] where current_block is the current block to be predicted, pred(current_block) is the prediction of the current block, and ref_block is the reference block constructed using a conventional motion compensation (MC) process and used for temporal prediction of the current block.

[0100] Note that the set of adjacent reconstructed samples and the set of reference samples ( Figure 8 The samples shown by small square blocks in the figure have the same number and the same pattern. Hereinafter, "left samples" will refer to a set of neighboring reconstructed samples (or a set of reference samples) located immediately to the left of the current block, and "top samples" will refer to a set of neighboring reconstructed samples (or a set of reference samples) located immediately above the top of the current block (or adjacent to the top of the current block). "Sample set" will refer to a combination of each of the "left samples" and "top samples" sets.

[0101] In the case of bidirectional prediction, the LIC process can be applied twice, as Fig. 9 The example shown in Fig. 9 In , the LIC process is applied to both reference 0 prediction (LIST-0) and reference 1 prediction (LIST_1), as Fig. 9 300 to 340 in FIG. Next, Fig. 9 At 350 in , the two predictions are combined using, for example, the default weighting (P = (P0 + P1 + 1) >> 1) or the bi-directional prediction weighted average (BPWA) (P = (g0.P0 + g1.P1 + (1 << (s-1))) >> s)). This method will be referred to as "method a".

[0102] is referred to herein as "Method B" and Fig.10 In another embodiment example shown in FIG. 1 , in the case of bidirectional prediction, Fig.10 In , two predictions, ie, reference 0 and reference 1 predictions, are determined at 300 and then combined at 360 before a single LIC process is applied at 320 , 330 , 340 , and 370 .

[0103] In an example of another embodiment referred to herein as "Method c" (Method c based on Method b), in the case of bidirectional prediction, the LIC-0 and LIC-1 parameters are derived directly, for example:

[0104] dist=∑ r∈Vcur,s∈Vref (rec cur(r) -a0.rec ref0(s)-a1.rec ref1(s) -b) 2 (9)

[0105] In the selection Figure 8 Some restrictions may be applied when using the left and top samples shown in for deriving LIC parameters. For example, the restrictions may be one or more of the following:

[0106] 1) The sum of the number of samples on the left and top should be a power of 2 in order to implement the division using a right shift.

[0107] 2) Left side (N L ) and top (N T )The number of samples is the same and equal to Ns (N = 2.Ns):

[0108] n = min(cuHeight, cuWidth)

[0109] x=log2(n)

[0110] Ns=2 x (10)

[0111] 3) The step size between samples on the left (stepV) or top (stepH) is equal to:

[0112] stepV=cuHeight>>log2(Ns) (11)

[0113] stepH=cuWidth>>log2(Ns)

[0114] Typically, LIC, WP, BCW, and DMVR as described above are separate tools. However, these tools can be combined in various ways. According to one aspect of the present disclosure, combining these tools can involve adjusting at least one of these tools to obtain improved or optimal performance. For example, Fig.16 The embodiment shown in may involve determining at 1610 whether motion vector refinement, such as provided by a DMVR tool, is activated. If not (a negative or "no" result at 1610), then processing, such as encoding or decoding of video or picture data, continues at 1640. If there is a positive result at 1610 (a "yes" result at 1610), then a determination is made at 1620 whether a second process is activated (e.g., LIC, WP, or BCW). If not (a "no" result at 1620), then processing continues at 1640. If the result at 1620 is positive (e.g., "yes"), then processing continues at 1630, where, as a result of the tool combination being activated, the motion vector refinement process is modified or adjusted as described below.

[0115] Fig.17Another example of an embodiment is shown, which is Fig.16 A variation of the embodiment shown in FIG. Fig.17 In the example, activation of the DMVR is determined at 1710, followed by activation of the LIC at 1720. In addition, Fig.17 The features of the embodiment shown in are similar to those described above with respect to Fig.16 Those features described in the embodiments of the present invention and will not be described again.

[0116] As described above, one aspect of the present disclosure may involve adjustments to at least one of the described tools to obtain improved or optimal performance when combining tools such as those described herein. For example, the absolute difference sum (SAD) method used to derive a DMVR predictor may not be suitable for the application of tools such as LIC, WP or BCW in combination with DMVR. For example, LIC or WP or BCW can correct some default values ​​that the DMVR process may have reduced or attempted to minimize using a SAD function. The encoder can apply a motion vector search algorithm using an additional, different from or modified cost determination method or function with respect to a function such as SAD. For example, when LIC or WP or BCW is activated, a function of a modified form of SAD such as mean removed SAD (MRSAD) can be applied to attempt to optimize the motion search for which LIC or WP or BCW will not be able to correct, that is, to optimize the result of the combination of MOTION+(LIC or WP or BCW). However, using DMVR with SAD after performing a motion search using MRSAD may not be optimal and may even be counterproductive. One approach to addressing these issues may be to prevent or disable the combination of these tools, for example, when LIC or WP or BCW is ON, then DMVR is not allowed. However, doing so may be undesirable or suboptimal because the benefits of DMVR are not provided.

[0117] In general, one aspect of the invention may be directed to improving motion vector refinement, such as DMVR, in the case of activated illumination compensation, such as LIC or WP or BCW. At least one embodiment may be directed to using mean removed sum of absolute values ​​(MRSAD) instead of using sum of absolute values ​​(SAD) in the DMVR derivation process when LIC or WP or BCW is activated in conjunction with DMVR. Once the DMVR prediction is available, the LIC process is calculated.

[0118] According to one aspect of the present disclosure, generally, at least one embodiment may involve using MRSAD in the DMVR derivation process when LIC or WP or BCW is activated. For example, Fig.12 An embodiment of a bidirectional prediction process is described. Fig.12In , at 1210, the input to the process is the motion vector pair of the L0 and L1 reference lists. If at 1220, it is determined that DMVR is not applied, then at 1250 a "conventional" bi-directional prediction process is applied, followed by the LIC process if LIC is enabled or activated, at 1260. If it is determined at 1220 that DMVR is applied, then at 1230, its application depends on the LIC condition. Instead of LIC, WP or BCW values ​​can also be used as conditions. If LIC is not applied, then at 1240, SAD is used as described above to derive DMVR, followed by bi-directional prediction using the refined motion vectors at 1250. If LIC is applied as determined at 1230, DMVR is applied at 1245 using MRSAD as a cost function, followed by motion compensation at 1250.

[0119] MRSAD can be used as Figure 7 occurs as shown. That is, the MRSAD of the five points (center and P1 to P4) are first compared. If the MRSAD of the center position is the smallest, the integer sample stage of DMVR is terminated. Otherwise, another position P5 is checked. The position with the smallest MRSAD is then selected as the center position for the second iterative search. The process of the second iterative search is the same as that of the first iterative search. The MRSAD calculated in the first iteration can be reused in the second iteration, so only the MRSAD of three additional points needs to be further calculated. As previously described, the integer sample search is followed by a fractional sample refinement of the parameter error surface equation using the MRSAD values.

[0120] MRSAD between two blocks of the same size first eliminates the constant offset between the two blocks before calculating the difference between the two blocks. It is different from the sum of absolute differences (SAD), where constant changes between the two blocks can affect a large difference. The purpose of MRSAD is to be more robust to lighting changes. In our case, this is relevant to use this measurement because the LIC or WP or BCW process will correct for lighting changes. First calculate the average difference D between the two blocks B1 and B2 as follows:

[0121]

[0122] where i is the block pixel coordinate belonging to B, and N is the number of pixels in the block.

[0123] The MRSAD is then calculated by summing the absolute differences of the block pixels minus the mean difference:

[0124]

[0125] In general, at least one example of an embodiment may involve calculating LIC parameters prior to or in parallel with a DMVR process, such as Fig.13 The example in Fig.13 In the example, after obtaining the input of two motion vectors at 1310, LIC parameters are determined at 1325, 1350 and 1360, while the DMVR process is performed at 1320, 1330, 1340, 1345, 1370 and 1380. The LIC parameters generated at 1360 are provided to 1390, where LIC is applied if it is activated or enabled. Fig.13 The exemplary embodiments in enable the calculation of LIC parameters in parallel with the DMVR process. Doing so may improve operational efficiency and / or improve performance, reduce costs, etc.

[0126] In general, another example of at least one embodiment may involve calculating the block average difference D only for the first position in the first iteration of the DMVR process and reusing it for adjacent positions. Doing so may enable calculation of the average block difference for each test position to be avoided, thereby potentially improving operational efficiency and / or improving performance, reducing costs, etc.

[0127] Fig.15 Another example of an embodiment is shown. Fig.15 An example involves motion vector refinement such as DMVR using a tool such as WP mentioned above. Fig.15 , at 1510, the input to the process is a pair of motion vectors for the L0 and L1 reference lists. If it is determined at 1520 that motion vector refinement, such as DMVR, is not applied ("No" at 1520), then at 1550 it is determined whether the reference is weighted. If so ("Yes" at 1550), then at 1565 bidirectional weighted motion compensation occurs and the predicted CU is output. If the reference is not weighted ("No" at 1550), then at 1560 "conventional" bidirectional predictive motion compensation is applied, and then the resulting predicted CU is output. If it is determined at 1520 that motion vector refinement, such as DMVR, is applied ("Yes" at 1520), then its application depends on whether the reference is weighted at 1530, such as with WP weighting. If it is weighted ("Yes" at 1530), then at 1545 motion vector refinement is derived based on using MRSAD as described above, and then at 1550 operations as described above continue. If it is determined at 1530 that the reference is not weighted (“NO” at 1530 ), then at 1540 the SAD as described above is used to derive a motion vector refinement, followed by continuing at 1550 as described above.

[0128] More generally, at least one other example of an embodiment may involve modifying or adjusting motion vector refinement, such as a DMVR process, based on an indicator, such as information encoded / decoded or inferred in the bitstream, or a flag, or an index (e.g., as in the previous embodiment, if LIC is on, the inferred flag is true). Fig.18 An example of such an embodiment is shown in FIG. Fig.18 In 1810, it is determined that activation of decoder-side motion vector refinement DMVR occurs. The DMVR process may include a refinement function such as SAD as described above. If it is determined in 1810 that the DMVR process is activated, then in 1820, the refinement function may be modified based on an indicator, such as information or a flag or an index. For example, the indicator may indicate that the refinement function is modified to MRSAD instead of SAD, such as when the LIC tool is activated as described above. Alternatively, the indicator may indicate a tool that is activated together with DMVR in one or more tools such as LIC, WP, and BCW, wherein the combination of the tools results in the modification of the refinement function as described above, for example, selecting MRSAD instead of SAD as the refinement function. After 1820, the picture information is encoded / decoded based on the DMVR process including the modified refinement function.

[0129] Fig.19 Shown as Fig.18 Another example of a variation of the embodiment shown in FIG. Fig.19 , operation determines at 1910 whether to activate a DMVR process, such as provided by a DMVR tool. If the DMVR is not activated (a negative or "no" result at 1910), then processing such as encoding or decoding of video or picture data continues at 1970. If there is a positive result at 1910 (a "yes" result at 1910), then the DMVR process is activated. As described above, the DMVR process can include a refinement function such as SAD. At 1920, it is determined (inferred from the bitstream or read / determined by the encoder) as described above with respect to Fig.18 The indicator is described, wherein the indicator indicates whether a refinement function, such as SAD, is modified or adjusted. The indicator value is tested at 1930. If the result is negative (a "no" result at 1930), processing continues at 1940, where a DMVR process including a refinement function such as SAD continues. If the result at 1930 is positive (e.g., "yes"), processing continues at 1950, where the refinement function is modified based on the indicator, for example, it is selected to be MRSAD. In a variation, the indicator can be a function index having multiple values ​​mapped to corresponding multiple functions, and the multiple functions can be selected to modify or adjust the refinement function included in the DMVR process. Fig.19In the example of , an index equal to 0 may indicate or correspond to a selection of a refinement function of SAD, while an index equal to 1 may indicate or correspond to a selection of a refinement function of MRSAD. The selection of the refinement function based on the indicator or index modifies the refinement function. Then, at 1960, a DMVR process including the modified refinement function occurs. At 1970, picture information is encoded / decoded based on the DMVR process including the modified refinement function.

[0130] This document describes various examples of embodiments, features, models, methods, etc. Many of these examples are described as being specific and, at least to illustrate individual characteristics, are often described in a manner that may appear restrictive. However, this is for the purpose of clarity of description and does not limit application or scope. In fact, various examples of the embodiments, features, etc. described herein can be combined and interchanged in various ways to provide additional examples of the embodiments.

[0131] In general, the examples of the embodiments described and contemplated in this document may be implemented in many different forms. Figure 1 and 2 And the following Fig.14 Some embodiments are provided, but other embodiments are contemplated and are not intended to be construed in detail. Figure 1 , 2 The discussion of 14 does not limit the breadth of implementation. At least one embodiment generally provides examples related to video encoding and / or decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream or signal. These and other embodiments can be implemented as methods, apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream or signal generated according to any of the described methods.

[0132] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0133] The terms HDR (high dynamic range) and SDR (standard dynamic range) are used in this disclosure. Those terms generally convey a specific value of dynamic range to one of ordinary skill in the art. However, it is also intended that in additional embodiments, references to HDR are understood to mean "higher dynamic range" and references to SDR are understood to mean "lower dynamic range". Such additional embodiments are not constrained by any specific value of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range".

[0134] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0135] Various methods and other aspects described in this document may be used to modify modules of a video encoder and / or decoder, such as Figure 1 The motion compensation and / or motion estimation modules 170 and 175 of the encoder 100 shown in FIG. Figure 2 275 of the decoder 200 shown in FIG. 275. Furthermore, aspects of the present invention are not limited to VVC or HEVC, and may be applied to, for example, other standards and recommendations (whether previously existing or developed in the future), as well as extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described herein may be used alone or in combination.

[0136] For example, various numerical values ​​are used in this document. The specific values ​​are for example purposes, and the described aspects are not limited to these specific values.

[0137] Fig.14 A block diagram of an example of a system in which various aspects and embodiments can be implemented is shown. System 1000 can be implemented as a device including various components described below, and is configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1000 can be implemented individually or in combination in a single integrated circuit, multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed on multiple ICs and / or discrete components. In various embodiments, system 1000 is coupled to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 1000 is configured to implement one or more aspects described herein.

[0138] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 1010 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an EEPROM, a ROM, a PROM, a RAM, a DRAM, an SRAM, a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device, and / or a network-accessible storage device.

[0139] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module (or modules) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated into the processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0140] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of the various items during the execution of the processes described in this document. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams or signals, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0141] In several embodiments, memory within the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as MPEG-2, HEVC, or VVC (universal video coding).

[0142] As shown in block 1130, input to the elements of system 1000 may be provided through various input devices. Such input devices include, but are not limited to: (i) an RF portion that receives an RF signal transmitted over the air, for example, by a broadcaster; (ii) a composite input terminal; (iii) a USB input terminal; and / or (iv) an HDMI input terminal.

[0143] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band); (ii) down-converting the selected signal; (iii) again band-limiting to a narrower frequency band to select a signal frequency band that may be referred to as a channel in some embodiments; (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (for example, cable) medium, and filter to the desired frequency band again by filtering, down-conversion and perform frequency selection. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions. Adding element can include inserting element between existing element, for example, inserting amplifier and analog-to-digital converter. In various embodiments, the RF part includes antenna.

[0144] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 1010. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device.

[0145] The various components of the system 1000 may be provided within an integrated housing. Within the integrated housing, the various components may be interconnected and transfer data between using a suitable connection arrangement 1140 such as an internal bus known in the art including an I2C bus, wiring, and printed circuit boards.

[0146] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. Communication interface 1050 may include, but is not limited to, a Pass The communication channel 1060 is a transceiver for transmitting and receiving data. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0147] In various embodiments, data is streamed to the system 1000 using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signals of these embodiments are received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communications. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications. Other embodiments provide streaming data to the system 1000 using a set-top box that passes data through an HDMI connection of an input block 1130. Still other embodiments provide streaming data to the system 1000 using an RF connection of an input block 1130.

[0148] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripherals 1120. In various examples of various embodiments, the other peripherals 1120 include one or more of a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 1000. In various embodiments, control signals are transmitted between the system 1000 and the display 1100, the speaker 1110, or other peripherals 1120 using signaling such as AV. Link, CEC, or other communication protocols, which enables device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 1000 via dedicated connections through the corresponding interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and the speaker 1110 can be integrated into a single unit in an electronic device (e.g., a television) along with other components of the system 1000. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T con) chip.

[0149] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more of the other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0150] These embodiments may be implemented by computer software implemented by the processor 1010 or a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, as a non-limiting example, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0151] Various generalized and specialized embodiments are also supported and considered in the present disclosure.Examples of embodiments according to the present disclosure include, but are not limited to, the following.

[0152] In general, at least one example of an embodiment may relate to a method for encoding picture information, comprising: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on an indicator; and encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0153] Generally, at least one example of an embodiment may relate to a method for decoding picture information, comprising: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on an indicator; and decoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0154] In general, at least one example of an embodiment may be directed to an apparatus for encoding picture information, comprising: one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0155] In general, at least one example of an embodiment may be directed to an apparatus for decoding picture information, comprising: one or more processors. The one or more processors are configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0156] In general, at least one example of an embodiment may relate to a method as described herein, wherein modifying the refinement function comprises modifying a cost function associated with a decoder-side motion vector refinement process.

[0157] In general, at least one example of an embodiment may relate to an apparatus as described herein, wherein the one or more processors configured to modify a refinement function comprises the one or more processors configured to modify a cost function associated with a decoder-side motion vector refinement process.

[0158] In general, at least one example of an embodiment may relate to a method as described herein, wherein modifying the cost function includes using a mean sum of absolute differences (MRSAD) cost function during a decoder-side motion vector refinement process.

[0159] Generally, at least one example of an embodiment may involve an apparatus as described herein, wherein one or more processors are configured to modify a cost function including the one or more processors being configured to use a mean removed sum of absolute differences (MRSAD) cost function during a decoder-side motion vector refinement process.

[0160] Generally, at least one example of an embodiment may relate to a method as described herein, wherein the refinement function includes a cost function; and modifying the refinement function includes selecting the cost function from a group including a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function.

[0161] Generally, at least one example of an embodiment may relate to an apparatus as described herein, wherein a refinement function comprises a cost function; and one or more processors configured to modify the refinement function comprises the one or more processors configured to select the cost function from a group comprising a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function.

[0162] Generally, at least one example of an embodiment may relate to a method or apparatus as described herein, wherein the indicator comprises one or more of information or indices or flags indicating that at least one of the following is activated during activation of a decoder-side motion vector refinement process: a local illumination compensation (LIC) process; or a weighted prediction (WP) process; or a coding unit (CU) level weighted bidirectional prediction (BCW) process.

[0163] Generally, at least one example of an embodiment may relate to a method for encoding a picture, comprising: determining activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and encoding the picture based on the modified motion vector refinement process and the second process.

[0164] Generally, at least one example of an embodiment may relate to a method for decoding a picture, comprising: determining activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and decoding the picture based on the modified motion vector refinement process and the second process.

[0165] Generally, at least one example of an embodiment may relate to an apparatus for encoding a picture, comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modify the motion vector refinement process based on the activation and the second process; and encode the picture based on the modified motion vector refinement process and the second process.

[0166] Generally, at least one example of an embodiment may involve an apparatus for decoding a picture, comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a motion vector refinement process and a second process different from the motion vector refinement process; modify the motion vector refinement process based on the activation and the second process; and decode the picture based on the modified motion vector refinement process and the second process.

[0167] In general, at least one example of an embodiment may be directed to a method or apparatus comprising a motion vector refinement process as described herein, wherein the motion vector refinement process comprises a decoding-side motion vector refinement process (DMVR).

[0168] Generally, at least one example of an embodiment may relate to a method or apparatus comprising a second process as described herein, wherein the second process comprises at least one of: a local illumination compensation (LIC) process or a weighted prediction process (WP) or a coding unit (CU) level weighted bidirectional prediction (BCW).

[0169] In general, at least one example of an embodiment may be directed to a method that includes modifying a motion vector refinement process as described herein, wherein modifying the motion vector refinement process includes modifying a cost function associated with the motion vector refinement process.

[0170] Generally, at least one example of an embodiment may involve an apparatus comprising one or more processors configured to modify a motion vector refinement process as described herein, wherein the one or more processors configured to modify the motion vector refinement process comprises the one or more processors configured to modify a cost function associated with the motion vector refinement process.

[0171] In general, at least one example of an embodiment may be directed to a method that includes modifying a cost function associated with a motion vector refinement process as described herein, wherein modifying the cost function includes using a mean sum of absolute differences (MRSAD) cost function during the motion vector refinement process.

[0172] Generally, at least one example of an embodiment may involve an apparatus comprising one or more processors configured to modify a cost function associated with a motion vector refinement process as described herein, wherein the one or more processors configured to modify the cost function includes using a mean removed sum of absolute differences (MRSAD) cost function during the motion vector refinement process.

[0173] Generally, at least one example of an embodiment may relate to a method for encoding a picture, comprising: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation and the LIC process; and encoding the picture based on the modified DMVR process and the LIC process.

[0174] Generally, at least one example of an embodiment may relate to a method for decoding a picture, comprising: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation and the LIC process; and decoding the picture based on the modified DMVR process and the LIC process.

[0175] Generally, at least one example of an embodiment may relate to an apparatus for encoding a picture, comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modify the DMVR process based on the activation and the LIC process; and encode the image based on the modified DMVR process and the LIC process.

[0176] Generally, at least one example of an embodiment may relate to an apparatus for decoding a picture, comprising one or more processors, wherein the one or more processors are configured to: determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modify the DMVR process based on the activation and the LIC process; and decode the image based on the modified DMVR process and the LIC process.

[0177] In general, at least one example of an embodiment may be directed to a method that includes modifying a DMVR process as described herein, wherein modifying the DMVR process includes modifying a cost function associated with the DMVR process.

[0178] Generally, at least one example of an embodiment may involve an apparatus comprising one or more processors configured to modify a DMVR process as described herein, wherein the one or more processors configured to modify the DMVR process comprises the one or more processors configured to modify a cost function associated with a motion vector refinement process.

[0179] In general, at least one example of an embodiment may be directed to a method that includes modifying a cost function associated with a DMVR process as described herein, wherein modifying the cost function includes using a mean removed sum of absolute differences (MRSAD) cost function during the DMVR process.

[0180] Generally, at least one example of an embodiment may relate to an apparatus comprising one or more processors configured to modify a cost function associated with a DMVR process as described herein, wherein the one or more processors configured to modify the cost function associated with the DMVR process comprises the one or more processors being configured to use a mean removed sum of absolute differences (MRSAD) cost function during the DMVR process.

[0181] Generally, at least one example of the embodiments may relate to a computer program product comprising instructions which, when executed by a computer, cause the computer to perform any one or more of the methods described herein.

[0182] Generally, at least one example of the embodiments may involve a non-transitory computer-readable medium storing executable program instructions to cause a computer executing the instructions to perform any one or more of the methods described herein.

[0183] In general, at least one example of an embodiment may involve a signal comprising data generated according to any one or more of the methods described herein.

[0184] In general, at least one example of an embodiment may involve a bitstream formatted to contain syntax elements and coded picture information according to any one or more of the methods described herein.

[0185] Generally, at least one example of an embodiment may relate to a device comprising an apparatus according to any embodiment of the apparatus as described herein, and at least one of: (i) an antenna configured to receive a signal comprising data representing image information; (ii) a band limiter configured to limit the received signal to a frequency band comprising data representing image information; and (iii) a display configured to display an image from the image information.

[0186] In general, at least one example of an embodiment may be directed to a device as described herein, wherein the device includes one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cellular phone, a tablet computer, or other electronic devices.

[0187] In the present disclosure, various implementations involve decoding. As used in the present application, "decoding" may include, for example, all or part of a process performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of the various implementations described in the present application, such as extracting a picture from a tiled (packed) picture, determining an upsampling filter to use and then upsampling the picture, and flipping the picture back to its intended orientation.

[0188] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or generally to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0189] In addition, various implementations involve encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream or signal. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also or alternatively include processes performed by encoders of the various implementations described in this application.

[0190] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or generally to a broader encoding process will become clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0191] Note that the syntax elements used here are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0192] When the figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when the figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0193] Various embodiments relate to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, usually given a constraint on computational complexity. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options, including all considered modes or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to save coding complexity, in particular, to calculate approximate distortion based on prediction or prediction residual signals rather than reconstructed signals. A mixture of these two methods can also be used, for example, by using approximate distortion only for some possible coding options and full distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both decoding costs and associated distortions.

[0194] The implementations and aspects described herein may be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed may also be implemented in other forms (e.g., a device or program). For example, a device may be implemented with appropriate hardware, software, and firmware. The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0195] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations in various places throughout this document are not necessarily all referring to the same embodiment.

[0196] Additionally, this document may refer to “obtaining” various information. Obtaining information may include, for example, one or more of determining information, estimating information, calculating information, predicting information, or retrieving information from a memory.

[0197] Additionally, this document may refer to “accessing” various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0198] Additionally, this document may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is often referred to in one way or another.

[0199] It should be understood that, for example, in the case of "A / B", "A and / or B", and "at least one of A and B", the use of any of the following " / ", "and / or", and "at least one of..." is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to include the selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to many of the listed items, as will be apparent to one of ordinary skill in this and related arts.

[0200] In addition, as used herein, the word "signal" especially refers to a corresponding decoder indicating something. For example, in some embodiments, the encoder signals a specific one of multiple parameters for refinement. In this way, in an embodiment, the same parameters are used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to signal information to the corresponding decoder. Although the foregoing relates to the verb form of the word "signal", the word "signal" can also be used as a noun in this article.

[0201] As will be appreciated by those skilled in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream or signal of the described embodiment. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0202] Various embodiments have been described. Embodiments may include any of the following features or entities, alone or in any combination, across various claim categories and types:

[0203] • providing in an encoder and / or decoder encoding / decoding picture information based on determining activation of a decoder-side motion vector refinement process comprising a refinement function, wherein said refinement function is modified based on the indicator.

[0204] providing, in an encoder and / or decoder, encoding / decoding picture information based on determining activation of a decoder-side motion vector refinement process comprising a refinement function, wherein the refinement function is modified based on an indicator, and comprising one or more of the following:

[0205] o the refinement function comprises a cost function, or

[0206] o modifying the refinement function comprises modifying a cost function associated with a decoder-side motion vector refinement process, or

[0207] o Modifying the cost function includes using a mean sum of absolute differences (MRSAD) cost function during a decoder-side motion vector refinement process, or

[0208] o modifying the refinement function comprises selecting the cost function from the group consisting of a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function, or

[0209] o The indicator includes one or more of information or indexes or flags indicating that at least one of the following is activated during activation of the decoder-side motion vector refinement process: a local illumination compensation (LIC) process; or a weighted prediction process (WP); or a coding unit (CU) level weighted bidirectional prediction (BCW) process.

[0210] • Provision is made in the encoder and / or decoder for applying motion vector refinement when illumination compensation is activated.

[0211] • Provision is made in the encoder and / or decoder for applying decoder side motion vector refinement (DMVR) when local illumination compensation (LIC) is activated.

[0212] • Provision is made in the encoder and / or decoder for applying motion vector refinement based on a modified cost function when illumination compensation is activated.

[0213] - Provision is made in the encoder and / or decoder for applying decoder side motion vector refinement (DMVR) when Weighted Prediction (WP) is activated.

[0214] • Provision is made in the encoder and / or decoder for applying motion vector refinement based on a modified cost function when WP is activated.

[0215] • Provision is made in the encoder and / or decoder for applying decoder-side motion vector refinement (DMVR) when CU-level weighted bi-directional prediction (BCW) is activated.

[0216] • Provision is made in the encoder and / or decoder for applying motion vector refinement based on a modified cost function when BCW is activated.

[0217] • Provision is made in the encoder and / or decoder for applying a decoder side motion vector refinement (DMVR) based on a modified cost function when local illumination compensation (LIC) or WP or BCW is activated.

[0218] • Provision is made in the encoder and / or decoder for applying motion vector refinement based on a modified Sum of Absolute Difference (SAD) cost function when illumination compensation (eg LIC or WP or BCW) is activated.

[0219] Provision is made in the encoder and / or decoder for applying motion vector refinement based on a modified sum of absolute differences (SAD) cost function when illumination compensation is activated, wherein the motion vector refinement comprises decoder side motion vector refinement (DMVR) and the modified sum of absolute differences cost function comprises mean removed SAD (MRSAD) and the illumination compensation comprises local illumination compensation (LIC) or WP or BCW.

[0220] • Provision is made in the encoder and / or decoder for applying decoder side motion vector refinement (DMVR) when local illumination compensation (LIC) is activated, wherein the LIC process is calculated once the DMVR prediction is available.

[0221] - Provision is made in the encoder and / or decoder for applying decoder side motion vector refinement (DMVR) based on whether local illumination compensation (LIC) or WP or BCW is activated, wherein DMVR being applied comprises applying a modified cost function when LIC or WP or BCW is activated.

[0222] providing in the encoder and / or decoder for applying decoder side motion vector refinement (DMVR) based on whether local illumination compensation (LIC) or WP or BCW is activated, wherein DMVR being applied comprises applying a modified cost function when LIC or WP or BCW is activated, and the modified cost function comprises a mean removed absolute difference and cost function.

[0223] provision in the encoder and / or decoder for applying decoder side motion vector refinement (DMVR) based on the value of a flag / index coded in the bitstream at CU, slice, SPS, PPS level, wherein the DMVR being applied comprises applying a modified cost function depending on the value of the coded information, and the modified cost function comprises a mean removed absolute difference and cost function.

[0224] • Provision is made in the encoder and / or decoder for applying motion vector refinement when illumination compensation is activated based on providing improved compression efficiency.

[0225] • Inserting syntax elements in the signaling that enable the encoder and / or decoder to provide for applying motion vector refinement when illumination compensation is activated as described herein.

[0226] • Select a combination of motion vector refinement and illumination compensation to apply at the decoder based on these syntax elements.

[0227] A bitstream or signal comprising one or more of the described syntax elements or their variants.

[0228] • Insertion of syntax elements in the signaling that enable the decoder to provide motion vector refinement and illumination compensation in a manner corresponding to that used by the encoder.

[0229] • Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof.

[0230] • A TV, set-top box, cell phone, tablet or other electronic device providing application motion vector refinement and illumination compensation according to any of the described embodiments.

[0231] • A TV, set-top box, cellular phone, tablet or other electronic device that performs motion vector refinement and illumination compensation according to any of the described embodiments, and displays (e.g., using a monitor, screen or other type of display) the resulting image.

[0232] • A TV, set-top box, cellular phone, tablet or other electronic device that tunes (eg, using a tuner) a channel to receive a signal including an encoded image, and performs motion vector refinement and illumination compensation according to any of the described embodiments.

[0233] • A TV, set-top box, cellular phone, tablet or other electronic device that receives over the air (eg, using an antenna) a signal comprising an encoded image and performs motion vector refinement and illumination compensation according to any of the described embodiments.

[0234] • A computer program product storing program code which, when executed by a computer, implements motion vector refinement and illumination compensation according to any of the described embodiments.

[0235] • A non-transitory computer-readable medium containing executable program instructions to cause a computer executing the instructions to implement motion vector refinement and illumination compensation according to any of the described embodiments.

[0236] Various other generalized and specialized embodiments are also supported and contemplated in the present disclosure.

Claims

1. A method for decoding a picture, the method comprising: determining whether decoder-side motion vector refinement (DMVR) and local illumination compensation (LIC) are activated for a current block of the picture; as well as Based on the determination that DMVR and LIC are activated for the current block: determining one or more LIC parameters associated with decoding the current block; Refining one or more motion vectors associated with the current block; as well as decoding the current block of the picture based on the one or more LIC parameters and the one or more refined motion vectors; Wherein said one or more LIC parameters are determined before or during said refinement of said one or more motion vectors.

2. The method of claim 1, wherein the one or more LIC parameters include weights and offsets to be applied to reference blocks of the current block.

3. The method of claim 1 , wherein the determining of the one or more LIC parameters is performed in parallel with the refining of the one or more motion vectors. 4 . The method of claim 1 , wherein the one or more motion vectors are refined based on at least a mean removed sum of absolute differences (MRSAD) cost function.

5. The method of claim 1 , wherein refining the one or more motion vectors comprises: A constant offset is calculated for a first search point associated with the decoding of the current block, and the constant offset is reused for a second search point associated with the decoding of the current block.

6. A video decoding device, comprising: The processor is configured as: determining whether decoder-side motion vector refinement (DMVR) and local illumination compensation (LIC) are activated for a current block of a picture; as well as Based on the determination that DMVR and LIC are activated for the current block: determining one or more LIC parameters associated with decoding the current block; Refining one or more motion vectors associated with the current block; as well as decoding the current block of the picture based on the one or more LIC parameters and the one or more refined motion vectors; Wherein said one or more LIC parameters are determined before or during said refinement of said one or more motion vectors. 7 . The video decoding apparatus of claim 6 , wherein the one or more LIC parameters include weights and offsets to be applied to reference blocks of the current block.

8. The video decoding device of claim 6, wherein the processor is configured to determine the one or more LIC parameters in parallel with refining the one or more motion vectors.

9. The video decoding apparatus of claim 6, wherein the one or more motion vectors are refined based on at least a mean removed sum of absolute differences (MRSAD) cost function.

10. The video decoding device of claim 6, wherein the processor is configured to refine the one or more motion vectors comprises: The processor is configured to calculate a constant offset for a first search point associated with the decoding of the current block and to reuse the constant offset for a second search point associated with the decoding of the current block.

11. A video encoding device, comprising: The processor is configured as: determining whether decoder-side motion vector refinement (DMVR) and local illumination compensation (LIC) are activated for a current block of a picture; as well as Based on the determination that DMVR and LIC are activated for the current block: determining one or more LIC parameters associated with decoding the current block; Refining one or more motion vectors associated with the current block; and encoding the current block of the picture based on the one or more LIC parameters and the one or more refined motion vectors; Wherein said one or more LIC parameters are determined before or during said refinement of said one or more motion vectors. 12 . The video encoding apparatus of claim 11 , wherein the one or more LIC parameters include weights and offsets to be applied to reference blocks of the current block.

13. The video encoding device of claim 11, wherein the processor is configured to refine the one or more motion vectors comprises: The processor is configured to calculate a constant offset for a first search point associated with the encoding of the current block and to reuse the constant offset for a second search point associated with the encoding of the current block.

14. A method for encoding a picture, the method comprising: determining whether decoder-side motion vector refinement (DMVR) and local illumination compensation (LIC) are activated for a current block of the picture; as well as Based on the determination that DMVR and LIC are activated for the current block: determining one or more LIC parameters associated with decoding the current block; Refining one or more motion vectors associated with the current block; and encoding the current block of the picture based on the one or more LIC parameters and the one or more refined motion vectors; Wherein said one or more LIC parameters are determined before or during said refinement of said one or more motion vectors.

15. A video decoding device, comprising: The processor is configured as: calculating an average reference sample value associated with a first sample of a current video block, wherein the average reference sample value is calculated based on a weighted average of two reference blocks associated with the current video block; reconstructing the first sample of the current video block based on the average reference sample; as well as The average reference sample is reused for a second sample of the current video block.

Images