Motion vector derivation in video encoding and decoding

Abstract

To provide an encoding / decoding method and apparatus for performing motion vector refinement.SOLUTION: Video processing involves encoding and / or decoding a picture based on determining an activation of a processing mode that involves a motion vector refinement process and a second process other than the motion vector refinement process, modifying the motion vector refinement process based on the activation and the second process, and encoding and / or decoding the picture based on the modified motion vector refinement process and the second process.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates to video encoding and decoding. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes typically use prediction and transformation to exploit spatial and temporal redundancy in the video content. Generally, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and then the difference between the original and predicted picture block, often denoted as prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transformation, quantization, and entropy coding. Summary of the Invention

[0003] At least one example of an embodiment involving a method for encoding picture information is provided, the method including: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on an indicator; and encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0004] At least one example of an embodiment is provided that involves a method for decoding picture information, the method including: determining activation of a decoder-side motion vector refinement process that includes a refinement function; modifying the refinement function based on an indicator; and decoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0005] At least one example of an embodiment is provided involving an apparatus for encoding picture information, the apparatus including one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0006] At least one example of an embodiment is provided involving an apparatus for decoding picture information, the apparatus including one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0007] At least one example of an embodiment involving a method for encoding picture information is provided, the method including determining activation of a processing mode involving a motion vector refinement process and a motion vector refinement function, modifying the motion vector refinement process based on the motion vector refinement function, and encoding at least a portion of the picture information based on the processing mode and the modified motion vector refinement process.

[0008] At least one example of an embodiment involving a method for decoding picture information is provided, the method including determining activation of a processing mode involving a motion vector refinement process and a motion vector refinement function, modifying the motion vector refinement process based on the motion vector refinement function, and decoding at least a portion of the picture information based on the processing mode and the modified motion vector refinement process.

[0009] At least one example of an embodiment is provided involving an apparatus for encoding picture information, the apparatus including one or more processors configured to determine activation of a processing mode involving a motion vector refinement process and a motion vector refinement function, modify the motion vector refinement process based on the motion vector refinement function, and encode at least a portion of the picture information based on the processing mode and the modified motion vector refinement process.

[0010] At least one example of an embodiment is provided involving an apparatus for decoding picture information, the apparatus including one or more processors configured to determine activation of a processing mode involving a motion vector refinement process and a motion vector refinement function, modify the motion vector refinement process based on the motion vector refinement function, and decode a picture based on the modified motion vector refinement process.

[0011] At least one example of an embodiment is provided involving a method for encoding a picture, the method including: determining activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and encoding the picture based on the modified motion vector refinement process and the second process.

[0012] At least one other example of an embodiment involving a method for decoding a picture is provided, the method including: determining activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process; modifying the motion vector refinement process based on the activation and the second process; and decoding the picture based on the modified motion vector refinement process and the second process.

[0013] At least one other example of an embodiment involving an apparatus for encoding a picture is provided, the apparatus including one or more processors configured to determine activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process, modify the motion vector refinement process based on the activation and the second process, and encode the picture based on the modified motion vector refinement process and the second process.

[0014] At least one other example of an embodiment involving an apparatus for decoding a picture is provided, the apparatus including one or more processors configured to determine activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process, modify the motion vector refinement process based on the activation and the second process, and decode the picture based on the modified motion vector refinement process and the second process.

[0015] At least one other example of an embodiment involving a method for encoding a picture is provided, the method including determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modifying the DMVR process based on the activation and the LIC process, and encoding the picture based on the modified DMVR process and the LIC process.

[0016] At least one other example of an embodiment involving a method for decoding a picture is provided, the method including determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modifying the DMVR process based on the activation and the LIC process, and decoding the picture based on the modified DMVR process and the LIC process.

[0017] At least one other example of an embodiment involving an apparatus for encoding a picture is provided, the apparatus including one or more processors configured to determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modify the DMVR process based on the activation and the LIC process, and encode the picture based on the modified DMVR process and the LIC process.

[0018] At least one other example of an embodiment involving an apparatus for decoding a picture is provided, the apparatus including one or more processors configured to determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modify the DMVR process based on the activation and the LIC process, and decode the picture based on the modified DMVR process and the LIC process.

[0019] The foregoing presents a simplified summary of the subject matter in order to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the subject matter. It is not intended to identify key / critical elements of embodiments or to delineate the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description provided below. [Brief explanation of the drawings]

[0020] The present disclosure may be better understood by considering the following detailed description in conjunction with the accompanying figures. [Figure 1] 1 provides a block diagram illustrating an example embodiment of a video encoder. [Figure 2] 1 provides a block diagram illustrating an example embodiment of a video decoder. [Figure 3] 1 illustrates an aspect of the present disclosure involving a coding tree unit (CTU). [Figure 4] 1 illustrates an embodiment of the present disclosure with a CTU and a coding unit (CU). [Figure 5] An embodiment of decoding-side motion vector refinement (DMVR) will be described. [Figure 6] 1 provides a flow diagram illustrating an embodiment of a DMVR search procedure. [Figure 7] 10 shows an example of a DMVR integer luminance sample search pattern. [Figure 8] An example of illumination compensation parameters, e.g., local illumination compensation (LIC) parameters, derived from neighboring reconstructed samples and corresponding reference samples transformed with motion vectors (MVs) for both a square CU (top diagram) and a rectangular CU (bottom diagram) is shown. [Figure 9] 10 provides a flow diagram illustrating an example of the derivation of LIC parameters and their application to predicted L0 and predicted L1, respectively. [Figure 10] 10 provides a flow diagram showing an example of deriving LIC parameters and applying them to prediction from combined L0 and L1. [Figure 11] 10 shows an example of the positioning of the left and top reference samples of a rectangular CU. [Figure 12] 1 provides a flow diagram illustrating an example of bi-prediction with DMVR and LIC. [Figure 13] 1 provides a flow diagram illustrating an example of derivation of LIC parameters based on decoded bi-predictive motion vectors before DMVR refinement. [Figure 14] 1 provides a block diagram illustrating an example of an embodiment of an apparatus according to various aspects described herein. [Figure 15] 1 provides a flow diagram illustrating an example embodiment involving bi-prediction with motion vector refinement, such as DMVR and weighted prediction (WP). [Figure 16] 1 provides flow diagrams illustrating various examples of embodiments according to the present disclosure; [Figure 17] 1 provides flow diagrams illustrating various examples of embodiments according to the present disclosure; [Figure 18] 1 provides flow diagrams illustrating various examples of embodiments according to the present disclosure; [Figure 19] 1 provides flow diagrams illustrating various examples of embodiments according to the present disclosure;

[0021] It should be understood that the drawings are intended to illustrate examples of various aspects, features, and embodiments according to the present disclosure, and are not necessarily the only possible configuration. Like reference designators throughout the various views refer to the same or similar features. DETAILED DESCRIPTION OF THE INVENTION

[0022] Turning now to the figures, Figure 1 shows an embodiment of a video encoder 100, such as an HEVC encoder. HEVC is a compression standard developed by the Joint Collaborative Team on Video Coding (JCT-VC) (see, e.g., "ITU-T H.265 TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (10 / 2014), SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS, Infrastructure of audiovisual services—Coding of moving video, High efficiency video coding, Recommendation ITU-T H.265"). Figure 1 may also show an encoder that has improvements made to the HEVC standard or that employs technology similar to HEVC, such as an encoder based on or improved upon the Joint Exploration Model (JEM) being developed by the Joint Video Experts Team (JVET), e.g., an encoder associated with a development effort called Versatile Video Coding (VVC).

[0023] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "picture" and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side while "decoded" is used on the decoder side.

[0024] The HEVC specification distinguishes between "blocks" and "units," where a "block" addresses a specific region of the sample array (e.g., luma, Y) and a "unit" contains collocated blocks of all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and prediction data (e.g., motion vectors) associated with the block.

[0025] For coding, a picture is divided into square coding tree blocks (CTBs) with configurable sizes, and a series of coding tree blocks are grouped into slices. A coding tree unit (CTU) contains the CTB of a coded color component. The CTB is the root of a quadtree division into coding blocks (CBs), which can be divided into one or more prediction blocks (PBs), forming the root of a quadtree division into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a prediction unit (PU) and a tree-structured set of transform units (TUs), where a PU contains prediction information for all color components and a TU contains a residual coding syntax structure for each color component. The sizes of the CB, PB, and TB for the luma component apply to the corresponding CU, PU, ​​and TU. In this application, the term "block" may be used to refer to any of the CTU, CU, PU, ​​TU, CB, PB, and TB. Additionally, "block" can also be used to refer to macroblocks and partitions as specified in H.264 / AVC or other video coding standards, or more generally to refer to arrays of data of various sizes.

[0026] In the encoder 100 of FIG. 1, a picture is encoded by the encoder elements as described below. The picture to be encoded is processed in units of CUs. Each CU is encoded using either intra mode or inter mode. When a CU is encoded in intra mode, intra prediction is performed (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder determines whether intra mode or inter mode is used to encode the CU (105) and indicates the intra / inter decision with a prediction mode flag. A prediction residual is calculated by subtracting the predicted block from the original image block (110).

[0027] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may also skip the transform and apply quantization directly to the untransformed residual signal on a 4x4 TU basis. The encoder may also bypass both the transform and quantization, i.e., the residual is coded directly without applying a transform or quantization process. In direct PCM coding, no prediction is applied, and coded unit samples are coded directly into the bitstream.

[0028] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined with the predicted block (155) to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0029] Figure 2 shows a block diagram of an embodiment of a video decoder 200, such as an HEVC decoder. In the embodiment of the decoder 200, the bitstream is decoded by the decoder elements described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass as described in Figure 1, which performs video decoding as part of encoding the video data. Figure 2 may also show a decoder in which improvements have been made to the HEVC standard or which employs HEVC-like technology, such as a decoder based on JEM or improved JEM.

[0030] In particular, the decoder's input includes a video signal or bitstream that may be generated by a video encoder, such as the video encoder 100 of FIG. 1. First, the signal or bitstream is entropy decoded (230) to obtain transform coefficients, motion vectors, and other coding information. The transform coefficients are inversely quantized (240) and inversely transformed (250) to decode the prediction residual. The decoded prediction residual is combined with a predicted block (255) to reconstruct an image block. The predicted block may be obtained from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275) (270). Advanced motion vector prediction (AMVP) and merge mode techniques may be used to derive motion vectors for motion compensation, which may use interpolation filters to calculate interpolated values ​​of sub-integer samples of the reference block. An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0031] The HEVC video compression standard uses motion-compensated temporal prediction to exploit the redundancy that exists between successive pictures of a video. To do so, a motion vector is associated with each prediction unit (PU). Each coding tree unit (CTU) is represented by a coding tree (CT) in the compressed domain. This is a quadtree decomposition of the CTU, with each leaf called a coding unit (CU), as shown in Figure 3.

[0032] Each CU is then given some intra- or inter-prediction parameters (prediction information). To do this, it is spatially divided into one or more prediction units (PUs), and each PU is assigned some prediction information. The intra- or inter-coding mode is assigned at the CU level, as shown in FIG. 4, which illustrates an example of dividing a coding tree unit into coding units, prediction units, and transform units. To code a CU, a prediction block or prediction unit (PU) is constructed from neighboring reconstructed samples (intra-prediction) or from a previously reconstructed picture stored in a decoded picture buffer (DPB) (inter-prediction). Next, a residual sample calculated as the difference between the original sample and the PU sample is transformed and quantized.

[0033] Inter prediction uses motion-compensated temporal prediction to exploit the redundancy that exists between consecutive pictures in a video. To do so, a motion vector is associated with a PU and a reference index 0 (refIdx0) that indicates which of multiple reference pictures listed in the list specified by LIST_0 to use.

[0034] HEVC uses two modes to encode motion data. These are called adaptive motion vector prediction (AMVP) and merge, respectively. AMVP involves signaling the reference picture(s) used to predict the current PU, the motion vector predictor index (obtained from a list of two predictors), and the motion vector differential. Merge mode involves signaling and decoding the index of several motion data collected in a list of motion data predictors. The list consists of five candidates and is created in the same way on the decoder and encoder sides. Thus, merge mode aims to derive motion information retrieved from the merge list. The merge list typically contains motion information related to several spatial and temporal surrounding blocks available in their decoded state when the current PU is being processed.

[0035] In other codecs, such as the codec developed by the JVET (Joint Exploration Team) group for the Versatile Video Coding (VVC) reference software called the Joint Exploration Model (JEM) and known as the VVC Test Model (VTM), some modes used in inter prediction (e.g., bidirectional inter prediction or B-mode) can compensate for illumination changes using transmitted parameters. In B-mode, the current block is associated with two motion vectors, which specify two reference blocks in two different images. The predictor block that allows a residual block for the current block to be calculated is the average of the two reference blocks. Several generalizations of bidirectional inter prediction have been proposed, in which the weights associated with each reference block are different; weighted prediction (WP) can be considered a generalization of bidirectional inter prediction in some aspects. In WP, the residual block is calculated as the difference between the current block and either a weighted version of the reference block in the case of unidirectional inter prediction or a weighted average of the two reference blocks in the case of bidirectional inter prediction. WP can be enabled at the sequence level in the sequence header (called a sequence parameter set (SPS) in VVC) or at the picture level in the picture header (called a picture parameter set (PPS) in VVC). WP defines weights and offsets for each group of CTUs (e.g., generally at the slice header level) associated with each component of each reference picture in each list of reference pictures (L0 and L1).

[0036] While WP is enabled in the sequence header (SPS) and picture header (PPS), and the associated weights and offsets are specified in the slice header, a new mode called CU-level weighted bi-prediction (BCW) allows signaling weights at the block level.

[0037] Some additional temporal prediction tools, with associated parameters determined on the decoder side, provide functions such as motion vector refinement and compensation for issues such as illumination variations. Such additional tools can include, for example, motion vector refinement, such as decoder-side motion vector refinement (DMVR), and illumination compensation, such as local illumination compensation (LIC). One purpose of DMVR may be to further refine motion vectors by using a predictive approach such as bilateral matching prediction. One purpose of LIC may be to compensate for illumination changes that may occur between a used prediction block and its reference block through motion-compensated temporal prediction. Both of these tools may involve, at least in part, decoder-side processes to derive the parameters used for prediction.

[0038] More specifically, an approach to improving the accuracy of merge mode MVs may involve applying bilateral matching (BM)-based decoder-side motion vector refinement. In bi-predictive operation, a refined MV is searched around an initial MV in reference picture lists L0 and L1. The BM method calculates the distortion between two candidate luminance blocks in reference picture lists L0 and L1 based on an approach such as sum of absolute differences (SAD). As shown in the example of Figure 5, the SAD between block 500 and block 501 (a red block) is calculated based on each MV candidate around the initial MV. The MV candidate with the lowest SAD becomes the refined MV and is used to generate a bi-predictive signal.

[0039] Approaches for applying DMVR include that DMVR may be applied to CUs coded in the following modes: ·CU level merge mode with bi-predictive MV. · A mode in which one is a past reference picture and one is a future reference picture for the current picture. A mode in which the distance from both reference pictures to the current picture (e.g., the difference in Picture Order Count (POC)) is the same. ·Modes where CU has 64 or more luma samples and CU height is 8 or more luma samples.

[0040] The refined MVs derived by the DMVR process may be used to generate inter-predicted samples and may also be used for temporal motion vector prediction for future picture coding, while the original MVs may be used, for example, for the deblocking process and may also be used for spatial motion vector prediction for future CU coding. Additional features of one or more approaches to DMVR are described below.

[0041] As shown in the example illustrated in Figure 5, the search points surround the initial MV, and the MV offset follows the MV difference mirroring rule, i.e., every point checked by the DMVR, indicated by the candidate MV pair (MV0, MV1) in Figure 5, follows the following two equations: MV0'=MV0+MV offset (1) MV1' = MV1 - MV offset (2) Here, MV offset represents the refinement offset between the initial MV and the refined MV in one of the reference pictures. The refinement search range may be, for example, two integer luma samples from the initial MV.

[0042] Figure 6 is a flow diagram illustrating an example of a search process for a DMVR such as that shown in Figure 5. As shown in the example of Figure 6, the search includes an integer sample offset search stage and a fractional sample refinement stage.

[0043] To reduce the search complexity, a fast search method with an early termination mechanism can be applied to the integer sample offset search stage. Instead of a full search, e.g., 25 points, a two-iteration search scheme can be applied to reduce the SAD checkpoint. As shown in Figure 7, up to six SADs are checked in the first iteration. First, the SADs of five points (center and P1-P4) are compared. If the SAD of the center position is the smallest, the integer sample stage of the DMVR terminates. Otherwise, another position P5 (determined by the SAD distribution of P1-P4) is checked. Then, the position (among P1-P5) with the smallest SAD is selected as the center position for the second iteration. The search process for the second iteration is the same as that for the first iteration. The SAD calculated in the first iteration can be reused in the second iteration; therefore, only the SADs of three additional points need to be further calculated.

[0044] Fractional sample refinement can be performed after the integer sample search. To reduce computational complexity, fractional sample refinement can be derived by using a parametric error surface equation instead of an additional search with SAD comparison. Fractional sample refinement can be conditionally invoked based on the output of the integer sample search stage. If the integer sample search stage finishes with the center having the smallest SAD in either the first or second iteration, fractional sample refinement is further applied.

[0045] Parametric error surface based sub-pixel offset estimation uses the central location cost and the costs at the central four adjacent locations to fit a 2D parabolic error surface equation of the form: E(x,y)=A(xx min ) 2 +B(yy min ) 2 +C (3) where (x_min,y_min) corresponds to the fractional position with the minimum cost, and C corresponds to the minimum cost value. By solving the above equation using the cost values ​​of the five search points, (x_min,y_min) is calculated as follows:

number

[0046] The codec may provide, for example, that the resolution of the motion vector (MV) can be 1 / 16 luma samples. Samples at fractional positions are interpolated using an 8-tap interpolation filter. In DMVR, the search points surround the first fractional pel motion vector (MV) at integer sample offsets, so samples at these fractional positions need to be interpolated for the DMVR search process. To reduce computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect is that by using a bilinear filter, with a 2-sample search range, DVMR does not access more reference samples than a regular motion compensation process. After the refined motion vector (MV) is obtained by the DMVR search process, a regular 8-tap interpolation filter is applied to generate the final prediction. To avoid accessing more reference samples than the regular MC process, samples that are not required for the interpolation process based on the original motion vector but are required for the interpolation process based on the refined motion vector are padded from these available samples.

[0047] The maximum unit size of the DMVR search process may be limited to, for example, 16 x 16. If the width and / or height of a CU is larger than the maximum size, for example, larger than 16 luma samples, the CU may be further divided into sub-blocks having widths and / or heights equal to the maximum size, for example, 16 luma samples.

[0048] Some modes used in inter prediction (or B mode) can compensate for illumination changes using transmitted parameters. In B mode, the current block is associated with two motion vectors, specifying two reference blocks in two different images. The predictor block that allows to calculate the residual block for the current block is the average of the two reference blocks. The predictor biPred of the current block is calculated as follows:

number

[0049] Several generalizations of bidirectional inter prediction have been proposed, in which the weights w0 and w1 differ, and weighted prediction (WP) may be considered a generalization of bidirectional inter prediction in some aspects. In WP, a residual block is calculated as the difference between a current block and either a weighted version of a reference block in the case of unidirectional inter prediction or a weighted average of two reference blocks in the case of bidirectional inter prediction. WP can be enabled at the sequence level in the sequence header (called a sequence parameter set (SPS) in VVC) or at the picture level in the picture header (called a picture parameter set (PPS) in VVC). WP assigns a weight w0 for each group of CTUs (e.g., generally at the slice header level) associated with each component of each reference picture i in each list (L0 and L1) of reference pictures stored in the DPB. i and offset Off i If the current block is coded in bidirectional WP, the predicted sample pred(x,y) at position (x,y) for the current block is calculated as follows: pred(x,y)=((w0.pred0(x,y)+w1.pred1(x,y)+Off 01 )≫(shift+1)) Off 01 =(Off0+Off1+1)≪shift. where pred i (x,y) is a list of reference images L stored in the DPB and spatially corresponding to pred(x,y). i is the motion compensated predictor sample obtained at w i is the weight, and Off i is the offset value.

[0050] In order to maintain the improved numerical precision when weighting samples, intermediate sample values ​​with improved bit depth precision can be stored and calculated. In this case, the final (desired) bit depth sample prediction precision (bitDepth) is obtained by a final right bit shift at the end of the prediction calculation process. For example, the reference picture of the DPB is stored with precision bitDepth, while the intermediate motion compensation samples are stored in the intermediate buffer with improved precision (bitDepth+sp). One purpose of the shift value shift in the two above equations is to ensure this intermediate improved bit depth precision. It should be noted that a similar intermediate bit depth precision enhancement process is generally used for all prediction tools that use a sample weighting process.

[0051] While WP is enabled in the sequence header (SPS) and picture header (PPS), and the associated weights and offsets are specified in the slice header, a new mode called bi-prediction with CU-level weights (BCW) allows signaling weights at the block level. When the BCW mode is applied to the current block, the predictor sample pred(x,y) of the current block is calculated as follows: pred(x,y)=((8-w).pred0(x,y)+w.pred1(x,y)+4)≫3 where pred_0(x,y) is a motion compensated predictor sample obtained from the list L0 of reference images stored in the DPB and spatially corresponding to pred(x,y), pred1(x,y) is a motion compensated predictor sample obtained from the list L1 of reference images stored in the DPB and spatially corresponding to pred(x,y), and w is a weight obtained from a set of five weights (w∈{-2, 3, 4, 5, 10}). The weight w is determined in one of two ways: 1) for non-merge CUs, the weight index bcw_idx is signaled after the motion vector differential; or 2) for merge CUs, the weight index bcw_idx is inferred from neighboring blocks based on merge candidate indexes.

[0052] The use of a tool such as illumination compensation, e.g., local illumination compensation (LIC), may be signaled at the CU level, such as through a flag (LIC flag) associated with each coding unit (CU) coded in inter mode. When this tool is activated, the decoder calculates several prediction parameters based on several reconstructed picture samples located to the left and / or top of the current block to be predicted and reference picture samples located to the left and / or top of the motion compensation block, as illustrated by the example shown in Figure 8. Hereinafter, a set of samples consisting of both samples located in the top row of the current block and samples located in the left column of the current block is referred to as an "L-shape" associated with the current block. For example, in Figure 8, one example of an L-shape includes small squares at the top and left of the current block.

[0053] LIC can be based on a linear model with parameters (a, b), corresponding to weights and offsets, for example. The parameters (a, b) can be determined, for example, based on minimizing the error, or local distortion, distance "dist", between the current sample and a linearly modified reference sample, which can be defined as follows (Equation 6):

number

[0054] In some implementations, additional restrictions may be imposed on the selection of reconstructed samples and reference samples used in the L-shape, such as that reconstructed samples belonging to neighboring blocks are not coded in intra-mode or coded with intra-mode constructed prediction (e.g., CIIP).

[0055] The values ​​of (a,b) can be obtained using least squares minimization as follows (Equation 7):

number

[0056] When the LIC parameters are obtained by the encoder or decoder of the current CU, the prediction pred(current_block) of the current CU includes cases such as, for example, the following unidirectional prediction: pred(current_block)=axref_block+b(8) where current_block is the current block to predict, pred(current_block) is the prediction of the current block, and ref_block is the reference block constructed by the normal motion compensation (MC) process and used for temporal prediction of the current block.

[0057] Note that the sets of adjacent reconstructed samples and the sets of reference samples (samples indicated by small square blocks in FIG. 8) have the same number and the same pattern. In the following, "left samples" refers to the set of adjacent reconstructed samples (or the set of reference samples) located immediately to the left of the current block, and "top samples" refers to the set of adjacent reconstructed samples (or the set of reference samples) located immediately above (or adjacent to) the top of the current block. A "sample set" refers to one combination of each of the set of "left samples" and the set of "top samples."

[0058] In the case of bi-prediction, the LIC process may be applied twice, as shown in the example illustrated in Figure 9. In Figure 9, the LIC process is applied to both the reference 0 prediction (LIST-0) and the reference 1 prediction (LIST_1), as indicated by 300-340 in Figure 9. Then, at 350 in Figure 9, the two predictions are combined, for example, using an initial weighting (P=(P0+P1+1)>>1) or a bi-prediction weighted average (BPWA): P=(g0.P0+g1.P1+(1<<(s-1)))>>s). This method will be referred to as "Method a."

[0059] In another embodiment example, referred to herein as "method b" and shown in FIG. 10, in the bi-predictive case, two predictions, i.e., reference 0 and reference 1 predictions, are determined at 300 and then combined at 360 before applying a single LIC process at 320, 330, 340, and 370 in FIG. 10.

[0060] In another embodiment example, referred to herein as "method c" (method c based on method b), in the case of bi-prediction, the LIC-0 and LIC-1 parameters are derived directly, for example, as follows:

number

[0061] Figure 17 shows another example of an embodiment that is a variation of the embodiment shown in Figure 16. In Figure 17, a determination is made to activate a DMVR at 1710, followed by a determination to activate a LIC at 1720. Otherwise, the features of the embodiment shown in Figure 17 are similar to those described above with respect to the embodiment of Figure 16 and will not be described again.

[0062] As mentioned above, aspects of the present disclosure may involve adapting at least one of the described tools to obtain improved or optimal performance when combining tools such as those described herein. For example, the sum of absolute differences (SAD) approach used to derive a DMVR predictor may not be suitable for applying tools such as LIC, WP, or BCW in combination with DMVR. For example, LIC, WP, or BCW may correct some initial values ​​that the DMVR process may already reduce or attempt to minimize using the SAD function. An encoder may apply a motion vector search algorithm using a cost determination approach or a function other than, different from, or modified with respect to a function such as SAD. For example, in an effort to optimize motion search for those for which LIC, WP, or BCW cannot correct, i.e., optimize the results of the motion + (LIC or WP or BCW) combination, a modified form of SAD function such as mean-removed SAD (MRSAD) may be applied when LIC, WP, or BCW is activated. However, using DMVR with SAD after a motion search using MRSAD may be non-optimal and even counterproductive in some cases. One approach to addressing such issues would be to prevent or prohibit the combination of these tools, e.g., DMVR is not allowed when LIC, WP, or BCW are on, but doing so is likely undesirable or suboptimal because the benefits of DMVR would not be provided.

[0063] Generally, one aspect of the present disclosure may involve improving motion vector refinement, such as DMVR, when illumination compensation, such as LIC, WP, or BCW, is activated. At least one embodiment may involve using mean removed sum of absolute values ​​(MRSAD) when LIC, WP, or BCW is activated in combination with DMVR, instead of using SAD in the DMVR derivation process. Once a DMVR prediction is available, the LIC process is calculated.

[0064] In accordance with aspects of the present disclosure, generally, at least one embodiment may involve using MRSAD in the DMVR derivation process when LIC, WP, or BCW is activated. For example, an embodiment of a bi-prediction process is shown in FIG. 12. In FIG. 12, the input to the process is a pair of motion vectors for the L0 reference list and the L1 reference list at 1210. If it is determined at 1220 that DMVR is not applied, a "normal" bi-prediction process is applied at 1250, and then, if LIC is enabled or activated, the LIC process is applied at 1260. If it is determined at 1220 that DMVR is applied, its application depends on the LIC condition at 1230. Instead of LIC, a WP value or BCW value may also be used as a condition. If LIC is not applied, a DMVR is derived at 1240 using SAD as described above, and then bi-prediction is performed using the refined motion vector at 1250. If LIC is applied as determined at 1230 , DMVR is applied at 1245 using MRSAD as the cost function, followed by motion compensation at 1250 .

[0065] As shown in Figure 7, the application of MRSAD may occur. That is, first, the MRSADs of five points (center and P1-P4) are compared. If the MRSAD of the center position is the smallest, the integer sample stage of the DMVR ends. Otherwise, another position, P5, is checked. The position with the smallest MRSAD is then selected as the center position for the second iteration search. The process for the second iteration search is the same as that for the first iteration search. The MRSAD calculated in the first iteration can be reused in the second iteration; therefore, only the MRSADs of three additional points need to be further calculated. As before, the integer sample search is followed by fractional sample refinement using the parametric error surface equation for the MRSAD value.

[0066] MRSAD between two blocks of the same size first removes any constant offset between the two blocks and then calculates the difference between these two blocks. This differs from Sum of Absolute Differences (SAD), in which constant changes between the two blocks can have a significant impact on the difference. The purpose of MRSAD is to be more robust to illumination changes. In this case, it is reasonable to use this method because the LIC or WP or BCW process compensates for illumination changes. The average difference D between two blocks B1 and B2 is first calculated as follows:

number

[0067] The MRSAD is then calculated by accumulating the absolute differences of the block pixels minus the average difference.

number

[0068] In general, another example of at least one embodiment may involve the block average difference D being calculated only for the first position in the first iteration of the DMVR process and reused for adjacent positions. Doing so may allow for avoiding the calculation of the average block difference for each tested position, which may allow for improved efficiency of operation and / or improved performance, reduced cost, etc., in some cases.

[0069] Another example of an embodiment is shown in Figure 15. The example of Figure 15 involves motion vector refinement, such as DMVR, using a tool such as WP as described above. In Figure 15, at 1510, the input to the process is a pair of motion vectors for the L0 reference list and the L1 reference list. If at 1520 it is determined that motion vector refinement, e.g., DMVR, is not applied ("No" at 1520), then at 1550 a determination is made as to whether the reference is weighted. If so ("Yes" at 1550), then bi-directional weighted motion compensation occurs at 1565, and a predicted CU is output. If the reference is not weighted ("No" at 1550), then "normal" bi-predictive motion compensation is applied at 1560, and then the resulting predicted CU is output. If at 1520 it is determined that motion vector refinement, e.g., DMVR, is applied ("Yes" at 1520), then at 1530 its application depends on whether the reference is weighted, such as with WP. If it is weighted ("yes" at 1530), then at 1545 a motion vector refinement is derived based on using the MRSAD as described above, and then operation continues as described above at 1550. If it is determined at 1530 that the reference is not weighted ("no" at 1530), then at 1540 a motion vector refinement is derived using the SAD as described above, and then operation continues at 1550 as described above.

[0070] More generally, at least one other example of an embodiment may involve a motion vector refinement, e.g., a DMVR process, that is modified or adapted and encoded / decoded or inferred in the bitstream based on an indicator, e.g., information, or a flag, or an index (e.g., inferred to be true if LIC is on the flag, as in the previous embodiment). An example of such an embodiment is shown in FIG. 18. In FIG. 18, a decision to activate a decoder-side motion vector refinement, DMVR, is made at 1810. The DMVR process may include a refinement function, such as SAD, as described above. If it is determined at 1810 that the DMVR process is activated, the refinement function may be changed at 1820 based on the indicator, e.g., information, or a flag, or an index. For example, the indicator may indicate that the refinement function should be changed to MRSAD instead of SAD, such as when the LIC tool is activated, as described above. Alternatively, the indicator may indicate which of one or more tools, such as LIC, WP, and BCW, is activated along with DMVR, where the combination of tools results in a change in the refinement function as described above, for example, selecting MRSAD rather than SAD as the refinement function. After 1820, the picture information is encoded / decoded based on the DMVR process including the changed refinement function.

[0071] FIG. 19 shows another example of an embodiment that is a variation of the embodiment shown in FIG. 18. In FIG. 19, the operation at 1910 determines whether to activate a DMVR process such as provided by a DMVR tool. If the DMVR is not activated (a negative or "no" result at 1910), processing continues at 1970, such as encoding or decoding video or picture data. If the result at 1910 is positive (a "yes" result at 1910), the DMVR process is activated. As noted above, the DMVR process may include a refinement function such as SAD. At 1920, an indicator such as that described above with respect to FIG. 18 is determined (inferred or read from the bitstream / determined by the encoder), where the indicator indicates whether the refinement function, e.g., SAD, is modified or adapted. At 1930, the indicator value is verified. If the result is negative (a "no" result at 1930), processing continues at 1940 to a DMVR process that includes a refinement function such as SAD. If the result at 1930 is affirmative (e.g., "yes"), processing continues at 1950, where the refinement function is modified, e.g., selected to be MRSAD, based on the indicator. In a variant, the indicator may be a function index having multiple values ​​mapped to corresponding multiple functions, which may be selected to modify or adapt the refinement function included in the DMVR process. In the example of FIG. 19, an index equal to 0 may indicate or correspond to the selection of a SAD refinement function, while an index equal to 1 may indicate or correspond to the selection of an MRSAD refinement function. The selection of the refinement function based on the indicator or index modifies the refinement function. Then, at 1960, a DMVR process including the modified refinement function occurs. At 1970, picture information is encoded / decoded based on the DMVR process including the modified refinement function.

[0072] This document describes various example embodiments, features, models, approaches, and the like. Many of these examples are specifically described and often described in a manner that may be considered limiting, at least to illustrate their individual characteristics. However, this is for clarity of description only and not to limit application or scope. Indeed, the various example embodiments, features, and the like described herein can be combined and interchanged in various ways to provide further example embodiments.

[0073] Generally, example embodiments described and contemplated herein can be implemented in many different forms. While Figures 1 and 2 above and Figure 14 described below provide some examples, other embodiments are contemplated, and discussion of Figures 1, 2, and 14 does not limit the breadth of implementations. At least one embodiment provides examples generally related to video encoding and / or video decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream or signal. These and other embodiments can be implemented as a method, an apparatus, a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium storing a bitstream or signal generated according to any of the described methods.

[0074] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side while "decoded" is used on the decoder side.

[0075] In this disclosure, the terms HDR (high dynamic range) and SDR (standard dynamic range) are used. These terms often convey specific values ​​of dynamic range to those skilled in the art. However, additional embodiments are also contemplated in which reference to HDR is understood to mean "higher dynamic range" and reference to SDR is understood to mean "lower dynamic range." Such additional embodiments are not constrained by the specific values ​​of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range."

[0076] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be varied or combined.

[0077] Various methods and other aspects described herein can be used to modify modules of a video encoder and / or video decoder, such as the motion compensation and / or motion estimation modules 170 and 175 of the encoder 100 shown in FIG. 1 and the motion compensation module 275 of the decoder 200 shown in FIG. 2. Furthermore, the aspects are not limited to VVC or HEVC, but can be applied, for example, to other standards and recommendations, whether existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described herein can be used individually or in combination.

[0078] For example, various numerical values ​​are used in this document. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0079] FIG. 14 shows a block diagram of an example system in which various aspects and embodiments can be implemented. System 1000 can be embodied as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 1000, either singly or in combination, can be embodied in a single integrated circuit, multiple ICs, and / or separate components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or separate components. In various embodiments, system 1000 is communicatively coupled to other similar systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the aspects described herein.

[0080] The system 1000 includes at least one processor 1010 configured to execute loaded instructions, for example, to implement various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits, as is known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile and / or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device, and / or a network-accessible storage device.

[0081] System 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded video or decoded video, which may include its own processor and memory. Encoder / decoder module 1030 represents a module or modules that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding and a decoding module. Furthermore, encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated within processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0082] Program code loaded into the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded into the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams or signals, matrices, variables, and intermediate or final results from the processing of equations, expressions, operations, and computational logic.

[0083] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be the memory 1020 and / or the storage device 1040, and may be, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video coding and decoding operations, such as MPEG-2, HEVC, or Versatile Video Coding (VVC).

[0084] Input to the elements of system 1000 may be provided through various input devices as shown in block 1130. Such input devices include, but are not limited to, (i) an RF section that receives RF signals transmitted over the air by, for example, a broadcast station, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0085] In various embodiments, the input devices of block 1130 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements that (i) select a desired frequency (also referred to as selecting a signal or band-limiting a signal to a frequency band), (ii) downconvert the selected signal, (iii) band-limit again to a narrower frequency band to select a signal frequency band, which in certain embodiments may be referred to as a channel (for example), (iv) demodulate the downconverted, band-limited signal, (v) perform error correction, and (vi) demultiplex to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner that performs a variety of these functions, including downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements perform frequency selection by receiving, filtering, downconverting, and filtering again to the desired frequency band an RF signal transmitted over a wired (e.g., cable) medium. In various embodiments, the order of the above (and other) elements is rearranged, some of these elements are removed, and / or other elements that perform similar or different functions are added. Adding elements may include inserting elements between existing elements, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0086] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, e.g., Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within processor 1010. Similarly, aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 1010. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including processor 1010 and encoder / decoder 1030, which operate in combination with memory and storage elements to process the data stream, for example, for display on an output device.

[0087] The various elements of system 1000 may be provided within an integrated housing in which the various elements can be interconnected and transmit data therebetween using a suitable connection arrangement 1140, e.g., an internal bus as is well known in the art, including an I2C bus, wiring, and a printed circuit board.

[0088] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented in a wired and / or wireless medium, for example.

[0089] In various embodiments, data is streamed to system 1000 using a Wi-Fi network, such as IEEE 802.11. The Wi-Fi signal in these embodiments is received via communication channel 1060 and communication interface 1050, which are adapted for Wi-Fi communication. Communication channel 1060 in these embodiments is typically connected to an access point or router, which provides access to external networks, including the Internet, enabling streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 1000 using a set-top box that delivers data via an HDMI connection in input block 1130. Still other embodiments provide streamed data to system 1000 using an RF connection in input block 1130.

[0090] System 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. In various exemplary embodiments, other peripheral devices 1120 include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 1000. In various embodiments, control signals are transmitted between system 1000 and display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV Link, CEC, or other communications protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 using communications channel 1060 via communications interface 1050. The display 1100 and speakers 1110 may be integrated into a single unit along with the other components of the system 1000 in an electronic device, such as a television. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0091] Alternatively, display 1100 and speakers 1110 may be separate from one or more of the other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments in which display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0092] The embodiments may be performed by computer software implemented by the processor 1010 or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type appropriate to the technical environment and may be implemented using any suitable data storage technology, such as, by way of non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type appropriate to the technical environment and may include, by way of non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0093] Various generalized and specific embodiments are also supported and contemplated throughout this disclosure. Examples of embodiments according to the present disclosure include, but are not limited to, the following:

[0094] Generally, at least one example of an embodiment may involve a method for encoding picture information, the method including determining activation of a decoder-side motion vector refinement process including a refinement function, modifying the refinement function based on an indicator, and encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0095] Generally, at least one example of an embodiment may involve a method for decoding picture information, the method including determining activation of a decoder-side motion vector refinement process including a refinement function, modifying the refinement function based on an indicator, and decoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0096] Generally, at least one example of an embodiment may involve an apparatus for encoding picture information, the apparatus including one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0097] Generally, at least one example of an embodiment may involve an apparatus for decoding picture information, the apparatus including one or more processors configured to: determine activation of a decoder-side motion vector refinement process including a refinement function; modify the refinement function based on an indicator; and encode at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

[0098] Generally, at least one example of an embodiment may involve a method described herein, wherein modifying the refinement function includes modifying a cost function associated with a decoder-side motion vector refinement process.

[0099] Generally, at least one example of an embodiment may involve an apparatus as described herein, wherein the one or more processors configured to modify the refinement function include one or more processors configured to modify a cost function associated with a decoder-side motion vector refinement process.

[0100] Generally, at least one example of an embodiment may involve the method described herein, wherein modifying the cost function includes using a mean removed absolute sum of difference (MRSAD) cost function during the decoder-side motion vector refinement process.

[0101] Generally, at least one example of an embodiment may involve an apparatus as described herein, wherein the one or more processors configured to modify the cost function include one or more processors configured to use a mean removed absolute sum of difference (MRSAD) cost function during a decoder-side motion vector refinement process.

[0102] Generally, at least one example of an embodiment may involve the method described herein, wherein the refinement function comprises a cost function, and modifying the refinement function comprises selecting a cost function from a group comprising a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function.

[0103] Generally, at least one example of an embodiment may involve an apparatus as described herein, wherein the refinement function comprises a cost function, and wherein the one or more processors configured to modify the refinement function comprise one or more processors configured to select a cost function from a group comprising a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function.

[0104] Generally, at least one example of an embodiment may involve a method or apparatus as described herein, wherein the indicator includes one or more of information or indexes or flags indicating activation of at least one of a local illumination compensation (LIC) process, or a weighted prediction process (WP), or a coding unit (CU) level weighted bi-prediction (BCW) process during activation of a decoder-side motion vector refinement process.

[0105] Generally, at least one example of an embodiment may involve a method for encoding a picture, the method including determining activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process, modifying the motion vector refinement process based on the activation and the second process, and encoding the picture based on the modified motion vector refinement process and the second process.

[0106] Generally, at least one example of an embodiment may involve a method for decoding a picture, the method including determining activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process, modifying the motion vector refinement process based on the activation and the second process, and decoding the picture based on the modified motion vector refinement process and the second process.

[0107] Generally, at least one example of an embodiment may involve an apparatus for encoding a picture including one or more processors, wherein the one or more processors are configured to determine activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process, modify the motion vector refinement process based on the activation and the second process, and encode the picture based on the modified motion vector refinement process and the second process.

[0108] Generally, at least one example of an embodiment may involve an apparatus for decoding a picture, the apparatus including one or more processors, the one or more processors being configured to determine activation of a processing mode involving a motion vector refinement process and a second process other than the motion vector refinement process, modifying the motion vector refinement process based on the activation and the second process, and decoding the picture based on the modified motion vector refinement process and the second process.

[0109] Generally, at least one example of an embodiment may involve a method or apparatus that includes a motion vector refinement process as described herein, where the motion vector refinement process includes a decoding-side motion vector refinement process (DMVR).

[0110] Generally, at least one example of an embodiment may involve a method or apparatus including a second process described herein, wherein the second process includes at least one of a local illumination compensation (LIC) process, or a weighted prediction process (WP), or a coding unit (CU) level weighted bi-prediction (BCW).

[0111] Generally, at least one example of an embodiment may involve a method that includes modifying a motion vector refinement process described herein, where modifying the motion vector refinement process includes modifying a cost function associated with the motion vector refinement process.

[0112] Generally, at least one example of an embodiment may involve an apparatus including one or more processors configured to modify a motion vector refinement process as described herein, wherein the one or more processors configured to modify the motion vector refinement process include one or more processors configured to modify a cost function associated with the motion vector refinement process.

[0113] Generally, at least one example of an embodiment may involve a method that includes modifying a cost function associated with a motion vector refinement process described herein, where modifying the cost function includes using a mean removed absolute sum of difference (MRSAD) cost function during the motion vector refinement process.

[0114] Generally, at least one example of an embodiment may involve an apparatus that includes one or more processors configured to modify a cost function associated with a motion vector refinement process described herein, wherein the one or more processors configured to modify the cost function include using a mean removed absolute sum of difference (MRSAD) cost function during the motion vector refinement process.

[0115] Generally, at least one example of an embodiment may involve a method for encoding a picture, the method including determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modifying the DMVR process based on the activation and the LIC process, and encoding the picture based on the modified DMVR process and the LIC process.

[0116] Generally, at least one example of an embodiment may involve a method for decoding a picture, the method including determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modifying the DMVR process based on the activation and the LIC process, and decoding the picture based on the modified DMVR process and the LIC process.

[0117] Generally, at least one example of an embodiment may involve an apparatus for encoding a picture, the apparatus including one or more processors configured to determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process, modify the DMVR process based on the activation and the LIC process, and encode the picture based on the modified DMVR process and the LIC process.

[0118] Generally, at least one example of an embodiment may involve an apparatus for decoding a picture, the apparatus including one or more processors configured to: determine activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modify the DMVR process based on the activation and the LIC process; and decode the picture based on the modified DMVR process and the LIC process.

[0119] Generally, at least one example of an embodiment may involve a method that includes modifying a DMVR process described herein, where modifying the DMVR process includes modifying a cost function associated with the DMVR process.

[0120] Generally, at least one example of an embodiment may involve an apparatus including one or more processors configured to modify the DMVR process described herein, wherein the one or more processors configured to modify the DMVR process include one or more processors configured to modify a cost function associated with a motion vector refinement process.

[0121] Generally, at least one example of an embodiment may involve a method that includes modifying a cost function associated with the DMVR process described herein, where modifying the cost function includes using a mean removed sum of absolute differences (MRSAD) cost function during DMVR.

[0122] Generally, at least one example embodiment may involve an apparatus that includes one or more processors configured to modify a cost function associated with a DMVR process described herein, wherein the one or more processors configured to modify the cost function associated with the DMVR process include one or more processors configured to use a mean removed sum of absolute differences (MRSAD) cost function during the DMVR process.

[0123] In general, at least one example of an embodiment may involve a computer program product including instructions that, when executed by a computer, cause the computer to perform any one or more of the methods described herein.

[0124] Generally, at least one example embodiment may involve a non-transitory computer-readable medium storing executable program instructions, the instructions causing a computer executing the instructions to perform any one or more of the methods described herein.

[0125] In general, at least one example embodiment may involve a signal containing data generated according to any one or more of the methods described herein.

[0126] Generally, at least one example embodiment may involve a bitstream formatted to include syntax elements and encoded image information according to any one or more of the methods described herein.

[0127] Generally, at least one example of an embodiment may involve a device including an apparatus according to any embodiment of the apparatus described herein and at least one of: (i) an antenna configured to receive a signal, the signal including data representing image information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the image information; and (iii) a display configured to display an image from the image information.

[0128] Generally, at least one example embodiment may involve a device described herein, where the device includes one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a mobile phone, a tablet, or other electronic device.

[0129] Throughout this disclosure, various implementations involve decoding. As used herein, "decoding" can encompass, for example, all or part of the processes performed on a received encoded sequence to generate a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by decoders in various implementations described herein, such as extracting a picture from a tiled (packed) picture, determining an upsample filter to use, then upsampling the picture, and flipping the picture back to its intended orientation.

[0130] As a further example, in one embodiment, "decoding" refers to entropy decoding only, in another embodiment, "decoding" refers to differential decoding only, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process generally will be clear based on the context of a particular description and will be well understood by one of ordinary skill in the art.

[0131] Various implementations also involve encoding. Similar to the above discussion of "decoding," "encoding," as used herein, can encompass all or part of the processes performed on, for example, an input video sequence to generate an encoded bitstream or signal. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforming, quantizing, and entropy encoding. In various embodiments, such processes also, or alternatively, include processes performed by the encoders of the various implementations described herein.

[0132] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process in general will be clear based on the context of a particular description and will be well understood by one of ordinary skill in the art.

[0133] It should be noted that the syntax elements used herein are descriptive terms, and therefore they do not preclude the use of other syntax element names.

[0134] Where a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, where a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0135] Various embodiments refer to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often given computational complexity constraints. Rate-distortion optimization is typically formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are various approaches to solving the rate-distortion optimization problem. For example, an approach may involve a thorough evaluation of the coding cost and associated distortion of the reconstructed signal after coding and decoding, and may be based on extensive testing of all encoding options, including all considered modes or coding parameter values. To reduce encoding complexity, faster approaches may also be used, particularly to calculate approximate distortion based on a prediction or prediction residual signal rather than a reconstructed signal. These two approaches may also be used in combination, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques for performing the optimization, but the optimization does not necessarily involve a thorough evaluation of both the coding cost and associated distortion.

[0136] The implementations and aspects described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed in the context of only a single implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, a processor, which refers generally to processing devices including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, for example, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0137] References to "one embodiment" or "one embodiment," or "one implementation" or "one implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in one embodiment" or "in one implementation" or "in one implementation," as well as any other variations, in various places throughout this document are not necessarily all referring to the same embodiment.

[0138] Additionally, this specification may refer to "obtaining" various pieces of information. Obtaining information may include, for example, one or more of determining information, evaluating information, calculating information, predicting information, or retrieving information from memory.

[0139] Additionally, this document may refer to "accessing" various pieces of information, which may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or evaluating information.

[0140] Additionally, this document may refer to "receiving" various portions of information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" is typically included in some manner, such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, judging information, predicting information, or evaluating information.

[0141] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as there are items listed, as would be apparent to one of ordinary skill in the art.

[0142] Also, as used herein, the word "signaling" refers, among other things, to instructing a corresponding decoder. For example, in certain embodiments, an encoder signals a specific one of multiple parameters for refinement. In this manner, in embodiments, the same parameters are used on both the encoder and decoder sides. Thus, for example, an encoder can transmit a specific parameter to a decoder (explicit signaling), so that the decoder can use the same specific parameter. Conversely, if the decoder already has a specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling) to enable the decoder to easily recognize and select the specific parameter. By avoiding the transmission of any actual function, bit savings are realized in various embodiments. It should be appreciated that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various embodiments. Although the above relates to the verb form of the word "signaling," the word "signaling" can also be used as a noun herein.

[0143] As will be apparent to those skilled in the art, implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bitstream or signal of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is well known. The signal can be stored on a processor-readable medium.

[0144] Various embodiments have been described. The embodiments may include any of the following features or entities, alone or in any combination, across various different claim categories and types. In the encoder and / or decoder, providing picture information to encode / decode based on determining activation of a decoder-side motion vector refinement process including a refinement function, where the refinement function is changed based on the indicator. In the encoder and / or decoder, providing picture information to encode / decode based on determining activation of a decoder-side motion vector refinement process including a refinement function, wherein the refinement function is modified based on the indicator; The refinement function includes a cost function, or modifying the refinement function comprises modifying a cost function associated with a decoder-side motion vector refinement process, or modifying the cost function includes using a Mean Removed Sum of Absolute Differences (MRSAD) cost function during the decoder-side motion vector refinement process, or modifying the refinement function includes selecting a cost function from a group including a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function, or o The indicator includes, or includes, one or more of, information or indexes or flags indicating activation of at least one of a Local Illumination Compensation (LIC) process, or a Weighted Prediction process (WP), or a Coding Unit (CU) level Weighted Bi-Prediction (BCW) process during activation of a decoder-side motion vector refinement process. In the encoder and / or decoder, providing for applying motion vector refinement when illumination compensation is activated. · Providing in the encoder and / or decoder to apply decoder-side motion vector refinement (DMVR) when local illumination compensation (LIC) is activated. In the encoder and / or decoder, applying motion vector refinement based on a modified cost function when illumination compensation is activated. · Providing that in the encoder and / or decoder, when Weighted Prediction (WP) is activated, decoder-side motion vector refinement (DMVR) is applied. In the encoder and / or decoder, providing for applying motion vector refinement based on a modified cost function when WP is activated. · Providing that in the encoder and / or decoder, when CU-level weighted bi-prediction (BCW) is activated, decoder-side motion vector refinement (DMVR) is applied. In the encoder and / or decoder, when BCW is activated, providing for applying motion vector refinement based on a modified cost function. In the encoder and / or decoder, providing applying decoder-side motion vector refinement (DMVR) based on a modified cost function when local illumination compensation (LIC) or WP or BCW is activated. In the encoder and / or decoder, providing applying motion vector refinement based on a modified sum of absolute differences (SAD) cost function when illumination compensation (e.g., LIC or WP or BCW) is activated. In an encoder and / or decoder, when illumination compensation is activated, providing applying motion vector refinement based on a modified sum of absolute differences (SAD) cost function, wherein the motion vector refinement comprises decoder-side motion vector refinement (DMVR), the modified sum of absolute differences cost function comprises mean-removed SAD (MRSAD), and the illumination compensation comprises local illumination compensation (LIC) or WP or BCW. In an encoder and / or decoder, providing for applying decoder-side motion vector refinement (DMVR) when local illumination compensation (LIC) is activated, where the LIC process is calculated once a DMVR prediction is available. In an encoder and / or decoder, providing applying decoder-side motion vector refinement (DMVR) based on whether local illumination compensation (LIC) or WP or BCW is activated, including applying a modified cost function when LIC or WP or BCW is activated. In an encoder and / or decoder, providing applying decoder-side motion vector refinement (DMVR) based on whether local illumination compensation (LIC) or WP or BCW is activated, including applying a modified cost function when LIC or WP or BCW is activated and the modified cost function includes a mean removed absolute difference sum cost function. Providing, in an encoder and / or decoder, applying decoder-side motion vector refinement (DMVR) based on flag / index values ​​coded in the bitstream at CU, slice, SPS, PPS level, where the applied DMVR comprises applying a modified cost function depending on the value of the coded information, where the modified cost function comprises a mean removed absolute difference sum cost function. In an encoder and / or decoder, applying motion vector refinement when illumination compensation is activated based on which improved compression efficiency is provided. Inserting syntax elements into the signaling that enable the encoder and / or decoder to provide for applying motion vector refinement when illumination compensation is activated, as described herein. Based on these syntax elements, select the combination of motion vector refinement and illumination compensation to apply at the decoder. A bitstream or signal containing one or more of the listed syntax elements, or variations thereof. Inserting syntax elements into the signaling that allow the decoder to provide motion vector refinement and illumination compensation in a manner that corresponds to the method used by the encoder. Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described syntax elements, or variations thereof. A television, set-top box, mobile phone, tablet, or other electronic device that provides for applying motion vector refinement and illumination compensation according to any of the described embodiments. A television, set-top box, mobile phone, tablet, or other electronic device that performs motion vector refinement and illumination compensation according to any of the described embodiments and displays the resulting image (e.g., using a monitor, screen, or other type of display). A television, set-top box, mobile phone, tablet, or other electronic device that tunes a channel (e.g., using a tuner) to receive a signal containing the encoded image and performs motion vector refinement and illumination compensation according to any of the described embodiments. A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image wirelessly (e.g., using an antenna) and performs motion vector refinement and illumination compensation according to any of the described embodiments. A computer program product storing program code that, when executed by a computer, implements motion vector refinement and illumination compensation according to any of the described embodiments. A non-transitory computer-readable medium comprising executable program instructions that, when executed by a computer, cause motion vector refinement and illumination compensation to be performed in accordance with any of the described embodiments.

[0145] Various other generalized and specialized embodiments are also supported and contemplated throughout this disclosure.

Claims

1. 1. A method for encoding picture information, comprising: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on the indicator; and encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

2. 1. A method for decoding picture information, comprising: determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on the indicator; and decoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

3. 1. An apparatus for encoding picture information, comprising: one or more processors, the one or more processors determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on the indicator; encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

4. 1. An apparatus for decoding picture information, comprising: one or more processors, the one or more processors determining activation of a decoder-side motion vector refinement process including a refinement function; modifying the refinement function based on the indicator; encoding at least a portion of the picture information based on the decoder-side motion vector refinement process and the modified refinement function.

5. The method of claim 1 or 2, wherein modifying the refinement function comprises modifying a cost function associated with the decoder-side motion vector refinement process.

6. 5. The apparatus of claim 3, wherein the one or more processors configured to modify the refinement function comprise the one or more processors configured to modify a cost function associated with the decoder-side motion vector refinement process.

7. The method of claim 5 , wherein modifying the cost function comprises using a mean removed sum of absolute differences (MRSAD) cost function during the decoder-side motion vector refinement process.

8. 7. The apparatus of claim 6, wherein the one or more processors configured to modify the cost function comprise the one or more processors configured to use a mean removed absolute sum of differences (MRSAD) cost function during the decoder-side motion vector refinement process.

9. the refinement function comprises a cost function; 6. The method of claim 1, wherein modifying the refinement function comprises selecting the cost function from a group comprising a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function.

10. the refinement function comprises a cost function; 7. The apparatus of claim 3, wherein the one or more processors configured to modify the refinement function comprise the one or more processors configured to select the cost function from a group comprising a sum of absolute differences (SAD) cost function and a mean removed sum of absolute differences (MRSAD) cost function.

11. 11. The method or apparatus of claim 1, wherein the indicator comprises one or more of information, an index, or a flag indicating activation of at least one of a local illumination compensation (LIC) process, or a weighted prediction process (WP), or a coding unit (CU) level weighted bi-prediction (BCW) process during activation of the decoder-side motion vector refinement process.

12. 1. A method for encoding a picture, comprising: determining activation of a processing mode involving a motion vector refinement process and a second process other than said motion vector refinement process; modifying the motion vector refinement process based on the activation; and encoding the picture based on the modified motion vector refinement process.

13. 1. A method for decoding a picture, comprising: determining activation of a processing mode involving a motion vector refinement process and a second process other than said motion vector refinement process; modifying the motion vector refinement process based on the activation; and decoding the picture based on the modified motion vector refinement process.

14. 1. An apparatus for encoding a picture, comprising: one or more processors, the one or more processors: determining activation of a processing mode involving a motion vector refinement process and a second process other than said motion vector refinement process; modifying the motion vector refinement process based on the activation; encoding the picture based on the modified motion vector refinement process.

15. 1. An apparatus for decoding a picture, comprising: one or more processors, the one or more processors: determining activation of a processing mode involving a motion vector refinement process and a second process other than said motion vector refinement process; modifying the motion vector refinement process based on the activation; and decoding the picture based on the modified motion vector refinement process.

16. 16. The method of claim 12 or 3 or the apparatus of claim 14 or 15, wherein the motion vector refinement process comprises a decode-side motion vector refinement process (DMVR).

17. 17. The method or apparatus of claim 12, wherein the second process comprises at least one of a local illumination compensation (LIC) process, or a weighted prediction process (WP), or a coding unit (CU) level weighted bi-prediction (BCW) process.

18. 18. The method of any one of claims 12, 13, 16 or 17, wherein modifying the motion vector refinement process comprises modifying a cost function associated with the motion vector refinement process.

19. 18. The apparatus of claim 14, wherein the one or more processors configured to modify the motion vector refinement process comprise the one or more processors configured to modify a cost function associated with the motion vector refinement process.

20. 20. The method of claim 18, wherein modifying the cost function comprises using a mean removed sum of absolute differences (MRSAD) cost function during the motion vector refinement process.

21. 20. The apparatus of claim 19, wherein the one or more processors configured to modify the cost function associated with the motion vector refinement process comprise the one or more processors configured to use a mean removed sum of absolute differences (MRSAD) cost function during the motion vector refinement process.

22. 1. A method for encoding a picture, comprising: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation; and and encoding the picture based on the modified DMVR process.

23. 1. A method for decoding a picture, comprising: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation; and and decoding the picture based on the modified DMVR process.

24. 1. An apparatus for encoding a picture, comprising: one or more processors, the one or more processors: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation and the LIC process; encoding the picture based on the modified DMVR process and the LIC process.

25. 1. An apparatus for decoding a picture, comprising: one or more processors, the one or more processors: determining activation of a processing mode involving a decoder-side motion vector refinement (DMVR) process and a local illumination compensation (LIC) process; modifying the DMVR process based on the activation and the LIC process; and decoding the picture based on the modified DMVR process and the LIC process.

26. The method of claim 22 or 23, wherein modifying the DMVR process includes modifying a cost function associated with the DMVR process.

27. 26. The apparatus of claim 24 or 25, wherein the one or more processors configured to modify the DMVR process comprise the one or more processors configured to modify a cost function associated with the motion vector refinement process.

28. 27. The method of claim 26, wherein modifying the cost function comprises using a mean removed sum of absolute differences (MRSAD) cost function during the DMVR process.

29. 28. The apparatus of claim 27, wherein the one or more processors configured to modify the cost function associated with the DMVR process comprise the one or more processors configured to use a mean removed sum of absolute differences (MRSAD) cost function during the DMVR process.

30. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1, 2, 5-7, 9, 11, 12, 15 or 17.

31. 19. A non-transitory computer readable medium storing executable program instructions, the instructions causing a computer executing the instructions to perform the method of any one of claims 1, 2, 5-7, 9, 11, 12, 15 or 17.

32. A signal comprising data generated according to the method of any one of claims 1, 5-7, 9, 11, 15 or 17.

33. A bitstream formatted to include syntax elements and encoded image information according to the method of any one of claims 1, 5 to 7, 9, 11, 15 or 17.

34. A device, A device according to any one of claims 3 to 6, 8, 10, 13, 14, 16 and 18; 1. A device comprising at least one of: (i) an antenna configured to receive a signal, the signal including data representing image information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the image information; and (iii) a display configured to display an image from the image information.

35. 24. The device of claim 23, wherein the device comprises one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a mobile phone, a tablet, or other electronic device.

Images