Motion compensation in video encoding and decoding

Abstract

To provide an encoding and / or decoding method and device improving motion compensation prediction.SOLUTION: An encoding or decoding method includes: processing video information based upon a motion model including encoding units including a plurality of sub-blocks of an affine motion model etc., to generate motion compensation information; determining at least one model parameter of a linear model for lighting change of the video information based upon a plurality of motion vectors included in the motion compensation information to acquire a local lighting compensation model; and encoding or decoding the video information based upon the motion compensation information and the local lighting compensation model.SELECTED DRAWING: Figure 18

Description

[Technical Field]

[0001] Technical Field [1] This disclosure includes video encoding and decoding. [Background technology]

[0002] background [2] To achieve high compression efficiency, image and video coding schemes, such as those defined by the High Efficiency Video Coding (HEVC) standard, typically employ prediction and transform coding to exploit spatial and temporal redundancy in video content. Intra- or inter-prediction is typically used to exploit intra- or inter-frame correlation, and the difference between the original block and the predicted block, referred to as the prediction error or prediction residual, is often transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded using an inverse process corresponding to prediction, transform, quantization, and entropy coding. Recent additions to video compression technology include various versions of the Joint Exploration Model (JEM) reference software and / or documentation being developed by the Joint Video Exploration Team (JVET). Efforts such as the JEM aim to further improve upon existing standards such as HEVC. Summary of the Invention

[0003] overview [3] In general, an example embodiment of a method or, e.g., an apparatus including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the plurality of motion vectors including a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and estimating a local distortion based on the set of reconstructed samples.

[0004] [4] Another example embodiment of a method or, e.g., an apparatus including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a first set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. and a second set of motion vectors associated with the first set of motion vectors, and obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0005] [5] Another example embodiment of a method or, for example, an apparatus including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located in an upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0006] [6] Another example embodiment of a method or, for example, an apparatus including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and estimating local distortion based on the set of reconstructed samples.

[0007] [7] Another example embodiment of a method or, for example, an apparatus including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each one of the sub-blocks included in the first row of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0008] [8] Another example embodiment of a method or, e.g., an apparatus including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a first set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. and a second set of motion vectors associated with the first set of motion vectors, and obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0009] [9] Another example embodiment of a method or, e.g., an apparatus including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; The first motion vector is associated with a first sub-block among a plurality of sub-blocks located in the upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector, and evaluating a local distortion based on the set of reconstructed samples.

[0010]

[10] Another example embodiment of a method or, e.g., an apparatus including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and estimating local distortion based on the set of reconstructed samples.

[0011]

[11] Generally, embodiments may include a method of encoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model.

[0012]

[12] In general, another embodiment may include an apparatus for encoding video information, the apparatus including one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0013]

[13] In general, another embodiment may include a method of decoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the video information based on the motion compensation information and the local illumination compensation model.

[0014]

[14] In general, another embodiment may include an apparatus for decoding video information, the apparatus including one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0015]

[15] In general, another embodiment may include a bitstream formatted to include coded video information, where the coded video data is encoded by processing the coded video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the coded video information based on the motion compensation information and the local illumination compensation model.

[0016]

[16] In general, one or more embodiments may provide a computer-readable storage medium, e.g., a non-volatile computer-readable storage medium, having stored thereon instructions for encoding or decoding video data according to the methods or apparatus described herein. One or more of the embodiments may also provide a computer-readable storage medium having stored thereon a bitstream generated by the methods or apparatus described herein. One or more of the embodiments may also provide methods and apparatus for transmitting or receiving a bitstream generated by the methods or apparatus described herein.

[0017] BRIEF DESCRIPTION OF THE DRAWINGS

[17] The present disclosure may be better understood from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0018] [Figure 1]

[18] shows a block diagram of an example embodiment of a video encoder. [Figure 2]

[19] shows a block diagram of an example embodiment of a video decoder. [Figure 3]

[20] A block diagram of an example embodiment of a system for providing video encoding and / or decoding is shown. [Figure 4]

[21] shows an example of division of a coding tree unit (CTU) into coding units (CUs), prediction units (PUs), and transform units (TUs) in, for example, HEVC. [Figure 5]

[22] gives an example of an affine motion model such as that used in the Joint Exploratory Model (JEM). [Figure 6]

[23] presents an example of a 4x4 sub-CU based affine motion vector field. [Figure 7]

[24] shows an example of the motion vector (MV) prediction process for an affine AMVP CU. [Figure 8]

[25] shows examples of motion vector prediction candidates in affine integration mode. [Figure 9]

[26] present the spatial derivation of affine motion field control points in the affine integration mode. [Figure 10]

[27] demonstrate the use of nearby samples to derive parameters associated with a Local Illumination Compensation (LIC) model. [Figure 11]

[28] shows an example of deriving the parameters of the LIC model in a bidirectional prediction model. [Figure 12]

[29] shows an example of neighborhood samples arranged in different L-shapes around a block of a reference picture that corresponds to a sub-block of a current coding unit based on a motion vector associated with the sub-block. [Figure 13]

[30] Some example embodiments of a video encoder are shown. [Figure 14]

[31] illustrates another example embodiment of a portion of a video encoder. [Figure 15]

[32] shows an example of neighborhood samples arranged in a pseudo-L shape around a block in a reference picture that corresponds to a sub-block in a current coding unit based on a motion vector associated with the sub-block. [Figure 16]

[33] illustrates another example embodiment of a portion of a video encoder. [Figure 17]

[34] shows an example of neighborhood samples arranged in different L-shapes around a block of a reference picture that corresponds to a group of sub-blocks of a current coding unit based on a motion vector associated with the group of sub-blocks. [Figure 18]

[35] illustrates another example of some embodiments of a video encoder. [Figure 19]

[36] shows an example of some embodiments of a video decoder. [Figure 20]

[37] Another example of an embodiment of a video decoder is shown. [Figure 21]

[38] Another example of an embodiment of a video decoder is shown. [Figure 22]

[39] Another example of an embodiment of a video decoder. [Figure 23]

[40] Another example of some embodiments of a video encoder is shown. [Figure 24]

[41] Another example of some embodiments of a video encoder is shown. [Figure 25]

[42] shows an example embodiment of a motion vector at the center of a coding unit. [Figure 26]

[43] shows another example of an embodiment of a motion vector at the center of a coding unit. [Figure 27]

[44] shows another example of an embodiment of a motion vector at the center of a coding unit. [Figure 28]

[45] presents an example embodiment for determining parameters of a local illumination compensation model. DETAILED DESCRIPTION OF THE INVENTION

[0019]

[46] In the various figures, like reference signs refer to the same or similar features.

[0020] Detailed Description

[47] Recent efforts to advance video compression technology, such as those associated with the Joint Search Model (JEM) being developed by the Joint Video Search Team (JVET), provide advanced features and tools. For example, such development efforts include providing support for additional motion models that improve temporal prediction. One such motion model is the affine motion model, which is described in more detail below. Support is also provided for tools such as block-based local illumination compensation (LIC). The LIC tool involves applying the LIC model to predict possible illumination variations between a prediction block used in motion-compensated prediction and a corresponding reference block. Various aspects and embodiments described herein include motion models and tools such as, for example, the affine motion model and the LIC tool.

[0021]

[48] ​​This document describes a wide variety of aspects, including tools, features, embodiments, models, methods, and the like. Many of these aspects are described with specificity and often in a manner that may sound limiting, at least to illustrate individual features. However, this is for clarity of description and does not limit the applicability or scope of the aspects. In fact, all of the various aspects can be combined and interchanged to provide further aspects. Furthermore, aspects can be similarly combined and interchanged with aspects described in the previous figures.

[0022]

[49] The aspects described and contemplated in this document can be implemented in many different forms. While Figures 1, 2, and 3 below and other figures throughout this document provide some embodiments, other embodiments are contemplated, and discussion of Figures 1, 2, and 3 does not limit the scope of the embodiments. At least one aspect generally relates to encoding and decoding video, and at least one other aspect generally relates to transmitting the generated or encoded bitstream. These and other aspects can be embodied as a method, an apparatus, or a computer-readable storage medium. For example, the computer-readable storage medium can be a non-transitory computer-readable medium. The computer-readable storage medium can store instructions for encoding or decoding video data according to any of the described methods and / or a bitstream generated according to any of the described methods.

[0023]

[50] In this application, the terms "reconstructed" and "decoded" can be used interchangeably, the terms "pixel" and "sample" can be used interchangeably, and the terms "image," "picture," and "frame" can be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while the term "decoded" is used on the decoder side.

[0024]

[51] Various methods are described above, each method including one or more steps or actions for achieving the described method. Except where a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be varied or combined.

[0025]

[52] Various methods and other aspects described in this document may be used to modify one or more modules of a video encoder and / or decoder, such as, for example, the motion estimation module 170, the motion compensation module 175, and / or the motion compensation module 275 of the JVET or HEVC encoder 100 and decoder 200 shown in Figures 1 and 2, respectively. Moreover, the aspects are not limited to JVET or HEVC, but may be applied, for example, to other standards and recommendations, whether already in existence or developed in the future, and extensions of any such standards and recommendations (including JVET and HEVC). Unless otherwise stated or technically excluded, the aspects described in this document may be used individually or in combination.

[0026]

[53] Various numerical values ​​may be used in this document. Any specific numerical values ​​are examples, and the described aspects are not limited to these specific values.

[0027] 1, 2, and 3 respectively show block diagrams of example embodiments of encoder 100, decoder 200, and system 1000. While variations of encoder 100, decoder 200, and system 1000 are contemplated, the examples described below are provided and explained for purposes of clarity without listing all possible or anticipated variations.

[0028]

[55] In Figure 1, before encoding, a video sequence may undergo pre-coding processing (101), for example, applying a color transform to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a more robust signal distribution for compression (e.g., using histogram equalization of one of the color components). Metadata can be associated with the pre-processing and attached to the bitstream.

[0029]

[56] In encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is divided (102) into, for example, CUs. Each unit is coded, for example, using either intra mode or inter mode. If the unit is coded in intra mode, encoder 100 performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder determines (105) whether intra mode or inter mode should be used to code the unit, and indicates the intra / inter decision, for example, with a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0030]

[57] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is coded directly without applying the transform or quantization processes.

[0031]

[58] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the predicted block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0032]

[59] Figure 2 shows a block diagram of an example video decoder 200. In the decoder 200, the bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass described in Figure 1. As noted above, the encoder 100 in Figure 1 also generally performs video decoding as part of encoding the video data, for example, to provide a basis for further prediction.

[0033]

[60] In particular, the decoder's input includes a video bitstream, which may be generated by a video encoder such as video encoder 100 of FIG. 1. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coding information. Picture partition information indicates how the picture is partitioned. Thus, the decoder can partition the picture according to the decoded picture partition information (235). The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined with a prediction block (255) to reconstruct an image block. The prediction block may be obtained from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0034]

[61] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., converting from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding processing (101). The post-decoding processing may use metadata derived in the pre-encoding processing and signaled in the bitstream.

[0035]

[62] Figure 3 shows a block diagram of a system in which various aspects and embodiments can be implemented. System 1000 can be implemented as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics devices, and servers. System 1000, as shown in Figure 3, can be communicatively coupled to other similar systems and displays via communication channels known to those skilled in the art to thereby implement one or more of the various aspects described herein.

[0036]

[63] The system 1000 may include at least one processor 1010 configured to execute loaded instructions that implement one or more of the various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 may include at least one memory 1020 (e.g., a volatile memory device, a non-volatile memory device). The system 1000 may include a storage device 1040, which may include non-volatile memory including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, a magnetic disk drive, and / or an optical disk drive. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device, and / or a network-accessible storage device. The system 1000 may include an encoder / decoder module 1030 configured to process data to provide encoded or decoded video.

[0037]

[64] The encoder / decoder module 1030 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as known to those skilled in the art.

[0038]

[65] Program code loaded into the processor 1010 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded into the memory 1020 for execution by the processor 1010. According to an embodiment, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during execution of the processes described herein, including, but not limited to, input video, decoded video, bitstreams, expressions, formulas, matrices, variables, operations, and operational logic.

[0039]

[66] System 1000 may include a communication interface 1050 that enables communication with other devices over a communication channel 1060. Communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data from communication channel 1060. The communication interface may include, but is not limited to, a modem or a network card, and the communication channel may be implemented within a wired and / or wireless medium. The various components of system 1000 may be connected or communicatively coupled together using a variety of suitable connections, including, but not limited to, an internal bus, a wire, and a printed circuit board.

[0040]

[67] As described in more detail below, aspects and embodiments according to the present disclosure may relate to features of the systems shown in Figures 1, 2, and 3, such as motion estimation features, e.g., motion compensation features, such as module 175 of Figure 1 and module 170 of Figure 1 and / or module 275 of Figure 2.

[0041]

[68] For clarity of explanation, the following detailed description describes aspects with reference to embodiments including video compression technologies such as HEVC, JEM, and / or H.266, etc. However, the described aspects are also applicable to other video processing technologies and standards.

[0042]

[69] In the HEVC video compression standard, a picture is divided into so-called coding tree units (CTUs), and each CTU is represented by a coding unit (CU) in the compressed domain. Each CU is then given intra or inter prediction parameters (prediction information). To do so, it is spatially divided into one or more prediction units (PUs), and each PU is assigned some prediction information. The intra or inter coding mode is assigned at the CU level. A diagram of the division of the coding tree units into coding units, prediction units, and transform units is shown in Figure 4.

[0043]

[70] In inter-coding mode, motion-compensated temporal prediction is used to exploit the redundancy that exists between successive pictures of a video. To do so, exactly one motion vector (MV) is assigned to each PU in HEVC. Therefore, in HEVC, the motion model linking a PU and its reference block involves only translation.

[0044]

[71] In at least one version of the Joint Search Model (JEM) developed by the JVET (Joint Video Search Team) group, CUs are no longer divided into PUs or TUs, and some motion information (prediction information in inter mode) is directly assigned to each CU. In JEM, CUs can be divided into sub-CUs, and motion vectors for each sub-CU can be calculated. Furthermore, several richer motion models are supported to improve temporal prediction. One of the new motion models introduced in JEM is the affine model, which basically involves using an affine model to represent motion vectors in a CU. The motion model used is shown in Figure 5. The affine model can be used to generate a motion field within a CU for motion estimation. For example, the affine motion field includes motion vector component values ​​for each position (x, y) within the considered block, as defined by Equation 1:

number

[0045]

[72] In practice, to keep complexity at a reasonable level, one can calculate motion vectors for each 4x4 sub-block (sub-CU) of the considered CU, as shown, for example, in Figure 6. Affine motion vectors are calculated from control point motion vectors at the center of each sub-block. As a result, temporal prediction of a CU in the affine model involves motion-compensated prediction of each sub-block using its own motion vector.

[0046]

[73] Affine motion compensation can be used in at least two ways, for example, in JEM with affine AMVP (Advanced Motion Vector Prediction) or AF_AMVP and affine synthesis. -Affine AMVP (AF_AMVP) CUs in AMVP mode with a size larger than 8x8 can be predicted in affine AMVP mode. This is signaled through a flag in the bitstream. The generation of an affine motion field for that inter-CU involves determining a control point motion vector (CPMV), which is obtained by the decoder through the summation of a motion vector differential and a control point motion vector prediction (CPMVP). A CPMVP is a pair of motion vector candidates taken from lists (A, B, C) and (D, E), respectively, as shown in Figure 7, which shows an example of the motion vector prediction process for an affine AMVP CU. -Affine integration In affine synthesis mode, a CU-level flag indicates whether the synthesis CU uses affine motion compensation. If so, the first available nearby CU coded in affine mode is selected from an ordered set of candidate positions (A, B, C, D, E), as shown in Figure 8, which illustrates motion vector prediction candidates in affine synthesis mode.

[0047]

[74] Once the first neighboring CU in affine mode is obtained, three motion vectors from the top left corner, top right corner, and bottom left corner of the neighboring CU are calculated, as shown in Figure 9.

number

number

number

[0048]

[75] In JEM, block-based Local Illumination Compensation (LIC) can also be applied. The LIC tool essentially aims to predict possible illumination variations between a predicted block and a reference block used through motion-compensated prediction. For each inter-mode uncoded CU, a LIC flag is signaled or implicitly derived to indicate the use of LIC. The LIC tool is based on a linear model of illumination changes using a scaling factor a and an offset b, called LIC parameters. In codecs such as JEM, the LIC tool is disabled for inter-mode coded CUs that utilize the affine motion compensation described above.

[0049]

[76] In inter mode, block-based local illumination compensation (LIC) allows for the correction of block prediction samples obtained through motion compensation by taking into account spatial or temporal local illumination variations. It is based on a model of illumination variation, such as a first-order linear model of illumination variation using a scaling factor a and an offset b. As shown in Figure 10, the LIC parameters (a and b) can be estimated by comparing a set of reconstructed samples surrounding the current block ("current blk") located in a neighborhood Vcur with a set of reconstructed samples located in a neighborhood Vref(MV) of a reference block ("reference blk") in a reference picture (the set can have various sizes depending on the application). MV represents the motion vector between the current block and the reference block. Typically, Vcur and Vref(MV) are included in the samples located in an L-shape (top, left, and upper left) around the current block and the reference block, respectively.

[0050]

[77] The LIC parameters can be selected based on any of a variety of techniques. For example, the LIC parameters can be selected based on minimizing local distortion. One technique for minimizing local distortion includes minimizing the mean square error (MSE) difference between the samples at Vcur and the corrected samples at Vref(MV). In one example, the LIC model is a linear model, i.e., LIC(x)=a * x+b, where the parameters a and b can be determined based on minimizing the MSE difference as in Equation 3.

number

number

[0051]

[78] For bi-prediction, the LIC parameters (a0, b0) and (a1, b1) for Vref0 (MV0) and Vref1 (MV1), respectively, can be derived independently. In other embodiments, the derivations may be dependent or interdependent. For example, as shown in Figure 11, for bi-prediction, the L-shapes for references 0 and 1 are compared with the L-shape of the current block to derive the LIC parameters.

[0052]

[79] If a CU is coded using integration mode, the LIC flag can be copied from a neighboring block, similar to the motion information copying in integration mode. Otherwise, the LIC flag of the CU is signaled to indicate whether LIC is applied or not. However, in methods such as JEM, if a CU uses an affine model (affine AMVP, affine integration), the LIC flag is always set to false.

[0053]

[80] At least one embodiment includes enabling a block-based local illumination compensation (LIC) tool when affine motion prediction is utilized for an inter-mode coded coding unit (CU).

[0054]

[81] At least one embodiment includes activating the LIC tool for inter-mode coded CUs that utilize an affine model for representing motion vectors.

[0055]

[82] At least one embodiment may include, for example, activating a LIC tool for inter-mode coded CUs using an affine model, including LIC flag determination and corresponding LIC parameter derivation rules for affine AMVP and affine synthesis.

[0056]

[83] At least one embodiment includes a method for activating the LIC tool and creating rules related to inter-mode coding using affine motion prediction in a manner that provides good compression efficiency (rate-distortion performance) while minimizing the increase in coding design complexity.

[0057]

[84] At least one embodiment comprises: Determining the LIC flag for inter-mode coded CUs using affine motion prediction. In the case of affine AMVP, an iterative loop over the LIC tools can be applied to determine the LIC flag, which is signaled in the bitstream. In other cases, in affine integration, the LIC flag can be obtained based on neighboring blocks, e.g., derived from affine control points associated with neighboring blocks, similar to the motion information copy in integration mode. [Encoder / Decoder]

[0058]

[85] At least one embodiment comprises: Based on the determination that the LIC flag is true, providing for the derivation of corresponding LIC parameters. One or more features can be involved. For example, using one motion vector of the first sub-block or any other sub-block (e.g., the center sub-block), or taking into account multiple motion vectors of sub-blocks in the first row / first column, e.g., all motion vectors associated with sub-blocks in the first row / first column or motion vectors associated with a subset of all motion vectors in the first row and / or first column. As another example, generating a pair of LIC parameters that is unique for the entire CU. As another example, deriving multiple pairs of LIC parameters. [Encoder / Decoder]

[0059]

[86] At least one embodiment improves correction of block predicted samples based on consideration of illumination variations, for example, by enabling LIC tools for CUs that are inter-mode coded using affine motion prediction.

[0060]

[87] Various embodiments described herein provide improved coding efficiency.

[0061]

[88] At least one embodiment is based on the inventors' recognition that techniques involving LIC deactivated in an affine model cannot fully incorporate the potential performance resulting from block predicted samples via affine motion compensation due to the lack of compensation by taking illumination variations into account.

[0062]

[89] In general, at least one embodiment may include deriving LIC parameters at the decoder side according to one or more embodiments that derive the parameters at the encoder side without having to code extra bits into the bitstream, thereby introducing extra burden on the bitrate. However, embodiments may include one or more syntax elements that are inserted into the signaling.

[0063]

[90] In at least one embodiment, an affine model can be used to calculate affine MVs for each 4x4 sub-block of a CU. This produces a set of reconstructed samples at Vref(MV), which can be different for each sub-block, with its own motion vector. An example of a 16x16 CU using an affine model is provided in Figure 12.

[0064]

[91] Figure 12 shows an example of different L-shaped neighborhood samples in 16 × 16 CUs using the affine model.

[0065]

[92] The current top-left sub-block ("current sub-blk0") is associated with MV0, and its corresponding reconstructed sample is located at Vref(MV0) in the neighborhood of the reference block ("reference blk0") in the reference picture, while MV i The current bottom left sub-block (current sub-block i ), the reconstructed sample is a reference block ("reference blk") in the reference picture.i ") near Vref(MV i ) are placed at Vref(MV0) and Vref(MV i ) can generate different L-shapes around the associated reference block. As described in more detail below, the LIC parameter derivation can be adapted to an affine model according to various embodiments.

[0066]

[93] Figure 13 shows an example embodiment of a method, e.g., in an encoder, for determining the LIC flag of a CU in an inter slice. As can be seen, additional inter modes for utilizing the LIC tool include affine synthesis mode and affine AMVP mode. In the affine synthesis mode, the LIC flag is inferred from neighboring blocks. In the affine AMVP mode, a performance or quality metric, e.g., a rate-distortion search, can be evaluated for the current CU. In the example of Figure 13, such evaluation includes deriving LIC parameters in step 1340, performing motion compensation in step 1350, applying LIC in step 1360, and calculating a cost, e.g., a rate-distortion cost, in step 1370. As shown in Figure 13, these steps are repeated in a loop over the LIC flags, i.e., LIC flag on and off. Thus, each possible LIC flag value is evaluated from a rate-distortion perspective for the current CU in both affine AMVP mode and AMVP mode. Once the LIC flag is determined, the flag can be signaled in the bitstream, for example, using existing syntax for signaling LIC usage, thereby avoiding the added overhead of adding syntax, or by adding syntax if appropriate for a particular environment, situation, or application.

[0067]

[94] According to at least one embodiment, similar to the affine AMVP mode, the LIC flag of the current CU can be derived from MVP candidates and / or from neighboring MVPs.

[0068]

[95] At least one embodiment can derive the LIC parameters for the entire CU using only one motion vector. For example, this motion vector is the motion vector MV0(v 0x ,v 0y At least one embodiment can include taking a set of reconstructed samples arranged in the same L-shape around the reference block and calculating unique LIC parameters, which are then applied to all sub-blocks in the current block.

[0069]

[96] An embodiment of a method for deriving LIC parameters of an affine model using one motion vector is shown in Figure 14. In step 300, the LIC flag is evaluated. If the LIC flag is false, a motion compensation process is performed in step 303A, followed by a performance metric such as a rate-distortion (RD) analysis in step 305. A motion vector (MV) is provided in step 305 based on the metric analysis results. If the LIC flag is true in step 300, processing continues with step 301, where the motion vector MV0(v 0x ,v 0y ) is calculated via Equation 1. Step 301 is followed by step 302, where LIC parameters are derived by minimizing the local distortion according to Equation 3 using a set of corrected samples at Vref(MV0). Step 302 is followed by block 306, which includes a motion compensation process at 303B (the same motion compensation process as described above with respect to 303A), followed by applying LIC at 304 based on the LIC parameters derived at 302. A loop over 306 is performed for each sub-block, where the LIC parameters obtained at 302 are utilized at 304 to correct for illumination changes for each sub-block. After loop over 306 has been performed for all sub-blocks, i.e., after running through steps 303B and 304, processing continues at step 305 as described above.

[0070]

[97] As explained, the example embodiment shown in Figure 14 may include one motion vector corresponding to the first sub-block. However, in at least one embodiment, the one motion vector may be calculated based on any sub-block other than the first sub-block, such as the central sub-block, or may be calculated for neighboring blocks, etc. In the example of one MV at the "center" of a coding unit, the one MV may be obtained or calculated based on one or more sub-blocks depending on the selected embodiment. Specific examples of embodiments obtaining one MV at the center of a CU are shown in Figures 25, 26, and 27.

[0071]

[98] For example, in Figure 25, the MV associated with the sub-block indicated by the dashed circle within the sub-block may be the "center" or central MV based on the selection of the sub-block that contains the point at the center of the coding unit, i.e., point (W / 2, H / 2), where W and H are the width and height of the coding unit, respectively. As another example, the MV at the "center" may be obtained by combining multiple MVs in the vicinity of the center, i.e., point (W / 2, H / 2). As an example, the embodiment shown in Figure 26 combines the MVs of four sub-blocks around point (W / 2, H / 2), labeled MV1, MV2, MV3, and MV4 in Figure 26, by averaging the four MVs to provide one motion vector that can be considered the center or central MV.

number

number

[0072]

[99] Embodiments such as those described above can address different L-shapes around a reference block that may be generated due to each sub-block having its own affine motion vector. However, LIC parameters calculated via only one motion vector may not be optimal for all of the sub-blocks because the illumination variation between each sub-block and its reference block may be different. Generally, at least one embodiment derives LIC parameters for the entire CU by considering multiple motion vectors as a benchmark. More specifically, instead of a perfect L-shape around a reference block, a "pseudo-L-shape" generated by several potentially disconnected patches is used. A pseudo-L-shape may be generated because multiple motion vectors may reference each reconstructed sample that does not form a continuous L-shape data configuration.

[0073]

[0100] For example, as shown in FIG. 15, for the current top-left sub-block ("current sub-blk0"), the corresponding reconstructed sample in Vref(MV0) above the reference block MV0 is used as the top-left corner patch of the "pseudo-L-shaped" sub-block. Next, the sub-blocks located in the first row of the CU (corresponding to reference blocks MV0, MV1, MV2, and MV3) generate the top patch of the "pseudo-L-shaped" sub-block (Vref(MV0), Vref(MV1), Vref(MV2), Vref(MV3)). Furthermore, the left patch of the "pseudo-L-shaped" sub-blocks in the first column (corresponding to reference blocks MV0, MV4, and MV5) are formed by using the reconstructed sample on the left side of the reference block of the sub-block. An additional "Vref(MV0)" is formed to the left of MV0 in the reference block. Note that Vref(MV5) is shown as twenty blocks because the sub-blocks in the picture have the same motion vector MV5. Using the "pseudo-L" the LIC parameters can be derived and then applied to the entire CU, for example.

[0074]

[0101] For example, one approach to minimizing local distortion using multiple motion vectors and choosing LIC parameters can be based on minimizing the MSE difference, similar to that described above with respect to Equation 3, modified for multiple motion vectors in Equation 5.

number

[0075]

[0102] At least one embodiment of a method for deriving LIC parameters using multiple motion vectors is shown in Figure 16. In step 401, motion vectors for sub-blocks in the first row and first column are generated via Equation 1 by looping over the sub-blocks in the first row and first column. In step 402, if the LIC flag is true, then LIC parameters may be derived by minimizing local distortion using Equation 4 using a "pseudo-L-shape" around the reference block. Processing in steps 403 and 404 proceeds in a similar manner as described above with respect to steps 303 and 304 of Figure 14.

[0076]

[0103] In at least one embodiment, to reduce complexity, only two motion vectors, e.g., top and left separately, can be used to generate a "pseudo-L" shape: the first motion vector is from the sub-block in the top row, e.g., the middle position of the first row, and the second motion vector is from the sub-block in the first column, e.g., the middle position of the first column.

[0077]

[0104] In at least one embodiment, a "pseudo-L" shape may be generated using a subset of a plurality of motion vectors associated with the first row and / or first column of sub-blocks of a coding unit. For example, the pseudo-L shape may be formed based on reconstructed samples obtained using motion vectors associated with one or more of the subset of sub-blocks in the first row of sub-blocks or the subset of sub-blocks in the first column of sub-blocks. That is, a first set of motion vectors may include motion vectors associated with each sub-block in the first row of sub-blocks of the coding unit, and a second set of motion vectors may include motion vectors associated with each sub-block in the first column of sub-blocks included in the coding unit. The pseudo-L shape may be generated based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, i.e., reconstructed samples generated based on the first subset, the second subset, or both the first and second subsets. In the above example involving sub-blocks in the middle of the first row and / or the middle of the first column of sub-blocks, the described first and / or second subsets of motion vectors may each include one motion vector associated with each sub-block in the middle of the first row and / or the first column.

[0078]

[0105] In at least one embodiment described above, only one pair of LIC parameters (a and b) is derived and used for the entire CU. In at least one other embodiment, the LIC tool for the affine model can include multiple or multiple sets of LIC parameters, e.g., multiple pairs of LIC parameters for a linear LIC model can be generated for use with an affine motion model to more accurately correct predicted samples. Because the LIC parameters can be derived at the decoder side as well, adding multiple sets of LIC parameters does not require encoding additional syntax bits into the bitstream and does not represent an additional burden on the bitrate.

[0079]

[0106] As an example, FIG. 17 shows another embodiment in which 1) some sub-blocks can be grouped into larger sub-blocks (referred to herein as "LIC groups"), 2) the CU can be divided into multiple LIC groups, for example, four LIC groups (top left, top right, bottom left, bottom right), 3) a pair of LIC parameters associated with each LIC group can be derived, and 4) during motion compensation, illumination changes of samples in a sub-block are corrected using the corresponding LIC parameters of the LIC group to which the sub-block belongs.

[0080]

[0107] One or more embodiments described herein may also be applied when calculating a pair of LIC parameters for each LIC group. In an embodiment, an L-shape around the current LIC group (Vcur_tl / Vcur_tr / Vcur_bl / Vcur_br) is applied instead of an L-shape around the current CU (Vcur). In an embodiment, a CU may be divided into a number of LIC groups greater than or less than four. In an embodiment, a CU may be divided into a different number of LIC groups depending on the size of the CU. For example, if the size of a CU is 16x16 or 32x32, four LIC groups are generated, and if it is 64x64, the CU may be divided into eight LIC groups.

[0081]

[0108] Another example of some embodiments of an encoder is shown in Figure 18. In Figure 18, video information, such as video data including picture portions, is processed at step 1810 based on an affine motion model to generate motion compensation information. At step 1820, a local illumination compensation (LIC) model is obtained, e.g., parameters of a linear model are derived according to one or more aspects or embodiments as described herein. Then, at step 1830, the video information is encoded to generate coded video information based on the motion compensation information and the LIC model.

[0082]

[0109] An example embodiment of a portion or part of a decoder according to one or more aspects of the present disclosure is shown in Figure 19. In Figure 19, coded video information, such as video data including coded picture portions, is processed in step 1910 based on an affine motion model to generate motion compensation information. In step 1920, a local illumination compensation (LIC) model is obtained, e.g., parameters of a linear model are derived according to one or more aspects or embodiments as described herein. Then, in step 1930, the video information is decoded to generate decoded picture portions based on the motion compensation information and the LIC model.

[0083]

[0110] Figures 20, 21, and 22 show additional examples of embodiments of the decoder portion. In Figure 20, in step 2010, motion compensation processing is performed for an inter-mode coded current CU. Step 2020 determines whether the inter mode is a unified mode. If so, in step 2030, the LIC flag is inferred as described herein. If not, in step 2080, the LIC flag is decoded. Both steps 2020 and 2080 are followed by step 2040, where the state of the LIC flag is tested. If the LIC flag is false, LIC is not applied, and processing for LIC ends in step 2070. If the LIC flag is determined to be true in step 2040, LIC parameters are derived in step 2050. In step 2060, LIC is applied based on all parameters of the current CU, after which processing ends in step 2070.

[0084]

[0111] In Figure 21, the LIC flag of the current CU is tested in step 2110. If it is false, LIC processing is disabled or not activated for the current CU, and motion compensation processing is performed in step 2170, followed by LIC-related processing ending in step 2160. If the LIC flag is true in step 2110, a motion vector corresponding to one sub-block, e.g., MV0 corresponding to the first sub-block, is calculated, identified, or obtained. Next, in step 2130, the motion vector and associated reference block, e.g., V ref The LIC model is obtained by deriving LIC parameters based on (MV0). Next, motion compensation is applied in step 2140, and LIC is applied in step 2150. Steps 2140 and 2150 are repeated as a loop over all sub-blocks to apply the LIC model to all sub-blocks based on parameters determined from one motion vector. After looping over all sub-blocks is complete, the process ends in step 2160.

[0085]

[0112] In FIG. 22 , the LIC flag of the current CU is tested in step 2210. If false, LIC processing is disabled or not activated for the current CU, and motion compensation processing is performed in step 2270, followed by LIC-related processing terminating in step 2260. If the LIC flag is true in step 2210, motion vectors corresponding to a plurality of sub-blocks, e.g., a subset of sub-blocks in the first row and / or a subset of sub-blocks in the first column, are calculated, determined, or obtained in step 2220 by looping over subsets included in one or more subsets of sub-blocks. Step 2220 is followed by step 2230, in which LIC parameters are derived based on a plurality of reference blocks associated with the plurality of motion vectors. As mentioned above, the plurality of reference blocks may have a data structure referred to as a pseudo-L-shape. Next, motion compensation processing is performed in step 2240, and LIC processing is applied in step 2250, where LIC is based on the LIC parameters determined in step 2230. Steps 2240 and 2250 are repeated in a loop over all sub-blocks to apply the LIC model to all sub-blocks. After looping over all sub-blocks is complete, processing ends in step 2260.

[0086]

[0113] One or more embodiments of deriving the LIC parameters proposed for the affine model as described herein can also be performed for other sub-CU-based motion vector predictions (i.e., "sub-block-based temporal merging candidates": alternative temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), and sub-block-based temporal motion vector prediction (SbTMVP)) when the LIC tool is activated.

[0087]

[0114] Additionally, although various aspects and embodiments have been described with respect to video information processing in a linear model associated with inter-mode or inter-coding and LIC, one or more aspects, embodiments, and features may also be applied to intra-mode or intra-coding. For example, in the case of a cross-component linear model (CCLM) in intra-coding, luma samples are used to predict corresponding chroma samples based on a linear model, and parameters of the linear model may be derived or obtained according to one or more aspects described herein with respect to LIC.

[0088]

[0115] Also, if there are several sub-blocks within one CU, each sub-block may have a different corresponding referenced sub-block, and therefore one or more of the described embodiments may also be applied to such a case.

[0089]

[0116] The embodiments may be performed by implemented computer software, such as by the processor 1010 of the system 1000 of FIG. 3 , by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 included in the example system 1000 shown in FIG. 3 may be of any type appropriate to the technical environment and may be implemented using any suitable data storage technology, such as, by way of non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type appropriate to the technical environment and may include, by way of non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0090]

[0117] The embodiments and aspects described herein may be embodied in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even when discussed only in the context of one embodiment (e.g., discussed only as a method), the discussed feature implementation may also be embodied in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. For example, a method may be implemented in an apparatus such as, for example, a processor, commonly referred to as a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices such as computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users.

[0091]

[0118] References to "one embodiment," "embodiment," "one implementation," or "embodiment," and other variations thereof, mean that the particular features, structures, characteristics, etc. described in connection with that embodiment are included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment," "in an embodiment," "in an implementation," or "in an embodiment," and any other variations thereof, in various places throughout this document do not necessarily all refer to the same embodiment.

[0092]

[0119] Additionally, this document may refer to "determining" various pieces of information. Determining information may include, for example, one or more of inferring information, calculating information, predicting information, or retrieving information from memory.

[0093]

[0120] Additionally, this document may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or inferring information.

[0094]

[0121] Additionally, this document may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" typically involves, in any event, operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or erasing information.

[0095]

[0122] As will be apparent to those skilled in the art, implementations can generate a wide variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bitstream of the described implementations. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. The signal can be transmitted over a wide variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium, for example, a non-transitory computer-readable medium.

[0096]

[0123] In general, at least one embodiment may include a method for encoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model.

[0097]

[0124] At least one embodiment may include an apparatus for encoding video information, the apparatus comprising one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0098]

[0125] At least one embodiment may include a method for decoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the video information based on the motion compensation information and the local illumination compensation model.

[0099]

[0126] At least one embodiment may include an apparatus for decoding video information, the apparatus comprising one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0100]

[0127] At least one embodiment may include a method or apparatus described herein, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one motion vector included in the motion compensation information.

[0101]

[0128] An example of an embodiment of the method may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the plurality of motion vectors including a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and estimating the local distortion based on the set of reconstructed samples.

[0102]

[0129] and decoding the video information based on the motion compensation information and the local illumination compensation model, wherein the video information may include a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, wherein the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks, and wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and wherein determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0103]

[0130] Another example of an embodiment of the method may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located in an upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and estimating local distortion based on the set of reconstructed samples.

[0104]

[0131] Another example of an embodiment of the method may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating local distortion based on the set of reconstructed samples.

[0105]

[0132] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and a respective one of each sub-block included in the first row of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L-shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0106]

[0133] Another example of an embodiment of a method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. and a local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0107]

[0134] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located in an upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0108]

[0135] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0109]

[0136] An example embodiment of the apparatus may include one or more processors, wherein the one or more processors are configured to process video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, wherein the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on the group of motion vectors, and estimate a local distortion based on the set of reconstructed samples.

[0110]

[0137] Another example of an embodiment of the apparatus may include one or more processors configured to process video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, wherein the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. and a vector, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0111]

[0138]

[0013] Another example of an embodiment of the apparatus may include one or more processors, wherein the one or more processors are configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block among the plurality of sub-blocks located at an upper-left corner of the coding unit, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0112]

[0139] Another example of an embodiment of the apparatus may include one or more processors, wherein the one or more processors are configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of a coding unit, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0113]

[0140] Another example of an embodiment of the apparatus may include one or more processors configured to process video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtain a local illumination compensation model; and encode the video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and a respective one of each sub-block included in the first row of sub-blocks, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on the group of motion vectors, and estimate a local distortion based on the set of reconstructed samples.

[0114]

[0141] Another example of an embodiment of the apparatus may include one or more processors, wherein the one or more processors process video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtain a local illumination compensation model; and encode the video information based on the motion compensation information and the local illumination compensation model; the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks; and to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; and to determine the at least one model parameter, the one or more processors are further configured to: obtain a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; and estimate the local distortion based on the set of reconstructed samples.

[0115]

[0142]

[0013] Another example of an embodiment of the apparatus may include one or more processors configured to: process video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each of the plurality of sub-blocks; obtain a local illumination compensation model; and encode the video information based on the motion compensation information and the local illumination compensation model; wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit; and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate a local distortion based on the set of reconstructed samples.

[0116]

[0143] Another example of an embodiment of the apparatus may include one or more processors, wherein the one or more processors process video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtain a local illumination compensation model; and encode the video information based on the motion compensation information and the local illumination compensation model. and to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate the local distortion based on the set of reconstructed samples.

[0117]

[0144] An example embodiment of the method may include processing coded video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the plurality of motion vectors including a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and estimating the local distortion based on the set of reconstructed samples.

[0118]

[0145] Another example of an embodiment of a method includes processing coded video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the coded video information based on the motion compensation information and the local illumination compensation model; the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks, and obtaining a local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0119]

[0146] Another example of an embodiment of the method may include processing coded video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located in an upper-left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and estimating the local distortion based on the set of reconstructed samples.

[0120]

[0147] Another example of an embodiment of the method may include processing coded video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0121]

[0148] Another example of an embodiment of the method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and a respective one of each sub-block included in the first row of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L-shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0122]

[0149] Another example of an embodiment of a method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. a set of motion vectors, and obtaining a local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0123]

[0150] Another example of an embodiment of the method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located in an upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0124]

[0151] Another example of an embodiment of the method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0125]

[0152] An example embodiment of the apparatus may include one or more processors, wherein the one or more processors are configured to process the coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, wherein the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on the group of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0126]

[0153] Another example of an apparatus embodiment may include one or more processors configured to process coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks in the first column of sub-blocks. and a motion vector of the first set of motion vectors, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0127]

[0154]

[0010] Another example of an embodiment of the apparatus may include one or more processors, wherein the one or more processors are configured to process the coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0128]

[0155] Another example of an embodiment of the apparatus may include one or more processors, wherein the one or more processors are configured to process the coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and wherein, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0129]

[0156]

[0013] Another example of an embodiment of the apparatus may comprise one or more processors configured to process coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtain a local illumination compensation model; and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and a respective one of each sub-block included in the first row of sub-blocks, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on the group of motion vectors; and estimate the local distortion based on the set of reconstructed samples.

[0130]

[0157] Another example of an embodiment of the apparatus may include one or more processors configured to process coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. and a motion vector of the first set of motion vectors, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0131]

[0158]

[0010] Another example of an embodiment of the apparatus may comprise one or more processors configured to process coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtain a local illumination compensation model; and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate a local distortion based on the set of reconstructed samples.

[0132]

[0159]

[0013] Another example of an embodiment of the apparatus may comprise one or more processors configured to process coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtain a local illumination compensation model; and decode the coded video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate a local distortion based on the set of reconstructed samples.

[0133]

[0160] In a variation of at least one embodiment described herein that includes a coding unit having a plurality of subblocks including a first row of subblocks and a first column of subblocks, the first subset of motion vectors includes a first motion vector corresponding to a subblock located at a middle position in the first row of subblocks, and the second subset of motion vectors includes a second motion vector corresponding to a subblock located at a middle position in the first column of subblocks.

[0134]

[0161] In a variation of at least one embodiment described herein that includes multiple sub-blocks of a coding unit, the multiple sub-blocks may be divided into multiple groups of sub-blocks, and model parameters of the local illumination compensation model may be determined for each of the multiple groups of sub-blocks, and encoding or decoding based on the local illumination compensation model may include processing video information associated with each group of sub-blocks using each of the at least one model parameter determined for each group.

[0135]

[0162] In a variation of at least one embodiment described herein that includes grouping sub-blocks, at least one of the first number of sub-blocks in each group of sub-blocks and the second number of groups formed for the coding unit is selected based on the size of the coding unit.

[0136]

[0163] A variation of at least one embodiment described herein that involves encoding video information may include identifying a rate-distortion metric associated with application of a local illumination compensation model and providing a syntax element within the encoded video information having a value based on the rate-distortion metric.

[0137]

[0164] In a variation of at least one embodiment described herein that includes at least one model parameter of a linear model, the at least one model parameter may include a pair of first and second model parameters corresponding to a scaling factor and an offset.

[0138]

[0165] Another example of an embodiment may include a computer program product including computational instructions that, when executed by one or more processors, perform any of the methods described herein.

[0139]

[0166] Another example of an embodiment may include a non-transitory computer-readable medium having stored thereon executable program instructions that cause a computer to execute instructions to perform any of the methods described herein.

[0140]

[0167] Another example embodiment may include a bitstream formatted to include encoded video information generated by the methods described herein.

[0141]

[0168] Variations of the bitstream embodiments described herein may include encoded video information including an indicator indicating encoding of the video information based on a local illumination compensation model and an affine motion model, and picture information encoded based on a local illumination compensation model and an affine motion model.

[0142]

[0169] Another example of an embodiment may include a device comprising an apparatus described herein and at least one of: (i) an antenna configured to receive a signal, the signal including data representing video information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the video information; and (iii) a display configured to display an image from the video information.

[0143]

[0170] At least one embodiment may include a method or apparatus described herein, wherein a selected one of the plurality of sub-blocks is positioned at a middle position in a top row of the sub-blocks, and a third sub-block of the plurality of sub-blocks is positioned at a middle position in a left column of the sub-blocks.

[0144]

[0171] At least one embodiment may include a method or apparatus described herein, wherein at least one model parameter of the linear model includes a pair of first and second model parameters corresponding to a scaling factor and an offset, and processing the video information based on the linear model includes processing a plurality of sub-blocks of the current coding unit based on the linear model using the scaling factor and the offset.

[0145]

[0172] At least one embodiment may include a method or apparatus described herein, wherein at least one model parameter of the linear model includes a pair of first and second model parameters corresponding to a scaling factor and an offset, and processing the video information based on the linear model includes dividing a plurality of sub-blocks of a current coding unit into a plurality of groups of sub-blocks, determining a pair of model parameters for each of the plurality of groups of sub-blocks to generate a plurality of pairs of parameters, and processing each of the groups of sub-blocks of the video information based on the linear model using a respective one of the plurality of pairs of parameters.

[0146]

[0173] At least one embodiment may include a method or apparatus for processing multiple groups of sub-blocks as described herein, wherein at least one of a first number of sub-blocks in each group of sub-blocks and a second number of sub-groups formed in the current coding unit is selected based on the size of the current coding unit.

[0147]

[0174] At least one embodiment may include a method or apparatus for encoding video information as described herein, further comprising determining a rate-distortion metric based on application of a local illumination compensation model, and providing a syntax element within the encoded video information having a value based on the rate-distortion metric.

[0148]

[0175] At least one embodiment may include a bitstream formatted to include coded video information, the coded video information including an indicator indicating coding of the video information based on a local illumination compensation model and an affine motion model, and picture information coded based on the local illumination compensation model and the affine motion model.

[0149]

[0176] At least one embodiment may include a device including an apparatus according to any embodiment described herein and further including at least one of: (i) an antenna configured to receive a signal, the signal including data representing video information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the video information; and (iii) a display configured to display an image from the video information.

[0150]

[0177] In at least one embodiment, when the LIC tool is activated, the LIC parameters proposed for the affine model are derived for other sub-CU-based motion vector predictions (i.e., "subblock0-based temporal merging candidates": Alternative Temporal Motion Vector Prediction (ATMVP), Spatial-Temporal Motion Vector Prediction (STMVP), and Subblock-based Temporal Motion Vector Prediction (SbTMVP)).

[0151]

[0178] At least one embodiment involves using predictive coding and / or decoding to enable compensation.

[0152]

[0179] One or more embodiments described herein for deriving the proposed LIC parameters may be applied to the derivation of other parameters, such as scaling, offset, and / or selection parameters.

[0153]

[0180] At least one other embodiment may include modifying pixel values ​​of the predicted block (the block to which the motion vector points), which may be by a variety of filters, for example, illumination compensation and / or color compensation as described herein.

[0154]

[0181] At least one other embodiment may include predictor blocks based on motion vectors generated by various modes, including predictors determined by intra-coding.

[0155]

[0182] At least one embodiment includes enabling a block-based local illumination compensation (LIC) tool when affine motion prediction is utilized for inter-mode coded coding units (CUs).

[0156]

[0183] At least one embodiment includes activating the LIC tool for inter-mode coded CUs that utilize an affine model to represent motion vectors.

[0157]

[0184] At least one embodiment may include, for example, activating a LIC tool for inter-mode coded CUs that use an affine model, including LIC flag determination for affine AMVP and affine synthesis, and corresponding LIC parameter derivation rules.

[0158]

[0185] At least one embodiment includes how to activate the LIC tool and create rules related to inter-mode coding using affine motion prediction in a way that provides good compensation efficiency (rate-distortion performance) while minimizing the increase in coding design complexity.

[0159]

[0186] At least one embodiment comprises: Determining a LIC flag for an inter-mode coded CU using affine motion prediction, where in the case of affine AMVP, an iterative loop over LIC tools can be provided to determine the LIC flag, and the LIC flag can be signaled to the bitstream; in other cases, in affine integration, the LIC flag can be copied from a neighboring block, similar to the motion information copying in integration mode, e.g., determining the LIC flag based on at least one affine control point associated with the neighboring block. [Encoder / Decoder]

[0160]

[0187] At least one embodiment comprises: Based on the determination that the LIC flag is true, creating a rule for deriving the corresponding LIC parameters, which typically includes consideration of several aspects, such as using one motion vector for the first sub-block or multiple motion vectors for sub-blocks in the first row / column; as another example, generating a unique pair of LIC parameters for the entire CU; as another example, deriving multiple pairs of LIC parameters. [Encoder / Decoder]

[0161]

[0188] At least one embodiment improves the correction of block prediction samples based on consideration of illumination variations, for example, by enabling LIC tools for inter-mode coded CUs using affine motion prediction.

[0162]

[0189] The various embodiments described herein provide the advantage of improved coding efficiency.

[0163]

[0190] At least one embodiment is based on the inventors' recognition that approaches involving LIC being deactivated in an affine model cannot fully incorporate the performance potential due to block prediction samples via affine motion compensation not being corrected by taking illumination variations into account.

[0164]

[0191] In at least one embodiment, the LIC parameters can be derived in accordance with one or more embodiments that derive the parameters at the decoder side without requiring additional bits to be coded into the bitstream at the encoder side, thereby introducing no additional burden on the bitrate.

[0165]

[0192] At least one embodiment may include inserting signaling syntax elements that allow the decoder to derive parameters, such as LIC parameters, based on the embodiment used at the encoder side.

[0166]

[0193] In at least one embodiment, the method applied at the decoder may be selected based on one or more syntax elements inserted into the signaling.

[0167]

[0194] In at least one embodiment, a bitstream or signal includes one or more of the described syntax elements or variations thereof.

[0168]

[0195] At least one embodiment includes creating, transmitting, receiving, and / or decoding a bitstream or signal that includes one or more of the described syntax elements or variations thereof.

[0169]

[0196] A TV, set-top box, mobile phone, tablet, or other electronic device implementing any one of the described embodiments.

[0170]

[0197] A TV, set-top box, mobile phone, tablet, or other electronic device that implements any one of the described embodiments and displays the generated images (e.g., using a monitor, screen, or other type of display).

[0171]

[0198] A TV, set-top box, mobile phone, tablet, or other electronic device that tunes to a channel (e.g., using a tuner) to receive a signal containing encoded images and processes the images according to any one of the described embodiments.

[0172]

[0199] A TV, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image wirelessly (e.g., using an antenna) and processes the image according to any one of the described embodiments.

[0173]

[0200] Embodiments may include a computer program product comprising program code that, when executed, performs a method according to any embodiment described herein.

[0174]

[0201] Embodiments may include a computer readable medium storing program code that, when executed, performs a method according to any embodiment described herein.

[0175]

[0202] Various other generalized and specific inventions and claims are also supported and contemplated throughout this disclosure.

Claims

1. 1. A video decoding device, comprising: obtaining an affine motion model associated with the block, the affine motion model comprising a plurality of control point motion vectors; obtaining local illumination compensation information associated with the block based on the plurality of control point motion vectors of the affine motion model associated with the block; decoding the block based on the affine motion model and the local illumination compensation information; 1. A video decoding apparatus comprising: a processor configured to:

2. The processor: obtaining a prediction of the block based on the affine motion model; adjusting a prediction of the block based on the local illumination compensation information; and further configured to: The video decoding apparatus of claim 1 , wherein the block is decoded based on an adjusted prediction of the block.

3. The block includes a plurality of sub-blocks, The processor: obtaining a plurality of prediction sub-blocks based on the affine motion model; adjusting the plurality of prediction sub-blocks based on the local illumination compensation information; and further configured to: The video decoding apparatus of claim 1 , wherein the block is decoded based on the adjusted plurality of prediction sub-blocks.

4. The block includes a plurality of sub-blocks, The processor: determining a first motion vector associated with a second sub-block of the plurality of sub-blocks based on the plurality of control point motion vectors; deriving a set of local illumination compensation parameters based on the first motion vector and a template for the block; obtaining a plurality of prediction sub-blocks based on the affine motion model; applying the set of local illumination compensation parameters to the plurality of prediction sub-blocks; The video decoding device of claim 1 , further configured to:

5. The processor is configured to derive a set of local illumination compensation parameters based on the first motion vector and a template for the block, determining at least one sample associated with a reference block based on the first motion vector; comparing the at least one sample associated with the reference block to a template of the block; The video decoding device of claim 4 , further configured to:

6. The processor is configured to compare the at least one sample associated with the reference block with a template of the block, The video decoding apparatus of claim 5 , further configured to compare the at least one sample associated with the reference block with one or more samples included in the block.

7. The block includes a plurality of sub-blocks, The processor: determining a plurality of motion vectors associated with respective sub-blocks of the plurality of sub-blocks based on the plurality of control point motion vectors; obtaining a plurality of template samples based on the plurality of motion vectors associated with respective sub-blocks; deriving a set of local illumination compensation parameters based on the plurality of template samples and the block template; obtaining a plurality of prediction sub-blocks based on the affine motion model; applying the set of local illumination compensation parameters to the plurality of prediction sub-blocks; The video decoding device of claim 1 , further configured to:

8. The processor is configured to obtain the plurality of template samples based on the plurality of motion vectors associated with respective sub-blocks, 8. The video decoding apparatus of claim 7, further configured to determine at least one of the plurality of template samples, each corresponding to a respective one of the sub-blocks, based on the plurality of motion vectors.

9. The video decoding apparatus of claim 8 , wherein the plurality of template samples form a pseudo-L shape.

10. The processor is configured to derive the set of local illumination compensation parameters based on the plurality of template samples and the block template, 9. The video decoding apparatus of claim 8, further configured to compare each of the plurality of template samples with a corresponding sample included in the template and associated with a respective sub-block.

11. obtaining an affine motion model associated with the block, the affine motion model comprising a plurality of control point motion vectors; obtaining local illumination compensation information associated with the block based on the plurality of control point motion vectors of the affine motion model associated with the block; decoding the block based on the affine motion model and the local illumination compensation information; A method of video encoding, comprising:

12. obtaining a prediction of the block based on the affine motion model; adjusting a prediction of the block based on the local illumination compensation information; Further comprising: The method of claim 11 , wherein the block is decoded based on an adjusted prediction of the block.

13. The block includes a plurality of sub-blocks, The method comprises: obtaining a plurality of prediction sub-blocks based on the affine motion model; adjusting the plurality of prediction sub-blocks based on the local illumination compensation information; Further comprising: The method of claim 11 , wherein the block is decoded based on the adjusted prediction sub-blocks.

14. The block includes a plurality of sub-blocks, The method comprises: determining a first motion vector associated with a second sub-block of the plurality of sub-blocks based on the plurality of control point motion vectors; deriving a set of local illumination compensation parameters based on the first motion vector and a template for the block; obtaining a plurality of prediction sub-blocks based on the affine motion model; applying the set of local illumination compensation parameters to the plurality of prediction sub-blocks; The method of claim 11 further comprising:

15. Deriving a set of local illumination compensation parameters based on the first motion vector and the block template includes: determining at least one sample associated with a reference block based on the first motion vector; comparing the at least one sample associated with the reference block to a template of the block; The method of claim 14 further comprising:

16. Comparing the at least one sample associated with the reference block to a template of the block includes: The method of claim 15 , further comprising comparing the at least one sample associated with the reference block to one or more samples included in the block.

17. The block includes a plurality of sub-blocks, The method comprises: determining a plurality of motion vectors associated with respective sub-blocks of the plurality of sub-blocks based on the plurality of control point motion vectors; obtaining a plurality of template samples based on the plurality of motion vectors associated with respective sub-blocks; deriving a set of local illumination compensation parameters based on the plurality of template samples and the block template; obtaining a plurality of prediction sub-blocks based on the affine motion model; applying the set of local illumination compensation parameters to the plurality of prediction sub-blocks; The method of claim 11 further comprising:

18. Obtaining the plurality of template samples based on the plurality of motion vectors associated with respective sub-blocks includes:

18. The method of claim 17, further comprising determining at least one of the plurality of template samples, each corresponding to a respective one of the sub-blocks, based on the plurality of motion vectors.

19. The method of claim 18 , wherein the plurality of template samples form a pseudo-L shape.

20. 1. A non-transitory computer-readable medium storing executable program instructions that cause a computer executing the executable program instructions to perform a method, comprising: The method comprises: obtaining an affine motion model associated with the block, the affine motion model comprising a plurality of control point motion vectors; obtaining local illumination compensation information associated with the block based on the plurality of control point motion vectors of the affine motion model associated with the block; decoding the block based on the affine motion model and the local illumination compensation information; 1. A non-transitory computer-readable medium comprising:

Images