Motion compensation for video encoding and decoding

By incorporating an affine motion model and local illumination compensation, the patent addresses the challenge of illumination changes in video encoding, enhancing compression efficiency and quality.

JP2024164159AActive Publication Date: 2024-11-26INTERDIGITAL VC HOLDINGS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024141856
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2018-05-09
Filing Date
2024-08-23
Publication Date
2024-11-26
Estimated Expiration
2039-05-07

AI Technical Summary

Technical Problem

Existing video encoding technologies, such as those based on the HEVC standard, struggle to efficiently account for illumination changes and spatial variations in video frames, leading to suboptimal compression efficiency and quality.

Method used

Implementing an affine motion model and local illumination compensation (LIC) to generate motion compensation information, which includes determining model parameters based on motion vectors and reconstructed samples, and applying a linear model to correct for illumination changes.

Benefits of technology

Enhances video encoding efficiency by accurately accounting for illumination variations, resulting in improved compression performance and quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024164159000001_ABST
    Figure 2024164159000001_ABST
Patent Text Reader

Abstract

To provide an encoding and / or decoding method and device improving motion compensation prediction.SOLUTION: An encoding or decoding method includes: processing video information based upon a motion model including encoding units including a plurality of sub-blocks of an affine motion model etc., to generate motion compensation information; determining at least one model parameter of a linear model for lighting change of the video information based upon a plurality of motion vectors included in the motion compensation information to acquire a local lighting compensation model; and encoding or decoding the video information based upon the motion compensation information and the local lighting compensation model.SELECTED DRAWING: Figure 18
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Technical Field [1] This disclosure includes video encoding and decoding. [Background technology]

[0002] background [2] To achieve high compression efficiency, image and video coding schemes such as those defined by the High Efficiency Video Coding (HEVC) standard typically employ prediction and transform coding to exploit spatial and temporal redundancy in the video content. Typically, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and the difference between the original block and the predicted block, denoted as the prediction error or prediction residual, is often transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to the prediction, transform, quantization, and entropy coding. Recent additions to video compression techniques include various versions of the Joint Exploration Model (JEM) reference software and / or documentation being developed by the Joint Video Exploration Team (JVET). Efforts such as JEM aim to provide further improvements over existing standards such as HEVC. Summary of the Invention

[0003] overview [3] In general, an example embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the plurality of motion vectors including a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating a local distortion based on the set of reconstructed samples.

[0004] [4] Another example embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information including a first set of motion vectors associated with each one of the sub-blocks included in the first row of sub-blocks and a first set of motion vectors associated with each one of the sub-blocks included in the first column of sub-blocks. and a second set of motion vectors associated with the first set of motion vectors, and obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0005] [5] Another example of an embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks disposed in an upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating a local distortion based on the set of reconstructed samples.

[0006] [6] Another example of an embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0007] [7] Another example of an embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, where the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, where the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each respective one of the sub-blocks included in the first row of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0008] [8] Another example embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, the plurality of sub-blocks included in the coding unit including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information including a first set of motion vectors associated with each respective one of the sub-blocks included in the first row of sub-blocks and a first set of motion vectors associated with each respective one of the sub-blocks included in the first column of sub-blocks. and a second set of motion vectors associated with the first set of motion vectors, and obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0009] [9] Another example embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; The first motion vector is associated with a first sub-block of a plurality of sub-blocks located in an upper left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector, and evaluating a local distortion based on the set of reconstructed samples.

[0010]

[10] Another example of an embodiment of a method or apparatus, e.g., including one or more processors, may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating a local distortion based on the set of reconstructed samples.

[0011]

[11] In general, embodiments may include a method of encoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model.

[0012]

[12] In general, another embodiment may include an apparatus for encoding video information, the apparatus including one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0013]

[13] In general, another embodiment may include a method of decoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the video information based on the motion compensation information and the local illumination compensation model.

[0014]

[14] In general, another embodiment may include an apparatus for decoding video information, the apparatus including one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0015]

[15] In general, another embodiment may include a bitstream formatted to include coded video information, where the coded video information is coded by processing the coded video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the coded video information based on the motion compensation information and the local illumination compensation model.

[0016]

[16] In general, one or more embodiments may provide a computer readable storage medium, such as a non-volatile computer readable storage medium, having stored thereon instructions for encoding or decoding video data according to the methods or apparatus described herein. One or more of the embodiments may also provide a computer readable storage medium having stored thereon a bitstream generated by the methods or apparatus described herein. One or more of the embodiments may also provide methods and apparatus for transmitting or receiving a bitstream generated by the methods or apparatus described herein.

[0017] BRIEF DESCRIPTION OF THE DRAWINGS

[17] The present disclosure may be better understood from the consideration of the following detailed description in conjunction with the accompanying drawings. [Brief description of the drawings]

[0018] [Figure 1] 1 shows a block diagram of an example embodiment of a video encoder. [Diagram 2] 1 shows a block diagram of an example embodiment of a video decoder. [Diagram 3] 2 shows a block diagram of an example embodiment of a system for providing video encoding and / or decoding. [Figure 4]

[21] For example, an example of division of a coding tree unit (CTU) into coding units (CUs), prediction units (PUs), and transform units (TUs) in HEVC is given. [Diagram 5]

[22] gives an example of an affine motion model such as that used in the Joint Exploratory Model (JEM). [Figure 6]

[23] gives an example of a 4x4 sub-CU based affine motion vector field. [Figure 7]

[24] shows an example of the motion vector (MV) prediction process for an affine AMVP CU. [Figure 8]

[25] shows examples of motion vector prediction candidates in affine integration mode. [Figure 9]

[26] present the spatial derivation of affine motion field control points in the affine integration mode. [Figure 10]

[27] provide an example of the use of nearby samples to derive parameters associated with a Local Illumination Compensation (LIC) model. [Figure 11]

[28] We provide an example of derivation of the parameters of the LIC model in a bidirectional prediction model. [Figure 12]

[29] shows an example of neighborhood samples arranged in different L-shapes around a block of a reference picture that corresponds to a subblock of a current coding unit based on a motion vector associated with the subblock. [Figure 13] 1 illustrates some example embodiments of a video encoder. [Figure 14] 3 illustrates another example embodiment of a portion of a video encoder. [Figure 15]

[32] shows an example of neighborhood samples arranged in a pseudo-L shape around a block of a reference picture that corresponds to a subblock of a current coding unit based on a motion vector associated with the subblock. [Figure 16] 3 illustrates another example embodiment of a portion of a video encoder. [Figure 17]

[34] shows an example of neighborhood samples arranged in different L-shapes around a block of a reference picture that corresponds to a group of sub-blocks of a current coding unit based on a motion vector associated with the group of sub-blocks. [Figure 18] 3 illustrates another example of some embodiments of a video encoder. [Figure 19] 3 shows an example embodiment of a video decoder. [Figure 20] 3 shows another example of an embodiment of a portion of a video decoder. [Figure 21] 3 shows another example of an embodiment of a portion of a video decoder. [Figure 22] 3 shows another example of an embodiment of a portion of a video decoder. [Diagram 23] 4 illustrates another example of an embodiment of a video encoder. [Figure 24] 4 illustrates another example of some embodiments of a video encoder. [Diagram 25]

[42] shows an example embodiment of a motion vector at the center of a coding unit. [Figure 26]

[43] shows another example of an embodiment of a motion vector at the center of a coding unit. [Figure 27]

[44] shows another example of an embodiment of a motion vector at the center of a coding unit. [Figure 28]

[45] presents an example embodiment for determining parameters of a local illumination compensation model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019]

[46] In the various drawings, like reference symbols refer to the same or similar features.

[0020] Detailed Description

[47] Recent efforts to advance video compression techniques, such as those associated with the Joint Search Model (JEM) being developed by the Joint Video Search Team (JVET), provide advanced features and tools. For example, such development efforts include providing support for additional motion models that improve temporal prediction. One such motion model is the affine motion model, which is described in more detail below. Support is also provided for tools such as block-based local illumination compensation (LIC). The LIC tool includes applying the LIC model to predict illumination variations that may occur between a prediction block used in motion-compensated prediction and a corresponding reference block. Various aspects and embodiments described herein include motion models and tools, such as, for example, the affine motion model and the LIC tool.

[0021]

[48] ​​This document describes a wide variety of aspects, including tools, features, embodiments, models, methods, and the like. Many of these aspects are described with specificity and often in a manner that may sound limiting, at least to illustrate individual features. However, this is for clarity of description and does not limit the applicability or scope of the aspects. In fact, all of the various aspects can be combined and interchanged to provide further aspects. Additionally, aspects can be similarly combined and interchanged with aspects described in the previous figures.

[0022]

[49] The aspects described and contemplated in this document can be embodied in many different forms. Although Figures 1, 2, and 3 below and other figures provide some embodiments throughout this document, other embodiments are contemplated, and discussion of Figures 1, 2, and 3 does not limit the scope of the embodiments. At least one of the aspects generally relates to encoding and decoding video, and at least one other aspect generally relates to transmitting the generated or encoded bitstream. These and other aspects can be embodied as a method, an apparatus, or a computer-readable storage medium. For example, the computer-readable storage medium can be a non-transitory computer-readable medium. The computer-readable storage medium can store instructions for encoding or decoding video data according to any of the methods described and / or a bitstream generated according to any of the methods described.

[0023]

[50] In this application, the terms "reconstructed" and "decoded" can be used interchangeably, the terms "pixel" and "sample" can be used interchangeably, and the terms "image", "picture", and "frame" can be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while the term "decoded" is used on the decoder side.

[0024]

[51] Various methods are described above, each method including one or more steps or actions for achieving the described method. Except where a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be varied or combined.

[0025]

[52] Various methods and other aspects described herein may be used to modify one or more modules of a video encoder and / or decoder, such as, for example, the motion estimation module 170, the motion compensation module 175, and / or the motion compensation module 275 of the JVET or HEVC encoder 100 and decoder 200 shown in Figures 1 and 2, respectively. Moreover, the aspects are not limited to JVET or HEVC, but may be applied, for example, to other standards and recommendations, whether already in existence or developed in the future, and extensions of any such standards and recommendations (including JVET and HEVC). Unless otherwise noted or technically excluded, the aspects described herein may be used individually or in combination.

[0026]

[53] Various numerical values ​​may be used in this document. Any specific numerical values ​​are examples, and the described aspects are not limited to these specific values.

[0027]

[54] Figures 1, 2, and 3 respectively show block diagrams of example embodiments of an encoder 100, a decoder 200, and a system 1000. Variations of the encoder 100, the decoder 200, and the system 1000 are contemplated, but the examples described below are provided and explained for purposes of clarity without listing all possible or anticipated variations.

[0028]

[55] In Figure 1, before encoding, a video sequence may undergo pre-encoding processing (101), for example applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components in order to obtain a more compression-robust signal distribution (e.g., using histogram equalization of one of the color components). Metadata can be associated with the pre-processing and attached to the bitstream.

[0029]

[56] In the encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is divided (102) into, for example, CUs for processing. Each unit is coded, for example, using either intra mode or inter mode. If the unit is coded in intra mode, the encoder 100 performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) whether intra mode or inter mode should be used to code the unit, and indicates the intra / inter decision, for example, with a prediction mode flag. A prediction residual is calculated, for example, by subtracting (110) the prediction block from the original image block.

[0030]

[57] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., the residual is directly coded without applying the transform or quantization processes.

[0031]

[58] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the prediction block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0032]

[59] Figure 2 shows a block diagram of an example video decoder 200. In the decoder 200, the bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass described in Figure 1. As noted above, the encoder 100 in Figure 1 also generally performs video decoding as part of the encoding of the video data, e.g., to provide a further prediction basis.

[0033]

[60] In particular, the decoder's input includes a video bitstream, which may be generated by a video encoder, such as video encoder 100 of FIG. 1. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coding information. Picture partition information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partition information (235). The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined (255) with a predictive block to reconstruct an image block. The predictive block may result from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0034]

[61] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., converting from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding processing (101). The post-decoding processing may use metadata derived in the pre-encoding processing and signaled in the bitstream.

[0035]

[62] Figure 3 shows a block diagram of a system in which various aspects and embodiments can be implemented. System 1000 can be implemented as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics devices, and servers. System 1000 can be communicatively coupled to other similar systems and displays shown in Figure 3 via communication channels known to those skilled in the art, thereby implementing one or more of the various aspects described herein.

[0036]

[63] The system 1000 may include at least one processor 1010 configured to execute loaded instructions implementing one or more of the various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 may include at least one memory 1020 (e.g., volatile memory device, non-volatile memory device). The system 1000 may include storage 1040, which may include non-volatile memory including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drive, and / or optical disk drive. The storage 1040 may include, as non-limiting examples, internal storage, attached storage, and / or network accessible storage. The system 1000 may include an encoder / decoder module 1030 configured to process data to provide encoded or decoded video.

[0037]

[64] The encoder / decoder module 1030 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0038]

[65] Program code loaded into the processor 1010 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded into the memory 1020 for execution by the processor 1010. According to an embodiment, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during execution of the processes described herein, including, but not limited to, input video, decoded video, bitstreams, equations, formulas, matrices, variables, operations, and operational logic.

[0039]

[66] The system 1000 may include a communication interface 1050 to enable communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data from the communication channel 1060. The communication interface may include, but is not limited to, a modem or a network card, and the communication channel may be implemented in a wired and / or wireless medium. The various components of the system 1000 may be connected or communicatively coupled together using a variety of suitable connections, including but not limited to an internal bus, wires, and printed circuit boards.

[0040]

[67] As described in more detail below, aspects and embodiments according to the present disclosure may relate to features of the systems shown in Figures 1, 2, and 3, such as motion estimation features, e.g., module 175 of Figure 1 and motion compensation features, e.g., module 170 of Figure 1 and / or module 275 of Figure 2.

[0041]

[68] For clarity of explanation, the following detailed description describes aspects with reference to embodiments including video compression technologies such as, for example, HEVC, JEM, and / or H.266, however, the described aspects are applicable to other video processing technologies and standards.

[0042]

[69] In the HEVC video compression standard, a picture is divided into so-called coding tree units (CTUs), and each CTU is represented in the compressed domain by a coding unit (CU). Each CU is then given intra or inter prediction parameters (prediction information). To do so, it is spatially divided into one or more prediction units (PUs), and each PU is assigned some prediction information. The intra or inter coding mode is assigned at the CU level. A diagram of the division of the coding tree units into coding units, prediction units, and transform units is shown in Figure 4.

[0043]

[70] In inter-coding modes, motion-compensated temporal prediction is used to exploit the redundancy that exists between successive pictures of a video. To do so, exactly one motion vector (MV) is assigned to each PU in HEVC. Thus, in HEVC, the motion model linking a PU and its reference blocks involves only translations.

[0044]

[71] In at least one version of the Joint Search Model (JEM) developed by the JVET (Joint Video Search Team) group, CUs are no longer divided into PUs or TUs, and some motion information (prediction information in inter mode) is directly assigned to each CU. In JEM, CUs can be divided into sub-CUs, and motion vectors for each sub-CU can be calculated. In addition, several richer motion models are supported to improve temporal prediction. One of the new motion models introduced in JEM is the affine model, which basically involves using an affine model to represent motion vectors in a CU. The motion model used is shown in Figure 5. The affine model can be used to generate a motion field inside a CU for motion prediction. For example, the affine motion field includes motion vector component values ​​for each position (x,y) inside the considered block, as defined by Equation 1.

number

[0045]

[72] In practice, to keep the complexity at a reasonable level, one can calculate motion vectors for each 4x4 subblock (sub-CU) of the considered CU, for example as shown in Figure 6. Affine motion vectors are calculated from the control point motion vectors at the center positions of each subblock. As a result, the temporal prediction of a CU in the affine model involves a motion compensated prediction of each subblock with its own motion vector.

[0046]

[73] Affine motion compensation can be used in at least two ways, for example in JEM with affine AMVP (Advanced Motion Vector Prediction) or AF_AMVP and affine synthesis. -Affine AMVP (AF_AMVP) CUs in AMVP mode with a size larger than 8x8 can be predicted in affine AMVP mode. This is signaled through a flag in the bitstream. The generation of the affine motion field for that inter CU includes determining a control point motion vector (CPMV), which is obtained by the decoder through the summation of a motion vector differential and a control point motion vector prediction (CPMVP). A CPMVP is a pair of motion vector candidates taken from lists (A,B,C) and (D,E) respectively, as shown in Figure 7, which shows an example of a motion vector prediction process for an affine AMVP CU. -Affine integration In affine synthesis mode, a CU level flag indicates whether the synthesis CU uses affine motion compensation. If so, the first available neighboring CU coded in affine mode is selected from the ordered set of candidate positions (A, B, C, D, E), as shown in Figure 8, which shows motion vector prediction candidates in affine synthesis mode.

[0047]

[74] Once the first neighboring CU in affine mode is obtained, three motion vectors from the top left, top right, and bottom left corners of the neighboring CU are calculated as shown in Figure 9.

number

number

number

[0048]

[75] In JEM, block-based local illumination compensation (LIC) can also be applied. The LIC tool basically aims to predict possible illumination variations between a predicted block and a reference block used through motion compensated prediction. For each inter-mode non-coded CU, a LIC flag is signaled or implicitly derived to indicate the use of LIC. The LIC tool is based on a linear model of illumination changes using a scaling factor a and an offset b called the LIC parameters. In codecs such as JEM, the LIC tool is disabled for inter-mode coded CUs that use affine motion compensation as described above.

[0049]

[76] In inter mode, block-based local illumination compensation (LIC) allows to correct block prediction samples obtained through motion compensation by taking into account spatial or temporal local illumination variations. It is based on a model of illumination changes, such as a first-order linear model of illumination changes using a scaling factor a and an offset b. As shown in Fig. 10, the LIC parameters (a and b) can be estimated by comparing a set of reconstructed samples surrounding the current block ("current blk") located in a neighborhood Vcur with a set of reconstructed samples located in a neighborhood Vref(MV) of a reference block ("reference blk") in a reference picture (the set can have different sizes depending on the application). MV represents the motion vector between the current block and the reference block. Usually, Vcur and Vref(MV) are included in samples located in an L-shape (top, left, and top left) around the current block and the reference block, respectively.

[0050]

[77] The LIC parameters can be selected based on any of a variety of techniques. For example, the LIC parameters can be selected based on minimizing local distortion. One technique for minimizing local distortion can include minimizing the mean squared error (MSE) difference between samples at Vcur and corrected samples at Vref(MV). As an example, the LIC model can be a linear model, i.e., LIC(x)=a * x+b, where the parameters a and b can be identified based on minimizing the MSE difference as in Equation 3.

number

number

[0051]

[78] For bi-prediction, the LIC parameters (a0,b0) and (a1,b1) for Vref0 (MV0) and Vref1 (MV1), respectively, can be derived independently. In other embodiments, the derivation may be dependent or interdependent. For example, as shown in FIG. 11, for bi-prediction, the L-shapes for references 0 and 1 are compared to the L-shape of the current block to derive the LIC parameters.

[0052]

[79] If the CU is coded using integration mode, the LIC flag can be copied from the neighboring block in a similar manner to the motion information copy in integration mode; otherwise, the LIC flag of the CU is signaled to indicate whether LIC is applied or not. However, in JEM et al.'s approach, if the CU uses an affine model (affine AMVP, affine integration), the LIC flag is always set to false.

[0053]

[80] At least one embodiment includes enabling a block-based local illumination compensation (LIC) tool when affine motion prediction is utilized for an inter mode coded coding unit (CU).

[0054]

[81] At least one embodiment includes activating the LIC tool for inter mode coded CUs that utilize an affine model for representing motion vectors.

[0055]

[82] At least one embodiment may include, for example, activating a LIC tool for inter-mode coded CUs using an affine model, and may include LIC flag determination and corresponding LIC parameter derivation rules for affine AMVP and affine synthesis.

[0056]

[83] At least one embodiment includes a method for activating the LIC tool and creating rules related to inter-mode coding using affine motion prediction in a manner that provides good compression efficiency (rate-distortion performance) while minimizing the increase in coding design complexity.

[0057]

[84] At least one embodiment comprises: Determining the LIC flag for inter mode coded CUs using affine motion prediction. In case of affine AMVP, an iterative loop over LIC tools can be applied to determine the LIC flag, which is signaled in the bitstream. In other cases, in affine integration, the LIC flag can be obtained based on neighboring blocks, e.g. derived from affine control points associated with neighboring blocks, in a similar manner to the motion information copy in integration mode. [Encoder / Decoder]

[0058]

[85] At least one embodiment comprises: Based on the determination that the LIC flag is true, providing for derivation of the corresponding LIC parameters. One or more features may be involved. For example, using one motion vector of the first subblock or any other subblock (e.g., center subblock), or taking into account multiple motion vectors of subblocks in the first row / first column, e.g., all motion vectors associated with subblocks in the first row / first column or motion vectors associated with a subset of all motion vectors in the first row and / or first column. As another example, generating a pair of LIC parameters that are unique across the CU. As another example, deriving multiple pair of LIC parameters. [Encoder / Decoder]

[0059]

[86] At least one embodiment improves correction of block predicted samples based on consideration of illumination variations, for example, by enabling a LIC tool for CUs that are inter mode coded using affine motion prediction.

[0060]

[87] Various embodiments described herein provide improved coding efficiency.

[0061]

[88] At least one embodiment is based on the inventors' recognition that techniques involving LIC deactivated in an affine model cannot fully incorporate the potential performance attributable to block predicted samples via affine motion compensation due to the lack of compensation by taking into account illumination variations.

[0062]

[89] In general, at least one embodiment may include deriving the LIC parameters at the decoder side according to one or more embodiments that derive the parameters at the encoder side without the need to code extra bits into the bitstream, thereby introducing extra overhead to the bitrate. However, embodiments may include one or more syntax elements that are inserted into the signaling.

[0063]

[90] In at least one embodiment, an affine model can be used to compute an affine MV for each 4x4 subblock of a CU. This produces a set of reconstructed samples at Vref(MV), which can be different for each subblock, with its own motion vector. An example of a 16x16 CU using the affine model is provided in FIG.

[0064]

[91] In Figure 12, we show an example of different L-shaped neighborhood samples in a 16 × 16 CU using the affine model.

[0065]

[92] The current top-left sub-block ("current sub-blk0") is associated with MV0, and its corresponding reconstructed sample is located at the neighborhood Vref(MV0) of the reference block ("reference blk0") in the reference picture, while MV i The current bottom left sub-block (current sub-blk i ), the reconstructed sample is a reference block in the reference picture ("reference blki ") near Vref(MV i ) are placed at Vref(MV0) and Vref(MV i ) can generate different L-shapes around the relevant reference blocks. As described in more detail below, the LIC parameter derivation can be adapted to an affine model according to various embodiments.

[0066]

[93] Figure 13 shows an example of an embodiment of a method for determining a LIC flag for a CU in an inter slice, e.g., in an encoder. As can be seen, additional inter modes for utilizing the LIC tool include affine synthesis mode and affine AMVP mode. For affine synthesis mode, the LIC flag is inferred from neighboring blocks. For affine AMVP mode, a performance or quality metric, e.g., a rate-distortion search, can be evaluated for the current CU. In the example of Figure 13, such evaluation includes deriving LIC parameters in step 1340, performing motion compensation in step 1350, applying LIC in step 1360, and calculating a cost, e.g., a rate-distortion cost, in step 1370. As shown in Figure 13, these steps are repeated in a loop over the LIC flags, i.e., LIC flag on and off. Thus, each possible LIC flag value is evaluated from a rate-distortion perspective for the current CU in both affine AMVP mode and AMVP mode. Once the LIC flag is determined, the flag can be signaled in the bitstream, for example, using existing syntax for signaling LIC usage, thereby avoiding the added overhead associated with adding syntax, or by adding syntax if appropriate for a particular environment, situation, or application.

[0067]

[94] According to at least one embodiment, similar to the affine AMVP mode, the LIC flag of the current CU can be derived from MVP candidates and / or from neighboring MVPs.

[0068]

[95] At least one embodiment can derive the LIC parameters for the entire CU using only one motion vector. For example, this motion vector is the motion vector MV0(v 0x ,v 0y At least one embodiment may include taking a set of reconstructed samples arranged in the same L-shape around the reference block and calculating unique LIC parameters, which are then applied to all sub-blocks in the current block.

[0069]

[96] An embodiment of a method for deriving LIC parameters of an affine model using one motion vector is shown in Figure 14. In step 300, the LIC flag is evaluated. If the LIC flag is false, a motion compensation process is performed in step 303A, followed by a performance metric such as a rate-distortion (RD) analysis in step 305. A motion vector (MV) is provided in step 305 based on the metric analysis result. If the LIC flag is true in step 300, the process continues with step 301, where the motion vector MV0(v 0x ,v 0y ) is calculated via Equation 1. Step 301 is followed by step 302, where LIC parameters are derived by minimizing the local distortion according to Equation 3 using a set of corrected samples at Vref(MV0). Step 302 is followed by block 306, which includes a motion compensation process at 303B (the same motion compensation process as described above with respect to 303A), followed by applying LIC at 304 based on the LIC parameters derived at 302. A loop over 306 is performed for each sub-block, where the LIC parameters obtained at 302 are utilized at 304 to correct the illumination changes of each sub-block. After loop over 306 has been performed for all sub-blocks, i.e., after performing steps 303B and 304, processing continues to step 305 as described above.

[0070]

[97] As explained, the example embodiment shown in Figure 14 may include one motion vector corresponding to the first subblock. However, in at least one embodiment, the one motion vector may be calculated based on any subblock other than the first subblock, such as the center subblock, or may be calculated for neighboring blocks, etc. In the example of one MV at the "center" of the coding unit, the one MV may be obtained or calculated based on one or more subblocks depending on the embodiment selected. Specific examples of embodiments obtaining one MV at the center of a CU are shown in Figures 25, 26, and 27.

[0071]

[98] For example, in FIG. 25, the MV associated with the sub-block indicated by the dashed circle in the sub-block may be the "center" or central MV based on the selection of the sub-block that contains a point at the center of the coding unit, i.e., point (W / 2,H / 2), where W and H are the width and height of the coding unit, respectively. As another example, the MV at the "center" may be obtained by combining multiple MVs in the vicinity of the center, i.e., point (W / 2,H / 2). As an example, the embodiment shown in FIG. 26 combines the MVs of four sub-blocks around point (W / 2,H / 2), marked MV1, MV2, MV3, and MV4 in FIG. 26, by taking the average of the four MVs to provide one motion vector that may be considered the center or central MV.

number

number

[0072]

[99] The embodiments as described above can address different L-shapes around the reference block that may be generated due to each sub-block having its own affine motion vector. However, the LIC parameters calculated via only one motion vector may not be optimal for all of the sub-blocks because the illumination variation between each sub-block and its reference block may be different. In general, at least one embodiment derives the LIC parameters for the entire CU by considering multiple motion vectors as a benchmark. More specifically, instead of a perfect L-shape around the reference block, a "pseudo-L-shape" generated by several potential disconnected patches is used. The pseudo-L-shape may be generated because multiple motion vectors may refer to each reconstructed sample that does not form a continuous L-shape data configuration.

[0073]

[0100] For example, as shown in FIG. 15, for the current top-left sub-block ("current sub-blk0"), the corresponding reconstructed sample in Vref(MV0) above the reference block MV0 is used as the top-left corner patch of the "pseudo-L shape". Then, the sub-blocks located in the first row of the CU (corresponding to reference blocks MV0, MV1, MV2, and MV3) generate the top patch of the "pseudo-L shape" (Vref(MV0), Vref(MV1), Vref(MV2), Vref(MV3)). Furthermore, the left patch of the "pseudo-L shape" is formed by the sub-blocks in the first column (corresponding to reference blocks MV0, MV4, and MV5) by using the reconstructed sample of the left side of the reference block of the sub-block. An additional "Vref(MV0)" is formed to the left of MV0 in the reference. Note that Vref(MV5) is shown as twenty blocks because the sub-blocks in the picture have the same motion vector MV5. Using the "pseudo-L-shape", the LIC parameters can be derived and then applied, for example, to the entire CU.

[0074]

[0101] For example, one approach to minimizing local distortion with multiple motion vectors and selecting LIC parameters can be based on minimizing the MSE difference, similar to that described above with respect to Equation 3, modified for multiple motion vectors in Equation 5.

number

[0075]

[0102] At least one embodiment of a method for deriving LIC parameters using multiple motion vectors is shown in Figure 16. In step 401, motion vectors for sub-blocks in the first row and first column are generated via Equation 1 by looping over the sub-blocks in the first row and first column. In step 402, if the LIC flag is true, the LIC parameters may be derived by minimizing the local distortion using Equation 4 with a "pseudo-L-shape" around the reference block. Processing in steps 403 and 404 proceeds in a similar manner as described above with respect to steps 303 and 304 of Figure 14.

[0076]

[0103] In at least one embodiment, to reduce complexity, a "pseudo-L shape" can be generated using only two motion vectors, e.g., top and left separately: the first motion vector is from the sub-block in the top row, e.g., the middle position of the first row, and the second motion vector is from the sub-block in the first column, e.g., the middle position of the first column.

[0077]

[0104] In at least one embodiment, a "pseudo-L" shape may be generated using a subset of a plurality of motion vectors associated with a first row and / or a first column of subblocks of a coding unit. For example, the pseudo-L shape may be formed based on reconstructed samples obtained using motion vectors associated with one or more of a subset of subblocks in the first row of subblocks or a subset of subblocks in the first column of subblocks. That is, the first set of motion vectors may include motion vectors associated with each subblock in the first row of subblocks of the coding unit, and the second set of motion vectors may include motion vectors associated with each subblock in the first column of subblocks included in the coding unit. The pseudo-L shape may be generated based on reconstructed samples generated based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, i.e., the first subset, the second subset, or both the first and second subsets. In the above example involving a sub-block in the middle of the first row and / or a sub-block in the middle of the first column, the described first and / or second subsets of motion vectors may each include one motion vector associated with each sub-block in the middle of the first row and / or first column.

[0078]

[0105] In at least one embodiment described above, only one pair of LIC parameters (a and b) is derived and used for the entire CU. In at least one other embodiment, the LIC tool of the affine model can include multiple or multiple sets of LIC parameters, for example, multiple pairs of LIC parameters of a linear LIC model can be generated for use with an affine motion model to more accurately correct the predicted samples. Since the LIC parameters can be derived on the decoder side as well, adding multiple sets of LIC parameters does not require additional syntax bits to be coded in the bitstream and does not represent an additional burden on the bitrate.

[0079]

[0106] As an example, FIG. 17 shows another embodiment in which 1) some sub-blocks can be grouped into larger sub-blocks (referred to herein as “LIC groups”), 2) the CU can be divided into multiple LIC groups, for example, four LIC groups (top left, top right, bottom left, bottom right), 3) a pair of LIC parameters associated with each LIC group can be derived, and 4) during motion compensation, illumination changes of samples in a sub-block are compensated using the corresponding LIC parameters of the LIC group to which the sub-block belongs.

[0080]

[0107] One or more embodiments described herein may also be applied when calculating a pair of LIC parameters for each LIC group. In an embodiment, an L-shape around the current LIC group (Vcur_tl / Vcur_tr / Vcur_bl / Vcur_br) is applied instead of an L-shape around the current CU (Vcur). In an embodiment, a CU may be divided into a number of LIC groups greater than / less than four. In an embodiment, a CU may be divided into a different number of LIC groups depending on the size of the CU. For example, if the size of the CU is 16×16 or 32×32, four LIC groups are generated, and if it is 64×64, the CU may be divided into eight LIC groups.

[0081]

[0108] Another example of some embodiments of an encoder is shown in Fig. 18. In Fig. 18, video information, such as video data including picture portions, is processed in step 1810 based on an affine motion model to generate motion compensation information. In step 1820, a local illumination compensation (LIC) model is obtained, e.g., parameters of a linear model are derived according to one or more aspects or embodiments as described herein. Then, in step 1830, the video information is encoded to generate encoded video information based on the motion compensation information and the LIC model.

[0082]

[0109] An example of an embodiment of a portion or part of a decoder according to one or more aspects of the present disclosure is shown in Figure 19. In Figure 19, coded video information, such as video data including coded picture portions, is processed in step 1910 based on an affine motion model to generate motion compensation information. In step 1920, a local illumination compensation (LIC) model is obtained, e.g., parameters of a linear model are derived according to one or more aspects or embodiments as described herein. Then, in step 1930, the video information is decoded to generate a decoded picture portion based on the motion compensation information and the LIC model.

[0083]

[0110] Figures 20, 21, and 22 show additional examples of embodiments of the decoder part. In Figure 20, in step 2010, a motion compensation process is performed for the current CU that is inter-mode coded. Step 2020 determines whether the inter-mode is a merged mode. If so, in step 2030, the LIC flag is inferred as described herein. If not, in step 2080, the LIC flag is decoded. Both steps 2020 and 2080 are followed by step 2040, where the state of the LIC flag is tested. If the LIC flag is false, LIC is not applied and in step 2070, the process for LIC ends. If in step 2040, it is determined that the LIC flag is true, in step 2050, the LIC parameters are derived. In step 2060, LIC is applied based on all the parameters of the current CU, and then in step 2070, the process ends.

[0084]

[0111] In Figure 21, in step 2110, the LIC flag of the current CU is tested. If it is false, LIC processing is disabled or not activated for the current CU, and motion compensation processing is performed in step 2170, followed by LIC-related processing ending in step 2160. If the LIC flag is true in step 2110, a motion vector corresponding to one sub-block, e.g., MV0 corresponding to the first sub-block, is calculated, identified, or obtained. Next, in step 2130, the motion vector and the associated reference block, e.g., V ref The LIC model is obtained by deriving LIC parameters based on (MV0). Then motion compensation is applied in step 2140 and LIC in step 2150. Steps 2140 and 2150 are repeated as a loop over all sub-blocks to apply the LIC model to all sub-blocks based on parameters determined from one motion vector. After the loop over all sub-blocks is completed, the process ends in step 2160.

[0085]

[0112] In FIG. 22, in step 2210, the LIC flag of the current CU is tested. If it is false, LIC processing is disabled or not activated for the current CU, and motion compensation processing is performed in step 2270, followed by LIC-related processing ending in step 2260. If the LIC flag is true in step 2210, in step 2220, motion vectors corresponding to a plurality of sub-blocks, e.g., a subset of sub-blocks in the first row and / or a subset of sub-blocks in the first column, are calculated, determined, or obtained by looping over subsets included in one or more subsets of sub-blocks. Step 2220 is followed by step 2230, in which LIC parameters are derived based on a plurality of reference blocks associated with the plurality of motion vectors. As mentioned above, the plurality of reference blocks may have a data structure referred to as pseudo-L-shaped. Then, motion compensation processing is performed in step 2240, and LIC processing is applied in step 2250, where the LIC is based on the LIC parameters determined in step 2230. Steps 2240 and 2250 are repeated in a loop over all sub-blocks to apply the LIC model to all sub-blocks. After looping over all sub-blocks is completed, the process ends in step 2260.

[0086]

[0113] One or more embodiments of deriving the LIC parameters proposed for the affine model as described in this specification can also be performed for other sub-CU-based motion vector predictions (i.e., "sub-block-based temporal merging candidates": Alternative Temporal Motion Vector Prediction (ATMVP), Spatio-Temporal Motion Vector Prediction (STMVP), and Sub-block-based Temporal Motion Vector Prediction (SbTMVP)) when the LIC tool is activated.

[0087]

[0114] In addition, although various aspects and embodiments have been described with respect to video information processing in an inter-mode or inter-coding and a linear model associated with LIC, one or more aspects, embodiments, and features may also be applied to intra-mode or intra-coding. For example, in the case of a cross-component linear model (CCLM) in intra-coding, a luma sample is used to predict a corresponding chroma sample based on a linear model, and parameters of the linear model may be derived or obtained according to one or more aspects described herein with respect to LIC.

[0088]

[0115] Also, when there are several sub-blocks within one CU, each sub-block may have a corresponding different referenced sub-block, so one or more embodiments described may also be applied to such a case.

[0089]

[0116] The embodiments may be performed by implemented computer software, for example by the processor 1010 of the system 1000 of FIG. 3, by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 included in the example system 1000 shown in FIG. 3 may be of any type suitable for the technical environment, and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 1010 may be of any type suitable for the technical environment, and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0090]

[0117] The embodiments and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of one embodiment (e.g., discussed only as a method), the implementation of the discussed feature may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. For example, a method may be implemented in an apparatus such as, for example, a processor, which is generally referred to as a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices, such as computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0091]

[0118] References to "one embodiment," "embodiment," "one implementation," or "implementation," as well as other variations thereof, mean that the particular features, structures, characteristics, etc. described in connection with that embodiment are included in at least one embodiment. Thus, the phrases "in one embodiment," "in an embodiment," "in an implementation," or "in an embodiment" appearing in various places throughout this document, as well as any other variations thereof, do not necessarily all refer to the same embodiment.

[0092]

[0119] Additionally, this document may refer to "determining" various pieces of information. Determining the information may include, for example, one or more of inferring the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0093]

[0120] Additionally, this document may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or inferring information.

[0094]

[0121] Additionally, this document may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves, in any event, an operation such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or erasing information.

[0095]

[0122] As will be apparent to one of ordinary skill in the art, the implementations can generate a wide variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions to perform a method or data generated by one of the described implementations. For example, the signal can be formatted to carry a bit stream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information that the signal carries can be, for example, analog information or digital information. The signal can be transmitted over a wide variety of different wired or wireless links, as is known. The signal can be stored in a processor-readable medium, for example, a non-transitory computer-readable medium.

[0096]

[0123] In general, at least one embodiment may include a method for encoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model.

[0097]

[0124] At least one embodiment may include an apparatus for encoding video information, the apparatus comprising one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0098]

[0125] At least one embodiment may include a method for decoding video information, the method including processing the video information based on an affine motion model to generate motion compensation information, obtaining a local lighting compensation model, and decoding the video information based on the motion compensation information and the local lighting compensation model.

[0099]

[0126] At least one embodiment may include an apparatus for decoding video information, the apparatus having one or more processors, the one or more processors configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model.

[0100]

[0127] At least one embodiment may include a method or apparatus described herein, wherein obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one motion vector included in the motion compensation information.

[0101]

[0128] An example of an embodiment of the method may include processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the plurality of motion vectors including a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0102]

[0129] Another example of an embodiment of the method includes processing video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and decoding the video information based on the motion compensation information and the local illumination compensation model, where the video information may include a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, where the motion compensation information includes a first set of motion vectors associated with each one of the sub-blocks in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks in the first column of sub-blocks, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, where determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0103]

[0130] Another example of an embodiment of the method may include processing video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector, and evaluating a local distortion based on the set of reconstructed samples.

[0104]

[0131] Another example of an embodiment of the method may include processing the video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector, and evaluating a local distortion based on the set of reconstructed samples.

[0105]

[0132] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, where the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, where the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each respective one of the sub-blocks included in the first row of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0106]

[0133] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information, obtaining a local illumination compensation model, and encoding the video information based on the motion compensation information and the local illumination compensation model, where the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each respective one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each respective one of the sub-blocks included in the first column of sub-blocks. and obtaining a local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0107]

[0134] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located in an upper-left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating a local distortion based on the set of reconstructed samples.

[0108]

[0135] Another example of an embodiment of the method may include processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating a local distortion based on the set of reconstructed samples.

[0109]

[0136] An example embodiment of the apparatus may comprise one or more processors, where the one or more processors are configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, where the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, where the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors, and estimate a local distortion based on the set of reconstructed samples.

[0110]

[0137] Another example of an embodiment of the apparatus may include one or more processors configured to process video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information including a first set of motion vectors associated with each one of the sub-blocks in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks in the first column of sub-blocks. and a vector for estimating the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0111]

[0138] Another example of an embodiment of the apparatus may include one or more processors, where the one or more processors are configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, where the video information includes a coding unit having a plurality of sub-blocks, and the first motion vector is associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0112]

[0139] Another example of an embodiment of the apparatus may include one or more processors, where the one or more processors are configured to process the video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and encode the video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0113]

[0140] Another example of an embodiment of the apparatus may comprise one or more processors, wherein the one or more processors are configured to process the video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtain a local illumination compensation model; and encode the video information based on the motion compensation information and the local illumination compensation model, wherein, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, wherein the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each one of the sub-blocks included in the first row of sub-blocks, and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and estimate the local distortion based on the set of reconstructed samples.

[0114]

[0141] Another example of an embodiment of the apparatus may include one or more processors, the one or more processors processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model; the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with a respective one of the sub-blocks in the first row of sub-blocks and a second set of motion vectors associated with a respective one of the sub-blocks in the first column of sub-blocks; and, to obtain a local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; and,

[0115]

[0142] Another example of an embodiment of the apparatus may include one or more processors configured to process video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with each one of the plurality of sub-blocks; obtain a local illumination compensation model; and encode the video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, and where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate a local distortion based on the set of reconstructed samples.

[0116]

[0143] Another example of an embodiment of the apparatus may include one or more processors for processing video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and encoding the video information based on the motion compensation information and the local illumination compensation model. and, to obtain a local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and, to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate a local distortion based on the set of reconstructed samples.

[0117]

[0144] An example embodiment of the method may include processing the coded video information based on an affine motion model to generate motion compensation information; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the plurality of motion vectors including a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0118]

[0145] Another example of a method embodiment includes processing encoded video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the encoded video information based on the motion compensation information and the local illumination compensation model; the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in the first row of sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in the first column of sub-blocks, and obtaining a local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0119]

[0146] Another example of an embodiment of the method may include processing the coded video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector, and evaluating the local distortion based on the set of reconstructed samples.

[0120]

[0147] Another example of an embodiment of the method may include processing the encoded video information based on an affine motion model to generate motion compensation information, obtaining a local illumination compensation model, and decoding the encoded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector, and evaluating a local distortion based on the set of reconstructed samples.

[0121]

[0148] Another example of an embodiment of the method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, where the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, where the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each respective one of the sub-blocks included in the first row of sub-blocks, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and evaluating the local distortion based on the set of reconstructed samples.

[0122]

[0149] Another example of an embodiment of the method may include processing the coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, and the motion compensation information includes a first set of motion vectors associated with each respective one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each respective one of the sub-blocks included in the first column of sub-blocks. a set of motion vectors, and obtaining a local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming a pseudo-L shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and evaluating the local distortion based on the set of reconstructed samples.

[0123]

[0150] Another example of an embodiment of the method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating the local distortion based on the set of reconstructed samples.

[0124]

[0151] Another example of an embodiment of the method may include processing coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; obtaining a local illumination compensation model; and decoding the coded video information based on the motion compensation information and the local illumination compensation model, where obtaining the local illumination compensation model includes determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, and determining the at least one model parameter includes obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; and evaluating a local distortion based on the set of reconstructed samples.

[0125]

[0152] An example embodiment of the apparatus may comprise one or more processors, where the one or more processors are configured to process the coded video information based on an affine motion model to generate motion compensation information, to obtain a local illumination compensation model, and to decode the coded video information based on the motion compensation information and the local illumination compensation model, where for obtaining the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information, where the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, where the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each sub-block included in the first column of sub-blocks, where for determining the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors, and to estimate the local distortion based on the set of reconstructed samples.

[0126]

[0153] Another example of an embodiment of the apparatus may include one or more processors configured to process coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, the video information including a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information including a first set of motion vectors associated with each one of the sub-blocks in the first row of sub-blocks and a second set of motion vectors associated with each one of the sub-blocks in the first column of sub-blocks. and a motion vector of the first set of motion vectors, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0127]

[0154] Another example of an embodiment of the apparatus may comprise one or more processors, where the one or more processors are configured to process the coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the video information including a coding unit having a plurality of sub-blocks, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at a top-left corner of the coding unit, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0128]

[0155] Another example of an embodiment of the apparatus may comprise one or more processors, where the one or more processors are configured to process the coded video information based on an affine motion model to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0129]

[0156] Another example of an embodiment of the apparatus may comprise one or more processors configured to process the coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtain a local illumination compensation model; and decode the coded video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors, where the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks, where the plurality of motion vectors include a group of motion vectors associated with each sub-block included in the first row of sub-blocks and each respective one of the sub-blocks included in the first row of sub-blocks, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L shape based on the group of motion vectors; and estimate the local distortion based on the set of reconstructed samples.

[0130]

[0157] Another example of an embodiment of the apparatus may include one or more processors configured to process coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, the plurality of sub-blocks included in the coding unit including a first row of sub-blocks and a first column of sub-blocks, the motion compensation information including a first set of motion vectors associated with each respective one of the sub-blocks included in the first row of sub-blocks and a second set of motion vectors associated with each respective one of the sub-blocks included in the first column of sub-blocks. and a motion vector of the first set of motion vectors, wherein to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming a pseudo-L-shape based on one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors, and estimate the local distortion based on the set of reconstructed samples.

[0131]

[0158] Another example of an embodiment of the apparatus may comprise one or more processors configured to process coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks; obtain a local illumination compensation model; and decode the coded video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a first sub-block of the plurality of sub-blocks located at an upper-left corner of the coding unit, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector; and estimate a local distortion based on the set of reconstructed samples.

[0132]

[0159] Another example of an embodiment of the apparatus may comprise one or more processors configured to process coded video information based on a coding unit including a plurality of sub-blocks to generate motion compensation information including a plurality of motion vectors associated with a respective one of the plurality of sub-blocks, obtain a local illumination compensation model, and decode the coded video information based on the motion compensation information and the local illumination compensation model, where to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of the coding unit, where to determine the at least one model parameter, the one or more processors are further configured to obtain a set of reconstructed samples forming an L-shape based on the first motion vector, and estimate a local distortion based on the set of reconstructed samples.

[0133]

[0160] In a variation of at least one embodiment described herein that includes a coding unit having a plurality of subblocks including a first row of subblocks and a first column of subblocks, the first subset of motion vectors includes a first motion vector corresponding to a subblock located at a mid-position in the first row of subblocks, and the second subset of motion vectors includes a second motion vector corresponding to a subblock located at a mid-position in the first column of subblocks.

[0134]

[0161] In a variation of at least one embodiment described herein that includes multiple sub-blocks of a coding unit, the multiple sub-blocks may be divided into multiple groups of sub-blocks, and model parameters of the local illumination compensation model may be determined for each of the multiple groups of sub-blocks, and encoding or decoding based on the local illumination compensation model may include processing video information associated with each group of sub-blocks using each of the at least one model parameter determined for each group.

[0135]

[0162] In a variation of at least one embodiment described herein that involves grouping sub-blocks, at least one of the first number of sub-blocks in each group of sub-blocks and the second number of groups formed for the coding unit is selected based on the size of the coding unit.

[0136]

[0163] A variation of at least one embodiment described herein that involves encoding video information may include identifying a rate-distortion metric associated with application of a local illumination compensation model and providing a syntax element in the encoded video information having a value based on the rate-distortion metric.

[0137]

[0164] In a variation of at least one embodiment described herein that includes at least one model parameter of a linear model, the at least one model parameter may include a pair of first and second model parameters corresponding to a scaling factor and an offset.

[0138]

[0165] Another example of an embodiment may include a computer program product including computational instructions that, when executed by one or more processors, perform any of the methods described herein.

[0139]

[0166] Another example of an embodiment may include a non-transitory computer readable medium having executable program instructions stored thereon that cause a computer to execute instructions to perform any of the methods described herein.

[0140]

[0167] Another example of an embodiment may include a bitstream formatted to include encoded video information generated by the methods described herein.

[0141]

[0168] Variations of the bitstream embodiments described in this specification may include encoded video information including an indicator indicating encoding of the video information based on a local illumination compensation model and an affine motion model, and picture information encoded based on the local illumination compensation model and the affine motion model.

[0142]

[0169] Another example of an embodiment may include a device comprising an apparatus as described herein and at least one of: (i) an antenna configured to receive a signal, the signal including data representing video information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the video information; and (iii) a display configured to display an image from the video information.

[0143]

[0170] At least one embodiment may include a method or apparatus described herein, wherein a selected one of the plurality of sub-blocks is positioned at a middle position of a top row of the sub-blocks and a third sub-block of the plurality of sub-blocks is positioned at a middle position of a left column of the sub-blocks.

[0144]

[0171] At least one embodiment may include a method or apparatus described herein, wherein at least one model parameter of the linear model includes a pair of first and second model parameters corresponding to a scaling factor and an offset, and processing the video information based on the linear model includes processing a plurality of sub-blocks of the current coding unit based on the linear model using the scaling factor and the offset.

[0145]

[0172] At least one embodiment may include a method or apparatus described herein, wherein at least one model parameter of the linear model includes a pair of first and second model parameters corresponding to a scaling factor and an offset, and processing the video information based on the linear model includes dividing a plurality of sub-blocks of a current coding unit into a plurality of groups of sub-blocks, determining a pair of model parameters for each of the plurality of groups of sub-blocks to generate a plurality of pairs of parameters, and processing each of the groups of sub-blocks of the video information based on the linear model using a respective one of the plurality of pairs of parameters.

[0146]

[0173] At least one embodiment may include a method or apparatus for processing multiple groups of sub-blocks as described herein, where at least one of a first number of sub-blocks in each group of sub-blocks and a second number of sub-groups formed in the current coding unit are selected based on a size of the current coding unit.

[0147]

[0174] At least one embodiment may include a method or apparatus for encoding video information as described herein, further comprising determining a rate-distortion metric based on application of a local illumination compensation model, and providing a syntax element within the encoded video information having a value based on the rate-distortion metric.

[0148]

[0175] At least one embodiment may include a bitstream formatted to include encoded video information, the encoded video information including an indicator indicating encoding of the video information based on a local illumination compensation model and an affine motion model, and picture information encoded based on the local illumination compensation model and the affine motion model.

[0149]

[0176] At least one embodiment may include a device including an apparatus according to any embodiment described herein and further including at least one of: (i) an antenna configured to receive a signal, the signal including data representing video information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the video information; and (iii) a display configured to display an image from the video information.

[0150]

[0177] In at least one embodiment, when the LIC tool is activated, the LIC parameters proposed for the affine model are derived for other sub-CU-based motion vector predictions (i.e., "subblock0-based temporal merging candidates": Alternative Temporal Motion Vector Prediction (ATMVP), Spatial-Temporal Motion Vector Prediction (STMVP), and Subblock-based Temporal Motion Vector Prediction (SbTMVP)).

[0151]

[0178] At least one embodiment involves using predictive coding and / or decoding to enable compensation.

[0152]

[0179] One or more embodiments described herein for deriving the proposed LIC parameters may be applied to the derivation of other parameters, such as scaling, offset, and / or selection parameters.

[0153]

[0180] At least one other embodiment may include modifying pixel values ​​of the prediction block (the block to which the motion vector points), where the modification may be by a variety of filters, such as illumination compensation and / or color compensation as described herein.

[0154]

[0181] At least one other embodiment may include predictor blocks based on motion vectors generated by various modes, including predictors determined by intra-coding.

[0155]

[0182] At least one embodiment includes enabling a block-based local illumination compensation (LIC) tool when affine motion prediction is utilized for an inter-mode coded coding unit (CU).

[0156]

[0183] At least one embodiment includes activating the LIC tool for inter mode coded CUs that utilize an affine model to represent motion vectors.

[0157]

[0184] At least one embodiment includes, for example, activating a LIC tool for inter mode coded CUs that use an affine model, and may include LIC flag determination for affine AMVP and affine integration and corresponding LIC parameter derivation rules.

[0158]

[0185] At least one embodiment includes how to activate the LIC tool and create rules related to inter-mode coding using affine motion prediction in a manner that provides good compensation efficiency (rate-distortion performance) while minimizing the increase in coding design complexity.

[0159]

[0186] At least one embodiment comprises: determining a LIC flag for an inter mode coded CU using affine motion prediction, where in case of affine AMVP, an iterative loop over the LIC tool can be provided to determine the LIC flag, and the LIC flag can be signaled to the bitstream, and in other cases, in affine integration, the LIC flag can be copied from a neighboring block, similar to the motion information copying in integration mode, e.g., determining the LIC flag based on at least one affine control point associated with the neighboring block; determining [Encoder / Decoder]

[0160]

[0187] At least one embodiment comprises: Based on the determination that the LIC flag is true, creating a rule for deriving the corresponding LIC parameters, which typically includes consideration of several aspects, such as using one motion vector for the first sub-block or multiple motion vectors for the sub-blocks in the first row / column, as another example, a unique pair of LIC parameters is generated for the entire CU, as another example, multiple pairs of LIC parameters are derived. [Encoder / Decoder]

[0161]

[0188] At least one embodiment improves the correction of block prediction samples based on consideration of illumination variations, for example, by enabling a LIC tool for inter mode coded CUs using affine motion prediction.

[0162]

[0189] The various embodiments described herein provide the advantage of improved coding efficiency.

[0163]

[0190] At least one embodiment is based on the inventors' recognition that approaches involving LIC being deactivated in an affine model cannot fully incorporate potential performance due to block prediction samples via affine motion compensation not being corrected by taking into account illumination variations.

[0164]

[0191] In at least one embodiment, the LIC parameters can be derived in accordance with one or more embodiments that derive the parameters at the decoder side without requiring additional bits to be coded into the bitstream at the encoder side, thereby introducing no additional burden on the bitrate.

[0165]

[0192] At least one embodiment may include inserting signaling syntax elements that allow the decoder to derive parameters, such as LIC parameters, based on the embodiment used at the encoder side.

[0166]

[0193] In at least one embodiment, the method applied at the decoder may be selected based on one or more syntax elements inserted into the signaling.

[0167]

[0194] In at least one embodiment, a bitstream or signal includes one or more of the described syntax elements or variations thereof.

[0168]

[0195] At least one embodiment includes creating, transmitting, receiving, and / or decoding a bitstream or signal that includes one or more of the described syntax elements or variations thereof.

[0169]

[0196] A TV, set-top box, mobile phone, tablet, or other electronic device implementing any one of the described embodiments.

[0170]

[0197] A TV, set-top box, mobile phone, tablet, or other electronic device that implements any one of the described embodiments and displays the generated images (e.g., using a monitor, screen, or other type of display).

[0171]

[0198] A TV, set-top box, mobile phone, tablet, or other electronic device that tunes to a channel (e.g., using a tuner) to receive a signal containing encoded images and processes the images according to any one of the described embodiments.

[0172]

[0199] A TV, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image wirelessly (e.g., using an antenna) and processes the image according to any one of the described embodiments.

[0173]

[0200] Embodiments may include a computer program product comprising program code that, when executed, performs a method according to any embodiment described herein.

[0174]

[0201] Embodiments may include a computer readable medium storing program code that, when executed, performs a method according to any embodiment described herein.

[0175]

[0202] Various other generalized and specific inventions and claims are also supported and contemplated throughout this disclosure.

Claims

1. 1. A method comprising: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, deriving the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information; The video information includes a coding unit having a plurality of sub-blocks, the coding unit including a first row of sub-blocks and a first column of sub-blocks. the plurality of motion vectors includes a group of motion vectors associated with a respective one of each of the sub-blocks in a first row of the sub-blocks and each of the sub-blocks in a first column of the sub-blocks; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

2. 1. A method comprising: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

3. 1. A method comprising: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; the video information includes a coding unit having a plurality of sub-blocks; The first motion vector is associated with a first sub-block among the plurality of sub-blocks arranged in an upper left corner of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

4. 1. A method comprising: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; the first motion vector is associated with a center of a coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

5. 1. A method comprising: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors; the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; the plurality of motion vectors includes a group of motion vectors associated with a respective one of each of the sub-blocks in a first row of the sub-blocks and each of the sub-blocks in a first column of the sub-blocks; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

6. 1. A method comprising: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

7. 1. A method comprising: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; The first motion vector is associated with a first sub-block among the plurality of sub-blocks arranged in an upper left corner of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

8. 1. A method comprising: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; the first motion vector is associated with a center of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

9. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; and, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors comprised in the motion compensation information; the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks; The plurality of motion vectors includes a group of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and each sub-block in a first column of the sub-blocks, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

10. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; [0023] the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; To obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; To determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

11. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector comprised in the motion compensation information; the video information includes a coding unit having a plurality of sub-blocks; The first motion vector is associated with a first sub-block of the plurality of sub-blocks located in a top left corner of the coding unit, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

12. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of a coding unit, and to determine the at least one model parameter, the one or more processors further comprising: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

13. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors; the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; The plurality of motion vectors includes a group of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and each sub-block in a first column of the sub-blocks, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

14. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; [0023] the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; To obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; To determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

15. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; [0023] To obtain the local illumination compensation model, the one or more processors: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector comprised in the motion compensation information; The first motion vector is associated with a first sub-block of the plurality of sub-blocks located in a top left corner of the coding unit, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

16. An apparatus comprising: One or more processors wherein the one or more processors: processing the video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; encoding the video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector comprised in the motion compensation information; the first motion vector is associated with a center of the coding unit; To determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

17. 1. A method comprising: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, deriving the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors included in the motion compensation information; the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks; the plurality of motion vectors includes a group of motion vectors associated with a respective one of each of the sub-blocks in a first row of the sub-blocks and each of the sub-blocks in a first column of the sub-blocks; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

18. 1. A method comprising: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

19. 1. A method comprising: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; the video information includes a coding unit having a plurality of sub-blocks; The first motion vector is associated with a first sub-block among the plurality of sub-blocks arranged in an upper left corner of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

20. 1. A method comprising: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; the first motion vector is associated with a center of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

21. 1. A method comprising: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors; the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; the plurality of motion vectors includes a group of motion vectors associated with a respective one of each of the sub-blocks in a first row of the sub-blocks and each of the sub-blocks in a first column of the sub-blocks; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

22. 1. A method comprising: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The method includes:

23. 1. A method comprising: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; The first motion vector is associated with a first sub-block among the plurality of sub-blocks arranged in an upper left corner of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

24. 1. A method comprising: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; Including, Obtaining the local illumination compensation model comprises determining at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information; the first motion vector is associated with a center of the coding unit; Determining the at least one model parameter comprises: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The method includes:

25. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; and, to obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a plurality of motion vectors comprised in the motion compensation information; The video information includes a coding unit having a plurality of sub-blocks, the coding unit including a first row of sub-blocks and a first column of sub-blocks. The plurality of motion vectors includes a group of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and each sub-block in a first column of the sub-blocks, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

26. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; [0023] the video information includes a coding unit having a plurality of sub-blocks including a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; To obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; To determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

27. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector comprised in the motion compensation information; the video information includes a coding unit having a plurality of sub-blocks; The first motion vector is associated with a first sub-block of the plurality of sub-blocks located in a top left corner of the coding unit, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

28. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on an affine motion model to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector included in the motion compensation information, the first motion vector being associated with a center of a coding unit, and to determine the at least one model parameter, the one or more processors further comprising: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

29. 1. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on at least one of the plurality of motion vectors; the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; The plurality of motion vectors includes a group of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and each sub-block in a first column of the sub-blocks, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on said group of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

30. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; [0023] the plurality of sub-blocks included in the coding unit include a first row of sub-blocks and a first column of sub-blocks; the motion compensation information includes a first set of motion vectors associated with a respective one of each sub-block in a first row of the sub-blocks and a second set of motion vectors associated with a respective one of each sub-block in a first column of the sub-blocks; To obtain the local illumination compensation model, the one or more processors are further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on one or more of a first subset of the first set of motion vectors or a second subset of the second set of motion vectors; To determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming a pseudo-L shape based on the one or more of the first subset of the first set of motion vectors or the second subset of the second set of motion vectors; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

31. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; [0023] To obtain the local illumination compensation model, the one or more processors: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector comprised in the motion compensation information; The first motion vector is associated with a first sub-block of the plurality of sub-blocks located in a top left corner of the coding unit, and to determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

32. An apparatus comprising: One or more processors wherein the one or more processors: processing the coded video information based on a coding unit comprising a plurality of sub-blocks to generate motion compensation information comprising a plurality of motion vectors associated with respective ones of the plurality of sub-blocks; Obtaining a local illumination compensation model; decoding the encoded video information based on the motion compensation information and the local illumination compensation model; and to obtain the local illumination compensation model, the one or more processors are configured to: and further configured to determine at least one model parameter of a linear model of illumination changes in the video information based on a first motion vector comprised in the motion compensation information; the first motion vector is associated with a center of the coding unit; To determine the at least one model parameter, the one or more processors: obtaining a set of reconstructed samples forming an L-shape based on the first motion vector; estimating local distortion based on the set of reconstructed samples; and The apparatus further configured to:

33. A method according to any one of claims 2, 6, 18 and 22 or an apparatus according to any one of claims 10, 14, 26 and 30, wherein the first subset of motion vectors includes a first motion vector corresponding to a first sub-block of the plurality of sub-blocks located at a middle position in a first row of the sub-blocks, and the second subset of motion vectors includes a second motion vector corresponding to a second sub-block of the plurality of sub-blocks located at a middle position in a first column of the sub-blocks.

34. A method according to any one of claims 1 to 8 or 17 to 24 or an apparatus according to any one of claims 9 to 16 or 25 to 32, wherein the plurality of sub-blocks of a current coding unit are divided into a plurality of groups of sub-blocks, and the at least one model parameter is determined for each of the plurality of groups of sub-blocks, and wherein encoding or decoding based on the local illumination compensation model comprises processing video information associated with each group of sub-blocks using a respective one of the at least one model parameter determined for each group.

35. 35. The method or apparatus of claim 34, wherein at least one of a first number of sub-blocks in each group of sub-blocks and a second number of groups formed for a coding unit are selected based on a size of the coding unit.

36. A method or device for encoding video information according to any one of claims 1 to 16, comprising: determining a rate-distortion metric associated with application of the local illumination compensation model; providing a syntax element within the encoded video information having a value based on the rate-distortion metric; The method or apparatus further comprises:

37. 29. A method or apparatus for encoding video information according to any one of claims 1 to 4, 9 to 12, 17 to 20 or 25 to 28, wherein the affine motion model comprises an affine synthesis motion model including determining a local illumination compensation flag based on at least one affine control point associated with a neighbouring block.

38. 38. A method or apparatus according to any preceding claim, wherein the at least one model parameter of the linear model comprises a pair of first and second model parameters corresponding to a scaling factor and an offset.

39. A computer program product comprising computational instructions which, when executed by one or more processors, perform the method according to any one of claims 1 to 8, 17 to 24 and 33 to 38.

40. A non-transitory computer readable medium having stored thereon executable program instructions that cause a computer to execute instructions to perform the method of any one of claims 1-8, 17-24, and 33-38.

41. A bitstream formatted to include encoded video information produced by the method of any one of claims 1 to 8, 17 to 24 and 33 to 38.

42. 42. A bitstream as claimed in claim 41, wherein the encoded video information comprises: an indicator indicating encoding of the video information based on the local illumination compensation model and an affine motion model; picture information coded based on the local illumination compensation model and the affine motion model; A bitstream containing

43. A device, comprising: A device according to any one of claims 9 to 16 and 25 to 32, at least one of: (i) an antenna configured to receive a signal, the signal including data representing video information; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the video information; and (iii) a display configured to display an image from the video information; 16. A device comprising: