Local luminance compensation and merge slope adjustment

Abstract

In one embodiment, a video codec receives a local luminance compensation (LIC) linear model generated using samples of two adjacent, different blocks, which is a linear model of local luminance compensation (LIC), which is a linear model of local luminance compensation (LIC) generated using samples of the two adjacent, different blocks of samples of the two adjacent, different blocks of samples of the two adjacent, different blocks of samples. The linear model includes a scaling factor and a y-axis intercept. The video codec adjusts the linear model by applying an offset to adjust a scaling factor (or slope) of the linear model. The scaling factor or y-axis intercept of the adjusted linear model may be tailored to remain within a prescribed range. The offset may be selected from a set of predefined values for a syntax element signaled in a bitstream. The applied offset may be inherited from a previously encoded location. When a current block is encoded by bidirectional prediction, the linear model may be one of a first and a second linear model used to generate a predictor of the current block.

Description

TECHNICAL FIELD

BACKGROUND

[0003] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT-like coding architecture. The basic unit of compression, called a coding unit (CU), is a 2Nx2N block of pixels, which can be recursively split into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0004] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Expert Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from a reconstructed signal, which is derived from coded picture regions. The prediction residual signal is processed by a block transform. The transform coefficients are quantized and entropy coded in the bitstream along with other additional information. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal by inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by in-loop filtering to remove coding artifacts. The decoded pictures are stored in a frame buffer for predicting future pictures in the input video signal.

[0005] In VVC, a coded picture is divided into non-overlapping square regions represented by associated coding tree units (CTUs). The leaf nodes of the coding tree correspond to coding units (CUs). A coded picture can be represented by multiple slices, each of which includes an integer number of CTUs. The individual CTUs in a slice are processed in a raster scan order. Bi-predictive (B) slices can be decoded using either intra prediction or inter prediction, using up to two motion vectors and reference indices to predict the sample values of each block. Predictive (P) slices are decoded using either intra prediction or inter prediction, using up to one motion vector and reference index to predict the sample values of each block. Intra (I) slices are decoded using only intra prediction.

[0006] A CTU can be partitioned into one or more non-overlapping CUs using a quadtree (QT) with nested multi-type-tree (MTT) structure to adapt to various local motion and texture characteristics. A CU can be further partitioned into smaller CUs using one of five partition types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0007] Each CU contains one or more prediction units (PUs). A prediction unit is the basic unit for signaling predictor information together with associated CU syntax. A specified prediction process is used to predict the values of the associated pixel samples within a PU. Each CU can contain one or more transform units (TUs) for representing a prediction residual block. A transform unit (TU) consists of one transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, each TB corresponding to one residual sample block from one color component. Integer transforms are applied to the transform blocks. The level values of the quantized coefficients are entropy coded in the bitstream together with other additional information. The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined as two-dimensional sample arrays specifying monochromatic components associated with CTUs, CUs, PUs, and TUs, respectively. Thus, a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. Similar relationships exist for CUs, PUs, and TUs.

[0008] For each inter predicted CU, the motion parameters include the motion vector, the reference picture index and the reference picture list usage index, and additional information for inter predicted sample generation. The motion parameters can be signaled in an explicit or implicit manner. When a CU is coded in skip mode, the CU is associated with one PU and there is no significant residual coefficient, no coded motion vector delta or reference picture index. A merge mode is specified, in which the motion parameters of the current CU are derived from neighboring CUs, including spatial and temporal candidates, and additional plans introduced in VVC. The merge mode can be applied to any inter predicted CU. An alternative to the merge mode is the explicit transmission of the motion parameters, in which the motion vector, the corresponding reference picture index for each reference picture list and the reference picture list usage flag and other needed information are explicitly signaled in each CU.

[0009] Intra block copy (IBC) or current picture referencing (CPR) refers to encoding a block of pixels by referencing a pixel position in the same current picture using a block vector.

[0010] Bi-prediction with CU-level weight (BCW) is a coding tool for enhancing bi-prediction. BCW allows applying different weights before combining the L0 and L1 predictions to generate the bi-prediction of a CU. For a CU coded by BCW, one weight parameter w is signaled for L0 and L1 predictions, such that the bi-prediction result P bi-pred is calculated based on w. In some embodiments, the index of the weight (instead of the weight itself) is explicitly signaled for inter modes. For merge mode, the BCW index is inherited from the selected merge candidate or set to a default value, indicating equal weights.

[0011] In the advanced motion vector prediction (AMVP) mode, the motion vector predictor (MVP) candidate is determined based on template matching (TM) error to select the candidate that achieves the minimum difference between the template of the current block and the template of the reference block, and then TM is performed only for that specific MVP candidate to refine the MV. The TM process can refine that MVP candidate using an iterative search according to the adaptive motion vector resolution (AMVR) mode search pattern. SUMMARY The following summary is illustrative only and is not intended to be limiting in any regard. That is, the following summary is intended to introduce the concepts, highlights, benefits and advantages of the novel and non-obvious technology described herein. Selective embodiments will be further described in the detailed description. Thus, the following summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used in determining the scope of the claimed subject matter.

[0013] Some embodiments of the disclosure provide a method of using local illumination compensation (LIC). A video codec receives a local illumination compensation (LIC) linear model generated using samples neighboring two different blocks. The linear model includes a scaling factor (slope) and a y-axis intercept. The video codec adjusts the linear model by applying an offset to adjust the scaling factor (or slope) of the linear model. The video codec can apply the adjusted linear model to an initial predictor of a current block to generate a final predictor of the current block, where the initial predictor includes a reference block identified by a motion vector or a block vector of the current block.

[0014] The samples neighboring the two different blocks can be samples in a reference template neighboring a reference block and samples in a current template neighboring a current block (if the linear model is derived for the current block). In some embodiments, the linear model is inherited from a previously coded position, or the linear model is selected from a list of candidate linear models associated with different candidate positions, such that the linear model is generated using samples in template regions neighboring two different previously coded blocks.

[0015] The offset can be selected from a predefined set of values of a syntax element signaled in a bitstream. The predefined set of values can be {0, 1 / 8, and -1 / 8}. In some embodiments, the applied offset is inherited from a previously coded position, which can be selected from a list of candidates. In some embodiments, the applied offset is used to adjust the scaling factor (or slope) of the linear model. The scaling factor or the y-axis intercept of the adjusted linear model can be clipped to remain within a specified range.

[0016] When the current block is coded using bi-prediction, the linear model can be one of the first and second linear models used to generate a predictor for the current block. The selected offset can be used to adjust the first linear model but not the second linear model. In other embodiments, the first and second offsets are selected from a plurality of predefined offsets and used to adjust the first and second linear models, respectively. The adjusted first linear model is applied to a first predictor identified based on a first motion vector (e.g., L0 MV) of the current block, and the second linear model is applied to a second predictor identified based on a second motion vector (e.g., L1 MV) of the current block. In some embodiments, the first and second linear models are derived iteratively, and the selected offset is used to adjust the last derived linear model. BRIEF DESCRIPTION OF DRAWINGS The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this disclosure. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. It is understood that the drawings are not necessarily to scale, as some components can be shown exaggerated or in a schematic form in the interest of clarity.

[0018] Figure 1 Local illumination compensation (LIC) is conceptually illustrated.

[0019] Figure 2 Sub-block mode of LIC is conceptually illustrated.

[0020] Figure 3 LIC for bi-predicted blocks is illustrated.

[0021] Figure 4A -B illustrates the effect of slope adjustment parameters for CCLM.

[0022] Figure 5A -B illustrates slope adjustment applied to LIC model.

[0023] Figure 6 An example of a video encoder that can implement LIC is illustrated.

[0024] Figure 7 Portions of a video encoder that implement LIC parameter adjustment are illustrated.

[0025] Figure 8 A process of encoding a block of pixels by adjusting the slope of the LIC model is conceptually illustrated.

[0026] Figure 9 An example of a video decoder that can implement LIC is illustrated.

[0027] Figure 10 This demonstrates a portion of the video decoder that implements LIC parameter adjustment.

[0028] Figure 11 This conceptually demonstrates the process of decoding pixel blocks by adjusting the slope of the LIC model.

[0029] Figure 12 An electronic system is conceptually shown, in which certain embodiments of this disclosure are implemented.

Detailed Implementation Methods

[0031] I. Local illumination compensation (LIC) LIC is an inter-frame prediction technique used to model the local brightness variation between the current block and its predicted block as a function of the brightness variation between the current block template and the reference block template. The parameters of this function can be represented by a scaling factor α and an offset β, forming a linear equation α*p[x]+β. This linear equation is a linear model used to compensate for brightness variations, where p[x] is a reference sample pointing from the MV to position x on the reference picture. The offset parameter β is also the y-intercept of the LIC linear equation.

[0032] Figure 1 Local Luminance Compensation (LIC) is conceptually demonstrated. As shown in the figure, inter-frame prediction is performed on the current block using a motion vector (MV) pointing to a reference block. Neighbor reconstruction samples of the current block (current block template) and neighbor reconstruction samples of the reference block (reference block template) are used to compute the parameters of the linear model α*p[x]+β. The linear model is applied to the inter-frame prediction predictor to generate the final compensation predictor.

[0033] When wrap around motion compensation is enabled, the motion vector (MV) is considered with wrap around offset when clipping. Since a and b can be derived based on the current block template and the reference block template, no additional signaling overhead is needed except for signaling one LIC flag in AMVP mode to indicate the usage of LIC. To derive the LIC linear model parameters, linear least square method is used, which requires each coding unit (CU) to perform the following operations: Multiplication: 2 * min(width, height) + 4 Addition: 4 * min(width, height) + 4 Shift: 12 To apply the linear model, 1 multiplication and 1 addition operation can be used for each sample in the reconstruction stage where the prediction is added to the residual. When in-loop luma reshaping is used, the inverse reshaping operation can be applied to the neighboring samples of the current CU before the LIC parameter derivation, since the neighboring samples of the current CU are in the reshaped domain, but the reference picture samples are in the original (un-reshaped) domain.

[0034] The following are the syntax tables related to LIC: Table 1: LIC syntax in SPS Table 2: LIC syntax in slice header Table 3: LIC syntax in coding unit (CU) sps_lic_enabled_flag equal to 0 indicates that local illumination compensation is disabled. sps_lic_enabled_flag equal to 1 indicates that local illumination compensation is enabled.

[0035] sh_lic_enabled_flag equal to 1 indicates that local illumination compensation is enabled in one tile group. sh_lic_enabled_flag equal to 0 indicates that local illumination compensation is disabled in one tile group.

[0036] lic_flag[ x0 ][ y0 ] equal to 1 indicates that for the current coding unit, local illumination compensation is used to derive the prediction samples of the current coding unit when decoding the P or B tile group. lic_flag[ x0 ][ y0 ] equal to 0 indicates that the coding unit is not predicted by applying local illumination compensation. When lic_flag[ x0 ][ y0 ] is not present, it is inferred to be equal to 0.

[0037] a. Subblock LIC The LIC model can be derived in non-subblock mode or subblock (or affine) mode. In non-subblock mode, the LIC model is derived based on the top and left boundary pixels of the whole block and the derived single model is applied to the whole block. In subblock (or affine) mode, the motion vectors (MVs) of all small subblocks are calculated from two or three CPMVs, but the derivation of the LIC model is based on the reference blocks of all top and left boundary subblocks and the derived single model is applied to the whole block. In other words, the subblocks are not independent when the LIC model is derived.

[0038] Figure 2 The subblock mode of LIC is conceptually illustrated. In the figure, the current block 200 is coded (encoded or decoded) in subblock mode. The boundary subblocks of the current block are labeled A to G. Each boundary subblock has its own MV (MV A to MV G ) pointing to its own reference block (A' to G'). The LIC model is then derived based on the templates (regions 221-227 in the reference picture) of the reference blocks.

[0039] In certain embodiments, specific restrictions are imposed on the application of LIC. For example, in the encoder, using the flag skipRDCheckForLIC to indicate that, in the case of integer motion vector (IMV) mode, if (1) the rate-distortion (RD) cost of non-LIC IMV AMVP mode is 1.2 times worse than the current best RD cost, or (2) the block is smaller than 32 luma samples, then no LIC test is performed. Other restrictions on LIC include: LIC is not used together with bi-prediction in merge mode Geometric mode, intra block copy (IBC) mode, CIIP mode are not used together with LIC Bi-prediction is not used together with LIC If a slice is non-intra and LIC is enabled at picture level, for each quantization parameter (QP) value to be tested, 4 additional RD checks are added: inserting inter with different IMVs (0~3) and LIC as the mode to be tested.

[0040] b. Bi-prediction LIC In certain embodiments, the LIC mode is extended to bi-prediction coding units (CUs). Specifically, two different linear models are applied to two prediction blocks, which are then combined to generate bi-prediction samples for the current CU, i.e., Equation (1) and Equation (2) where P0 is the L0 prediction (e.g., the L0 reference block identified by the L0 motion vector), and P1 is the L1 prediction for the current block (e.g., the L1 reference block identified by the L1 motion vector); and and and denote the scaling factor and offset for L0 and L1, respectively; denotes the weight for the weighted combination of L0 and L1 predictions (indicated by the BCW index at the CU level).

[0041] Figure 3 Bi-prediction LIC is illustrated. A bi-prediction current block 300 is illustrated, with an L0 motion vector (MV0) and an L1 motion vector (MV1). The current block has a current template region 320. MV0 identifies an L0 predictor / reference block 310 with a reference template region 330. MV1 identifies an L1 predictor / reference block 311 with a reference template region 331. In certain embodiments, the L0 LIC parameters and and the L1 LIC parameters and are iteratively derived based on the samples of the current template region 320, the L0 reference template region 330, and the L1 reference template region 331.

[0042] The derivation scheme for the LIC mode is iteratively applied and reused when deriving the L0 and L1 parameters. Specifically, first the L0 template prediction is minimized and the difference between the template T and the updated template T' to derive the L0 parameters (L0 and , ) and subtract the corresponding samples in T from the samples in to obtain the updated template T'. Then, the L1 parameters (L1 and ) are computed to minimize the difference between the L1 template prediction and the updated template T'. Finally, the L0 parameters are optimized again in the same way.

[0043] In some embodiments, the derivation process of the bi-directional LIC parameters is as follows: (1) For the prediction candidates from the L0 list, the LIC parameters are defined in the same way as in the uni-directional case, derived using the neighboring reconstructed template and the unmodified L0 reference template, and applied to the unmodified L0 reference template to obtain the LIC-modified L0' reference template. (2) The LIC-modified L1' reference template is computed as follows: L1' reference template = 2 x neighboring reconstructed template - L0' reference template. (3) For the candidates from the L1 list, the LIC parameters can be defined using the unmodified L1 reference template and the samples in the LIC-modified L1' reference template. The defined parameters are applied to the unmodified L1 reference template to obtain the LIC-modified L1" reference template. (4) The LIC-modified L0" reference template is computed as follows: L0" reference template = 2 x neighboring reconstructed template - L1" reference template. (5) For the candidates from the L0 list, the refined LIC parameters can be defined using the unmodified L0 reference template and the samples in the modified L0" reference template, and applied to the unmodified L0 reference template.

[0044] In some embodiments, a flag is signaled for AMVP bi-prediction CUs to indicate the LIC mode, while for merge-related inter CUs, the flag is obtained by inheritance. In addition, when decoder-side motion vector refinement (DMVR) (including multiple DMVR, adaptive DMVR, and affine DMVR) and bi-directional optical flow (BDOF) are applied, the LIC function is disabled.

[0045] c. Overlapped Block Motion Compensation (OBMC) with LIC In some embodiments, overlapped block motion compensation (OBMC) can be applied to inter blocks coded with LIC mode. In some embodiments, to reduce complexity, OBMC is only applied to the top and left boundaries of a CU, while OBMC is always disabled for the boundaries of an intra-subblock of a LIC CU. In addition, when a neighboring block is coded with LIC, its LIC parameters are applied to generate the corresponding prediction samples for OBMC of the current block.

[0046] d. Slope adjustment for LIC LIC estimates a set of parameters (scaling factor and offset) for a current coding unit (CU) to be coded. In some embodiments, an encoder estimates a slope (or scaling factor) adjustment and then signals the estimated slope adjustment to a decoder. In some embodiments, the slope adjustment is estimated in both the encoder and the decoder, so no additional signaling is needed.

[0047] In some embodiments, there is a set of predefined slope adjustment values (e.g., ±0.95, ±0.8, ±0.6) that can be applied to make the slope adjustment. In some embodiments, the adjustment value is added to the slope. In other embodiments, the slope is multiplied by the adjustment value. In yet other embodiments, the combination of the adjusted slope and the offset is tested as an option, and the best option is selected based on a certain criterion.

[0048] In some embodiments, the "adjustment step" is predefined. The best option based on a certain criterion is signaled to the decoder. In other embodiments, all possible combinations are tested as options at both the encoder and the decoder, so the best option can be determined or identified without additional signaling.

[0049] In some embodiments, the slope adjustment is determined based on certain criteria, such as block size, relationship / correspondence between block width / height, etc.

[0050] In some embodiments, the slope adjustment can be signaled separately for each color component (luma or chroma). In some embodiments, the slope adjustment can be signaled for only one component (e.g., Y). In some embodiments, the slope adjustment can be shared by multiple components (e.g., Cb and Cr can share the same slope adjustment). In some embodiments, the slope adjustment is defined once and then shared among all color components (e.g., the slope adjustment is defined for the Y component and shared among all Y, Cb, Cr components).

[0051] In some embodiments, a separate syntax element can be defined in the bitstream to indicate whether the slope adjustment signaling is enabled or not. In some embodiments, the syntax element is defined in the bitstream at the sequence level (e.g., sequence parameter set or SPS). In some embodiments, the syntax element is defined in the bitstream at the frame / picture / tile / coding tree unit (CTU) / coding unit (CU) / prediction unit (PU) level (e.g., PPS or PH or SH).

[0052] e. Slope adjustment for linear model prediction The slope adjustment for LIC is similar to the slope adjustment in cross-component linear model (CCLM) prediction. CCLM employs a model with two parameters to map luma values to chroma values. The slope parameter "a" and the bias parameter "b" define the mapping as follows: chromaVal = a * lumaVal + b In some embodiments, an adjustment value "u" for the slope parameter is signaled to update the model to the following form: chromaVal = a' * lumaVal + b' where a' = a + u b' = b - u * y r With this selection, the mapping function is tilted or rotated around the point with luma value y r In some embodiments, the average of the reference luma samples is used as y r when the model is established, in order to provide a meaningful adjustment to the model. Figure 4A -B illustrates the effect of the slope adjustment parameter on the CCLM. Figure 4A shows the CCLM model without slope adjustment. Figure 4B illustrates the CCLM model updated by slope adjustment.

[0053] In some embodiments, local illumination compensation (LIC) with slope adjustment is provided, in which the LIC parameters are updated by an adjustment parameter in a similar way as the slope adjustment for CCLM. The adjustment parameter is signaled in AMVP mode. Figure 5A -B shows the effect of slope adjustment applied to the LIC model. The LIC model is derived by regression on the reference template samples and the current template samples. Figure 5A shows the LIC model without slope adjustment, in which the scaling factor a is the slope of the model and the offset parameter b is the y-axis intercept. Figure 5BThe LIC model with slope adjustment is shown. Specifically, the slope a is updated to a new scaling factor a' after adding an adjustment parameter u.

[0054] f. IBC-LIC model merge mode IBC-LIC model merge mode refers to coding (encoding or decoding) a current block by inheriting the LIC model from a previously coded block that is spatially neighboring or non-neighboring to the current block using a candidate list. More specifically, the LIC model of IBC-LIC model merge mode can be obtained by: (a) Constructing a model candidate list consisting of model parameters of spatially neighboring and non-neighboring neighbors, history candidates, and a default model. In some embodiments, the size of the candidate list is twelve. The LIC models are collected from previously coded (neighboring and non-neighboring) blocks that use IBC-LIC and IBC-LIC model merge. In addition, a history IBC-LIC model table of size six can be maintained, similar to the HMVP table. The LIC models of spatial neighbors and the models in the history IBC-LIC model table can be added to the IBC-LIC model merge candidate list. If the candidate list is not full, the default model and scaled model are added to the list. To avoid redundant models, a pruning operation can also be performed.

[0055] (b) Calculating the offset for each model candidate. For a certain inherited IBC-LIC model parameter set (a, b), the offset can be calculated from the template as: totalDiff = ∑ (Rec - (a*Ref + b)) offset = totalDiff / N where Rec and Ref are the pixels in the current block template and the reference block template, and N is the total number of pixels in the template region. Then b is modified as b = b + offset.

[0056] ​(c) Select one IBC-LIC model from the candidate list and signal its index in the bitstream. In some embodiments, a flag is signaled to indicate whether IBC-LIC model merge mode is applied. If the flag is true, further signal an index to indicate which candidate model is used for the current block. More specifically, the proposed mode is explicitly signaled in both IBC-AMVP and IBC-Merge modes. In IBC-AMVP mode, the flag is signaled when IBC-LIC flag is true. In IBC-Merge mode, the flag is signaled only when the current block is not coded as IBC-CIIP, IBC-GPM, TM-Merge or skip mode. If IBC-LIC model merge flag is true, regular IBC-LIC flag inheritance is not applied and the current block is treated as regular IBC-LIC by other blocks.

[0057] g. LIC slope adjustment for bi-directional LIC In some embodiments, the offset value is selected from a predefined set of offsets. The selected offset value is used to adjust the scaling parameter of the bi-directional prediction LIC linear model. (Thus, in this context, the "offset value" is an adjustment applied to the scaling factor a, not the y-axis intercept b). In some embodiments, the offset value is added to the scaling parameter of only one LIC linear model. The LIC linear model can be either the L0 or L1 LIC linear model. If the bi-directional prediction LIC linear model is iteratively derived (e.g., the L0 linear model is first derived, and then the L1 model is derived based on the derived L0 linear model), the offset value is added to the scaling parameter of the last derived LIC linear model. An index is signaled in the bitstream to indicate the selected offset value from the predefined set of offsets to optimize the bi-prediction LIC linear model.

[0058] In some embodiments, the syntax elements related to LIC scaling parameter adjustment are signaled for bi-directional LIC. Specifically, the offset value (e.g., the above Figure 5BThe "u" shown in the equations above is used to adjust the scaling parameters, which is similar to the slope adjustment for the cross-component linear model, and will be signaled in the bitstream for bi-directional LIC. In addition, a set of candidate adjustment offset values can be predefined, and the selected value is signaled in the bitstream by an index. In some embodiments, two slope offset values are signaled for L0 and L1 respectively, one for L0 and the other for L1. In some embodiments, only one slope adjustment offset value is signaled and applied to both directions. In certain embodiments, a flag is signaled separately to indicate only one adjustment offset value is used. In certain embodiments, only one offset value is signaled for one list (L0 or L1), and the offset value for the second list (L1 or L0) is a scaled value of the signaled offset value (the specific scaling depends on the POC distance between the current picture and the L0 and L1 reference pictures). In some embodiments, the sign of the offset value is signaled independently by the encoder and decoded by the decoder. In this case, the sign and the absolute value of the adjustment can be coded / decoded separately.

[0059] In certain embodiments, the adjustment is performed before the second set of LIC parameters is derived in bi-prediction. In some implementations, the adjustment is performed only for one set of LIC parameters, and the second set of LIC parameters is recalculated taking into account the updated first set of parameters. In certain implementations, the adjustment is applied to the prediction results of both lists (L0 and L1) simultaneously. In certain implementations, a first adjustment is applied to one set of LIC parameters, and a second adjustment can be applied to the other set of LIC parameters subsequently.

[0060] In some embodiments, the adjustment is performed after the LIC scaling and offset parameter calculation for L0 and L1 prediction is completed. In certain embodiments, the adjustment is performed only for one set of LIC parameters, and the second set of LIC parameters is recalculated taking into account the updated first set of parameters. In certain implementations, the adjustment is applied to the prediction results of both lists (L0 and L1) simultaneously. In certain implementations, a first adjustment is applied to one set of LIC parameters, and a second adjustment can be applied to the other set of LIC parameters subsequently.

[0061] In certain embodiments, the adjustment is performed in an iterative derivation process. In some embodiments, the adjustment is applied independently to each predictor. For example, in bi-prediction LIC, the parameters can be derived in the following order: L0 -> L1 -> L0" or L0 -> L1 -> L0" -> L1" or L0 -> L0" -> L1 or L0 -> L0" -> L1 -> L1" or L0 -> L1 -> L1" -> L0", etc.

[0062] In some embodiments, the adjusted scaling (a) and / or y-axis intercept (b) values are clipped after adjustment to remain within a specified range.

[0063] In some embodiments, the set of predefined offset values is {-1 / 8, 0, 1 / 8} or consistent with the predefined offset value set for single-prediction LIC slope adjustment, determined according to QP or neighboring coding unit (CU) selected LIC offset values.

[0064] In some embodiments, the offset value for adjusting the scaling parameter of bi-prediction LIC model is only signaled in inter or affine AMVP mode. In certain implementations, the offset value signaling is only performed when AMVR mode is disabled or a particular AMVR MV resolution is selected. In certain embodiments, the offset value is only signaled when the BCW weight is equal weight or the BCW weight is a particular weight. In some implementations, the offset value is only used to adjust one or two color components (i.e., the offset value is only used to adjust part of the color components instead of all color components).

[0065] h. Merge list for LIC adjustment In some embodiments, LIC linear model adjustment can be inherited from neighboring blocks to the current block (e.g., from spatial neighboring and non-neighborhood, temporal, temporal-translation, history candidate, and default model adjustment). In certain implementations, a predictor list is independently maintained for each list (L0 and L1) respectively. In certain embodiments, LIC parameter adjustment can be unrestricted by list or prediction direction (e.g., list is shared between L0 and L1).

[0066] In some embodiments, a lic_adj_merge_flag is signaled to indicate whether to apply the adjusted merge list. In certain implementations, lic_adj_merge_L0 / L1 flags are used to indicate whether to merge the LIC parameters for L0 / L1. In one implementation, lic_adj_mrg_idx_L0 / L1 can be used to identify the index of the LIC adjustment obtained from the adjusted merge list separately for L0 / L1. In certain implementations, lic_adj_mrg_idx can be used to identify the index of the LIC adjustment obtained from the adjusted merge list, which is used for both L0 and L1. In certain implementations, lic_adj_mrg_idx can be used to identify the index of the LIC adjustment obtained from the adjusted merge list, which is used for the predictor of L0 or L1, and the second predictor uses a scaled version of the adjustment.

[0067] In some embodiments, the adjustment parameters can be signaled to the decoder, and the final LIC parameters are equal to a weighted (equal or unequal) sum of the computed scaling factors and / or offsets and the corresponding adjustment values from the merge list, and the result is clipped.

[0068] In some embodiments, not only the adjustment parameters can be inherited, but also the LIC scaling factors and / or offsets can be inherited in bi-predictive LIC. In some embodiments, the LIC model candidate list can include model parameters from spatially neighboring and non-neighboring neighborhoods, temporal, temporal-translation, history candidates, and default models. For some embodiments, all the above methods can be extended to the LIC merge list.

[0069] II. Example Video Encoder Figure 6 An example of a video encoder 600 that can implement LIC is shown. As shown, video encoder 600 receives input video signals from video source 605 and encodes the signals into a bitstream 695. Video encoder 600 has a number of components or modules for encoding the signals from video source 605, including at least some selected from among a transform module 610, a quantization module 611, an inverse quantization module 614, an inverse transform module 615, an intra-estimation module 620, an intra-prediction module 625, a motion compensation module 630, a motion estimation module 635, a loop filter 645, a reconstructed picture buffer 650, an MV buffer 665, an MV prediction module 675, and an entropy encoder 690. Motion compensation module 630 and motion estimation module 635 are part of an inter-prediction module 640.

[0070] In some embodiments, modules 610 - 690 can be software instruction modules executed by one or more processing units (e.g., processors) of a computing or electronic device. In some embodiments, modules 610 - 690 can be hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. While modules 610 - 690 are shown as independent modules, some modules can be combined into a single module.

[0071] Video source 605 provides an original video signal that presents pixel data for each video frame without compression. Subtractor 608 calculates a difference between the original video pixel data of video source 605 and the predicted pixel data 613 from either motion compensation module 630 or intra prediction module 625 as a prediction residual 609. Transform module 610 converts the difference (or residual pixel data or residual signal 608) into transform coefficients (e.g., by performing a discrete cosine transform, or DCT). Quantization module 611 quantizes the transform coefficients into quantized data (or quantized coefficients) 612, which is encoded into bitstream 695 by entropy encoder 690.

[0072] Inverse quantization module 614 inverse quantizes the quantized data (or quantized coefficients) 612 to obtain transform coefficients, which inverse transform module 615 inverse transforms to generate a reconstructed residual 619. The reconstructed residual 619 is added to the predicted pixel data 613 to generate reconstructed pixel data 617. In some embodiments, reconstructed pixel data 617 is temporarily stored in a line buffer (not shown) for intra prediction and spatial MV prediction. The reconstructed pixels are filtered by loop filter 645 and stored in reconstructed picture buffer 650. In some embodiments, reconstructed picture buffer 650 is a memory external to video encoder 600. In some embodiments, reconstructed picture buffer 650 is a memory internal to video encoder 600.

[0073] Intra estimation module 620 performs intra prediction based on reconstructed pixel data 617 to generate intra predicted data. The intra predicted data is to be provided to entropy encoder 690 to be encoded into bitstream 695. The intra predicted data is also used by intra prediction module 625 to generate predicted pixel data 613.

[0074] Motion estimation module 635 performs inter prediction on reference pixel data of previously coded frames stored in reconstructed picture buffer 650 by generating MVs. The MVs are provided to motion compensation module 630 to generate predicted pixel data.

[0075] Rather than encode full actual MVs in the bitstream, video encoder 600 generates predicted MVs using MV prediction and encodes the difference between the MVs used for motion compensation and the predicted MVs as residual motion data and stores in bitstream 695.

[0076] MV prediction module 675 generates predicted MVs based on previously coded (i.e., motion compensation MVs used to perform motion compensation) for previous video frames. MV prediction module 675 retrieves reference MVs for previous video frames from MV buffer 665. Video encoder 600 stores MVs generated for current video frames in MV buffer 665 as reference MVs for generating predicted MVs.

[0077] The MV prediction module 675 creates a predicted MV using reference MVs. The predicted MV can be computed through spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensated MV (MC MV) of the current block is encoded by the entropy coder 690 into the bitstream 695.

[0078] The entropy coder 690 uses entropy coding techniques, such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, to encode various parameters and data into the bitstream 695. The entropy coder 690 encodes various table elements, flags, and quantized transform coefficients 612 and residual motion data as syntax elements into the bitstream 695. The bitstream 695 is then stored in a storage device or transmitted to a decoder via a communication medium, such as a network.

[0079] The loop filter 645 performs filtering or smoothing operations on the reconstructed pixel data 617 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 645 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is also performed before the loop filter.

[0080] Figure 7 Portions of the video encoder 600 that implement LIC parameter adjustment are shown. A current block can be initially encoded through inter prediction or intra prediction (motion compensation module 630 or intra prediction module 625) to generate an initial predictor 715. When the current block is encoded through inter prediction, the initial predictor 715 can be a reference block in a reference picture identified by a motion vector (MV). When the current block is encoded through bi-directional inter prediction, the initial predictor 715 can include two reference blocks identified by two MVs as a L0 predictor and a L1 predictor.

[0081] The LIC model constructor 705 uses the reconstructed samples retrieved from the reconstructed picture buffer 650 to derive the LIC model 710. In some embodiments, when the current block is coded with bi-prediction, the model constructor 705 constructs one L0 model applied to the L0 predictor and one L1 model applied to the L1 predictor. The L0 model and the L1 model can be iteratively derived based on the reconstructed samples in the neighboring blocks, the template region of the L0 predictor, and the template region of the L1 predictor, as described in Section I.b above.

[0082] The video encoder 600 can then apply the LIC model 710 to the initial predictor 715 to generate the LIC predictor 725. When the LIC model 710 includes an L0 model and an L1 model, the outputs of the two models can be combined as the LIC predictor 725 according to Equation (1). This LIC predictor 725 can be used to predict the pixel data 613 as the final predictor for the current block. Once generated, the LIC model 710 and its derivation-related information can be stored in the LIC model memory 735 for inheritance by subsequent blocks.

[0083] In some embodiments, the scaling parameter a and / or the y-axis intercept parameter b of the LIC model 710 can be adjusted by adding an offset value. The offset value can be selected from a set of predetermined values, such as {-1 / 8, 0, 1 / 8}, by an offset selection index. Such an index can be signaled in the bitstream 695 by the entropy encoder 690. The offset value, when added to the scaling parameter a, adjusts the slope of the LIC model. In some embodiments, the slope adjustment is applied only to the L0 model. In some embodiments, the slope adjustment is applied to both the L0 and L1 models. In some embodiments, two offset values are selected for the two models using two offset selection indices, respectively. Once used, the offset adjustment (or its selection index) of the LIC model can be stored in the LIC model memory 735 for inheritance by subsequent blocks.

[0084] The LIC model 710 can also be inherited from a previously coded block. In some embodiments, when the LIC linear model is inherited, the derivation-related information is also inherited in addition to the offset and scaling parameters of the model. The model derivation information can include the selection of the template region, the size of the template region, the LIC model type (linear model ax+b, LIC with position term, or multi-tap LIC), the multi-model flag, the classification method of the multi-model, the threshold of the multi-model, etc. The adjustment offset value for the LIC model can also be inherited from a previously coded block. When coding the current block using LIC, the encoder can inherit the LIC model itself, the adjustment offset, or both.

[0085] The LIC information inheritor 730 provides inherited information by retrieving information from the LIC model memory 735. The LIC information inheritor 730 can identify and obtain different types of candidates (corresponding to different candidate positions) from the history table, such as spatial candidates, non-adjacent spatial candidates, temporal candidates, and history candidates. The LIC information inheritor 730 can construct a candidate list including one or more candidate LIC linear models and / or one or more LIC parameter adjustment offsets. The candidate list can include different types of candidates. The history table for implementing history candidates can be stored in the LIC model memory 735 (or in the motion vector candidate memory, such as the MV buffer 665). The selection of the candidates is passed to the entropy encoder 690 for encoding to be signaled in the bitstream 695.

[0086] Figure 8 The process 800 of encoding a pixel block by adjusting a LIC model slope is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 600 perform the process 800 by executing instructions stored in a computer-readable medium. In some embodiments, the encoder 600 is implemented in an electronic device and performs the process 800.

[0087] At block 810, the encoder receives data of a current pixel block in a current picture to be encoded. At block 820, the encoder receives a linear model for local illumination compensation (LIC) generated using samples from two different blocks adjacent to the current block, the linear model including a scaling factor and a y-axis intercept.

[0088] The samples from two different blocks adjacent to the current block can be samples in a reference template adjacent to a reference block and samples in a current template adjacent to the current block (if the linear model is derived for the current block). In some embodiments, the linear model is inherited from a previously encoded position, or the linear model is selected from a list of candidate linear models associated with different candidate positions, such that the linear model is generated using samples from template regions adjacent to two different previously encoded blocks. The candidate list can include spatially adjacent and non-adjacent neighborhoods, temporal, temporal-translation, history candidates, and a default model.

[0089] At block 830, the encoder 600 adjusts the linear model by applying an offset value. The offset value can be selected from a predefined set of values of a syntax element signaled in a bitstream. The predefined set of values can be {0, 1 / 8, and -1 / 8}. In some embodiments, the applied offset value is inherited from a previously encoded position, which can be selected from a candidate list. In some embodiments, the applied offset value is used to adjust the scaling factor (or slope) of the linear model. The scaling factor or the y-intercept of the adjusted linear model can be clipped to remain within a prescribed range.

[0090] At block 840, the encoder generates a prediction using the adjusted linear model and produces prediction residuals, and uses to encode the current block. In particular, the encoder 600 can apply the adjusted linear model to an initial predictor of the current block to generate a final predictor of the current block, where the initial predictor includes a reference block identified by a motion vector or a block vector of the current block.

[0091] When the current block is encoded by bi-prediction, the linear model can be one of the first and second linear models used to generate a final predictor of the current block. The selected offset value can be used to adjust the first linear model but not the second linear model. The adjusted first linear model is applied to a first predictor identified based on a first motion vector (e.g., L0 MV) of the current block, and the second linear model is applied to a second predictor identified based on a second motion vector (e.g., L1 MV) of the current block. In some embodiments, the first offset value and the second offset value are selected from a plurality of predefined offset values, and are used to adjust the first and second linear models, respectively. In some embodiments, the first linear model and the second linear model are iteratively derived, and the selected offset value is used to adjust the last derived linear model (which can be the first linear model or the second linear model).

[0092] III. Example Video Decoder In some embodiments, the encoder can signal (or generate) one or more syntax elements in a bitstream so that the decoder can parse the one or more syntax elements from the bitstream.

[0093] FIG. 1 illustrates an example of a video decoder 900 that can implement local illumination compensation (LIC). As shown, the video decoder 900 is a picture decoding or video decoding circuit that receives a bitstream 995 and decodes the contents of the bitstream into pixel data for displaying video frames. The video decoder 900 has a plurality of components or modules for decoding the bitstream 995, including at least some components selected from among an inverse quantization module 911, an inverse transform module 910, an intra prediction module 925, a motion compensation module 930, a loop filter 945, a decoded picture buffer 950, a motion vector buffer 965, a motion vector prediction module 975, and some components selected from among a parser 990. The motion compensation module 930 is an inter prediction module 940.

[0094] In some embodiments, the modules 910 - 990 can be software instruction modules executed by a processing unit (e.g., a processor) of one or more computing devices. In some embodiments, the modules 910 - 990 can be hardware circuit modules implemented by integrated circuits of one or more electronic devices. Although the modules 910 - 990 are shown as independent modules, some of the modules can be combined into a single module.

[0095] The parser 990 (or entropy decoder) receives the bitstream 995 and performs initial parsing according to syntax defined by a video coding or picture coding standard. The parsed syntax elements include various table header elements, flags, and quantized data (or quantized coefficients) 912. The parser 990 parses the various syntax elements by using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0096] The inverse quantization module 911 de-quantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 910 performs inverse transform on the transform coefficients 916 to produce a reconstructed residual signal 919. The reconstructed residual signal 919 is added to the predicted pixel data 913 from the intra prediction module 925 or the motion compensation module 930 to produce decoded pixel data 917. The decoded pixel data is filtered by the loop filter 945 and stored in the decoded picture buffer 950. In some embodiments, the decoded picture buffer 950 is a memory external to the video decoder 900. In some embodiments, the decoded picture buffer 950 is a memory internal to the video decoder 900.

[0097] Intra prediction module 925 receives intra prediction data from bitstream 995 and generates predicted pixel data 913 from decoded pixel data 917 stored in decoded picture buffer 950 according to the data. In some embodiments, decoded pixel data 917 is also stored in a line buffer (not shown) for intra picture prediction and spatial motion vector prediction.

[0098] In some embodiments, the contents of decoded picture buffer 950 are used for display. Display device 905 can retrieve the contents directly from decoded picture buffer 950 for display, or retrieve the contents of decoded picture buffer to a display buffer. In some embodiments, display device receives pixel values from decoded picture buffer 950 through pixel transport.

[0099] Motion compensation module 930 generates predicted pixel data 913 from decoded pixel data 917 stored in decoded picture buffer 950 according to motion compensation motion vectors (MC MVs). These motion compensation motion vectors are decoded by adding residual motion data received from bitstream 995 to predicted motion vectors received from motion vector prediction module 975.

[0100] MV prediction module 975 generates predicted MVs based on reference MVs (e.g., motion compensation MVs used to perform motion compensation) generated for decoding previous video frames. MV prediction module 975 retrieves reference MVs for previous video frames from MV buffer 965. Video decoder 900 stores motion compensation MVs generated for decoding a current video frame in MV buffer 965 as reference MVs for generating predicted MVs. Loop filter 945 performs filtering or smoothing operations on decoded pixel data 917 to reduce encoding artifacts, particularly at boundaries of blocks of pixels. In some embodiments, the filtering or smoothing operations performed by loop filter 945 include a deblock filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is also performed before loop filter.

[0101] Figure 10A portion of the video decoder 900 implementing LIC parameter adjustment is shown. The current block can be initially coded by inter-prediction or intra-prediction (motion compensation module 930 or intra-prediction module 925) to generate an initial predictor 1015. When the current block is coded by inter-prediction, the initial predictor 1015 can be a reference block in a reference picture identified by a motion vector (MV). When the current block is coded by bi-directional inter-prediction, the initial predictor 1015 can include two reference blocks identified by two MVs as a L0 predictor and a L1 predictor.

[0102] The LIC model constructor 1005 derives the LIC model 1010 using reconstructed samples retrieved from the decoded picture buffer 950. In some embodiments, when the current block is coded by bi-prediction, the model constructor 1005 constructs a L0 model applied to the L0 predictor and a L1 model applied to the L1 predictor. The L0 model and the L1 model can be iteratively derived based on reconstructed samples in a template region neighboring the current block, the L0 predictor and the L1 predictor, as described in Section I.b above.

[0103] The video decoder 900 can then apply the LIC model 1010 to the initial predictor 1015 to generate a LIC predictor 1025. When the LIC model 1010 includes a L0 model and a L1 model, the outputs of the two models can be combined as the LIC predictor 1025 according to Equation (1). This LIC predictor 1025 can serve as the final predictor and the predicted pixel data 913 for the current block. Once generated, the LIC model 1010 and information related to its derivation can be stored in the LIC model memory 1035 for inheritance by subsequent blocks.

[0104] In some embodiments, the scaling parameter a and / or the y-axis intercept parameter b of the LIC model 1010 can be adjusted by adding an offset value. The offset value can be selected from a set of predetermined values (e.g., {-1 / 8, 0, 1 / 8}) by an offset selection index. Such an index can be signaled in the bitstream 995 and parsed by the entropy decoder 990. The offset value, when added to the scaling parameter a, adjusts the slope of the LIC model. In some embodiments, the slope adjustment is applied only to the L0 model. In some embodiments, the slope adjustment is applied to both the L0 and L1 models. In some embodiments, two offset selection indices are used to select two offset values for the two models, respectively. Once used, the offset adjustment (or its selection index) of the LIC model can be stored in the LIC model memory 1035 for inheritance by subsequent blocks.

[0105] The LIC model 1010 can be inherited from a previously encoded block. In some embodiments, when the LIC linear model is inherited, in addition to the offset and scaling parameters of the model, the related information is also inherited. The model derivation information can include the selection of the template region, the size of the template region, the LIC model type (linear model ax+b, LIC with position term, or multi-tap LIC), the multi-model flag, the classification method of the multi-model, the threshold of the multi-model, etc. The adjustment offset to the LIC model can also be inherited from the previously encoded block. When encoding the current block using LIC, the decoder can inherit the LIC model itself, the adjustment offset, or both.

[0106] The LIC information inheritor 1030 provides the inherited information by retrieving the information from the LIC model memory 1035. The LIC information inheritor 1030 can identify and fetch different types of candidates (corresponding to different candidate positions) from the history table, such as spatial candidates, non-adjacent spatial candidates, temporal candidates, and history candidates. The LIC information inheritor 1030 can construct a candidate list including one or more candidate LIC linear models and / or one or more LIC parameter adjustment offsets. The candidate list can include different types of candidates. The history table for implementing the history candidates can be stored in the LIC model memory 1035 (or in the motion vector candidate memory, such as the MV buffer 965). The selection of the candidate is received by the entropy decoder 990 by parsing the bitstream 995 and selected according to an index.

[0107] Figure 11 The process 1100 of decoding a pixel block by adjusting the slope of the LIC model is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 900 perform the process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, the decoder 900 is implemented in an electronic device and performs the process 1100.

[0108] At block 1110, the decoder receives data of a current pixel block in a current picture to be decoded. At block 1120, the decoder receives a linear model for local illumination compensation (LIC) that is generated using samples of two different blocks adjacent to the current pixel block, the linear model including a scaling factor and a y-axis intercept.

[0109] Samples adjacent to two different blocks can be samples in a reference template of a reference block adjacent to the current block and samples in a current template of the current block (if the linear model is derived for the current block). In some embodiments, the linear model is inherited from a previously coded location or the linear model is selected from a list of candidate linear models associated with different candidate locations such that the linear model is generated using samples in template regions adjacent to two different previously coded blocks. The candidate list can include spatially neighboring and non- neighboring neighborhoods, temporal, temporal-translation, history candidates, and default model.

[0110] At block 1130, the decoder 900 adjusts the linear model by applying an offset value. The offset value can be selected from a predefined set of values of a syntax element signaled in a bitstream. The predefined set of values can be {0, 1 / 8, and -1 / 8}. In some embodiments, the applied offset is inherited from a previously coded location selected from a candidate list. In some embodiments, the applied offset is used to adjust the scaling factor (or slope) of the linear model. The adjusted scaling factor or y-intercept of the linear model can be clipped to remain within a specified range.

[0111] At block 1140, the decoder reconstructs the current block by generating a prediction using the adjusted linear model. Specifically, the decoder applies the adjusted linear model to an initial predictor of the current block to generate a final predictor of the current block, where the initial predictor includes a reference block identified by a motion vector or block vector of the current block. The decoder can then provide the reconstructed current block for display as part of a reconstructed current picture.

[0112] When the current block is coded using bi-prediction, the linear model can be one of a first and a second linear model used to generate a final predictor of the current block. The selected offset value can be used to adjust the first linear model but not the second linear model. The adjusted first linear model is applied to a first predictor identified based on a first motion vector (e.g., L0 MV) of the current block, and the second linear model is applied to a second predictor identified based on a second motion vector (e.g., L1 MV) of the current block. In some embodiments, a first offset value and a second offset value are selected from a plurality of predefined offset values and used to adjust the first linear model and the second linear model, respectively. In some embodiments, the first linear model and the second linear model are iteratively derived, and the selected offset value is used to adjust the last derived linear model (which can be either the first linear model or the second linear model).

[0113] IV. Example Electronic System Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium). When these instructions are executed by one or more computational or processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, random access memories (RAMs), hard drives, erasable programmable read only memories (EPROMs), electrically erasable programmable read only memories (EEPROMs), etc. Computer readable media do not include carrier waves and electronic signals over wire, optical, or other forms of wired or wireless communication links.

[0114] In this specification, the word "software" is used to mean, among other things, the firmware residing in read-only memory or the applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, a plurality of software applications can be implemented as sub-parts of a larger program while remaining distinct software applications. In some embodiments, multiple software applications can also be implemented as a single program where the code is organized into separate parts to perform separate tasks. Finally, any combination of separate programs or a single program implementing multiple software applications can be utilized when practicing software applications described herein. In some embodiments, software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.

[0115] Figure 12 An electronic system 1200 is conceptually illustrated herein. The present disclosure is implemented in some embodiments in an electronic system 1200. The electronic system 1200 can be a computer (e.g., a desktop computer, a laptop computer, a tablet computer, etc.), a telephone, a PDA, or any other sort of electronic device. Such an electronic system includes various types of computer readable media and interfaces for various other types of computer readable media. Electronic system 1200 includes a bus 1205, a processing unit 1210, a graphics processing unit (GPU) 1215, a system memory 1220, a network 1225, a read only memory 1230, a permanent storage device 1235, an input device 1240, and an output device 1245.

[0116] Bus 1205 generally represents any type or form of mechanical, electrical, magnetic, or optical interface that is used to interconnect and enable data transference between the various components of electronic system 1200. For example, bus 1205 can be used to enable data transference between processing unit 1210 and GPU 1215, read only memory 1230, system memory 1220, and permanent storage device 1235.

[0117] From these various storage types, processing unit 1210 retrieves instructions to execute and processes data to be executed in order to perform the processes of the present disclosure. The processing unit can be a single processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by GPU 1215. GPU 1215 can offload various computations or supplement the picture processing provided by processing unit 1210.

[0118] Read only memory (ROM) 1230 stores static data and instructions that are used by processing unit 1210 and other modules of the electronic system. Permanent storage device 1235, on the other hand, is a read-and-write memory device. It is a non-volatile memory unit that stores instructions and data even when electronic system 1200 is off. Some embodiments of the present disclosure use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as permanent storage device 1235.

[0119] Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc. and its corresponding disk drive) as permanent storage device. Like permanent storage device 1235, system memory 1220 is a read-and-write memory device. However, unlike permanent storage device 1235, system memory 1220 is a volatile read-and-write memory, such as a random access memory. System memory 1220 stores some of the instructions and data that processing unit uses at runtime. In some embodiments, the processes of the present disclosure are stored in system memory 1220, permanent storage device 1235, and / or read-only memory 1230. For example, various storage units include instructions for processing multimedia clips in accordance with some embodiments. From these various storage types, processing unit 1210 retrieves instructions to execute and processes data to be executed in order to perform the processes of some embodiments.

[0120] Bus 1205 also connects to input and output devices 1240 and 1245. Input device 1240 enables a user to communicate information and select commands to the electronic system. Input device 1240 includes alphanumeric and other keys, a pointing device (also called a “cursor control device”), a camera (such as a webcam), a microphone or similar device to receive voice commands, etc. Output device 1245 displays information in a form digestible by the user and outputs data generated by the electronic system. Output device 1245 includes printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), and speakers or similar audio output devices. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0121] Finally, as Figure 12As shown, bus 1205 also couples electronic system 1200 to a network 1225 through a network adapter (not shown). In this manner, the computer can act as a client on the network. Any or all components of electronic system 1200 can be used in conjunction with the subject disclosure.

[0122] Certain embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions (also referred to as computer-readable or machine-readable instructions) in machine-readable or computer-readable media (also referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable / rewritable DVD discs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media can store the computer program instructions so as to be executed by at least one processing unit. Examples of computer programs or computer code include machine code, such as produced by a compiler, and files including high-level scripts that are executable by a computer, e- component, or microprocessor using an interpreter.

[0123] Although the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself. In other embodiments, the instructions are stored on memory devices coupled to the integrated circuit.

[0124] The terms “computer,” “server,” “processor,” and “memory,” as used in this specification and any claims of this application are intended to refer to electronic or other technological devices. These terms exclude personal computers or groups of persons. For purposes of this specification, the term display or displaying means displaying on an electronic device. The terms “computer readable medium,” “computer readable media,” and “machine readable medium,” as used in this specification and any claims of this application are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

[0125] While the present disclosure has been described with numerous specific details, one of ordinary skill in the art will recognize that the disclosure can be embodied in other specific forms without departing from the spirit of the disclosure. For example, software can be implemented in a machine-executable sequence of instructions, and / or in a graphical programming language, and / or in hardware description languages, and / or in hardware logic, and / or in any other format suitable to a skilled person for programming a machine. In addition, many of the figures (including Figure 8 and Figure 11 ) conceptually illustrate processes. The specific operations of these processes can not be performed in the exact order shown and described. The specific operations can not be performed in one continuous series of operations, and different specific operations can be performed in different embodiments. Furthermore, the process could be implemented using several sub-processes, or alternatively, as part of a larger macro process. Thus, one of ordinary skill in the art would understand that the disclosure is not to be limited by the foregoing illustrative details, but rather is to be defined by the following claims in combination with their due scope.

[0126] The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely examples, and that in fact many other architectures can be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermediate components. Likewise, any two components so associated can also be viewed as being "operably connected", or "operably coupled", to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being "operably couplable", to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and / or physically interacting components and / or wirelessly interactable and / or wirelessly interacting components and / or logically interacting and / or logically interactable components.

[0127] Also, to the extent that any meaning or definition of a freely used term herein conflicts, that term shall be interpreted by this detailed description.

[0128] Further, those skilled in the art will appreciate that, in general, the terms used herein, especially in the appended claims, such as, for example, the substantive of the appended claims, are generally intended as "open" terms, e.g., the term "comprising" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "including" should be interpreted as "including but not limited to," and the like. Those skilled in the art will further appreciate that, unless specifically stated in the claims, the particular number of a recited claim statement introduced by the term "at least one" is intentional and will be explicitly stated in the claim, and no such intention exists in the absence of such a statement. For example, to aid understanding, the following appended claims can contain the use of introductory phrases "at least one" and "one or more" to introduce claim statements. However, the use of these phrases should not be interpreted as implying that any particular claim limitation introduced by the indefinite article "a" or "an" is limited to containing only one such statement, even if the same claim includes the introductory phrase "one or more" or "at least one" and the indefinite article such as "a" or "an," e.g., "a" and / or "an" should be interpreted as "at least one" or "one or more"; the same applies to the definite article used to introduce claim statements. Moreover, even if a specific number of recited claim statements introduced by the term "at least one" is explicitly stated, those skilled in the art will recognize that such a statement should be interpreted as being at least the stated number, e.g., a simple statement of "two statements" without further modifier means at least two statements, or two or more statements. Furthermore, in those instances where a convention similar to "at least one A, B, and C, etc." is used, generally such structure is interpreted in a manner that those skilled in the art understand the convention, e.g., "a system having at least one A, B, and C" would include, but not be limited to, a system having only A, a system having only B, a system having only C, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc. In those instances where a convention similar to "at least one A, B, or C, etc." is used, generally such structure is interpreted in a manner that those skilled in the art understand the convention, e.g., "a system having at least one A, B, or C" would include, but not be limited to, a system having only A, a system having only B, a system having only C, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc. Those skilled in the art will further appreciate that almost any disjunctive word and / or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be interpreted to consider the possibility of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" would be interpreted to include the possibility of "A," or "B," or "A and B."

[0129] As can be seen from the foregoing, various embodiments of the present disclosure have been described herein for illustrative purposes, and various modifications can be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the following claims.

Claims

1. A video encoding method, comprising: Receive data for a current pixel block of a current image to be encoded or decoded into a video; Receive a linear model generated using samples from two neighboring different blocks, the linear model including a scaling factor and a y-intercept; Adjust the linear model with an offset; and The current block is encoded or decoded using this adjusted linear model.

2. The video coding method as described in claim 1, wherein the linear model is used for local illumination compensation (LIC).

3. The video encoding method as described in claim 1, wherein the samples adjacent to the two different blocks are samples in a reference template of a neighboring reference block and samples in a current template of a neighboring current block.

4. The video coding method of claim 1, wherein the samples adjacent to the two different blocks are samples in the template regions of two adjacent different previously coded blocks.

5. The video encoding method of claim 1, wherein the offset is used to adjust the scaling factor of the linear model.

6. The video encoding method of claim 1, wherein encoding or decoding the current block comprises: The adjusted linear model is applied to an initial predictor for the current block to generate a final predictor for the current block, wherein the initial predictor includes a reference block identified by a motion vector or a block vector of the current block; and The current block is encoded or decoded using this final predictor.

7. The video coding method of claim 1, wherein the linear model is one of a first and a second linear model for generating a predictor of the current block, wherein the selected offset is used to adjust the first linear model instead of the second linear model.

8. The video coding method of claim 7, wherein the adjusted first linear model is applied to a first predictor based on a first motion vector identifier of the current block, and the second linear model is applied to a second predictor based on a second motion vector identifier of the current block.

9. The video coding method of claim 7, wherein the first and second linear models are iteratively derived for the current block, and wherein the selected offset is used to adjust the last derived linear model.

10. The video coding method of claim 1, wherein the scaling factor of the adjusted linear model or the y-intercept is cropped to remain within a specified range.

11. The video encoding method of claim 1, wherein the offset is selected from a predefined set of values ​​for a syntax element of signaling in a bitstream.

12. The video encoding method of claim 11, wherein the predefined set of values ​​includes 0, 1 / 8 and -1 / 8.

13. The video coding method of claim 11, wherein the first and second linear models are applied to the first and second predictors identified by the first and second motion vectors of the current block, respectively, wherein a first offset and a second offset are selected from the plurality of predefined offsets and are respectively used to adjust the first and second linear models.

14. The video encoding method of claim 1, wherein the offset is inherited from a previously encoded position.

15. The video encoding method of claim 14, wherein the inherited offset is selected from a list of candidate positions.

16. The video coding method of claim 1, wherein the linear model is inherited from a previously encoded position.

17. The video coding method of claim 16, wherein the inherited linear model is selected from a list of candidate linear models associated with different candidate positions.

18. An electronic device comprising: A video encoding circuit configured to perform the following operations: Receive data for a current pixel block of a current image to be encoded or decoded into a video; Receive a linear model generated using samples from two neighboring different blocks, the linear model including a scaling factor and a y-intercept; Adjust the linear model with an offset; and The current block is encoded or decoded using an adjusted linear model, wherein the adjusted linear model is one of a first and a second linear model for generating a final predictor of the current block, the adjusted first linear model is applied to a first predictor based on a first motion vector identifier of the current block, and the second linear model is applied to a second predictor based on a second motion vector identifier of the current block.

19. A video decoding method, comprising: Receive the data to be decoded as a current pixel block of a current image in a video; The system receives a linear model generated using samples from two neighboring different blocks, the linear model including a scaling factor and a y-intercept; adjusts the linear model with an offset; and reconstructs the current block using the adjusted linear model, wherein the adjusted linear model is one of a first and a second linear model used to generate a final predictor for the current block, the adjusted first linear model being applied to a first predictor identified based on a first motion vector of the current block, and the second linear model being applied to a second predictor identified based on a second motion vector of the current block.

20. A video coding method, comprising: Receive the data to be encoded as a current pixel block of a current image in a video; The system receives a linear model generated using samples from two neighboring different blocks, the linear model including a scaling factor and a y-intercept; adjusts the linear model with an offset; and encodes the current block using the adjusted linear model, wherein the adjusted linear model is one of a first and a second linear model used to generate a final predictor for the current block, the adjusted first linear model being applied to a first predictor identified based on a first motion vector of the current block, and the second linear model being applied to a second predictor identified based on a second motion vector of the current block.

Images