Spatial local illumination compensation

Abstract

At least a method and an apparatus for efficiently encoding or decoding a video are presented. For example, parameters for local illumination compensation LIC of a current block being encoded / decoded in a picture are determined based on spatially adjacent reconstructed samples of at least one spatial reference block and corresponding spatially adjacent reconstructed samples, where the at least one spatial reference block is a spatial neighboring block of the current block in the picture. For example, a flag enables / disables spatial LIC for the current block. For example, the spatial LIC applies to either inter / intra / IBC prediction. For example, multiple spatial reference blocks are used in determining the spatial LIC parameters. For example, spatially adjacent reconstructed samples of multiple lines are used in determining the spatial / temporal LIC parameters.

Description

[Technical field]

[0001] At least one of the present embodiments relates generally to a method or apparatus for video encoding or decoding, and more particularly to a method or apparatus that includes applying spatial local illumination compensation. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes usually employ prediction, including motion vector prediction, and transformation to exploit spatial and temporal redundancy in video content. In general, intra- or inter-prediction is used to exploit correlation within or between frames, whereby the difference between an original image and a predicted image, often referred to as a prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy coding, quantization, transformation, and prediction.

[0003] Recent additions to video compression technology include various industry standards, reference software versions, and / or documentation, such as the Joint Video Exploration Model (JEM) and later the Versatile Video Coding (VVC) Test Model (VTM) being developed by the JVET (Joint Video Exploration Team) group, with the goal of further improving the existing HEVC (High Efficiency Video Coding) standard.

[0004] Existing methods for encoding and decoding show some limitations in compensating for illumination mismatch between different regions / blocks in the same slice / picture. This problem is especially pronounced for content that contains several sample values ​​where spatial illumination variations propagate progressively in inter / intra / IBC prediction. Therefore, there is a need to improve this state of the art. Summary of the Invention

[0005] The shortcomings and disadvantages of the prior art are resolved and addressed by the general aspects described herein.

[0006] According to a first aspect, a method is provided. The method includes: video decoding by determining parameters for local illumination compensation based on spatially adjacent reconstructed samples for a current block being decoded in a picture and corresponding spatially adjacent reconstructed samples of at least one spatial reference block, and decoding the current block using the determined parameters. Advantageously, the at least one spatial reference block is a spatially adjacent block of the current block in the picture.

[0007] According to another aspect, a second method is provided, which includes: video coding by determining, for a current block being coded in a picture, parameters for local illumination compensation based on spatially adjacent reconstructed samples of the current block and corresponding spatially adjacent reconstructed samples of at least one spatial reference block, and coding the current block using the local illumination compensation based on the determined parameters, advantageously the at least one spatial reference block being a spatially adjacent block of the current block in the picture.

[0008] According to another aspect, an apparatus is provided. The apparatus comprises one or more processors, the one or more processors being configured to implement the method for video decoding according to any of its variants. According to another aspect, the apparatus for video decoding comprises means for determining parameters for local illumination compensation based on spatially adjacent reconstructed samples of at least one spatial reference block for a current block being decoded in a picture and corresponding spatially adjacent reconstructed samples, and means for decoding the current block using the local illumination compensation based on the determined parameters. Advantageously, the at least one spatial reference block is a spatially adjacent block of the current block in the picture.

[0009] According to another aspect, another apparatus is provided. The apparatus comprises one or more processors, the one or more processors being configured to implement the method for video coding according to any of its variants. According to another aspect, the apparatus for video coding comprises means for determining, for a current block being coded in a picture, parameters for local illumination compensation based on spatially adjacent reconstructed samples of at least one spatial reference block and corresponding spatially adjacent reconstructed samples, and means for coding the current block using local illumination compensation based on the determined parameters. Advantageously, the at least one spatial reference block is a spatially adjacent block of the current block in the picture.

[0010] According to another general aspect of at least one embodiment, a syntax element is determined that indicates whether spatial local illumination compensation is applied to the current block.

[0011] According to another general aspect of at least one embodiment, the current block is coded with inter prediction, intra prediction, or IBC prediction.

[0012] According to another general aspect of at least one embodiment, the at least one spatial reference block is one of an upper neighboring block and a left neighboring block.

[0013] According to another general aspect of at least one embodiment, the at least one spatial reference block is any one of an upper neighboring block (B0), a left neighboring block (A0), a top right neighboring block (B1), a bottom left neighboring block (A1), and an upper left neighboring block (B2).

[0014] According to another general aspect of at least one embodiment, a syntax element is determined that indicates which spatial reference block is to be used in determining parameters for local illumination compensation.

[0015] According to another general aspect of at least one embodiment, the at least one spatial reference block is a neighboring block selected as a motion vector predictor MVP candidate in inter prediction.

[0016] According to another general aspect of at least one embodiment, the at least one spatial reference block is responsive to an intra-prediction mode used to encode the current block.

[0017] According to another general aspect of at least one embodiment, the at least one spatial reference block comprises a neighboring block selected as an intra block copy reference block.

[0018] According to another general aspect of at least one embodiment, the neighboring reconstructed samples are located at the left and top boundaries of the current block and at least one spatial reference block.

[0019] According to another general aspect of at least one embodiment, the adjacent reconstructed samples are located within a plurality of left and top reference lines of the current block and the at least one spatial reference block. According to another general aspect of at least one embodiment, the adjacent reconstructed samples are located across the reconstructed blocks of the current block and the at least one spatial reference block. According to another general aspect of at least one embodiment, the at least one spatial reference block comprises a first spatial reference block and a second spatial reference block, and the spatially adjacent reconstructed samples of the first spatial reference block and the spatially adjacent reconstructed samples of the second spatial reference block are averaged to determine a parameter of local illumination compensation.

[0020] According to another aspect, a third method is provided, which includes: video decoding by determining parameters for local illumination compensation based on spatially adjacent reconstructed samples for a current block being decoded in a picture and corresponding spatially adjacent reconstructed samples of at least one reference block; and decoding the current block using the local illumination compensation based on the determined parameters, wherein the neighboring reconstructed samples are located within a plurality of left and upper reference lines of the current block and the at least one reference block. According to another general aspect of at least one embodiment, the neighboring reconstructed samples are located throughout the reconstructed blocks of the current block and the at least one spatial reference block.

[0021] According to another aspect, a fourth method is provided, which includes: video coding by determining parameters for local illumination compensation based on spatially adjacent reconstructed samples for a current block being coded in a picture and corresponding spatially adjacent reconstructed samples of at least one reference block; and coding the current block using local illumination compensation based on the determined parameters, the neighboring reconstructed samples being located within a plurality of left and upper reference lines of the current block and the at least one reference block. According to another general aspect of at least one embodiment, the neighboring reconstructed samples are located throughout the reconstructed blocks of the current block and the at least one spatial reference block.

[0022] According to another general aspect of at least one embodiment, a device is provided that includes an apparatus according to any of the decoding embodiments and at least one of: (i) an antenna configured to receive a signal, the signal including a video block; (ii) a band limiter configured to limit the received signal to a frequency band including the video block; or (iii) a display configured to display an output representing the video block.

[0023] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that includes data content generated according to any of the described decoding embodiments or variations.

[0024] According to another general aspect of at least one embodiment, there is provided a signal including video data generated according to any of the described encoding embodiments or variations.

[0025] According to another general aspect of at least one embodiment, a bitstream is formatted to include data content generated according to any of the described decoding embodiments or variations.

[0026] According to another general aspect of at least one embodiment, a computer program product is provided that includes instructions that, when executed by a computer, cause the computer to perform any of the encoding / decoding embodiments or variations described.

[0027] These and other aspects, features, and advantages of the general aspects will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings. [Brief description of the drawings]

[0028] In the drawings, some example embodiments are illustrated. [Figure 1] 1 illustrates the concepts of coding tree unit (CTU) and coding unit (CU) for representing compressed VVC pictures. [Diagram 2] 1 illustrates the derivation of a local illumination compensation LIC parameter process using a corresponding template, in accordance with at least one embodiment. [Diagram 3] 1 shows an example video game picture with a light source that creates gradual lighting variations within the same picture. [Figure 4] 1 illustrates a general encoding method in accordance with a general aspect of at least one embodiment. [Diagram 5] 1 illustrates a general decoding method in accordance with a general aspect of at least one embodiment. [Figure 6] 1 illustrates a derivation of a spatial LIC parameter process using a reference template of an upper / left neighboring block for inter prediction, according to at least one embodiment. [Figure 7] 4 illustrates a decoding method according to a first embodiment in which spatial LIC is applied during decoding of inter blocks. [Figure 8] 1 illustrates a derivation of spatial LIC parameter process using average reference template of upper and left neighboring blocks for inter prediction, according to at least one embodiment. [Figure 9] 13 shows the locations of spatial MVP candidates for inter blocks. [Figure 10] 1 illustrates a derivation of a spatial LIC parameter process using a reference template of an upper right neighboring block for inter prediction, according to at least one embodiment. [Figure 11] 4 illustrates a decoding method according to a second embodiment in which spatial LIC is applied during decoding of inter blocks based on MVP candidates. [Figure 12] 13 shows the intra prediction direction in VVC. [Figure 13] 1 illustrates the derivation of spatial LIC parameter process using reference templates of above / left / top-right / bottom-left / top-left neighboring blocks for intra prediction, according to at least one embodiment. [Figure 14] 1 illustrates a matrix weighting intra prediction process in VVC. [Figure 15] 4 illustrates a decoding method according to a third embodiment in which spatial LIC is applied during decoding of intra blocks. [Figure 16] 1 illustrates a process of derivation of spatial LIC parameters using a reference template that includes a left boundary of a left neighboring block for intra prediction and a reference template that includes an upper boundary of an upper neighboring block for intra prediction, according to at least one embodiment. [Figure 17]1 illustrates a derivation of spatial LIC parameter process using multiple line reference templates of spatially neighboring blocks in accordance with at least one embodiment. [Figure 18] 1 illustrates a derivation of spatial LIC parameter process using multiple line reference templates of spatially neighboring blocks in accordance with at least one embodiment. [Figure 19] 1 illustrates a process for deriving spatial LIC parameters using a reference template that includes spatially adjacent blocks, according to at least one embodiment. [Figure 20] IBC prediction in VVC is shown. [Figure 21] 1 illustrates the derivation of a spatial LIC parameter process using a reference template indicated by a block vector for IBC prediction, in accordance with at least one embodiment. [Figure 22] 13 illustrates a decoding method according to a fourth embodiment in which spatial LIC is applied during decoding of an IBC block. [Figure 23] 1 shows a block diagram of an embodiment of a video encoder in which various aspects of the embodiments may be implemented. [Figure 24] 1 shows a block diagram of an embodiment of a video decoder in which various aspects of the embodiments may be implemented. [Diagram 25] 1 shows a block diagram of an exemplary device in which various aspects of the embodiments may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] It should be understood that the drawings and description are simplified to illustrate elements relevant to a clear understanding of the present principles, and for purposes of clarity, exclude many other elements that may be found in a typical encoding and / or decoding device. Although terms such as first and second may be used herein to describe various elements, it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another.

[0030] Various embodiments are described with respect to encoding / decoding an image, which may be applied to encode / decode portions of an image, such as slices or tiles, groups of tiles, or entire sequences of images.

[0031] Various methods have been described above, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for the proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0032] At least some embodiments relate to methods for encoding or decoding video, where spatial LIC allows for compensating for gradual lighting within the same picture.

[0033] FIG. 1 illustrates the concept of coding tree units (CTUs) and coding units (CUs) for representing compressed VVC pictures. In VVC, a picture is divided into so-called coding tree units (CTUs), and each CTU is represented by one or more coding units (CUs). For each CU, spatial prediction (or "intra prediction") and / or temporal prediction (also called "inter prediction" or "motion compensated prediction") is performed. Spatial prediction uses pixels from samples of already coded neighboring blocks (called reference samples) in the same video picture / slice to predict the current video block. Spatial prediction reduces spatial redundancy inherent in video signals. Temporal prediction uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction signal for a given video block is typically signaled by one or more motion vectors that indicate the amount and direction of motion between the current block and its reference block. Also, if multiple reference pictures are supported, for each video block, its reference picture index is further transmitted, which is used to identify which reference picture in the reference pictures the temporal prediction signal comes from. After spatial and / or temporal prediction, a mode decision block in the encoder selects the best prediction mode based, for example, on a rate-distortion optimization method. For ease of reference, the terms "CU" and "block" are used interchangeably throughout this text.

[0034] FIG. 2 illustrates the derivation of a Local Illumination Compensation (LIC) parameter process using a corresponding template according to at least one embodiment. In a temporal prediction process, Local Illumination Compensation (LIC) is a coding tool used to address the problem of local illumination variations that may exist between temporally neighboring pictures. The LIC tool is based on a linear model, where at least one of a scaling factor α and an offset β is applied to a reference sample to obtain a predicted sample of a current block. Specifically, the LIC tool is mathematically modeled by the following equation: P(x,y)=α P r (x+v x ,y+v y )+β (1) where P(x,y) is the predicted signal of the current block at coordinate (x,y), and P r (x+v x ,y+v y ) is the motion vector (v x ,v y ), and α and β are the corresponding scaling factors and offsets applied to the reference block.

[0035] In FIG. 2, when LIC is applied to a block, a minimum mean square error (LMSE) method is used to derive the values ​​of the LIC parameters (i.e., α and β) by minimizing the differences between neighboring samples of the current block (i.e., template T in FIG. 2) and their corresponding reference samples (T0 or T1) in the temporal reference pictures L0 reference picture and L1 reference picture, as shown in FIG. 2.

[0036]

number

[0037]

number

[0038]

number

[0039] Furthermore, when LIC is applied to a bidirectional block (i.e., predicted by two temporal predictive blocks), LIC parameters are derived and applied separately for each prediction direction, i.e., L0 and L1. As shown in FIG. 2, two reference templates T0 and T1 can be obtained based on two motion vectors MV0 and MV1. By separately minimizing the distortion between T0 and T and between T1 and T, the corresponding pairs of LIC parameters in two directions can be derived according to equations (2) and (3). Then, the final bidirectional predicted signal of the current block is generated by combining two LIC uni-predictive blocks, as shown below.

[0040]

number

[0041]

number

[0042]

number

[0043] When an inter block is predicted in merge mode, the LIC flag is included as part of the motion information in addition to the MV and reference index. When a merge candidate list is constructed, the LIC flag is inherited from the neighboring blocks for the merge candidate. Otherwise, the LIC flag is context coded using a single context, and when the LIC tool is not applicable, the LIC flag is not signaled.

[0044] However, it is desirable to increase the coding efficiency of some video content that contains some gradual lighting variations within the same picture. Such a situation may typically occur in some game video content or computer graphic images, where some lighting source is located somewhere in the picture and light propagates gradually across the picture. Figure 3 shows an example video game picture with a light source that generates gradual lighting variations within the picture. In such a case, the block to be coded may include some background content with luma values ​​that evolve gradually according to spatial location, and some local specific texture elements that may be considered as foreground information. Such gradual lighting variations within the same picture may also occur in natural images, and the present principles are compatible with any type of video content.

[0045] As mentioned above, LIC can be regarded as an extension of normal motion compensation prediction by addressing the illumination variation between different pictures in the motion compensation stage. Although the prior art LIC can compensate for the illumination mismatch between different pictures, it has not been applied or adapted to illumination compensation between different blocks in the same picture.

[0046] This is solved and addressed by the general aspects described herein, which are directed to determining parameters for local illumination compensation based on spatially adjacent reconstructed samples for a current block being decoded or being decoded in a picture and corresponding spatially adjacent reconstructed samples of at least one spatial reference block, where the at least one spatial reference block is a spatial neighboring block of the current block in the picture. Thus, assuming that one coding block and its spatial neighboring blocks in a picture have propagating illumination variations, after generating a prediction signal for the block, the present principles propose to apply spatial LIC to enhance the prediction. Since the reference block is not located in a temporal reference picture, but is instead located in the same picture, both the reference block search and the template used for spatial LIC parameter estimation are adjusted. Furthermore, a spatial LIC flag determination indicating the use of spatial LIC can also be defined. Furthermore, various embodiments of spatial local illumination compensation (spatial LIC) for inter / intra / IBC prediction, where different blocks have progressively propagating spatial illumination variations in the same picture, are disclosed. Furthermore, various embodiments of the shape of the template used in local illumination compensation (spatial / temporal LIC) are also disclosed.

[0047] 4 illustrates a general encoding method (100) according to a general aspect of at least one embodiment. The block diagram of FIG. 4 partially illustrates modules of an encoder or encoding method, as implemented, for example, in the example encoder of FIG.

[0048] According to a general embodiment, a coding method 100 is disclosed. The method comprises a step 11 of determining parameters for local illumination compensation based on spatially neighboring reconstructed samples for a current block being coded in a picture and on corresponding spatially neighboring reconstructed samples of at least one spatial reference block. Advantageously, the spatial reference block is a spatial neighboring block of the current block in the picture, as described in various embodiments below. The parameters determined for local illumination compensation thus make it possible to perform spatial LIC. The spatial LIC is applied to the prediction of the current block to compensate for the gradual illumination in the picture, resulting in a compensated prediction of the block. According to different embodiments, the prediction is one of inter prediction, intra prediction or intra block copy (IBC) prediction. According to another embodiment, a syntax element is determined indicating whether spatial local illumination compensation is applied to the current block or not. After spatial compensation of the prediction of the current block, the residual is calculated in the usual way, for example by subtracting the compensated prediction from the current block, and then the remaining processing (transformation, quantization, CABAC coding, etc.) is performed as in state-of-the-art coding methods, in a general coding step 12.

[0049] 5 illustrates a general decoding method (200) according to a general aspect of at least one embodiment. The block diagram of FIG. 5 partially illustrates modules of a decoder or decoding method implemented, for example, in the exemplary decoder of FIG.

[0050] According to a general embodiment, a decoding method 200 is disclosed. The method comprises a step 21 of determining parameters for local illumination compensation for a current block being decoded in a picture based on spatially neighboring reconstructed samples and corresponding spatially neighboring reconstructed samples of at least one spatial reference block. In terms of encoding, the spatial reference blocks are spatially neighboring blocks of the current block in the picture, as described in various embodiments below. According to different embodiments, spatial LIC is enabled / disabled for the current block using a dedicated flag, and spatial LIC is applied to one of inter-, intra- or IBC predictions of the current block. Then, the decoding 22 further comprises decoding residual values, for example by performing CABAC decoding, inverse quantization of the transform coefficients, then an inverse transform of the decoded coefficients, and adding the thus decoded residual values ​​to the compensated prediction to decode the current block.

[0051] Various embodiments of the general spatial LIC used in the encoding or decoding method are described below. According to various embodiments, a block (or CU) level spatial LIC flag is defined for an inter / intra / IBC block to indicate whether spatial LIC is applied to the block or not. If spatial LIC is applied to an inter / intra / IBC block, according to another particular embodiment, a linear model of spatial illumination variation is defined using α and an offset β, which are defined using a scaling factor. The estimate of spatial LIC parameters is derived by minimizing the difference between neighboring reconstructed samples of a current block (current template) and corresponding neighboring reconstructed samples of a spatial reference block (reference template) in the same picture. Various embodiments described below relate to the derivation of the CU level spatial LIC flag: - Selection of spatial neighboring blocks to be used as reference blocks for spatial LIC parameter estimation, generation of a template constituted by neighboring reconstructed samples and used for spatial LIC parameter estimation.

[0052] In the following, spatial LIC in inter prediction is described, its spatial LIC derivation, reference block decision, and template generation used for spatial LIC parameter estimation. Then, for spatial LIC in intra prediction, reference block decision and template generation are also described, especially the differences compared to spatial LIC in inter prediction. Then, for spatial LIC in IBC prediction, reference block decision is also described. Finally, a spatial reference block search for inter / inter prediction is proposed.

[0053] According to a first embodiment, spatial LIC is applied during encoding / decoding of inter blocks. Figure 6 illustrates the derivation of spatial LIC parameter process using reference template of upper / left neighboring block for inter prediction according to at least one embodiment.

[0054] According to the prior art LIC tool described above, LIC is applied to compensate for temporal illumination variations between different frames in inter prediction, hereinafter referred to as temporal LIC. When there may be some propagating illumination variations between some spatial blocks in the same frame, spatial LIC is proposed to further compensate for spatial illumination variations within the same frame in inter prediction.

[0055] According to a variant embodiment, the spatial LIC flag spatial_lic_flag is defined to indicate whether spatial LIC is applied or not. When an inter block is coded in merge mode, the spatial LIC flag is copied from the neighboring block in a similar manner to the motion information copy in merge mode, otherwise the spatial LIC flag is signaled for the block.

[0056] According to another variant, when spatial LIC is applied to CU, it is also defined using a scaling factor α and an offset β based on a linear model of spatial illumination variation. The estimate of the spatial LIC parameters is derived by minimizing the difference between the neighboring reconstructed samples of the current block (i.e., template T in FIG. 6) and the corresponding neighboring reconstructed samples of the spatial reference block in the same picture.

[0057] In FIG. 6, the top / left spatial neighboring block of the current block is used as the reference block, and the neighboring reconstructed samples of the top / left block (i.e., T in FIG. 6) are used as the reference blocks. A Or T L If the spatial neighboring blocks of the current block are available, the spatial LIC parameters (α A and β A ) is estimated using the LMSE-based LIC derivation as follows:

[0058]

number

[0059] Left space LIC parameter (α L and β LA similar estimation process for , when the left spatial neighboring blocks of the current block are available, is derived as follows:

[0060]

number

[0061] If only the upper or left spatial neighboring block is available, the upper spatial LIC parameter α A and β A ), or the left LIC parameter (α L and β L ) is applied to the normal motion compensated predicted samples to obtain the final predicted samples of the current block. P(x,y)=α A / L P r (x+v x ,y+v y )+β A / L .(9)

[0062] If both the upper and left spatial neighboring blocks are available, the upper and left spatial LIC parameters are A and T and T L and T separately, and then the final predicted samples of the current block are generated by applying the final spatial LIC parameters, which are obtained by averaging the top and left spatial LIC parameters, as shown below.

[0063]

number

[0064] FIG. 7 shows a decoding method according to a first embodiment, where spatial LIC is applied during the decoding of an inter block, for example using the top / left neighboring block. The input to the algorithm is the current CU to be decoded in the current inter picture. If the top or left spatial neighboring block of the current spatial neighboring block is available (step 1040), it consists in analyzing the spatial LIC flag spatial_lic_flag, which indicates the use of the proposed spatial LIC process in the current CU. In case of merge mode, spatial_lic_flag is inferred from the neighboring blocks (step 1051), in a similar way to the prior art LIC in merge mode. In case of AMVP mode, spatial_lic_flag is decoded from the bitstream (step 1052).

[0065] If spatial_lic_flag is false, then only the normal motion compensation decoding process, for example as specified by the VVC decoding process, is involved. If spatial_lic_flag is true, the next step 1070 consists of estimating the spatial LIC parameters using the available upper / left spatial neighboring blocks. If both the upper and left spatial neighboring blocks are available (step 1080), then in step 1090, the final spatial LIC parameters are obtained by averaging the upper and left spatial LIC parameters. Then, the final predicted samples of the current block are generated by applying the spatial LIC parameters to the normal motion compensation predicted samples, as shown in step 1100.

[0066] According to a variant of this embodiment, only the upper or left spatial LIC parameters are applied to the normal motion compensated prediction sample to obtain the final prediction sample of the current block, and the decision of which spatial reference block to use is made, for example, via a rate-distortion (RD) or sum of absolute difference (SAD) check. A flag lic_refblk_flag is signaled in the bitstream to indicate which spatial reference block and the corresponding spatial LIC parameter set are applied. When lic_refblk_flag is equal to 0, the left spatial LIC parameters are applied. Otherwise, the above spatial LIC parameters are applied.

[0067] As mentioned above, when both the upper and left spatial neighboring blocks are available, the upper and left spatial LIC parameters are derived separately. Then, the upper and left spatial LIC parameters are averaged to generate the final spatial LIC parameters, and applied to obtain the final predicted sample of the current block. If such a method needs to perform the LMSE-based spatial LIC derivation twice, it will bring about a non-negligible increase in complexity in both the encoder and the decoder.

[0068] According to another variant of this embodiment, in order to reduce the complexity of the proposed spatial LIC derivation, an improved spatial LIC algorithm is proposed for the case where both the upper and left spatial neighboring blocks are available. Figure 8 illustrates the derivation of spatial LIC parameters process using the average reference template of the upper and left neighboring blocks for inter prediction according to at least one embodiment. Specifically, instead of deriving the upper and left spatial LIC parameters separately, the reference template T ave First, the block T A and the left block T L is generated by averaging the reconstructed samples of the two templates in

[0069]

number

[0070] Then, an LMSE-based derivation is used to calculate the values ​​of the scaling factors α and the offset β reference template T ave and the template T of the current block are used for spatial LIC.

[0071]

number

[0072] Finally, the derived spatial LIC parameters are applied to the normal motion compensated prediction samples to obtain the final prediction samples of the current block based on the linear model shown in FIG.

[0073] Therefore, in this variant, only one spatial LIC parameter estimation needs to be performed to form the final predicted samples of the current block.

[0074] According to the second embodiment, motion vector prediction (MVP) candidates are used as reference blocks in inter prediction. Figure 9 shows the location of spatial MVP candidates in VVC. For inter prediction, MVs can be signaled in either merge mode or AMVP mode. Both signaling mechanisms basically utilize a motion vector prediction (MVP) list built from available motion information from spatial or temporal neighboring blocks of the currently coded block. The location of spatial MVP candidates is shown in Figure 9. The order of derivation is B0 (top), A0 (left), B1 (top right), A1 (bottom left) and B2 (top left). As explained before, instead of using only the top (B0) and left (A0) spatial neighboring blocks of the current block as reference blocks, other spatial neighboring blocks for MVP list construction are also considered as reference block candidates for spatial LIC. When spatial LIC is applied to the current block, once one of the five spatial candidates is selected as the best MVP candidate, the spatial LIC parameters are automatically derived with the corresponding selected spatial neighboring block.

[0075] FIG. 10 illustrates the derivation of spatial LIC parameters process using a reference template of the top right (B1) neighboring block for inter prediction, according to at least one embodiment. If the top right (B1) spatial neighboring block of the current block is selected, it is used as a reference block for spatial LIC, as shown in FIG. 10. The neighboring reconstructed sample (T AR (Figure 10) is used to estimate the spatial LIC parameters. The spatial LIC parameters (α AR and β AR ) is estimated using the LMSE-based LIC derivation as follows:

[0076]

number

[0077] Figure 11 shows a decoding method according to a second embodiment in which spatial LIC is applied during decoding of an inter block based on an MVP candidate. If the MVP is one of the five spatial MVP candidates (step 2050), the method includes parsing the spatial LIC flag spatial_lic_flag, which indicates the use of the proposed spatial LIC process in the current CU. In the case of merge mode, spatial_lic_flag is inferred from neighboring blocks in a manner similar to the prior art LIC in merge mode (step 2061). In the case of AMVP mode, spatial_lic_flag is decoded from the bitstream (step 2062).

[0078] If spatial_lic_flag is false, only the normal motion compensation decoding process is included. If spatial_lic_flag is true, the next step 2080 includes estimating spatial LIC parameters using the corresponding selected spatial neighboring blocks. Then, as shown in step 2090, the final predicted sample of the current block is generated by applying the spatial LIC parameters to the normal motion compensation predicted sample.

[0079] According to yet another variation of this embodiment, when spatial LIC is applied to the current block, instead of only one of the five spatial candidates being selected as the best MVP candidate, the spatial LIC parameters from these five spatial neighboring blocks are applied to obtain the final predicted sample of the current block. The decision of which spatial reference block set to use can be made via a rate-distortion (RD) or sum of absolute difference (SAD) check. An index lic_refblk_index is signaled in the bitstream, indicating which spatial reference block and the corresponding spatial LIC parameter set are applied.

[0080] According to the third embodiment, spatial LIC is applied during the encoding / decoding of intra blocks. As described above for inter prediction, spatial LIC is proposed to compensate for spatial illumination variations within the same frame. Although illumination variations can be gradually propagated across intra-coded frames, the intra blocks to be encoded / decoded can also include these gradually propagated spatial illumination variations.

[0081] As specified by VVC, planar and DC intra prediction modes are used to predict smooth and gradually changing regions, and angular prediction modes are used to capture different directional structures. However, even DC and planar intra prediction modes are intended for smooth and gradually changing content, and cannot properly handle some content with directional and gradually propagating illumination variations, and have similar limitations to other directional intra prediction modes. Therefore, the third embodiment proposes to apply spatial LIC to compensate spatial illumination variations for intra prediction.

[0082] As previously described for inter blocks, a spatial LIC flag, spatial_lic_flag, is defined and signaled for intra blocks to indicate whether spatial LIC is applied or not. When spatial LIC is applied, it also uses a scaling factor, α, and an offset β, based on a linear model of spatial illumination variation. An estimate of the spatial LIC parameters is also derived by minimizing the difference between the neighboring reconstructed samples of the current block and the corresponding neighboring reconstructed samples of the spatial reference block in the picture.

[0083] There are some differences between spatial LIC for inter prediction and spatial LIC for intra prediction with regard to selecting possible spatial reference blocks. For example, the spatial neighboring blocks used to estimate spatial LIC parameters are determined based on the intra prediction mode. Moreover, instead of considering both the top and left boundaries, only the top or left boundaries are used to construct a template, which is then used to estimate spatial LIC parameters. Moreover, according to yet another variant, the template is generated not only by the reconstructed samples in the neighboring first top / left line, but also, for example, by the reconstructed samples in the second / third top / left line, or even more top / left lines, or the entire reconstructed neighboring blocks. According to another variant, the proposed spatial LIC for intra prediction is activated only for some intra prediction modes (i.e., DC and planar modes).

[0084] According to a variant of the third embodiment, the spatial LIC is applied during encoding / decoding of an intra block based on the intra prediction mode. The spatial LIC parameters for intra prediction are estimated using an LMSE-based LIC derivation, using the adjacent reconstructed samples of the closest reconstructed spatial neighboring blocks (i.e., up / left / top-right / bottom-left / top-left in FIG. 9). According to a non-limiting example, the decision of which spatial neighboring blocks to use is made via a rate distortion (RD) check or a sum of absolute differences (SAD) check. An index lic_refblk_index indicating which spatial reference block and corresponding spatial LIC parameter set are applied is signaled in the bitstream.

[0085] FIG. 12 shows the intra prediction directions in VVC. VVC supports 95 direction prediction modes indexed from -14 to -1 and 2 to 80. For square CUs, only prediction modes 2 to 66 are used. These prediction modes correspond to different prediction directions from 45 degrees clockwise to -135 degrees. For rectangular blocks, wide-angle modes (-14 to -1 or 67 to 80) can be applied. For some flat blocks (W > H) and tall blocks (W < H), they use the wide-angle mode to replace an equal number of regular angle modes in the opposite direction. According to the variant, instead of indicating which spatial neighboring blocks are applied with additional syntax elements, the reference blocks in the spatial LIC for intra prediction can be determined based on the intra prediction mode (IPM). FIG. 13 shows the derivation of the spatial LIC parameter process using the reference templates of the up / left / top-right / bottom-left / top-left neighboring blocks for intra prediction according to the third embodiment, where the spatial reference blocks respond to the intra prediction mode used to encode the current block. Thus, - For non-angle modes, for planar (IPM equal to 0) and DC (IPM equal to 1), the adjacent reconstructed samples T A and T L in FIG. 13) are used to estimate the spatial LIC parameters. In the case of horizontal mode (IPM is 18) and the other 30 modes in the horizontal direction (IPM 3 to 33), only the left block is used as the reference block and its neighboring reconstructed samples. L In FIG. 13, it is used for spatial LIC parameter estimation. On the other hand, when the vertical mode IPM is 50) and the other 30 modes belong to the vertical direction (IPM 35 to 65), the upper block T in FIG. A Only the neighboring reconstructed samples of are used for spatial LIC parameter estimation. For diagonal modes that represent angles that are multiples of -45 degrees, ○ 45° mode (IPM is 2), adjacent reconstructed samples of the bottom left block (T in Fig. 13) BL ) is used for spatial LIC parameter estimation. ○ −45° mode (IPM is 34), adjacent reconstructed samples of the top left block (T in Fig. 13) AL ) is used. ○ −135° mode (IPM is 66), the neighboring reconstructed samples of the top right block (T_AR in FIG. 13) are used. - For wide-angle modes beyond the bottom-left direction (IPM-1 to IPM-14), the bottom-left block is used as the reference block and its neighboring reconstructed samples are used T BL Refer to Figure 13 for spatial LIC parameter estimation. On the other hand, for wide-angle modes beyond the upper right direction, IPM 67-80) and the upper right block (T_ AR ) adjacent reconstructed samples are used for spatial LIC parameter estimation.

[0086] The templates used to estimate the spatial LIC parameters are for the intra prediction mode IPM as shown in Table 1.

[0087] [Table 1]

[0088] According to another variant of the third embodiment, the intra prediction mode is matrix weighted intra prediction. Figure 14 shows the matrix weighted intra prediction process in VVC. In addition to traditional intra prediction, matrix weighted intra prediction (MIP) method is a new intra prediction technique added to VVC. To predict a sample of a rectangular block of width W and height H, MIP takes as input one line of H reconstructed adjacent boundary samples to the left of the block and one line of W reconstructed adjacent boundary samples above the block, if these reconstructed samples are available. The generation of the prediction signal is based on three steps: averaging, matrix vector multiplication, and linear interpolation, as shown in Figure 14. For each CU of intra mode, a flag mip_flag is sent to indicate whether MIP mode is applied or not.

[0089] When spatial LIC is applied to this intra-coded CU with MIP, the templates used to estimate the spatial LIC parameters are the same as for CUs with non-angular mode, and the top and left blocks (T A and T L ) are used.

[0090] Figure 15 shows a decoding method according to a third embodiment in which spatial LIC is applied during decoding of intra blocks. Similar to spatial LIC for inter prediction, it involves parsing the spatial LIC flag spatial_lic_flag decoded from the bitstream (steps 3303 / 3313). If spatial_lic_flag is false, only the normal intra prediction decoding process is involved. If spatial_lic_flag is true, the proposed spatial LIC process is performed on the decoded intra prediction of the current CU using the following steps:

[0091] If the block is intra predicted using MIP (step 3300), an estimation of the spatial LIC parameters using the spatially above and left neighboring blocks is performed (step 3314). If the block is intra predicted with conventional intra prediction, the template decision for the spatial LIC parameters is based on the intra prediction mode IPM (step 3304). The next step 3305 then consists of estimating the spatial LIC parameters using the corresponding selected template.

[0092] The final predicted samples of the current block are then generated by applying the spatial LIC parameters to the regular intra predicted samples, as shown in steps 3306 / 3315.

[0093] According to a variation of this embodiment, in the case of DC mode and planar mode, instead of using only the upper and left spatial neighboring blocks of the current block as reference blocks, the other three templates from the lower left, upper left, and upper right can also be used together for the spatial LIC parameters.

[0094] According to another variation of this embodiment, when the intra prediction mode belongs to the horizontal / vertical direction, instead of only one of the five spatial templates being selected as the template for estimating the spatial LIC parameters, two or three templates can be used together to calculate the spatial LIC parameters. For example, for the modes belonging to the horizontal direction (IPM 3-33), the left, bottom left, and top left blocks can be used as reference blocks, and their neighboring reconstructed samples (T L ,T BL and T AL 13) are used for spatial LIC parameter estimation. For the modes belonging to the vertical direction (IPM 35 to 65), the top, top right, and top left templates (T in FIG. 13) are used as reference templates for spatial LIC parameter estimation. A ,T AR and T AL) can be used. These templates work together in a similar manner as the left and top templates for DC and planar modes.

[0095] According to a further variant of the third embodiment, several shapes are disclosed for the template used to estimate the spatial LIC parameters. As mentioned before, the template used to estimate the spatial LIC parameters is always L-shaped around the current / reference block, which is composed by the adjacent reconstructed samples located at the left and top boundaries of the current / reference block. Instead of using this fixed L-shaped template, several more flexible template generation methods are proposed in this section.

[0096] According to a first variant, only the left or top boundary of the spatial reference block is used as the template. According to a previous variant of the third embodiment, the selection of the reference template is derived from the intra prediction mode IPM to enhance the different influence of the illumination variation from the left and top reference samples in some circumstances.

[0097] For the horizontal modes (IPM 3-33), the left reference template T L For the modes IPM 35-65) belonging to the vertical direction, the reference template T A The top of the reference template is considered. For either the left or top reference template, it includes both reconstructed samples located on the left and top boundaries. In order to better capture the propagation of illumination variations and to reduce the computational complexity of spatial LIC parameter estimation, only the reconstructed samples located on one boundary are used to construct the template.

[0098] 16 illustrates the derivation of spatial LIC parameters process using a reference template including the left boundary of the left neighboring block for intra prediction and a reference template including the top boundary of the top neighboring block for intra prediction, according to at least one embodiment. For example, for horizontal directional modes (IPM 3-33), only the reconstructed samples located on the left boundary are used to derive the template T of the current block. H Template T' in the left and reference blocks of FIG. H For vertical modes (IPM 35-65), we only consider the upper boundary and generate the template for the current block (T V ) and the template of the reference block) and (T' on the right of Figure 16 V )

[0099] According to a second variant, multiple reference lines of a spatial reference block are used as a template. Figure 17 shows the derivation of spatial LIC parameter process using multiple line reference templates of spatial neighboring blocks according to at least one embodiment. So far, the proposed template for spatial LIC uses only the reconstructed samples located on the nearest reference line (top / left boundary). For better capture and compensation of illumination mismatch, multiple reference lines are used to construct the template. As shown in Figure 17, an example of two reference lines is shown, and the neighboring reconstructed samples located on one additional top-left line are used to generate the template of the current block (T in Figure 17) and the template of the reference block (T' in Figure 17). To reduce the computational complexity, the template samples in the two reference lines are both subsampled (2:1 subsampling). It can be subsampled at the same position for both reference lines (in the upper example of Figure 17) or at an interlaced position (in the lower example of Figure 17).

[0100] According to another variant, the left border template is applied to the horizontal mode and the top border template is used for the vertical mode. The computational complexity is reduced with fewer samples in the template, but the estimation accuracy of the illumination variation may also be affected. Therefore, according to another variant of this embodiment, multiple reference lines from the left / top side only are applied to the horizontal / vertical mode. Figure 18 shows another derivation of the spatial LIC parameter process using multiple line reference templates of spatially neighboring blocks for intra prediction according to at least one embodiment. An example of two reference lines from the same side of only one spatial reference block is shown in Figure 18. For intra prediction mode, the left line is used for the horizontal mode (top example of Figure 18) and the right line is used for the vertical mode (bottom example of Figure 18).

[0101] According to another variant, a flag lic_mrl_flag is signaled in the bitstream indicating whether multiple reference lines are applied to construct the template. If lic_mrl_flag is false, only the conventional closest reference line (top / left boundary) is applied to generate the template.

[0102] According to another variant, the template with multiple reference lines is applied in spatial LIC parameter estimation for inter prediction. In fact, different aspects of the multiple line reference template are described for spatial LIC applied in intra prediction. However, this is for the purpose of clarity of description and does not limit the application or scope of these aspects to either intra prediction or spatial LIC. In fact, any of the different aspects can be combined and exchanged to provide a template with multiple reference lines applied in spatial LIC parameter estimation for inter prediction, or the template with multiple reference lines is applied in the prior art LIC parameter estimation for inter prediction.

[0103] According to another variant, the template includes the entire reconstructed neighboring block. Figure 19 shows the derivation of spatial LIC parameter process using reference template including spatial neighboring block according to at least one embodiment. In order to further improve the estimation accuracy of illumination variation, without considering computational complexity, the template is generated by using all reconstructed samples of neighboring block as available. As an example, either of the reconstructed left and upper neighboring blocks of the current block is used to generate the template of the current block (T in Figure 19), or either of the reconstructed left and upper neighboring blocks of the reference block constitutes the template of the reference block (T' in Figure 19).

[0104] According to a variant, only for small blocks (block size ≦8×8) the template is generated using the reconstructed neighboring blocks. Advantageously, this feature makes it possible to reduce the complexity of the variant of FIG.

[0105] According to another variant, the use of reconstructed neighboring blocks as templates is applied in the spatial LIC parameter estimation for inter prediction or in the prior art LIC parameter estimation for inter prediction.

[0106] According to the fourth embodiment, spatial LIC is applied during encoding / decoding of IBC blocks. Figure 20 shows IBC prediction in VVC. Intra block copy (IBC) is a screen content coding (SCC) tool implemented in VVC. For IBC prediction, block matching (BM) is performed in the encoder to find the optimal block vector (or motion vector) for each CU (as shown in Figure 20). Here, the block vector is used to indicate the displacement from the current block to a reference block already reconstructed in the current picture. IBC coded CUs are treated as a third prediction mode other than intra prediction mode or inter prediction mode. It is well known that IBC significantly improves the coding efficiency of screen content material (including game video content). Therefore, the fourth embodiment relates to applying spatial LIC to compensate spatial illumination variation for IBC prediction.

[0107] Compared with the spatial LIC for inter / intra prediction described above, the spatial reference block used for spatial LIC estimation for IBC prediction is the intra copy (i.e., template T in FIG. 21). IBC ) is the same reference block used for IBC(α IBC and β IBC The estimation process of the spatial LIC parameters for is derived as follows:

[0108]

number

[0109] FIG. 22 shows the decoding process according to the fourth basic embodiment, where spatial LIC is applied during the decoding of an IBC block. The input to the algorithm is the current IBC CU to be decoded in the current intra picture. It consists in analyzing the spatial LIC flag spatial_lic_flag, which indicates the use of the proposed spatial LIC process in the current CU (step 4030). If spatial_lic_flag is false, only the normal IBC predictive decoding process is involved. If spatial_lic_flag is true, the block vectors (bv x ,bv y ) is used for estimating the spatial LIC parameters (step 4050). Then, the final predicted samples of the current block are generated by applying the spatial LIC parameters to the IBC predicted samples, as shown in step 4060.

[0110] According to a fifth embodiment, the spatial reference block is searched in spatial LIC for intra-prediction and inter-prediction. As mentioned before, the spatial LIC parameters for intra / inter prediction are estimated using the nearest reconstructed spatial neighboring block (top / left / top-right / bottom-left / top-left as shown in the exemplary FIG. 13). According to yet another variant, some non-nearest spatial neighboring blocks while being within a predefined search area are considered as reference blocks for spatial LIC parameter estimation for intra / inter prediction. In this case, a spatial LIC search vector to indicate the displacement from the current block to the spatial reference block is signaled in the bitstream.

[0111] Further embodiments and information Various aspects are described in this application, including tools, features, embodiments, models, methods, and the like. Many of these aspects are described with specificity and in what may often sound definitive terms to at least indicate their individual characteristics. However, this is for purposes of clarity of description and does not limit the applicability or scope of the aspects. In fact, all of the different aspects can be combined and substituted to provide further aspects. Moreover, these aspects can also be combined and substituted with aspects described in previous applications as well.

[0112] The aspects described and contemplated in this application can be implemented in many different forms. Figures 23, 24, and 25 below provide some embodiments, but other embodiments are contemplated, and the descriptions of Figures 23, 24, and 25 do not limit the scope of implementations. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0113] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably.

[0114] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. In addition, terms such as "first", "second", etc. may be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding". The use of such terms does not imply a modified ordering of operations unless specifically required. Thus, in this example, the first decoding need not be performed before the second decoding, but may occur, for example, before, during, or during an overlapping time with the second decoding.

[0115] Various methods and other aspects described in the present application may be used to modify modules of the video encoder 100 and the video decoder 200 as shown in Figures 23 and 24, such as intra-prediction and / or inter-prediction modules, entropy coding modules, and / or decoding modules (160, 170, 260, 275). Furthermore, aspects of the present disclosure are not limited to VVC or HEVC, but may be applied to other standards and recommendations, such as existing or future developments, or extensions of any such standards and recommendations, including VVC and HEVC. Unless otherwise specified or technically precluded, aspects described in the present application may be used individually or in combination.

[0116] In this application, various numerical values ​​are used, such as the number of transforms, the number of transform levels, transform indexes, etc. The specific values ​​are for illustrative purposes and the described aspects are not limited to these specific values.

[0117] 23 illustrates an encoder 100. Variations of this encoder 100 are contemplated, but for clarity, the encoder 100 is described below without describing all possible variations.

[0118] Before being encoded, a video sequence may undergo pre-encoding processing (101), such as applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0119] In the encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is divided (102) into units, e.g., CUs, and processed. Each unit is coded, e.g., using either intra mode or inter mode. When a unit is coded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder decides (105) whether to use intra mode or inter mode to code the unit, and indicates the intra / inter decision, e.g., by a prediction mode flag. A prediction residual is calculated (110), e.g., by subtracting the predicted block from the original image block.

[0120] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is directly coded without applying a transform or quantization process.

[0121] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the prediction block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / Sample Adaptive Offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0122] Figure 24 shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs a decoding pass that is the reverse of the encoding pass as described in Figure 24. The encoder 100 also generally performs video decoding as part of encoding the video data.

[0123] In particular, the decoder's input includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coding information. Picture partition information indicates how the picture is partitioned. The decoder may therefore partition the picture according to the decoded picture partition information (235). The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined with the prediction block (255) to reconstruct an image block. A prediction block can be obtained (270) from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0124] The decoded picture may further undergo post-decoding processing (285), such as an inverse color conversion (e.g., from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding processing (101). The post-decoding processing may use metadata derived in the pre-encoding processing and signaled in the bitstream.

[0125] FIG. 25 illustrates a block diagram of an example of a system in which various aspects and embodiments are implemented. The system 5000 may be embodied as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include various electronic devices, such as, but not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of the system 5000, alone or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of the system 5000 are distributed across multiple ICs and / or separate components. In various embodiments, the system 5000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, the system 5000 is configured to implement one or more of the aspects described herein.

[0126] The system 5000 includes at least one processor 5010 configured to execute instructions loaded therein, for example to implement various aspects described herein. The processor 5010 can include embedded memory, input / output interfaces, and various other circuits known in the art. The system 5000 includes at least one memory 5020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 5000 includes a storage device 5040, which can include non-volatile memory and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash, magnetic disk drives, and / or optical disk drives. The storage devices 5040 may include, by way of non-limiting examples, internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0127] The system 5000 includes an encoder / decoder module 5030 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. The encoder / decoder module 5030 represents a module that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 5030 may be implemented as a separate element of the system 5000, or may be incorporated within the processor 5010 as a combination of hardware and software, as is known to those skilled in the art.

[0128] Program code loaded into the processor 5010 or the encoder / decoder 5030 to perform various aspects described herein may be stored in the storage device 5040 and subsequently loaded into the memory 5020 for execution by the processor 5010. According to various embodiments, one or more of the processor 5010, the memory 5020, the storage device 5040, and the encoder / decoder module 5030 may store one or more of various items during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, expressions, operations, and operational logic.

[0129] In some embodiments, memory internal to the processor 5010 and / or the encoder / decoder module 5030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 5010 or the encoder / decoder module 5030) is used for one or more of these functions. The external memory may be the memory 5020 and / or the storage device 5040, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations such as MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by JVET).

[0130] Inputs to the elements of system 5000 can be provided through various input devices, shown in block 5005. Such input devices include, but are not limited to, (i) a Radio Frequency (RF) section that receives, for example, an RF signal transmitted throughout a broadcast by a broadcaster, (ii) a Component (COMP) input terminal (or set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Although not shown in FIG. 25, another example includes composite video.

[0131] In various embodiments, the input devices of block 5005 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select a signal frequency band that may be referred to as a channel in a particular embodiment (for example), (iv) demodulating the down-converted band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or to baseband. In one embodiment of a set-top box, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and refiltering to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0132] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting the system 5000 to other electronic devices via USB and / or HDMI connections. It will be appreciated that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or implemented within the processor 5010, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, for example, in a separate interface IC or implemented within the processor 5010, as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, the processor 5010 and an encoder / decoder 5030, which operates in combination with memory and storage elements to process the data stream as desired for presentation on an output device.

[0133] The various elements of the system 5000 can be provided within an integrated housing in which the various elements can be interconnected and data can be transmitted between them using a suitable connection device 5015, such as an internal bus as is known in the art, including an Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0134] The system 5000 includes a communication interface 5050 that enables communication with other devices over a communication channel 5090. The communication interface 5050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 5090. The communication interface 5050 may include, but is not limited to, a modem or a network card, and the communication channel 5090 may be implemented within a wired and / or wireless medium, for example.

[0135] Data is streamed or otherwise provided to the system 5000 in various embodiments using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received over a communication channel 5090 and communication interface 5050 that are compatible with Wi-Fi communication. The communication channel 5090 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. In other embodiments, streaming data is provided to the system 5000 using a set-top box that delivers data via an HDMI connection in the input block 5005. In yet other embodiments, streaming data is provided to the system 5000 using an RF connection in the input block 5005. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0136] The system 5000 can provide output signals to various output devices including a display 5065, speakers 5075, and other peripheral devices 5085. The display 5065 in various embodiments includes one or more of a touch screen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display, for example. The display 5065 can be for a television, a tablet, a laptop, a mobile phone, or other device. The display 5065 can also be integrated with other components (e.g., as in a smartphone) or can be separate (e.g., an external monitor for a laptop). The other peripheral devices 5085 include one or more of a stand-alone digital video disc (or digital versatile disc, DVR) (for both terms), a disc player, a stereo system, and / or a lighting system, in various examples of embodiments. Various embodiments use one or more peripheral devices 5085 to provide functionality based on the output of the system 5000. For example, a disc player performs the function of playing the output of the system 5000 .

[0137] In various embodiments, control signals are communicated between the system 5000 and the display 5065, speakers 5075, or other peripheral devices 5085 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow control between devices with or without user intervention. The output devices can be communicatively coupled to the system 5000 via dedicated connections through the respective interfaces 5065, 5075, and 5085. Alternatively, the output devices can be connected to the system 5000 using a communication channel 5090 via the communication interface 5050. The display 5065 and speakers 5075 can be integrated into a single unit with other components of the system 5000 in an electronic device such as, for example, a television. In various embodiments, the display interface 5065 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0138] For example, if the RF portion of input 5005 is part of a separate set-top box, display 5065 and speakers 5075 may alternatively be separate from one or more of the other components. In various embodiments where display 5065 and speakers 5075 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0139] The embodiments may be performed by computer software implemented by the processor 5010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 5020 may be of any type suitable for the technology environment, and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and rib-bubble memories. The processor 5010 may be of any type suitable for the technology environment, and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0140] Various implementations include decoding. As used herein, "decoding" can encompass all or part of the processes performed on a received encoded sequence to provide a final output suitable for, for example, a display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as, for example, entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by the decoder of various implementations described herein, including, for example, deriving parameters for spatial LIC and applying spatial LIC to either inter-, intra-, or IBC prediction.

[0141] As a further example, in one embodiment, "decoding" refers to entropy decoding only, in another embodiment, "decoding" refers to differential decoding only, and in another embodiment, "decoding" refers to a combination of entropy and differential decoding. Whether the phrase "encoding process" is intended to refer to a task subset specifically or to the broader encoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0142] Various implementations involve encoding. Similar to the above discussion of "decoding," "encoding" as used herein can encompass all or part of the processes performed on an input video sequence to produce, for example, an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, such as, for example, partitioning, differential encoding, transforming, quantizing, and entropy encoding. In various embodiments, such processes also or alternatively include processes performed by the encoder in various implementations described herein, such as, for example, deriving parameters for spatial LIC and applying spatial LIC to either inter-, intra-, or IBC prediction.

[0143] As a further example, in one embodiment, "encoding" refers to entropy encoding only, in another embodiment, "encoding" refers to differential encoding only, and in another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to refer to a task subset specifically or to the broader encoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0144] It should be noted that the syntax elements used herein, e.g., spatial_lic_flag, lic_refblk_index, lic_mrl_flag, are descriptive terms, and therefore do not exclude the use of other syntax element names.

[0145] Where a figure is presented as a flow chart, it should be understood that the figure also provides a block diagram of the corresponding apparatus. Similarly, where a figure is presented as a block diagram, it should be understood that the figure also provides a flow chart of the corresponding method / process.

[0146] Various embodiments refer to rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is usually considered, often giving a computational complexity constraint. Rate-distortion optimization is usually formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate-distortion optimization problem. For example, these approaches may be based on an extensive test of all encoding options, including all considered modes or encoding parameter values, with a full evaluation of their encoding costs and associated distortion of the reconstructed signal after encoding and decoding. To keep the encoding complexity down, more rapid approaches may also be used, especially with the calculation of approximate distortion based on the prediction or prediction residual signal, rather than the reconstructed signal. A mixture of these two approaches may also be used, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a full evaluation of both the coding cost and the associated distortion.

[0147] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., an apparatus or a program). For example, an apparatus may be implemented in appropriate hardware, software, and firmware. The method may be implemented in a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0148] Reference to "one embodiment" or "an embodiment" or "one implementation" or "an implementation," or other variations thereof, means that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "an embodiment" or "in one implementation," or other variations thereof appearing in various places throughout this application are not necessarily all referring to the same embodiment.

[0149] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0150] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0151] Additionally, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" generally involves in some way, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0152] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," it should be understood that use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed alternative (A), or selection of only the second listed alternative (B), or selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). This may be expanded as many times as the number of items listed, as would be apparent to one of ordinary skill in the art and related arts.

[0153] Also, as used herein, the term "signaling" means to indicate something, especially to a corresponding decoder. For example, in a particular embodiment, an encoder signals a particular one of a plurality of parameters for a transform. Thus, in an embodiment, the same parameter is used at both the encoder and decoder sides. Thus, for example, an encoder can transmit a particular parameter to a decoder so that the decoder can use the same particular parameter (explicit signaling). In contrast, if the decoder already has the particular parameter as well as other parameters, a non-transmitting signaling (implicit signaling) can be used to simply allow the decoder to know and select the particular parameter. By avoiding the transmission of any actual function, bit savings are realized in various embodiments. It will be understood that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used in various embodiments to signal information to a corresponding decoder. The above relates to the verb form of the word "signal", which may also be used as a noun in this specification.

[0154] This disclosure has described various information, such as syntax, that may be transmitted or stored. This information may be packaged or arranged in various manners, including, for example, manners common in video standards, such as placing the information in an SPS, PPS, NAL unit, header (e.g., a NAL unit header or slice header), or SEI message. Other manners are also available, including, for example, manners common in system-level or application-level standards, such as placing the information in one or more of the following: • Session Description Protocol (SDP) is a format for describing multimedia communication sessions for purposes of session announcement and session invitation, described in, for example, RFCs and used in conjunction with Real-time Transport Protocol (RTP) transport. ● DASH MPD (Media Presentation Description) Descriptors, e.g. as used in DASH and transmitted over HTTP, Descriptors are associated with a representation or a collection of representations to provide additional characteristics to the content representation. RTP header extensions, e.g. as used during RTP streaming, and / or • The ISO Base Media File Format, such as that used in OMAF and in some specifications uses boxes, which are object-oriented building blocks defined by a unique type identifier and a length, also known as "atoms".

[0155] As will be apparent to one skilled in the art, implementations can result in a variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bit stream of the described embodiments. For example, such a signal can be formatted as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information that the signal carries can be, for example, analog information or digital information. As is known, the signal can be transmitted over a variety of different wired or wireless links. The signal can be stored in a processor-readable medium.

[0156] Several embodiments are described. The features of these embodiments may be provided alone or in any combination across various claim categories and types. Furthermore, the embodiments may include one or more of the following features, devices, or aspects, alone or in any combination across various claim categories and types. ● Apply spatial local illumination compensation for inter / intra / IBC prediction at the decoder and / or encoder to compensate for illumination mismatch between different blocks in the same picture. o A CU level spatial LIC flag, spatial_lic_flag, is defined for Inter / Intra / IBC blocks to indicate whether spatial LIC is applied to the block or not. When spatial LIC is applied (spatial_lic_flag(true) for Inter / Intra / IBC blocks, use a linear model of the spatial illumination variation, using scaling factor α and offset β. ○ - Estimates of the spatial LIC parameters are derived by minimizing the differences between neighboring reconstructed samples of the current block (current template) and corresponding neighboring reconstructed samples of a spatial reference block (reference template) in the same picture. Apply spatial LIC parameters to the Inter / Intra / IBC prediction samples to obtain the final prediction samples. Derive the CU level spatial LIC flag spatial_lic_flag in the decoder and / or encoder. For inter blocks, the spatial LIC flag is copied from the neighboring block if it is coded in merge mode, in a similar way to the motion information copy in merge mode, otherwise the spatial LIC flag is signaled, For Intra / IBC blocks, the spatial LIC flag is signaled, For intra blocks, the spatial LIC flag is only present for some intra prediction modes (i.e. DC mode and planar mode). Select spatial neighboring blocks to be used as reference blocks for spatial LIC parameter estimation in the decoder and / or encoder. For inter / intra blocks, the closest reconstructed spatial neighboring block is selected as the reference block, ○ Consider only the two closest available spatially adjacent blocks (above and to the left) ○ - if both the upper and left spatial neighboring blocks are available, they can both be applied as reference blocks; If both upper and left spatial neighboring blocks are available and only one reference block applies, add a flag lic_refblk_flag to indicate which one applies; Consider only the 5 closest spatially adjacent blocks available (top / left / top-right / bottom-left / top-left), If all five of these spatial neighbors are available and only one reference block applies, add the flag lic_refblk_index to indicate which reference block applies, For inter blocks, once one of the five spatial candidates is selected as the best MVP candidate, the block in which the selected spatial MVP candidate is located is selected as the reference block, For intra blocks, the reference block selection is based on the intra prediction mode. a spatial LIC search vector is signaled in the bitstream to indicate the displacement of the current block to the spatial reference block, taking into account some non-nearest spatial neighboring blocks while being within a predefined search area; For IBC blocks, the reference block used for intra-copy is selected as the reference block. generating templates composed of neighboring reconstructed samples for spatial LIC parameter estimation in the decoder and / or encoder; For Inter / Intra / IBC blocks, the template is constructed by the neighboring reconstructed samples located at the left and top boundaries of the current / reference block, For Inter / Intra / IBC blocks, the template is constructed by the neighboring reconstructed samples located within multiple left and top reference lines of the current / reference block, For Inter / Intra / IBC blocks, the template is constructed by the neighboring reconstructed samples located within multiple left and top reference lines of the current / reference block, For intra blocks, the template consists of the entire neighboring reconstructed blocks of the current / reference block. • Signaling information regarding the spatial LIC process to be applied at the decoder. • Derive information about the spatial LIC process to be applied from the template, the derivation being applied at the decoder and / or encoder. • Inserting syntax elements into the signaling that allow the decoder to identify the spatial LIC process to be used, such as a transform index. Based on these syntax elements, select at least one spatial LIC process to apply in the decoder. Applying modified spatial LIC to derive at least one prediction at the decoder. A bitstream or signal including one or more of the syntax elements described, or variations thereof. A bitstream or signal including syntax carrying information generated according to any of the described embodiments. • Inserting syntax elements into the signaling that allow the decoder to apply the spatial LIC process in a manner that corresponds to the method used by the encoder. ● Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described syntax elements, or variations thereof. ● Creating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments. A method, process, apparatus, instruction storage medium, data storage medium, or signal according to any of the described embodiments. A TV, set-top box, mobile phone, tablet, or other electronic device running a spatial LIC process adapted to correct predictions according to any of the described embodiments. ●A TV, set-top box, mobile phone, tablet, or other electronic device that performs a spatial LIC process adapted to correct predictions according to any of the described embodiments and displays the resulting images (e.g., using a monitor, screen, or other type of display). ●A TV, set-top box, mobile phone, tablet, or other electronic device that selects a channel for receiving a signal containing an encoded image (e.g., using a tuner) and performs a spatial LIC process adapted to modify a prediction according to any of the described embodiments. ●A TV, set-top box, mobile phone, tablet, or other electronic device that wirelessly receives a signal containing an encoded image (e.g., using an antenna) and performs a spatial LIC process adapted to modify a prediction according to any of the described embodiments.

Claims

1. A method for video decoding, comprising: for a current block in a picture, obtaining a prediction of said current block; determining, for the current block, parameters for local illumination compensation based on spatially neighboring reconstructed samples of at least one spatial reference block and corresponding spatially neighboring reconstructed samples; and decoding the current block by applying local illumination compensation to the prediction of the current block based on the determined parameters; A method, wherein the at least one spatial reference block is a spatial neighboring block of the current block in the same picture.

2. The method of claim 1, further comprising determining a syntax element indicating whether the local illumination compensation is applied to the current block.

3. The method of claim 2, wherein the current block is encoded in inter prediction.

4. The method described in claim 3, wherein the at least one spatial reference block is either an adjacent adjacent block above or an adjacent adjacent block to the left.

5. The method described in claim 3, wherein the at least one spatial reference block is any one of an adjacent block above, an adjacent block to the left, an adjacent block to the upper right, an adjacent block to the lower left, and an adjacent block to the upper left.

6. The method of claim 3, wherein the at least one spatial reference block is a neighboring block selected as a motion vector predictor MVP candidate.

7. The method of claim 2, wherein the current block is encoded using intra prediction.

8. The method described in claim 7, wherein at least one spatial reference block is an adjacent neighboring block that responds to the intra prediction mode used to encode the current block.

9. The method of claim 2, wherein the current block is encoded in intra block copy prediction.

10. The method of claim 9, wherein the at least one spatial reference block comprises the adjacent block selected as an intra block copy reference block.

11. A method for video encoding, comprising the steps of: for a current block in a picture, obtaining a prediction of said current block; determining, for the current block, parameters for local illumination compensation based on spatially neighboring reconstructed samples of the current block and corresponding spatially neighboring reconstructed samples of at least one spatial reference block; and encoding the current block by applying local illumination compensation to the prediction of the current block based on the determined parameters; The method, wherein the at least one spatial reference block is a spatial neighboring block of the current block in the same picture.

12. The method of claim 11, further comprising determining a syntax element indicating whether the local illumination compensation is applied to the current block.

13. The method of claim 12, wherein the current block is encoded in inter prediction.

14. The method of claim 13, wherein the at least one spatial reference block is either an adjacent adjacent block above or an adjacent adjacent block to the left.

15. The method described in claim 13, wherein the at least one spatial reference block is any one of an upper adjacent block, a left adjacent block, an upper right adjacent block, a lower left adjacent block, and an upper left adjacent block.

16. The method of claim 13, wherein the at least one spatial reference block is a neighboring block selected as a motion vector predictor MVP candidate.

17. The method of claim 12, wherein the current block is encoded using intra prediction.

18. The method described in claim 17, wherein at least one spatial reference block is an adjacent neighboring block that is responsive to the intra prediction mode used to encode the current block.

19. The method of claim 12, wherein the current block is encoded in intra block copy prediction.

20. The method of claim 19, wherein the at least one spatial reference block comprises the adjacent block selected as an intra block copy reference block.

21. An apparatus for video decoding, comprising one or more processors and at least one memory, the one or more processors comprising: for a current block in a picture, obtaining a prediction of said current block; determining, for the current block, parameters for local illumination compensation based on spatially neighboring reconstructed samples of at least one spatial reference block and corresponding spatially neighboring reconstructed samples; and decoding the current block by applying local illumination compensation to the prediction of the current block based on the determined parameters; The at least one spatial reference block is a spatially neighboring block of the current block in the same picture.

22. An apparatus for video encoding comprising one or more processors and at least one memory, the one or more processors comprising: for a current block in a picture, obtaining a prediction of said current block; determining, for the current block, parameters for local illumination compensation based on spatially neighboring reconstructed samples of at least one spatial reference block and corresponding spatially neighboring reconstructed samples; encoding the current block by applying local illumination compensation to the prediction of the current block based on the determined parameters; The at least one spatial reference block is a spatial neighboring block of the current block in the same picture.

Images