Local illumination compensation for video encoding or decoding

Abstract

In at least one embodiment, a method and apparatus for encoding / decoding a picture, comprising predicting at least one block, wherein the prediction comprises performing motion compensation and local illumination compensation based on a reference block, the local illumination compensation comprising applying a linear model based on the sum of absolute differences of adjacent reconstructed samples of the reference block and corresponding reference samples, wherein the adjacent reconstructed samples of the reference block and the corresponding reference samples are co-located according to an L-shape substantially adjacent to a block to be predicted, the L-shape comprising a row of pixels located on a top side of the prediction block and a column of pixels located on a left side of the prediction block, the co-location being determined according to a motion vector of the prediction block.

Description

Technical Field

[0001] At least one of the present embodiments generally relates to local illumination compensation for video encoding or decoding. Background Art

[0002] To achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original block and the predicted block (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded and decoded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy encoding and decoding, quantization, transformation, and prediction. Summary of the invention

[0003] One or more of the present embodiments provide a method and apparatus for encoding / decoding a picture using Local Illumination Compensation (LIC) parameters that are calculated using a derivation that provides reduced complexity and thus improves the performance of LIC related processes.

[0004] According to a first aspect of at least one embodiment, a video encoding method includes: predicting picture data of at least one block in a picture, wherein the prediction includes performing motion compensation and local illumination compensation based on a reference block, and the local illumination compensation includes applying a linear model based on the sum of absolute differences of adjacent reconstructed samples and corresponding reference samples of the reference block, wherein the adjacent reconstructed samples and the corresponding reference samples of the reference block are co-located according to an L-shape substantially adjacent to a block to be predicted, the L-shape including a row of pixels located on a top side of the prediction block and a column of pixels located on a left side of the prediction block, and the co-location is determined based on a motion vector of the prediction block.

[0005] According to a second aspect of at least one embodiment, a video decoding method includes: predicting picture data of at least one block in a picture, wherein the prediction includes performing motion compensation and local illumination compensation based on a reference block, and the local illumination compensation includes applying a linear model based on the sum of absolute differences of adjacent reconstructed samples and corresponding reference samples of the reference block, wherein the adjacent reconstructed samples and the corresponding reference samples of the reference block are co-located according to an L-shape that is substantially adjacent to the block to be predicted, and the L-shape includes a row of pixels located on the top side of the prediction block and a column of pixels located on the left side of the prediction block, and the co-location is determined based on the motion vector of the prediction block.

[0006] According to a third aspect of at least one embodiment, a device includes: an encoder for encoding picture data of at least one block in a picture or video, wherein the encoder is configured to predict the picture data of at least one block in the picture, wherein the prediction includes performing motion compensation and local illumination compensation based on a reference block, and the local illumination compensation includes applying a linear model based on the sum of absolute differences between adjacent reconstructed samples of the reference block and corresponding reference samples, wherein the adjacent reconstructed samples and the corresponding reference samples of the reference block are co-located according to an L-shape that is substantially adjacent to the block to be predicted, the L-shape including a row of pixels located on the top side of the prediction block and a column of pixels located on the left side of the prediction block, and the co-location is determined based on the motion vector of the prediction block.

[0007] According to a fourth aspect of at least one embodiment, a device includes: a decoder for decoding picture data of at least one block in a picture or video, wherein the decoder is configured to predict picture data of at least one block in a picture, wherein the prediction performs motion compensation and local illumination compensation based on a reference block, and the local illumination compensation includes applying a linear model based on the sum of absolute differences between adjacent reconstructed samples of the reference block and corresponding reference samples, wherein the adjacent reconstructed samples and the corresponding reference samples of the reference block are co-located according to an L-shape substantially adjacent to the block to be predicted, the L-shape including a row of pixels located on the top side of the prediction block and a column of pixels located on the left side of the prediction block, and the co-location is determined based on the motion vector of the prediction block.

[0008] According to variant aspects of the first, second, third and fourth aspects, the parameters of the linear model are calculated by:

[0009]

[0010] Where cur(r) is the adjacent reconstructed sample in the current picture, ref(s) is the reference sample constructed with motion compensation converted by the motion vector mv from the reference picture, and s=r+mv.

[0011] According to variant aspects of the first, second, third and fourth aspects, the parameter "a" is derived using an additional simple regularization term "corr" and is determined in the following manner:

[0012]

[0013] in And for example reg_shift=7.

[0014] According to a fifth aspect of at least one embodiment, a computer program comprising program code instructions executable by a processor is proposed, which computer program implements the steps of the method according to at least the first or second aspect.

[0015] According to a sixth aspect of at least one embodiment, a computer program product is proposed, which is stored on a non-transitory computer-readable medium and includes program code instructions executable by a processor, and the computer program product implements the steps of the method according to at least the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A block diagram of an example of a video encoder 100, such as a High Efficiency Video Codec (HEVC) encoder, is shown.

[0017] Figure 2 A block diagram of an example of a video decoder 200, such as a HEVC decoder, is shown.

[0018] Figure 3 Examples of codec tree units and codec trees in the compressed domain are shown.

[0019] Figure 4 An example of partitioning a CTU into coding units, prediction units, and transform units is shown.

[0020] Figure 5 An example of an "L-shape" for local illumination compensation is shown.

[0021] Figure 6 An example embodiment of a multi-model LIC is shown in which the models are split by a threshold.

[0022] Figure 7 A block diagram is shown of an example of a system in which various aspects and embodiments may be implemented.

[0023] Fig. 8A An example is shown in which a single model is determined based on extreme values.

[0024] Figure 8B Another example is shown in which a single model is determined based on the average extreme value.

[0025] Fig.9A The prediction method in the case of bidirectional prediction including the first method for deriving LIC parameters is shown.

[0026] Fig. 9B The prediction method in the case of bidirectional prediction including a second method for deriving LIC parameters is shown.

[0027] Fig. 10A An example of a multi-model discontinuity problem is shown.

[0028] Fig. 10B A first example of a technique to solve the multiple model discontinuity problem is shown where the models are constructed as lines passing through the (minimum; maximum) points of each subset.

[0029] Fig.11 An example embodiment of a LIC including a regularization step is shown.

[0030] Fig.12 A regularization function is shown in which parameters are adjusted when they are outside a given range.

[0031] Fig.13 The temporal depth associated with the layered picture coding principle is shown.

[0032] Fig.14 An example embodiment of regularization for handling multi-model LIC is shown.

[0033] Fig.15A A method for determining the discontinuity between two LIC models is shown.

[0034] Fig. 15B A method for calibrating a LIC model that ensures multi-model continuity is shown in accordance with at least one embodiment.

[0035] Fig.16 An example of a block diagram of multi-model LIC parameter correction is shown.

[0036] Fig.17 An example embodiment for partitioning two LIC models is shown.

[0037] Fig.18 A method for deriving LIC parameters using two LIC models is shown.

[0038] Fig.19 An example embodiment for determining single model parameters when a discontinuity problem occurs is shown.

[0039] Fig. 20 An example embodiment of a decision making process is shown. DETAILED DESCRIPTION

[0040] In at least one embodiment, video encoding or decoding uses LIC, and the calculation of LIC parameters is simplified by reducing its complexity, thereby improving video encoding or decoding performance.

[0041] In at least one embodiment, video encoding or decoding uses LIC that uses at least one linear model whose parameters are determined using a derivation process that includes a regularization process for improving the estimation of local illumination variations and a correction process for preventing discontinuity problems when using multiple linear models. Thus, visual artifacts are prevented.

[0042] Figure 1 A block diagram of an example of a video encoder 100, such as a High Efficiency Video Codec (HEVC) encoder, is shown. Figure 1 It is also possible to show an encoder in which the HEVC standard is improved or an encoder adopting a technique similar to HEVC, such as the JEM (Joint Exploration Model) encoder developed by JVET (Joint Video Exploration Team).

[0043] Before being encoded, the video sequence may undergo a pre-encoding process (101). This is performed, for example, by applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0044] In HEVC, to encode a video sequence having one or more pictures, the pictures are partitioned (102) into one or more slices, where each slice may include one or more slice segments. Slice segments are organized into codec units, prediction units, and transform units. The HEVC specification distinguishes between "blocks" and "units", where a "block" addresses a specific area in a sample array (e.g., luma, Y), and a "unit" includes all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and prediction data (e.g., motion vectors) associated with the block.

[0045] For the codec in HEVC, the picture is partitioned into codec tree blocks (CTBs) of square shape with configurable size, and a continuous set of codec tree blocks is grouped into slices. The codec tree unit (CTU) contains the CTB of the coded color component. The CTB is the root of the quadtree partitioned into codec blocks (CBs), and the codec block can be partitioned into one or more prediction blocks (PBs) and forms the root of the quadtree partitioned into transform blocks (TBs). Corresponding to the codec blocks, prediction blocks and transform blocks, the codec unit (CU) includes a tree-structured set of prediction units (PUs) and transform units (TUs), the PU includes prediction information for all color components, and the TU includes a residual codec syntax structure for each color component. The sizes of the CB, PB and TB of the luminance component are applicable to the corresponding CU, PU and TU. In the present application, the term "block" may be used to refer to any of, for example, CTU, CU, PU, ​​TU, CB, PB and TB. Additionally, "block" may also be used to refer to macroblocks and partitions specified in H.264 / AVC or other video codec standards, and more generally to data arrays of various sizes.

[0046] In the example of encoder 100, a picture is encoded by an encoder element as described below. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the CU and indicates the intra / inter decision through a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0047] The CU in intra mode is predicted from reconstructed neighboring samples within the same strip. A set of 35 intra prediction modes is available in HEVC, including DC, planar, and 33 angular prediction modes. The intra prediction reference is reconstructed from rows and columns adjacent to the current block. The reference extends over twice the block size in the horizontal and vertical directions using available samples from the previously reconstructed block. When an angular prediction mode is used for intra prediction, the reference samples can be copied along the direction indicated by the angular prediction mode.

[0048] The applicable luma intra prediction mode for the current block can be encoded using two different options. If the applicable mode is included in a constructed list of three most probable modes (MPMs), the mode is signaled by an index in the MPM list. Otherwise, the mode is signaled by a fixed-length binarization of the mode index. The three most probable modes are derived from the intra prediction modes of the top and left neighboring blocks.

[0049] For inter CU, the corresponding codec block is further divided into one or more prediction blocks. Inter prediction is performed at the PB level, and the corresponding PU contains information about how to perform inter prediction. Motion information (e.g., motion vector and reference picture index) can be signaled in two methods, namely, "merge mode" and "advanced motion vector prediction (AMVP)".

[0050] In merge mode, the video encoder or decoder assembles a candidate list based on already coded blocks, and the video encoder signals the index of one of the candidates in the candidate list. At the decoder side, motion vectors (MVs) and reference picture indices are reconstructed based on the signaled candidates.

[0051] In AMVP, a video encoder or decoder assembles a candidate list based on motion vectors determined from already encoded blocks. The video encoder then signals an index in the candidate list to identify the motion vector predictor (MVP) and signals a motion vector difference (MVD). On the decoder side, the motion vector (MV) is reconstructed as MVP+MVD. The applicable reference picture index is also explicitly encoded and decoded in the PU syntax for AMVP.

[0052] The prediction residual is then transformed (125) and quantized (130), including at least one embodiment for adapting the chroma quantization parameters described below. The transform is typically based on a separable transform. For example, a DCT transform is first applied in the horizontal direction and then in the vertical direction. In recent codecs such as JEM, the transforms used in the two directions may be different (e.g., DCT in one direction and DST in the other), which results in a wide variety of 2D transforms, whereas in previous codecs the variety of 2D transforms for a given block size was typically limited.

[0053] The quantized transform coefficients along with motion vectors and other syntax elements are entropy encoded (145) to output a bitstream. The encoder can also skip the transform and apply quantization directly to the untransformed residual signal on a 4x4 TU basis. The encoder can also bypass both transform and quantization, i.e., encode and decode the residual directly without applying the transform or quantization process. In direct PCM encoding, no prediction is applied, and the codec unit samples are encoded and decoded directly into the bitstream.

[0054] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to reconstruct the picture, for example, to perform deblocking / SAO (sample adaptive offset) filtering to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0055] Figure 2 1 shows a block diagram of an example of a video decoder 200, such as a HEVC decoder. In the example of the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described above. Figure 1 The encoding pass described in is reciprocal to the decoding pass, which performs video decoding as part of encoding the video data. Figure 2 Decoders in which improvements are made to the HEVC standard or decoders that employ techniques similar to HEVC, such as the JEM decoder, may also be shown.

[0056] Specifically, the input to the decoder includes a video bitstream that may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, picture partitioning information, and other codec information. The picture partitioning information indicates the size of the CTU, and the manner in which the CTU is partitioned into CUs and, if applicable, into PUs. The decoder may therefore partition (235) the picture into CTUs and each CTU into CUs based on the decoded picture partitioning information. The transform coefficients are dequantized (240), including for adapting at least one embodiment of the chroma quantization parameters described below, and inverse transformed (250) to decode the prediction residual.

[0057] The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block may be obtained (270) from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275). As described above, AMVP and merge mode techniques may be used to derive motion vectors for motion compensation, which may use an interpolation filter to calculate interpolated values ​​of sub-integer samples of a reference block. An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored at a reference picture buffer (280).

[0058] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0059] Figure 3 An example of a codec tree unit and a codec tree in the compressed domain is shown. In the HEVC video compression standard, a picture is partitioned into so-called codec tree units (CTUs), which are typically 64×64, 128×128, or 256×256 pixels in size. Each CTU is represented by a codec tree in the compressed domain. This is a quadtree of CTUs, where each leaf is called a codec unit (CU).

[0060] Figure 4 An example of partitioning a CTU into codec units, prediction units and transform units is shown. Each CU is then given some intra or inter prediction parameters (prediction information). For this purpose, it is spatially partitioned into one or more prediction units (PUs), each of which is assigned some prediction information. Intra or inter codec modes are assigned at the CU level.

[0061] In this application, the terms "reconstruction" and "decoding" can be used interchangeably, the terms "encoding" or "codec" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Typically, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side. The terms "block" or "picture block" can be used to refer to any of CTU, CU, PU, ​​TU, CB, PB and TB. In addition, the terms "block" or "picture block" can be used to refer to macroblocks, partitions, and sub-blocks specified in H.264 / AVC or other video codec standards, and more generally to sample arrays of many sizes.

[0062] Figure 5 An example of an “L-shape” for local illumination compensation is shown. In practice, emerging video compression tools such as those studied in the Joint Exploration Model (JEM) and the generic video codec reference software developed by the JVET (Joint Video Exploration Team) group [2] use some additional temporal prediction tools such as Local Illumination Compensation (LIC) with associated parameters determined at the decoder side.

[0063] Basically, the purpose of LIC is to compensate for illumination changes that may occur between a prediction block and its reference block used for temporal prediction by motion compensation. The use of LIC is usually signaled on CU level by a flag (LIC flag) associated with each codec unit (CU) coded in inter mode, or inferred from previously decoded blocks in case the current CU is coded in merge mode, for example. When the tool is activated, the decoder calculates some prediction parameters ( Figure 5 ). In the considered prior art codec (JEM), the use of LIC for a given block depends on a flag associated with this block, called LIC flag.

[0064] In the following, we will refer to the “L-shape” associated with the current block as the set consisting of samples on the rows above the current block and samples on the columns to the left of the current block, as Figure 5 In variations, more than one row (or column) may be used.

[0065] The first implementation of LIC uses a LIC model based on a simple linear correction of Equation 1 applied to the regular current block prediction:

[0066] Ycorr(x)=a.Ypred(x)+b (Equation 1)

[0067] where Ypred(x) is the predicted sample value at position x, Ycorr(x) is the illumination compensated predicted sample value at position x, and (a, b) are the LIC parameters.

[0068] The LIC parameters (a, b) are weights and biases based on minimizing the error between the current sample and the linearly modified reference sample, which are defined in Equation 2 as follows:

[0069] dist=∑ r∈Vcur,s∈Vref (cur(r)-a.ref(s)-b) 2 (Equation 2)

[0070] in:

[0071] cur(r) is the current image ( Figure 5 The adjacent reconstructed samples in the right side of

[0072] ref(s) is the image obtained from the reference image ( Figure 5 ), where s=r+mv, cur(r) and ref(r) are the co-located samples in the reconstructed L-shape and the reference L-shape, respectively.

[0073] The value of (a, b) is obtained using least squares minimization (LSM) as shown in Equation 3 below:

[0074]

[0075] Note that the value of N can be further adjusted (incrementally reduced) so that the sum term in Equation 3 remains below the maximum allowed integer storage value (e.g., sum term < 2 16 ). In addition, for large blocks, the subsampling of the top and left sample sets can be increased.

[0076] In case of additional conditions for selecting reconstructed samples, N may be equal to "numValid", or non-valid samples may be replaced with replicated valid samples.

[0077] The value of (a, b) can be obtained, for example, using a simpler calculation than least squares minimization, such as using extrema.

[0078] Once the encoder or decoder obtains the LIC parameters (a, b) of the current CU, the prediction of the current CU includes the following (unidirectional prediction case):

[0079] pred(current_block)=a×ref_block+b (Equation 4)

[0080] Where current_block is the current block to be predicted, pred(current_block) is the prediction of the current block, and ref_block is a reference block constructed using a conventional motion compensation (MC) process for temporal prediction of the current block. In a variant, in the case of bidirectional prediction, ref_block is a weighted sum of two reference blocks.

[0081] It should be noted that the adjacent reconstruction sample set and the reference sample set (see Figure 5 The gray samples in the left block have the same number and the same pattern. In the following, we refer to the neighboring reconstruction set (or reference sample set) located on the left side of the current block as "left samples", and refer to the neighboring reconstruction set (or reference sample set) located on the top side of the current block as "top samples". We refer to the combination of the "left sample" and "top sample" sets as "sample set".

[0082] Table 1 provides an estimate of the complexity of LIC parameter derivation according to Equation 3. Complexity is measured in this paper as the number of operations required to derive the LIC parameters. In this table, N=2 kcorresponds to the number of reconstructed and reference samples with a bit depth equal to "d". The first column identifies the required operation, the second column measures the number of bits required in memory, the third to sixth columns count the number of required sum, multiplication, shift (divide by 2), and integer division operations respectively, and the last row provides the total number of required operations.

[0083]

[0084]

[0085] Table 1

[0086] In at least a first embodiment, the LIC parameter calculation process uses the sum of absolute differences (SAD) as shown in Equation 6:

[0087]

[0088] Where cur(r) is the current image ( Figure 5 The adjacent reconstructed samples in the right side of the image are obtained by motion compensation from the reference image ( Figure 5 The adjacent reconstructed samples (cur(r)) and the reference samples (ref(s)) of the current block are co-located with respect to the L shape by the relationship "s=r+mv", as shown in Figure 5 shown.

[0089] Table 2 provides an estimate of the complexity of the LIC parameter derivation according to this first embodiment.

[0090]

[0091] Table 2

[0092] The use of the sum of absolute differences for the LIC parameter calculation results in significantly more efficiency. First, the memory requirements are reduced (column 2). The required complementary additions are largely compensated by the dramatic reduction in multiplications (column 4). Thus, this technique greatly improves the efficiency of LIC related calculations and more generally improves the efficiency of encoding or decoding.

[0093] In a variant embodiment, the parameter a of Equation 6 is determined using the regularization term "corr" as shown in Equation 7:

[0094]

[0095] In at least one embodiment, the regularization term is defined as shown in Equation 8:

[0096]

[0097] Here, reg_shift takes the value 7, for example.

[0098] Figure 7 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 1000 may be embodied as a device including various components described below, and is configured to perform one or more aspects of the various aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, encoders, transcoders, and servers. The elements of system 1000 may be embodied in a single integrated circuit, multiple ICs, and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed over multiple ICs and / or discrete components. In various embodiments, system 1000 is coupled to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 1000 is configured to implement one or more aspects of the various aspects described in this document.

[0099] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, the various aspects described in this document. The processor 1010 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an EEPROM, a ROM, a PROM, a RAM, a DRAM, an SRAM, a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0100] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that may be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000, or may be incorporated into the processor 1010 as a combination of hardware and software known to those skilled in the art.

[0101] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operation logic.

[0102] In several embodiments, memory inside the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be a memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC, or VVC (universal video codec).

[0103] Input to the elements of system 1000 may be provided through various input devices, as indicated at block 1130. Such input devices include, but are not limited to, (i) an RF section that receives an RF signal transmitted over the air, for example, by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0104] In various embodiments, the input device of block 1130 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with the following elements required: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal frequency band that (for example) may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a frequency band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted on wired (for example, cable) medium, and filter to the desired frequency band to perform frequency selection by filtering, down-conversion and filtering again. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions. Adding element can include inserting element between existing elements, such as, for example inserting amplifier and analog-to-digital converter. In various embodiments, the RF part includes antenna.

[0105] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, can be implemented as needed, for example, in a separate input processing IC or in the processor 1010. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed in a separate interface IC or in the processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 1010 and the encoder / decoder 1030, which operate in combination with memory and storage elements to process the data streams as needed for presentation on an output device.

[0106] The various elements of system 1000 may be disposed within an integrated housing in which the various elements may be interconnected and data transmitted therebetween using a suitable connection arrangement (eg, an internal bus known in the art, including an I2C bus, wiring, and printed circuit boards).

[0107] The system 1000 includes a communication interface 1050 capable of communicating with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0108] In various embodiments, a Wi-Fi network such as IEEE 802.11 is used to stream data to the system 1000. The Wi-Fi signals of these embodiments are received through a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box that delivers data through an HDMI connection of an input block 1130 to provide streaming data to the system 1000. Other embodiments use an RF connection of an input block 1130 to provide streaming data to the system 1000.

[0109] The system 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripherals 1120. In various examples of embodiments, the other peripherals 1120 include one or more of a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 1000. In various embodiments, control signals are communicated between the system 1000 and the display 1100, speakers 1110, or other peripherals 1120 using signaling such as AV.Link, CEC, or other communication protocols that implement device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to the system 1000 using a communication channel 1060 via a communication interface 1050. The display 1100 and speakers 1110 can be integrated into a single unit with other components of the system 1000 in an electronic device such as, for example, a television. In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0110] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and the speaker 1110 can be separated from one or more of the other components instead. In various embodiments where the display 1100 and the speaker 1110 are external components, an output signal can be provided via a dedicated output connection (including, for example, an HDMI port, a USB port, or a COMP output). The implementation described herein can be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of the implementation (for example, discussed only as a method), the implementation of the discussed features can also be implemented in other forms (for example, a device or a program). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device such as a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0111] Figure 6 An example embodiment of a multi-model LIC in which the models are divided by a threshold is shown. In such an embodiment, the two linear models use different LIC parameters ((a0, b0) and (a1, b1)), such as Figure 6 As shown, and determined, for example, using equation 3 above or using other methods. The first model and the first LIC parameters are used for the first subset of samples, and the second model and the second LIC parameters are used for the second subset of samples. Hereinafter, this mode is referred to as multi-model LIC (MM-LIC). This embodiment uses two linear models (a0, b0) and (a1, b1) and a threshold value on the reconstructed luminance sample values ​​to determine which model to use. The threshold is determined, for example, as the average value of the reconstructed luminance sample values.

[0112] Fig. 8A An example is shown in which a single model is determined based on extreme values. In this example, two reference samples (XA, XB) with a minimum value (Min) and a maximum value (Max) and associated reconstruction samples (YA, YB) are used to obtain the value of "a" (Equation 3b):

[0113]

[0114] Figure 8BAnother example in which a single model is determined based on an average extreme value is shown. In this example, the minimum and maximum values ​​are replaced by the average of two or more minimum values ​​or two or more maximum values, respectively. In the example shown in the figure, Min1 and Min2 are two minimum values. These elements are averaged to determine the values ​​of XA and YA. Max1 and Max2 are two maximum values. These elements are averaged to determine the values ​​of XB and YB. The value of "a" is then determined based on these corrected values ​​of XA, XB, YA, and YB. This allows increased robustness and is particularly effective in the case of outliers (e.g., due to noise). In the example shown, Max1 is clearly a contour line, and using only the values ​​of Min1 and Max1 to determine a linear model will result in a model that is not very accurate.

[0115] Fig.9A The prediction method in the case of bidirectional prediction including the first method for deriving LIC parameters is shown. In this case, the prediction is based on two references called reference 0 and reference 1 combined together. The LIC process is applied twice, first to the reference 0 prediction (LIC-0) and second to the reference 1 prediction (LIC_1). Next, the two predictions are combined together as usual using the default weighting (P=(P0+P1+1)>>1) or the bidirectional prediction weighted average (BPWA): P=(g0.P0+g1.P1+(1<<(s-1)))>>s).

[0116] Fig. 9B A prediction method is shown in the case of a bidirectional prediction including a second method for deriving LIC parameters. In this variant, referred to as method B, conventional predictions are first combined in step 360. Then, a single LIC process is applied in step 330 and a prediction is calculated in step 340 according to the determined LIC parameters.

[0117] Another method for parameter estimation for intra prediction is named Cross-Component Linear Model (CCLM). The prediction of a block of chrominance samples (Ycorr) can be constructed from a (scaled down) reconstructed block of luma samples (Ypred) corrected with a linear model. In this case, the same model as in Equation 1 is used, where:

[0118] Ypred(x) is the reconstructed luminance sample value at position x,

[0119] Ycorr(x) is the prediction of the chrominance sample value at position x.

[0120] The value of (a, b) is estimated based on the minimum and maximum values ​​(Minluma, Maxluma) of the reconstructed luma samples of the L-shape and the minimum and maximum values ​​Ycurr (Minchroma, Maxchroma) of the reconstructed chroma samples, as shown in Equation 5:

[0121] a=((〖Max〗_chroma-〖Min〗_chroma)) / ((〖Max〗_luma-〖Min〗luma))

[0122] b = [Min]_chroma - a. [Min]_luma (Equation 5)

[0123] like Figure 6 As shown, to improve LIC, two different LIC models can be used, with e.g. Fig. 8A 8B. The first model and the first LIC parameters are used for the first subset of samples, and the second model and the second LIC parameters are used for the second subset of samples. The division between the two sets is done with respect to a threshold.

[0124] Fig. 10A An example of the multi-model discontinuity problem is shown. This is a case of model discontinuity, in other words, when two lines generated by two models ((a0; b0) and (a1; b1)) do not intersect at the threshold. This discontinuity problem may cause visual artifacts and / or reduce coding efficiency and should be prevented.

[0125] Fig. 10B A first example of a technique for solving the multi-model discontinuity problem is shown where the models are constructed as lines passing through the (minimum; maximum) points of each subset. Correction is done by replacing Max0 and Min1 with the average value between the previous Max0 and Min1 (point T in the figure) and using these corrected values ​​to determine the LIC parameters for both models. This prevents discontinuity because the maximum value of the first model is equal to the minimum value of the second model (equal to T in the figure).

[0126] At least one problem faced by LIC is that the derivation of LIC parameters is based on a relatively small number of samples compared to the current block size. In addition, these samples are not co-located with the current block, making them likely to be poor estimates of local illumination variations. Furthermore, LIC parameter estimation also depends on the method used to derive (a, b). For example, in the case of Equation 3, if the denominator is low, some inconsistencies in the value of "a" may occur. In another example, if the numerator is low, inconsistencies in the value of "a" may occur. This is typically the case, for example, when the distribution of ref(s) or cur(r) is narrow. Therefore, in some cases, the derivation of LIC parameters includes some uncertainty. In order to reduce this uncertainty, a regularization process is inserted in the prediction stage so that potential problems are detected and the LIC parameters are corrected accordingly to prevent visual artifacts and / or improve coding efficiency.

[0127] Fig.11 An example embodiment of the LIC including a regularization step is shown. In such an embodiment, the overall principles including the LIC parameter derivation (330) remain unchanged, but once the LIC parameters have been calculated, a regularization function is applied in step 350 in order to potentially correct the previously determined LIC parameters. Such regularization or correction is particularly desirable when the derivation of the LIC parameters is uncertain, in other words, when the confidence level of the determined LIC parameters is too low.

[0128] Fig.12 A regularization function is shown in which parameters are adjusted when they are outside a given range. In this variant embodiment, the values ​​(a, b) are adjusted if "a" and / or "b" exceed some predefined thresholds (th_a, th_b) around the default (1, 0) values. For example: for a 10-bit sample, th_a=0.25 and th_b=100. In at least one embodiment, the predefined thresholds th_a and th_b are evenly distributed around a default value equal to 1.0.

[0129] In at least one embodiment:

[0130] If (a<1-th_a) or (a>1+th_a), adjust "a".

[0131] If (b<-th_b) or (b>th_b), adjust "b".

[0132] In a variation, the thresholds are not evenly distributed around the default (1,0) value:

[0133] If (a<1-th_a1) or (a>1+th_a2), adjust "a".

[0134] If (b<-th_b1) or (b>th_b2), adjust "b".

[0135] In at least one embodiment, the regularization function provides for adjustment of the parameter when the parameter is outside a given range. In such an embodiment, parameter a is first adjusted, and b is determined according to equation 1. In fact, with respect to equation 1, the derivation of the LIC parameter is based on the property r1:

[0136] DCrec=a.DCref+b(r1)

[0137] Where DCrec is the average value of the reconstructed samples and DCref is the average value of the reference samples. Therefore, in this embodiment:

[0138] If ((b>th_b)&&(a<1-th_a)), then

[0139] "a" is set to "1-th_a", and

[0140] Calculate "b" using (r1).

[0141] If ((b<-th_b)&&(a>1+th_a)), then

[0142] "a" is set to "1+th_a", and

[0143] "b" is calculated using (r1), and thus b = DCrec - a.DCref.

[0144] In at least another embodiment, the property (r1) is also verified using the adjusted value (a+da, b+db):

[0145] DCrec=(a+da).DCref+(b+db)(r2)

[0146] Given (r1), (r2) becomes:

[0147] Da=-db / DCref(r3)

[0148] According to this embodiment, a is adjusted to (a+da) and b is adjusted to (b+db), the adjustment "da" is calculated explicitly with (r3), and "db" is determined as:

[0149] If (b<-th_b), then db = -th_b-b

[0150] If (b>th_b), then db = th_b-b

[0151] These regularization adjustments ensure that the LIC parameter conforms to a certain range of values ​​and, therefore, the prediction step behaves correctly under good conditions.

[0152] In at least one embodiment, the values ​​of "th_a" and / or "th_b" are encoded in the bitstream. For example, they may be encoded in a sequence header, a picture header, a slice header, or a title header. In another embodiment, they are fixed or associated with a certain profile and / or level.

[0153] In at least another embodiment, the value of "th_a" and / or "th_b" is a function of at least a parameter (or set of parameters) P and can be obtained from the bitstream using a decoding process. For example, P may include a picture order count (POC) distance between a current picture (POCcur) and a reference picture (POCref).

[0154] For example: th_a = 0.25x (1 + 0.25x abs (POCcur – POCref))

[0155] Or: th_a = 0.25 x (1 + 0.25 x min(4; abs(POCcur – POCref)))

[0156] In another example, parameter P includes the temporal depth of the current picture. Fig.13 The temporal depth related to the principle of hierarchical picture coding and decoding is shown.

[0157] In at least one embodiment, parameter P includes the number of samples of the L shape actually used for estimating the LIC model (e.g., numValid). In this case, the values of "th_a" and / or "th_b" are functions of P.

[0158] For example: If (numValid < Nc), then

[0159] th_a = th_a1 and th_b = th_b1

[0160] Otherwise

[0161] th_a = th_a2 and th_b = th_b2

[0162] Where, for example, th_a1 = 0.3, th_a2 = 0.4, th_b1 = 0, th_b2 = 80, and Nc is the confidence threshold. This threshold may be different in terms of luminance and chrominance due to the different number of samples. In one example, the threshold 32 is used for luminance and the threshold 16 is used for chrominance. In a variant, NC is a function of the current block size, e.g., NC = 0.5 x (blockWidth + blockHeight).

[0163] In this embodiment, a low value of numValid indicates a reduced number of samples and thus implies a low confidence in the validity of the LIC model.

[0164] In the case of multi-model LIC, multiple regularization processes are required: one regularization process for each model ( Fig.14 element 350 in), because each individual model of the multi-model LIC is independent of other individual models.

[0165] Fig.14 An example embodiment for processing the regularization of multi-model LIC is shown. As introduced above, two different LIC models can be used and can lead to discontinuity problems. At step 380, a correction process is added to ensure the continuity of multiple models. This process can be implemented directly after the regularization process.

[0166] In at least one embodiment, the correction process (380) is applied only in merge mode. In practice, when the encoder detects a multi-model discontinuity problem, the LIC can be disabled for a block. This is performed by encoding the LIC flag as false for this block. However, in merge mode, the LIC flag is inherited from another neighboring block, and the encoder has less flexibility to avoid this problem unless recursively re-encoding the previous block.

[0167] In at least one embodiment, if the discontinuity size is above a discontinuity threshold (DT), a correction process (380) is applied.

[0168] In at least one embodiment, if the discontinuity size is above DT, the correction process (380) is applied only in merge mode.

[0169] The discontinuity size (DS) can be calculated by the decoder in different ways:

[0170] as the distance between the values ​​at the intersection between each linear model and the threshold line, e.g., Fig.15A shown.

[0171] As the difference between the maximum value of the first model M0 and the minimum value of the second model M1. DS = max0-min1

[0172] The value of the discontinuity threshold DT may be implicitly known by the decoder, encoded in the bitstream, calculated from other decoded parameters, or obtained by other means.

[0173] In at least one embodiment and as Fig. 15B As shown, Fig.14 The correction step 380 operates as follows:

[0174] For Yref="threshold", C0 and C1 are calculated as the values ​​given by models M0 and M1 respectively,

[0175] Calculate T as the average between these two values: T = (C0 + C1) / 2,

[0176] M0 and M1 are corrected when the line passes through (Avg0; T) and (T; Avg1), where Avg0 and Avg1 are the averages of the sample values ​​below and above the threshold, respectively.

[0177] In a variant embodiment of the previous embodiment, M0 and M1 are corrected to lines passing through (Min0; T) and (T; Max1), where Min0 is the minimum value of samples associated with the first model M0 and Max1 is the maximum value of samples associated with the second model M1.

[0178] Fig.16An example of a block diagram for multi-model LIC parameter correction is shown. This element corresponds to Fig.14 Block 380. First, in step 381, the discontinuity size DS between the two models is calculated. Then, in step 382, ​​the discontinuity size DS is compared with the discontinuity threshold DT. When DS>DT (branch "yes"), a discontinuity problem has been detected. In this case, a third step 383 includes correcting the LIC parameters of the two models using one of the above embodiments. When no discontinuity is detected (branch "no"), the LIC parameters are not corrected. Then, in step 340, a prediction is performed using the LIC parameters of the two models.

[0179] Fig.17 An example embodiment for partitioning two LIC models is shown. In such an embodiment, the two linear models use different LIC parameters ((a0, b0) and (a1, b1)), such as Figure 6 As shown, and determined, for example, using Equation 3 or using other methods. The first model and the first LIC parameters are used for a first subset of samples, and the second model and the second LIC parameters are used for a second subset of samples. In the following, this mode is referred to as multi-model LIC (MM-LIC).

[0180] The division between the two sets is done with respect to a threshold. Figure 6 In , NM0 and NM1 represent the number of reconstructed neighboring samples having values ​​below and above the threshold, respectively.

[0181] Determining a threshold for dividing the two models may be performed according to various embodiments.

[0182] In at least one embodiment, the threshold for dividing the two models is determined, for example, as the average value of the reconstructed samples ("cur(s)").

[0183] In at least one embodiment, the threshold is determined so that the number of samples used in each model (NM0 and NM1) is substantially the same. This increases the effectiveness of the LIC model. In fact, when the number of samples of one of the models is too small (NM0 is much larger than NM1 or NM0 is much smaller than NM1), the effectiveness of the corresponding LIC model (NM1 or NM0, respectively) is uncertain, and the LIC parameters derived using Equation 3 may be unreliable because they are based on too few samples. This can be done using Figure 6 This is done by taking the histogram shown and counting the number of samples. In a variation, this is done only if the size of the histogram (2 比特-深度-s (2 bit-depth–s This embodiment should only be used when )) is less than N=NM0+NM1.

[0184] In an alternative embodiment, a single LIC model is used when the number of samples of the models is not well balanced. This can be determined by dividing the number of samples of the model with the highest number of samples by the number of samples of the other model. If the ratio is greater than a threshold, the models are considered not well balanced, and in this case a single linear model is used. An example of a threshold is 10. In another example, this can be determined if a number of samples is below a predetermined value (e.g., NMi = 4 samples).

[0185] In at least one embodiment, by using a value less than the reconstruction sample range (e.g., 0...2 比特-深度 ) to achieve memory savings. This is done by using an appropriate scaling factor "s" corresponding to, for example, a right shift of the sample values. In this case, the histogram buffer size is reduced to 2 比特-深度-s , and the histogram step size is 2 s , thus saving memory.

[0186] In a variant embodiment, the lowest and highest values ​​of the histogram are saturated. This is shown by the values ​​Min range and Max range in the figure. All sample values ​​less than the Min range are set to the Min range, and all sample values ​​above the Max range are set to the Max range. When the potential range of thresholds is known, only a rough distribution is required around the possible thresholds, allowing some memory to be saved. For example, an estimate of the sample value range can be inferred from previously decoded samples in the same or previously decoded pictures, or the range value can be encoded in the bitstream.

[0187] In at least one embodiment, a single pass over the data is performed by aggregating the partial sums over a histogram: is calculated at the same intervals as the values ​​of the histogram. Once the values ​​such as The value of and the threshold are chosen, and the partial sums are accumulated with the histogram values ​​so that there is no need to loop again over the ref and cur values. This corresponds to an approximation of the sum of absolute differences, namely:

[0188]

[0189] Among them, n cur (h) is the number of reconstructed cur samples with values ​​∈ [h2s; (h+1).2s], and H = {h0, h1...} is the number of histogram values. The same approximation can be applied to the reference samples. This single pass is particularly interesting when the LIC parameters a and b are not obtained using least squares minimization (LSM) as shown in Equation 3, but by using the sum of absolute differences (SAD) as shown in Equation 6:

[0190]

[0191] Where cur(r) is the current image ( Figure 5 ), ref(s) is the adjacent reconstructed sample from the reference image ( Figure 5 The left side of the block is a reference sample constructed by motion compensation (converted by motion vector mv), and s = r + mv. The neighboring reconstructed samples (cur (r)) and the reference samples (ref (s)) of the current block are co-located with respect to the L shape by the relationship "s = r + mv", as shown in Figure 5 shown.

[0192] This embodiment is also applicable when the LIC parameters a and b are obtained using least squares minimization (LSM).The sum terms of the two models are simply added to obtain the sum term of the single model.

[0193] Fig.18 A method for deriving LIC parameters using two LIC models is shown. In this method, once a threshold is determined from the histogram at step 370, the value of the reconstructed sample is compared to the threshold at step 330 to determine whether the reconstructed sample is to be used to determine the LIC parameters of the first LIC model or the second LIC model. Then, at step 340, corresponding LIC parameters are determined for both models, and predictions are calculated using both models. These predictions are then combined (at step 360), for example using a simple average or a bi-prediction weighted average.

[0194] In at least one embodiment, the multi-model discontinuity problem is detected by the encoder, and the LIC feature can be disabled by the encoder for the block where the multi-model discontinuity occurs. This can be signaled by the encoder by encoding the LIC flag as false for the block. However, in merge mode, the LIC flag is inherited from another neighboring block, and the encoder has less flexibility to avoid this problem unless recursively re-encoding the previous block.

[0195] In at least one embodiment, in case of multi-model discontinuity of a block encoded in merge mode, the encoder does not use MM-LIC but uses a single LIC model.

[0196] In a variant embodiment, the discontinuity size (DS) is calculated by the decoder, and if the DS is above a discontinuity threshold, a single LIC model approach is selected, otherwise a multi-model LIC model is used. The discontinuity threshold is implicitly known to the decoder, either encoded in the bitstream, calculated from other decoding parameters, or obtained using other means.

[0197] Fig.19An example embodiment for determining the parameters of a single model when a discontinuity problem occurs is shown. In this embodiment, which corresponds to the case where a single LIC model approach is selected due to a discontinuity problem, the LIC parameters of the single model are derived directly from the average values ​​(Avg0 and Avg1, respectively) of the samples below and above the threshold that separates the two models. This has the advantage that the parameters of the single model are efficiently derived in a simple manner without the need to rescan the samples, and without the need to calculate equation 6 again, since the average values ​​Avg0 and Avg1 have already been calculated.

[0198] Fig. 20 An example embodiment of a decision process according to one embodiment is shown. The process is intended to decide whether LIC should use a single linear model or multiple linear models. First, the LIC parameters are determined for the case where two models are used. Then, at step 335, a decision is made to decide whether LIC should be used and which model to use. The decision is made based on the different elements mentioned above (unbalanced sample number, discontinuity problem). When a single model should be used, then at step 336, for example, using Fig. 9B The method shown determines LIC parameters for a single model and performs block prediction at step 342. When the decision is to select multi-model LIC, then block prediction is performed at step 340 using the LIC parameters calculated for both models.

[0199] The decision of whether LIC uses a single linear model or multiple linear models is valid for both the encoder and the decoder, but the decision not to use LIC is only valid on the encoder side.

[0200] Reference to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations thereof means that a particular feature, structure, characteristic, etc. described in conjunction with the embodiment is included in at least one embodiment. Therefore, the appearance of the phrase "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in various places throughout the specification do not necessarily refer to the same embodiment. In addition, the application or its claims may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory. In addition, the application or its claims may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, predicting information, or estimating information.

[0201] Additionally, this application or its claims may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information can include, for example, one or more of accessing information or retrieving information (e.g., from a memory or optical storage medium). Furthermore, during operations such as, for example, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is generally involved in one way or another.

[0202] It should be recognized that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to as many of the items listed as will be apparent to one of ordinary skill in this and related arts.

[0203] It will be apparent to those skilled in the art that implementations may generate various signals formatted to carry information that may, for example, be stored or transmitted. The information may include, for example, instructions for executing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be sent via a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

Claims

1. A video decoding method, comprising: identifying a plurality of adjacent reconstructed samples in a first L-shape adjacent to the current block; identifying a plurality of reference samples in a second L-shape adjacent to a reference block of the current block corresponding to the plurality of adjacent reconstructed samples in the first L-shape; determining a local illumination compensation parameter as a ratio between a sum of absolute first differences and a sum of absolute second differences, each first difference between adjacent reconstructed samples in the first L-shape and an average of the adjacent reconstructed samples, and each second difference between a reference sample in the second L-shape and an average of the reference samples; and Local illumination compensation is performed using the determined local illumination compensation parameters.

2. The method according to claim 1, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the adjacent reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and s=r+mv, mv is the motion vector.

3. The method according to claim 1, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the neighboring reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and corr is a regularization term, and s=r+mv, mv is a motion vector.

4. The method according to claim 3, wherein: The regularization term is determined as follows: where reg_shift is the shift value.

5. The method according to claim 4, wherein: The reg_shift value is equal to 7.

6. An apparatus comprising a decoder for decoding picture data of at least one block in a picture or video, wherein: The decoder is configured to: identifying a plurality of adjacent reconstructed samples in a first L-shape adjacent to the current block; identifying a plurality of reference samples in a second L-shape adjacent to a reference block of the current block corresponding to the plurality of adjacent reconstructed samples in the first L-shape; determining a local illumination compensation parameter as a ratio between a sum of absolute first differences and a sum of absolute second differences, each first difference between adjacent reconstructed samples in the first L-shape and an average of the adjacent reconstructed samples, and each second difference between a reference sample in the second L-shape and an average of the reference samples; and Local illumination compensation is performed using the local illumination compensation parameters.

7. The device according to claim 6, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the adjacent reconstructed sample in the current block in the first L-shape, ref(s) is the reference sample in the second L-shape constructed using motion compensation, N is the number of reconstructed samples and reference samples, and s=r+mv, mv is the motion vector.

8. The device according to claim 6, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the neighboring reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and corr is a regularization term, and s=r+mv, mv is a motion vector.

9. The device according to claim 8, wherein: The regularization term is determined as follows: where reg_shift is the shift value.

10. The device according to claim 9, wherein: The reg_shift value is equal to 7.

11. A video encoding method, comprising: identifying a plurality of adjacent reconstructed samples in a first L-shape adjacent to the current block; identifying a plurality of reference samples in a second L-shape adjacent to a reference block of the current block corresponding to the plurality of adjacent reconstructed samples in the first L-shape; determining a local illumination compensation parameter as a ratio between a sum of absolute first differences and a sum of absolute second differences, each first difference between adjacent reconstructed samples in the first L-shape and an average of the adjacent reconstructed samples, and each second difference between a reference sample in the second L-shape and an average of the reference samples; and The current block is encoded with local illumination compensation using the determined local illumination compensation parameters.

12. The method according to claim 11, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the adjacent reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and s=r+mv, mv is the motion vector.

13. The method according to claim 11, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the neighboring reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and corr is a regularization term, and s=r+mv, mv is a motion vector.

14. The method according to claim 13, wherein: The regularization term is determined as follows: where reg_shift is the shift value.

15. The method according to claim 14, wherein: The reg_shift value is equal to 7.

16. An apparatus comprising an encoder for encoding picture data of at least one block in a picture or video, wherein: The encoder is configured as: identifying a plurality of adjacent reconstructed samples in a first L-shape adjacent to the current block; identifying a plurality of reference samples in a second L-shape adjacent to a reference block of the current block corresponding to the plurality of adjacent reconstructed samples in the first L-shape; determining a local illumination compensation parameter as a ratio between a sum of absolute first differences and a sum of absolute second differences, each first difference between adjacent reconstructed samples in the first L-shape and an average of the adjacent reconstructed samples, and each second difference between a reference sample in the second L-shape and an average of the reference samples; and The current block is encoded with local illumination compensation using the local illumination compensation parameters.

17. The device according to claim 16, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the adjacent reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and s=r+mv, mv is the motion vector.

18. The device according to claim 16, wherein: The local illumination compensation parameters are calculated in the following way: Where cur(r) is the neighboring reconstructed sample of the current block in the first L-shape, ref(s) is the reference sample in the second L-shape, N is the number of reconstructed samples and reference samples, and corr is a regularization term, and s=r+mv, mv is a motion vector.

19. The device according to claim 18, wherein: The regularization term is determined as follows: where reg_shift is the shift value.

20. The device according to claim 19, wherein The reg_shift value is equal to 7.

21. A non-transitory computer-readable medium comprising program code instructions, which, when the program is executed by a computer, cause the computer to perform the method of any one of claims 1 to 5 or claims 11-15.

Images