Local illumination correction flag inheritance

Abstract

To provide a method and an apparatus for improving a compression efficiency in a video compression scheme.SOLUTION: A method and an apparatus for improving a compression efficiency in a video compression scheme allow for a flexible use of local illumination correction. The method includes individual local illumination correction for a component of a video block. The method also includes a flexible derivation method for illumination correction information. In one aspect, local illumination correction information can inherit from other remainders such as, e.g., a neighboring block.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates to video compression, and more particularly to video encoding and decoding. [Background technology]

[0002] Many attempts have been made to improve the coding efficiency of block-based codecs. Local illumination compensation (LIC) is a tool introduced for the purpose just mentioned. In the Joint Exploration Model (JEM) developed by the Joint Video Exploration Team (JVET) group and in the VVC reference software, additional temporal prediction tools with associated parameters determined at the decoder side are under investigation, including Local Illumination Compensation (LIC). Essentially, the purpose of LIC is to compensate for illumination changes that may occur between the predicted block and the reference block used through motion-compensated temporal prediction. Summary of the Invention

[0003] The just-mentioned and other shortcomings and disadvantages of the prior art are addressed by the presently described aspects, which are directed to methods and apparatus for managing the trade-off between coding efficiency and complexity provided by FRUC tools.

[0004] According to an aspect of the described embodiment, there is provided a method that includes determining local illumination compensation information to be used for one or more video components of a video block, encoding the video block using the local illumination compensation information, and indicating in a bitstream whether the illumination compensation information is used for the video block.

[0005] According to another aspect of the described embodiment, there is provided a second method that includes parsing a bitstream for syntax information indicating local illumination correction, and decoding at least one video block in the bitstream based on the syntax to conditionally apply local illumination correction to components of the one video block.

[0006] According to another aspect of the described embodiments, there is provided an apparatus including a memory and a processor, the processor being configured to encode or decode a portion of a video signal according to any of the methods described above.

[0007] According to another general aspect of at least one embodiment, there is provided an apparatus including an apparatus according to any of the decoding aspects and at least one of (i) an antenna configured to receive a signal, the signal including a video block; (ii) a band limiter configured to limit the received signal to a band of frequencies including the video block; or (iii) a display configured to display an output representing the video block.

[0008] According to another general aspect of at least one embodiment, there is provided a non-transitory computer-readable medium that includes data content generated according to any of the described encoding embodiments or variants.

[0009] According to another general aspect of at least one embodiment, there is provided a signal that includes video data generated according to any of the described encoding embodiments or variants.

[0010] According to another general aspect of at least one embodiment, a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variants.

[0011] According to another general aspect of at least one embodiment, there is provided a computer program product including instructions that, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variants.

[0012] The presently discussed and other aspects, features and advantages of the present principles will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0013] [Figure 1] 1 illustrates the concepts of a Coding Tree Unit and a Coding Tree, which represent a compressed HEVC picture. [Figure 2] In JEM, we illustrate that LIC parameters are derived from adjacent reconstructed samples and corresponding reference samples transformed by MV (top: square CU, bottom: rectangular CU). [Figure 3] Illustrates the LIC flag inference process in the case of merge mode in the encoder (left) and decoder (right). [Figure 4] The derivation and application of LIC parameters for each prediction L0 and L1 are illustrated. [Figure 5] Illustrates the derivation and application of LIC parameters to combined predictions from L0 and L1. [Figure 6] This illustrates how the LIC-flag may be inherited multiple times consecutively. [Figure 7] 1 illustrates a common video compression scheme. [Figure 8] 1 illustrates a common video decompression scheme. [Figure 9] 1 illustrates a flow diagram for one aspect for decoding a video block under the present principles. [Figure 10] This illustrates that the LIC-flag can be inherited multiple times consecutively. [Figure 11] 10 illustrates a flow diagram of another aspect for decoding a video block under the present principles. [Figure 12] Illustrates another example of how the LIC-flag can be inherited multiple times consecutively. [Figure 13] A processor-based system for encoding / decoding. [Figure 14] 1 illustrates one embodiment of a method for decoding video using at least one illumination correction flag. [Figure 15] 1 illustrates one embodiment of a method for encoding video with at least one lighting correction flag. [Figure 16] 1 illustrates one embodiment of an apparatus for encoding or decoding video with at least one illumination correction flag. DETAILED DESCRIPTION OF THE INVENTION

[0014] The domain of the aspects described herein is video compression and is intended to improve the video compression efficiency of modern video coding schemes. Based on a hybrid architecture, we aim to improve the compression efficiency compared to existing video compression systems and to implement efficient local illumination correction.

[0015] Block-Based Video Compression In HEVC (High Efficiency Video Coding, ITU-T H.265) or VVC (Versatile Video Codec of Joint Video Experts Team), to encode a picture, a frame is first divided into large blocks (CTUs = Coding Tree Units) and possibly further divided into smaller coding units (CUs), as depicted in Figure 1.

[0016] To encode a CU, a prediction block (PU = Prediction Unit) is constructed from adjacent reconstructed samples (intra prediction) or from previously reconstructed samples of a picture stored in a DPB (Decoded Pictures Buffer) (inter prediction). Residual samples, calculated as the difference between the original samples and the PU samples, are then transformed, quantized, and entropy coded.

[0017] In inter prediction, motion compensated temporal prediction is used to exploit the redundancy that exists between consecutive pictures in a video. To do so, a motion vector is associated with the PU, and a reference index 0 (refIdx0) is used to indicate which reference picture in LIST_0 to use.

[0018] Local Lighting Correction In the Joint Exploration Model (JEM) developed by the Joint Video Exploration Team (JVET) group and in the VVC reference software, additional temporal prediction tools with associated parameters determined at the decoder side are under investigation, including Local Illumination Compensation (LIC). Essentially, the purpose of LIC is to compensate for possible illumination changes between the predicted block and the reference block used throughout motion-compensated temporal prediction.

[0019] Typically, the use of LIC is signaled at the CU level through a flag (LIC flag) associated with each coding unit (CU) coded in inter mode. When the aforementioned tool is activated, the decoder calculates prediction parameters based on several reconstructed picture samples localized to the left and / or above the current block to be predicted and reference picture samples localized to the left and / or above the block to be motion compensated (FIG. 2). In the considered prior art codec (JEM), the use of LIC for a given block depends on a flag associated with the aforementioned block, called the LIC flag. LIC processing is performed on all image components (e.g., luma component, chroma component).

[0020] In the following, we will refer to the set of samples in the rows above the current block and / or the set of samples in the columns to the left of the current block as the "L-shape" associated with the current block, as depicted in gray in Figure 2. Another proposal would be to discard some reference samples (e.g., use only reference samples from coding units / blocks coded in inter-coding mode).

[0021] LIC flag inheritance The LIC flag can be either explicitly coded or inherited. When inherited, the LIC flag is derived from previously coded parameters, such as previously reconstructed LIC flag values, as illustrated in FIG. 3 for the encoder (401) and decoder (402).

[0022] In merge or skip mode, a list of candidate parameters is constructed (410). The parameters may include MV, reference index, uni- or bi-prediction, and LIC-flag. If the LIC-flag is true (430), the LIC process applies: LIC parameters are derived (440) and applied (450) to the predicted PU samples (420).

[0023] LIC model In the current approach, the LIC model is based on a simple linear correction applied to the normal current block prediction (Equation 1). Y corr (x)=aY pred (x)+b Equation 1. Linear LIC model However, Y pred (x) is the predicted sample value at location x, and Y corr (x) is the corrected predicted sample value at location x, and (a, b) are the LIC parameters (sometimes called scale and offset).

[0024] However, the described aspects are not limited to the simple model just described. For example, (Y corr = a.(Y pred ) 2 + by pred + c) or (Y corr = a.log(Y pred Other models such as a) + b) could be used, for example.

[0025] LIC parameter estimation In the case of the standard LIC model (Equation 1), the LIC parameters (a, b) are weights and offsets that can be determined based on minimizing the error between the current sample and a linearly corrected reference sample, and are defined as follows:

[0026]

number

[0027] where rec_cur(r) is the adjacent reconstructed sample in the current picture (Figure 2-right), rec_ref(s) is the reference sample constructed by MC from the reference picture (Figure 2-left), s = r + mv, and rec_cur(r) and rec_ref(r) are the co-located samples in the reconstructed L-shape and the reference L-shape, respectively.

[0028] The values ​​of (a, b) are obtained using least squares minimization (LSM) (Equation 3).

[0029]

number

[0030] For the sum term in Equation 3, the maximum integer storage number value allowed (for example, N<2 16 Note that the value of N may be further adjusted (e.g., incrementally decreased) to remain below ≡ ...

[0031] Once the LIC parameters are obtained by the encoder or decoder for the current CU, the prediction pred(current_block) of the current CU is as follows (in the case of unidirectional prediction): pred(current_block)=a×ref_block+b (Formula 1) where current_block is the current block to predict, pred(current_block) is the prediction of the current block, and ref_block is the reference block constructed by normal motion compensation (MC) processing and used for temporal prediction of the current block.

[0032] In the case of bi-prediction, the LIC process is applied twice: first to the prediction of reference 0 (LIST-0) and second to the prediction of reference 1 (LIST_1) (Figure 4). The two predictions are then combined together as usual, using the default weighting (P = (P0 + P1 + 1)>>1), or BPWA (bi-prediction weighted averaged): P = (g0.P0 + g1.P1 + (1<<(s-1)))>>s). The method just described is called Method a.

[0033] In a variant (method b), in the case of biprediction, the regular predictions are first combined and then one single LIC process is applied (Fig. 5).

[0034] Default LIC model The estimation / calculation of LIC parameters may lead to values ​​corresponding to the identity. In the case just mentioned, one can assume that for every "x" samples of the PU block, Y corr (x) = Y pred (x). In the case of the LIC model described by Equation 1, what has just been stated corresponds to (a;b)=(1;0). Typically, what has just been stated can occur when there is no illumination change between the current block and the reference block, or when no "L-shape" is available (e.g., a CU at the edge of the top-left picture / slice), or when the range of values ​​is too narrow to estimate the LIC parameters. In what follows, "default model" will mean the LIC model that does not modify the PU sample values.

[0035] The LIC flag indicates whether LIC processing is applied to the current CU for all components. However, it is possible that the estimated LIC parameters are default for at least one component. If the LIC flag is explicitly coded, the encoder may select the value of the LIC flag (true or false) according to the optimal rate-distortion tradeoff (410). However, if the LIC flag is inherited, the encoder cannot easily control the LIC processing because the LIC flag may be inherited multiple times consecutively, as depicted in FIG. 6.

[0036] The present invention improves the derivation of local illumination compensation (LIC) flag parameters by deriving component-based LIC flags. Additionally, LIC flag inheritance may be conditioned on the actual derived LIC parameter values.

[0037] Aspect 1: One defines one LIC-flag per component. C components (e.g., N for YUV or RGB) C = 3) and when the LIC flag is explicitly coded (e.g., in AMVP mode), one C Encode the LIC flag.

[0038] In a variant, one encodes one single LIC-flag value, and the LIC-flag for each component is equal to the encoded LIC-flag.

[0039] Aspect 2 (can be combined with Aspect 1): One defines another "LIC_flag" that means "LIC_flag_out." The normal "LIC_flag" indicates whether LIC processing (LIC parameter derivation and PU sample correction) is applied to the current CU. "LIC_flag_out" indicates the value of the LIC flag that can be inherited by another CU (550).

[0040] In a variant, one defines one value of "LIC_flag_out" per component C. The derivation (570) of "LIC_flag_out[C]" is a function of the LIC parameters calculated for component C. For example, if the LIC parameters calculated for component C are default, then LIC_flag_out[C]=false, otherwise LIC_flag_out[C]=true. If the normal "LIC_flag" is false, then "LIC_flag_out" is false.

[0041] In a variant, for the LIC model given by (Equation 1), if the scale value is zero, then LIC_flag_out[C]=false.

[0042] Embodiment 2.1 (can be combined with Embodiments 1 and 2): If the current block's normal "LIC_flag" is true, the derivation of "LIC_flag_out" can be a function of the "LIC_flag_out" information of the previous neighbors. For example, if the "LIC_flag_out" of most of the coded neighboring blocks is true, then the "LIC_flag_out" of the current block is set to true. As depicted in Figure 10, if the "LIC_flag_out" of neighbors N0 and N2 is true, but the "LIC_flag_out" of another neighbor N1 is false, then the "LIC_flag_out" of the current block is derived as true, which is the majority value among the neighbors.

[0043] Aspect 3 (can be combined with Aspects 1 and 2): In another variant of embodiment 2, if the value of the LIC parameter is near the default, then LIC_flag_out[C]=false. For example, in the case of the LIC model (Equation 1), "near the default" (default a is 1 (i.e., 32>>5)) can correspond to: abs(a-1) <th_a および abs(b)<th_b where (th_a;th_b) are predetermined values ​​(for example, 0.1 and 10<<(bitdepth_8)).

[0044] Embodiment 4 (can be combined with embodiments 1, 2, and 3): In another variant, the derivation (570) of "LIC_flag_out[C2]" is a function of the LIC parameters calculated for component C1, where C1≠C2, e.g., C1=luminance and C2 is one chrominance component (e.g., Cb or Cr).

[0045] Embodiment 5 (can be combined with embodiments 1, 2, 3, and 4): One defines one value of "LIC_flag_out" for each component type T, e.g., one for luma and one for chroma (components Cb and Cr).

[0046] "LIC_flag" is decoded or inferred depending on the type of component.

[0047] "LIC_flag_out" is used as before for the luma component, and for the chroma components, "LIC_flag_out" is inferred as false if at least one of the LIC parameters calculated for chroma Cb or Cr is default (or close to default). In a variant, "LIC_flag_out" of a chroma component is inferred as false if the LIC parameters calculated for chroma Cb and Cr are both default (or close to default, embodiment 3).

[0048] Aspect 6 (Aspect 1, 2, or 3): This embodiment is depicted in FIG.

[0049] "LIC_flag" is associated with "LIC_flag_counter". Combined with aspect 1, there is one "LIC_flag_counter[C]" per component. The value of "LIC_flag_counter" is initialized when decoding a CU with an explicitly coded LIC_flag. If "LIC_flag=true", then "LIC_flag_counter" is initialized as "MaxLicCount" (e.g., "MaxLicCount=2") (620). If "LIC_flag=false", then "LIC_flag_counter" is initialized as 0 (630). In merge mode, "LIC_flag_counter" is inherited (610).

[0050] (a) In combination with embodiment 2 or embodiment 3, if the LIC parameters are at the default (or close to the default), the "LIC_flag_counter" is decremented (640) (unless the "LIC_flag_counter" is zero).

[0051] CUs coded in merge mode inherit LIC_flag_counter. If LIC_flag_counter=0, LIC is disabled for the CU (650). In a variant, if the CU inherits LIC_flag, LIC_flag_counter is decremented (b). One has only (a), only (b), or (a)+(b) (in the case just mentioned, LIC_flag_counter can be decremented twice).

[0052] In the example of FIG. 12, "MaxLicCount=3". a.CUa has explicitly coded "LIC_flag" equal to true, the derived LIC parameter is not the default, and also "LIC_flag_counter" equal to 3. b. CUb is a merge and inherits "LIC_flag_counter" from the CU above (CUa). The derived LIC parameter is not the default, and "LIC_flag_counter" remains equal to 3. c.CUc is a merge and inherits the LIC_flag_counter from the left CU (CUb). The derived LIC parameters are default and the LIC_flag_counter is decremented and equal to 2. d.CUd is a merge and inherits the LIC_flag_counter from the left CU (CUc). The derived LIC parameters are default and the LIC_flag_counter is decremented and equal to 1. e.CUe is a merge and inherits the LIC_flag_counter from the left CU (CUd). The derived LIC parameter is the default, and the LIC_flag_counter is also decremented and equal to 0. f.CUf is a merge and inherits LIC_flag_counter from the left CU (CUe). LIC_flag_counter is equal to 0 and the LIC is also disabled (LIC_flag = false). g.CUg is a merge and inherits LIC_flag_counter from the left CU (CUf). LIC_flag_counter is equal to 0 and the LIC is also disabled (LIC_flag = false).

[0053] Aspect 7 (Aspect 1): In this embodiment, there is no LIC for the chrominance components: "LIC_flag" and "LIC_flag_out" for the chrominance components are always inferred as false.

[0054] Aspect 8: In this embodiment, there is no LIC propagation (inheritance) for the chrominance components. 'LIC_flag' is coded in the bitstream for the luma and chrominance components, and LIC is applied as usual. However, 'LIC_flag_out' is always inferred as false for the chrominance components, regardless of the LIC parameter value. As a result, LIC for the chrominance components is always false for the merged block.

[0055] Note that the following variants are possible with this embodiment: one single 'LIC_flag', one 'LIC_flag' per component, and one 'LIC_flag' per component type. In the case of one single 'LIC_flag', one flag is decoded per CU, and 'LIC_flag_out' is always false for chroma components and as explained above for luma components.

[0056] One embodiment of a method 1500 under the general aspects described herein is shown in Figure 15. The method begins at start block 1501, and control continues to block 1510 for determining local illumination compensation information to be used for one or more video components of a video block. Control continues from block 1510 to block 1520 for encoding the video block using the local illumination compensation information. Control continues from block 1520 to block 1530 for indicating in the bitstream whether such illumination compensation information is used for the video block.

[0057] One embodiment of a method 1400 under the general aspects described herein is shown in Figure 14. The method begins at start block 1401, and control continues to block 1410 for parsing a bitstream for syntax information indicating local illumination correction. Control continues from block 1410 to block 1420 for decoding at least one video block in the bitstream based on the syntax and conditionally applying local illumination correction to components of the one video block.

[0058] 16 illustrates one embodiment of an apparatus 1600 for compressing, encoding, or decoding video using an encoding or decoding tool. The apparatus includes a processor 1610 and may be interconnected to a memory 1620 through at least one port. Additionally, both the processor 1610 and the memory 1620 may have one or more additional interconnections to external connections.

[0059] Additionally, the processor 1610 is configured to either insert or receive information in the bitstream, and either compress, encode, or decode using various coding tools.

[0060] This application describes various aspects, including tools, features, aspects, models, approaches, and the like. Many of the just-described aspects are described with particularity and, in many cases, in a manner that may appear limiting, at least to illustrate their individual characteristics. However, the just-described statements are for purposes of clarity of description and do not limit the applicability or scope of the just-described aspects. Indeed, all of the different aspects can be combined and substituted to provide further aspects. Furthermore, aspects can likewise be combined and substituted with aspects described in earlier applications.

[0061] The aspects described and contemplated in this application can be implemented in many different ways. While Figures 7, 8, and 13 provide some embodiments, other embodiments are contemplated, and the discussion of Figures 7, 8, and 13 does not limit the breadth of implementations. Generally, at least one of the aspects relates to video encoding and decoding, and generally, at least one other aspect relates to transmitting the generated or encoded bitstream. The just-described and other aspects can be implemented as a method, an apparatus, a computer-readable medium having instructions stored thereon, and / or a computer-readable medium having a bitstream generated according to any of the described methods, for encoding or decoding video data according to any of the described methods.

[0062] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably. Usually, although not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0063] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the inherent operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0064] Various methods and other aspects described herein can be used to modify modules, such as the intra-prediction, entropy coding, and / or decoding modules (160, 360, 145, 330) of video encoder 100 and decoder 200 shown in Figures 7 and 8. Furthermore, the aspects are not limited to VVC or HEVC, but can be applied to other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any of the above standards and recommendations (including VVC and HEVC). Unless otherwise indicated or specifically technically prohibited, the aspects described herein can be used individually or in combination.

[0065] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes and the aspects described are not limited to the specific values ​​just mentioned.

[0066] 7 illustrates an encoder 100. Although variations of the encoder 100 are envisioned, the encoder 100 is described below for purposes of clarity without describing all contemplated variations.

[0067] Before being encoded, a video sequence may go through a pre-encoding process (101), such as applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or remapping the input picture components to make the signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata is associated with the pre-processing and can be bound to the bitstream.

[0068] In the encoder 100, a picture is encoded by the encoder elements described below. The picture to be encoded is divided (102) and processed, for example, in units of a CU. Each unit is encoded, for example, using either intra or inter mode. When a unit is encoded in intra mode, intra prediction (160) is performed. In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder determines (105) which one of intra or inter mode to use to encode the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (110) the predicted block from the original image block.

[0069] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., the residual is coded directly without applying either the transform or quantization process.

[0070] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the predicted block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / Sample Adaptive Offset (SAO) filtering to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0071] Figure 8 illustrates a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by the decoder elements described below. Generally, the video decoder 200 performs a decoding pass that is the inverse of the encoding pass described in Figure 7. Furthermore, the encoder 100 also generally performs video decoding as part of encoding the video data.

[0072] In particular, the decoder's input includes a video bitstream, which may be generated by the video encoder 100. First, the bitstream is entropy decoded (230) to obtain transform coefficients, motion vectors, and other coding information. Picture partition information indicates how the picture is divided. Therefore, the decoder may partition (235) the picture according to the decoded picture partition information. The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. An image block is reconstructed by combining (255) the decoded prediction residual with a predicted block. The predicted block may be obtained (270) from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0073] Additionally, the decoded picture may go through a post-decoding process (285), such as an inverse color transform (e.g., YCbCr 4:2:0 to RGB 4:4:4 conversion) or an inverse remapping that reverses the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata signaled in the bitstream that was derived in the pre-encoding process.

[0074] FIG. 13 illustrates a block diagram of an example system in which various aspects and embodiments can be implemented. System 1000 can be embodied as a device including various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television broadcast receivers, personal video recording systems, connected consumer electronics appliances, and servers. The elements of system 1000, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or separate components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various aspects, the system 1000 is configured to implement one or more aspects described in this document.

[0075] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein to implement various aspects described herein, for example. The processor 1010 can include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which can include non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drives, and / or optical disk drives. The storage devices 1040 may include, by way of non-limiting example, internal storage devices, attached storage devices (including detachable and non-detachable storage devices), and / or network-accessible storage devices.

[0076] System 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. Encoder / decoder module 1030 represents a module or modules that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated within processor 1010 as a combination of hardware and software, as known to those skilled in the art.

[0077] Program code loaded onto the processor 1010 or encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various aspects, one or more of the processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more various items during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing of equations, formulas, operations, and operational logic.

[0078] In some embodiments, memory internal to the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of the just-described functions. The external memory can be the memory 1020 and / or the storage device 1040, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as MPEG-2 (MPEG refers to the Moving Picture Experts Group, which is further called ISO / IEC 13818, which is further known as H.222, and which is further known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, which is further known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, JVET, a standard being developed by the Joint Video Experts Team).

[0079] Input to elements of system 1000 can be provided via various input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) an RF section that receives radio frequency (RF) signals, e.g., transmitted over the air by a broadcast station, (ii) a Component (COMP) input terminal (or set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples not shown in FIG. 13 include composite video. In various aspects, the input devices of block 1130 are associated with respective input processing elements known in the art. For example, the RF section can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or bandlimiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) bandlimiting again to a narrower band of frequencies (e.g., selecting a signal frequency band, which in some aspects may be referred to as a channel), (iv) demodulating the downconverted, bandlimited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various aspects includes one or more elements that perform the functions just described, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section can include, for example, a tuner that performs various functions, including downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and associated input processing elements receive RF signals transmitted over a wired medium (e.g., cable), filter, downconvert, and perform frequency selection by filtering again to the desired frequency band. Various embodiments rearrange the order of the above-mentioned (and other) elements, remove some of the just-mentioned elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0080] Additionally, the USB and / or HDMI terminals may include respective interface processors for coupling system 1000 to other electronic devices over USB and / or HDMI connections. It will be understood that various aspects of input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or, if desired, within processor 1010. Similarly, aspects of USB or HDMI interface processing may be implemented in a separate input processing IC or, if desired, within processor 1010. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 1010 and an encoder / decoder 1030 that operates in cooperation with memory and storage elements to process the data stream for presentation at an output device, if desired.

[0081] The various elements of system 1000 may be provided within an integrated housing in which the various elements are interconnected and capable of transmitting data therebetween using suitable interconnection arrangements, such as internal buses known in the art, including an I2C (Inter-IC) bus, wiring, and printed circuit boards.

[0082] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented within a wired and / or wireless medium, for example.

[0083] Data may be streamed or otherwise provided to system 1000 in various embodiments using a wireless network, such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE citing the Institute of Electrical and Electronics Engineers). In the embodiments just described, Wi-Fi signals are received via communication channel 1060 and communication interface 1050, which are adapted for Wi-Fi communication. Typically, communication channel 1060 in the embodiments just described is coupled to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 1000 using a set-top box that delivers data via an HDMI connection in input block 1130. Still other embodiments provide streamed data to system 1000 using an RF connection in input block 1130. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0084] The system 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, a tablet, a laptop, a cellular phone (mobile phone), or other device. Furthermore, the display 1100 can be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). In various example embodiments, the other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of the system 1000. For example, a disc player plays the output of the system 1000.

[0085] In various aspects, control signals are communicated between the system 1000 and the display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be coupled to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and speakers 1110 can be integrated with other components of the system 1000 in a single unit, e.g., in an electronic device such as a television. In various aspects, the display interface 1070 includes a display driver such as, e.g., a timing controller (TCon) chip.

[0086] Alternatively, the display 1100 and speakers 1110 can be separate from one or more other components, for example, if the RF portion of the input 1130 is part of a separate set-top box. In various aspects where the display 1100 and speakers 1110 are external components, the output signal can be presented via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0087] Embodiments may be performed by computer software implemented by the processor 1010, by hardware, or by a combination of hardware and software. By way of non-limiting example, embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, by way of non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, by way of non-limiting example, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0088] Various implementations include decoding. As used herein, "decoding" can include all or part of the processing performed on a received encoding sequence to generate a final output suitable for, for example, a display. In various aspects, such processing includes one or more of the processing typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various aspects, the processing also or alternatively includes the processing performed by the decoders of the various implementations described herein.

[0089] As a further example, in one embodiment "decoding" refers only to "entropy decoding," in another embodiment "decoding" refers only to differential decoding, and in another embodiment "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding" is intended to refer specifically to a subset of operations or more generally, it is believed that the decoding process will be clear based on the context of the particular description and will be well understood by those skilled in the art.

[0090] Various implementations include encoding. In a manner similar to the above discussion of "decoding," "encoding" as used herein can include all or part of the processing performed on an input video sequence, for example, to generate an encoded bitstream. In various aspects, such processing includes one or more of the processing typically performed by an encoder, such as segmentation, differential encoding, transform, quantization, and entropy encoding. In various aspects, the processing also or alternatively includes the processing performed by the encoders of the various implementations described herein.

[0091] As a further example, in one embodiment "encoding" refers only to "entropy decoding," in another embodiment "encoding" refers only to differential decoding, and in another embodiment "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding" is intended to refer specifically to a subset of operations or more generally, it is believed that the encoding process will be clear based on the context of the particular description and will be well understood by those skilled in the art.

[0092] Note that the syntax elements used herein are descriptive terms; as noted above, they do not preclude the use of other syntax element names.

[0093] When a diagram is provided as a flow diagram, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is provided as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0094] Various aspects may refer to parametric models or rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often subject to computational complexity constraints. This can be measured through the Rate Distortion Optimization (RDO) metric, or through LMS (Least Mean Square), MAE (Mean of Absolute Errors), or other such measures. Rate-distortion optimization is typically formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate-distortion optimization problem. For example, an approach may be based on extensive testing of all encoding options, including all considered modes or coding parameter values, along with a thorough evaluation of the coding cost and associated distortion of the reconstructed signal after encoding and decoding. Furthermore, faster approaches may be used to reduce encoding complexity, particularly with an approximated distortion calculation based on a predicted or prediction residual signal rather than a reconstructed one. A mixture of the two approaches just mentioned can also be used, for example, by using approximated distortion for only some of the possible encoding options and full distortion for other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches use any of a variety of techniques to perform optimization, but the optimization does not necessarily involve a complete evaluation of both the coding cost and the associated distortion.

[0095] Implementations and aspects described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even when described only in the context of a single form of implementation (e.g., described only as a method), the described feature implementation may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. A method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Furthermore, a processor also includes communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0096] Reference to "one embodiment" or "embodiment" or "one implementation" or "implementation" means that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment, as well as other variations. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" appearing in various places throughout this application are not necessarily all referring to the same embodiment, as well as any other variations.

[0097] Additionally, the application may refer to "determining" various portions of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0098] Additionally, the application may refer to "accessing" various portions of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0099] Additionally, the application may refer to "receiving" various portions of information. Receiving, as with "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" is typically included among operations such as, in some way or another, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0100] It should be understood that the use of any of the following, " / ," "and / or," and "at least one of," for example, "A / B," "A and / or B," and "at least one of A and B," is intended to include the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, for "A, B, and / or C" and "at least one of A, B, and C," the phrase is intended to include the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). What has just been said can be expanded beyond what has been listed, as will be apparent to those skilled in the art and related arts.

[0101] Furthermore, as used herein, the word "signaling" refers, among other things, to indicating something to a corresponding decoder. For example, in some aspects, an encoder signals a particular one of multiple transforms, coding modes, or flags. In this way, in some aspects, the same transform, parameter, or mode is used on both the encoder and decoder sides. Thus, for example, an encoder can transmit particular parameters to a decoder (explicit signaling), and the decoder can use the same particular parameters. Conversely, if the decoder already has certain parameters as well, signaling can be used without transmission (implicit signaling) to simply enable the decoder to know and select certain parameters. By avoiding the transmission of any actual function, bit savings are realized in various aspects. It is understood that signaling can be accomplished in various ways. For example, one or more syntax elements, flags, etc., can be used to signal information to a corresponding decoder in various aspects. Although the foregoing has concerned the verb form of the word "signaling," the word "signaling" can also be used herein as a noun.

[0102] As will be apparent to those skilled in the art, implementations can generate a variety of signals formatted to carry information that can be stored or transmitted, for example. For example, the information can include instructions for performing a method or data generated by one of the described implementations. For example, the signal can be formatted to carry a bitstream of the described manner. For example, the signal can be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. For example, formatting can include encoding a data stream and modulating a carrier wave with the encoded data stream. For example, the information carried by the signal can be analog or digital information. The signal can be transmitted over a variety of separate wired or wireless links, as is known. The signal can be stored in a processor-readable medium.

[0103] We describe several embodiments across various claim categories and types. The features of the embodiments just described may be provided alone or in any combination. Furthermore, embodiments may include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types. • A bitstream or signal containing one or more of the syntax elements described, or variations thereof. A bitstream or signal including syntax conveying information generated according to any of the described aspects. • Creating and / or transmitting and / or receiving and / or decoding according to any of the described aspects. - A method, process, apparatus, instruction storage medium, data storage medium, or signal according to any of the described aspects. Inserting syntax elements into the signaling that allow the decoder to determine the coding mode in a corresponding manner used by the encoder. • Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described syntax elements or variations thereof. ● A television, set-top box, mobile phone, tablet, or other electronic device that performs the conversion method(s) according to any of the described aspects. ●A television, set-top box, mobile phone, tablet, or other electronic device that performs the transformation method(s) determination and displays the resulting image (e.g., using a monitor, screen, or other type of display) according to any of the described aspects. ●A television, set-top box, mobile phone, tablet, or other electronic device that selects, band-limits, or tunes a channel (e.g., using a tuner), receives a signal containing an encoded image, and performs a conversion method(s) according to any of the described aspects. ●A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal over the airwaves (e.g., using an antenna) containing the encoded image and performs one or more conversion methods.

Claims

1. Obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for a current image block in a merge mode, the LIC flag indicating whether local illumination correction is applied to the current image block; calculating LIC model parameters for the current image block in response to determining that the LIC flag indicates that local illumination correction has been applied to the current image block; and decrementing a LIC flag counter for the current image block in response to determining that the LIC model parameters are near default values; decoding the current image block using the LIC model parameters; A method comprising:

2. 2. The method of claim 1, wherein for the current image block in the merge mode, obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag includes inheriting the LIC flag counter from a candidate block and deriving the LIC flag from the LIC flag counter.

3. The method of claim 2 , further comprising decoding an index that identifies the candidate block in the list of candidate blocks.

4. 3. The method of claim 2, wherein deriving the LIC flag from the LIC flag counter comprises setting the LIC flag to True in response to determining that the LIC flag counter is greater than zero.

5. 2. The method of claim 1, wherein obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for the current image block in the merge mode comprises obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for each component of the current image block in the merge mode.

6. 1. An apparatus comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors: Obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for a current image block in a merge mode, the LIC flag indicating whether local illumination correction is applied to the current image block; calculating LIC model parameters for the current image block in response to determining that the LIC flag indicates that local illumination correction has been applied to the current image block; and decrementing a LIC flag counter for the current image block in response to determining that the LIC model parameters are near default values; decoding the current image block using the LIC model parameters; An apparatus configured to:

7. 7. The apparatus of claim 6, wherein for the current image block in the merge mode, obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag includes inheriting the LIC flag counter from a candidate block and deriving the LIC flag from the LIC flag counter.

8. The apparatus of claim 7 , further comprising decoding an index that identifies the candidate block in the list of candidate blocks.

9. 8. The apparatus of claim 7, wherein deriving the LIC flag from the LIC flag counter comprises setting the LIC flag to True in response to determining that the LIC flag counter is greater than zero.

10. 7. The apparatus of claim 6, wherein obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for the current image block in the merge mode comprises obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for each component of the current image block in the merge mode.

11. Obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for a current image block in a merge mode, the LIC flag indicating whether local illumination correction is applied to the current image block; calculating LIC model parameters for the current image block in response to determining that the LIC flag indicates that local illumination correction has been applied to the current image block; and decrementing a LIC flag counter for the current image block in response to determining that the LIC model parameters are near default values; encoding the current image block using the LIC model parameters; A method comprising:

12. 12. The method of claim 11, wherein for the current image block in the merge mode, obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag includes inheriting the LIC flag counter from a candidate block and deriving the LIC flag from the LIC flag counter.

13. The method of claim 12 , further comprising encoding an index that identifies the candidate block in the list of candidate blocks.

14. 13. The method of claim 12, wherein deriving the LIC flag from the LIC flag counter comprises setting the LIC flag to True in response to determining that the LIC flag counter is greater than zero.

15. 12. The method of claim 11, wherein obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for the current image block in the merge mode comprises obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for each component of the current image block in the merge mode.

16. 1. An apparatus comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors: Obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for a current image block in a merge mode, the LIC flag indicating whether local illumination correction is applied to the current image block; calculating LIC model parameters for the current image block in response to determining that the LIC flag indicates that local illumination correction has been applied to the current image block; and decrementing a LIC flag counter for the current image block in response to determining that the LIC model parameters are near default values; encoding the current image block using the LIC model parameters; An apparatus configured to:

17. 17. The apparatus of claim 16, wherein for the merge mode current image block, obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag includes inheriting the LIC flag counter from a candidate block and deriving the LIC flag from the LIC flag counter.

18. The apparatus of claim 17 , further comprising encoding an index that identifies the candidate block in the list of candidate blocks.

19. 18. The apparatus of claim 17, wherein deriving the LIC flag from the LIC flag counter comprises setting the LIC flag to True in response to determining that the LIC flag counter is greater than zero.

20. 17. The apparatus of claim 16, wherein obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for the current image block in the merge mode comprises obtaining a local illumination correction (LIC) flag and a LIC flag counter associated with the LIC flag for each component of the current image block in the merge mode.

Images