Local illumination correction flag inheritance

Abstract

A method and apparatus for improving compression efficiency in a video compression scheme enables flexible use of local illumination correction. The method includes individual local illumination corrections for components of a video block. The method also includes a flexible derivation method for illumination correction information. In one aspect, the local illumination correction information can be inherited from other remaining blocks, such as neighboring blocks.

Description

[Technical Field]

[0001] This disclosure relates to video compression, and more specifically, to video encoding and decoding. [Background technology]

[0002] Many attempts have been made to improve the encoding efficiency of block-based codecs. Local illumination compensation (LIC) is a tool introduced for the purposes described above. Additional time prediction tools with decoder-determined related parameters, including Local Illumination Compensation (LIC), are being researched in the Joint Exploration Model (JEM) developed by the Joint Video Exploration Team (JVET) group and in the VVC reference software. Essentially, the purpose of LIC is to compensate for possible illumination changes between the predicted block and the reference block used through motion-compensated time prediction. [Overview of the Initiative]

[0003] The prior art described above, as well as other shortcomings and disadvantages, are addressed by the described embodiment, which is directed toward methods and apparatus for managing the trade-off between coding efficiency and complexity provided by the FRUC tool.

[0004] A method is provided in accordance with the aspects of the described embodiment. The method includes the steps of determining local illumination compensation information to be used for one or more video components of a video block, encoding the video block using the local illumination compensation information, and indicating in a bitstream whether the aforementioned illumination compensation information is used for the video block.

[0005] A second method is provided according to another aspect of the described embodiment. The method includes the steps of parsing a bitstream for syntax information indicating local illumination correction, and decoding at least one video block in the bitstream based on the aforementioned syntax, and conditionally applying local illumination correction to the components of the aforementioned one video block.

[0006] There is an apparatus provided according to another aspect of the described embodiment. The apparatus includes memory and a processor. The processor can be configured to encode or decode a portion of the video signal by any of the methods described above.

[0007] According to another general aspect of at least one embodiment, there is a device provided that includes an apparatus that follows one of the embodiments of decoding, and at least one of the following: (i) an antenna configured to receive a signal, the signal of which includes a video block; (ii) a band limiter configured to limit the received signal to a frequency band including the video block; or (iii) a display configured to display an output representing the video block.

[0008] There is a non-temporary computer-readable medium provided, which contains data content generated according to either the encoding method or variant described, according to another general aspect of at least one aspect of the medium.

[0009] There is a signal provided, which includes video data generated according to either the encoding method or variant described, according to another general aspect of at least one aspect of the said method.

[0010] According to another general aspect of at least one of the embodiments, the bitstream is formatted to include data content generated according to either the described encoding aspect or variant.

[0011] According to another general aspect of at least one aspect, there is a computer program product provided which, when the program is executed by a computer, includes instructions causing the computer to perform either the described mode of decoding or a variant thereof.

[0012] The aspects, features, and advantages of this principle described above will become apparent from the following detailed description of exemplary embodiments, which will be read in conjunction with the attached drawings. [Brief explanation of the drawing]

[0013] [Figure 1] This illustrates the concepts of a coding tree unit and a coding tree, which represent a compressed HEVC picture. [Figure 2] In JEM, the LIC parameter is illustrated as being derived from adjacent reconstructed samples and the corresponding reference sample converted by MV (top: square CU, bottom: rectangular CU). [Figure 3] The LIC flag inference process in merge mode is illustrated in the encoder (left) and decoder (right). [Figure 4] The derivation and application of LIC parameters for each prediction L0 and L1 are illustrated as examples. [Figure 5] This example illustrates the derivation and application of LIC parameters to predictions combined from L0 and L1. [Figure 6] Illustrate how the LIC-flag can be inherited continuously multiple times. [Figure 7] Illustrate a general video compression scheme. [Figure 8] Illustrate a general video decompression scheme. [Figure 9] Illustrate a flowchart for one aspect of decoding a video block under this principle. [Figure 10] Illustrate that the LIC-flag can be inherited continuously multiple times. [Figure 11] Illustrate a flowchart of another aspect for decoding a video block under this principle. [Figure 12] Illustrate another example of the LIC-flag being able to be inherited continuously multiple times. [Figure 13] A processor-based system for encoding / decoding. [Figure 14] Illustrate one aspect of a method for decoding video using at least one illumination correction flag. [Figure 15] Illustrate one aspect of a method for encoding video using at least one illumination correction flag. [Figure 16] Illustrate one aspect of an apparatus for encoding or decoding video using at least one illumination correction flag.

Embodiments for Carrying out the Invention

[0014] The area of the aspects described in this specification is video compression, and it is intended to improve the video compression efficiency of the latest video encoding scheme. Based on a hybrid architecture, it aims to improve the compression efficiency compared to existing video compression systems and to implement efficient local illumination correction.

[0015] Block-based video compression In HEVC (High Efficiency Video Coding, ITU-T H.265) or VVC (Versatile Video Codec of Joint Video Experts Team), to encode a picture, the frame is first divided into large blocks (CTU = Coding Tree Unit), as shown in Figure 1, and then, if possible, further divided into smaller coding units (Coding Unit: CU).

[0016] To encode the CU, the Prediction Unit (PU) is constructed from adjacent reconstructed samples (intra-prediction) or from previously reconstructed samples of a picture stored in a Decoded Pictures Buffer (DPB) (inter-prediction). The residual sample, calculated as the difference between the original sample and the PU sample, is then transformed, quantized, and entropy-encoded.

[0017] In interpretation, motion-compensated time prediction is used to take advantage of the redundancy present between consecutive pictures in a video. To do this, the motion vector is associated with the PU, and a reference index 0 (refIdx0) is used to indicate which reference picture in LIST_0 is being used.

[0018] Local illumination correction In the JEM (Joint Exploration Model) developed by the JVET (Joint Video Exploration Team) group and in the VVC reference software, additional time prediction tools with related parameters determined on the decoder side, including Local Illumination Compensation (LIC), are under research. Essentially, the purpose of LIC is to compensate for possible illumination changes between the predicted block and the reference block used through motion-compensated time prediction.

[0019] Typically, the use of LIC is signaled at the CU level through a flag (LIC flag) associated with each encoded unit (CU) encoded in intermode. When the tool described above is activated, the decoder calculates prediction parameters based on several reconstructed picture samples localized to the left and / or above the predicted current block, and a reference picture sample localized to the left and / or above the motion-compensated block (Figure 2). In the prior art codec considered (JEM), the use of LIC for a given block depends on the flag associated with the block, called the LIC flag. LIC processing is performed on all image components (e.g., luminance components, chrominance components).

[0020] In the following, we will refer to the set of samples in the top row of the current block and / or the left column of the current block, as shown in gray in Figure 2, as an "L-shape" associated with the current block. Another suggestion would be to discard some reference samples (for example, using only reference samples from encoded units / blocks encoded in inter-encoded mode).

[0021] LIC flag inheritance The LIC flag can be either explicitly encoded or inherited. If inherited, the LIC flag is derived from a previously encoded parameter, such as a previously reconstructed LIC flag value, as illustrated in Figure 3 for the encoder (401) and decoder (402).

[0022] In merge mode or skip mode, a list of parameter candidates is constructed (410). The parameters may include MV, reference index, uni-prediction or bi-prediction, LIC-flag. If the LIC-flag is true (430), the LIC process applies the following. The LIC parameters are derived (440) and applied to the predicted PU samples (420) (450).

[0023] LIC model In the latest approach, the LIC model is based on a simple linear correction (Equation 1) applied to normal current block prediction. Y corr (x)=a.Y pred (x)+b Equation 1. Linear LIC model However, Y pred (x) is the predicted sample value at position x, Y corr (x) is the corrected predicted sample value at position x, and (a,b) are the LIC parameters (sometimes called scale and offset).

[0024] However, the described aspects are not limited to the simple model described above. For example, other models such as (Y corr = a.(Y pred ) 2 + b.Y pred + c) or (Y corr = a.log(Y pred ) + b) could be used for example.

[0025] LIC parameter estimation In the case of the standard LIC model (Equation 1), the LIC parameters (a,b) are weights and offsets and can be determined based on minimizing the error between the current sample and the linearly corrected reference sample, and are defined as follows.

[0026]

number

[0027] However, rec_cur(r) is an adjacent reconstructed sample in the current picture (Figure 2-right), and rec_ref(s) is a reference sample constructed by MC from the reference picture (Figure 2-left), where s = r + mv, and rec_cur(r) and rec_ref(r) are samples located at the same position in the reconstructed L-shape and the reference L-shape, respectively.

[0028] The values ​​of (a,b) are obtained using the least squares method (LSM) (Equation 3).

[0029]

number

[0030] For the sum term in Equation 3, the maximum integer storage number value allowed (for example, N < 2 16 Note that in order to remain below this value, the value of N may be further adjusted (for example, incremented and decremented). Furthermore, subsampling of the upper and left sample sets can also be incremented for larger blocks.

[0031] Once the LIC parameters are obtained for the current CU by the encoder or decoder, the prediction pred(current_block) for the current CU is as follows (for unidirectional prediction): pred(current_block)=a×ref_block+b (formula 1) However, current_block is the current block to predict, pred(current_block) is the prediction for the current block, and ref_block is a reference block constructed by normal motion compensation (MC) processing and used for time prediction of the current block.

[0032] In the case of bi-prediction, the LIC process is applied twice: first to the prediction for reference 0 (LIST-0) and second to the prediction for reference 1 (LIST_1) (Figure 4). The two predictions are then combined as usual using the default weighting (P = (P0 + P1 + 1)>>1) or BPWA (bi-prediction weighted averaged): P = (g0.P0 + g1.P1 + (1<<(s-1)))>>s). The method described above is called method a.

[0033] In the variant (method b), in the case of biprediction, the normal prediction is first combined, and then a single LIC process is applied (Figure 5).

[0034] Default LIC Model The estimation / calculation of LIC parameters may lead to values ​​corresponding to identity. In the case mentioned above, one would be Y for all "x" samples in the PU block. corr (x) = Y pred (x) is given. In the case of the LIC model described by Equation 1, what has just been stated corresponds to (a;b)=(1;0). Typically, what has just been stated can occur if there is no illumination change between the current block and the reference block, or if an "L-shape" is not available (e.g., CU at the top-left edge of the picture / slice), or if the range of values ​​is too narrow to estimate the LIC parameters. In the following, "default model" will mean an LIC model that does not modify the PU sample values.

[0035] The LIC flag indicates whether LIC processing is applied to the current CU for all components. However, it is possible that the estimated LIC parameters are the default for at least one component. If the LIC flag is explicitly encoded, the encoder may select a value (true or false) for the LIC flag according to the optimal rate distortion trade-off (410). However, if the LIC flag is inherited, the encoder cannot easily control the LIC processing because the LIC flag may be inherited multiple times consecutively, as depicted in Figure 6.

[0036] This invention improves the derivation of local illumination compensation (LIC) flag parameters by deriving component-based LIC flags. Additionally, LIC flag inheritance may be conditioned on the actually derived LIC parameter values.

[0037] Appearance 1: One approach is to define one LIC-flag for each component. C Components (for example, N for YUV or RGB) C =3) and if the LIC flag is explicitly encoded (e.g., AMVP mode), then one is N C Encode the LIC flag.

[0038] In the variant, one encodes a single LIC-flag value, and each component of the LIC-flag is equal to the encoded LIC-flag.

[0039] Embodiment 2 (can be combined with Embodiment 1): One option is to define another "LIC_flag" which means "LIC_flag_out". The normal "LIC_flag" indicates whether LIC processing (LIC parameter derivation and PU sample correction) is applied to the current CU. "LIC_flag_out" indicates the value of the LIC flag that can be inherited by another CU (550).

[0040] In the variant, one is to define one value for "LIC_flag_out" for each component C. The derivation of "LIC_flag_out[C]" (570) is a function of the LIC parameters calculated for component C. For example, if the LIC parameters calculated for component C are the default values, then LIC_flag_out[C]=false; otherwise, LIC_flag_out[C]=true. If the normal "LIC_flag" is false, then "LIC_flag_out" is also false.

[0041] In the variant, for the LIC model given by (Equation 1), if the scale value is zero, then LIC_flag_out[C]=false.

[0042] Embodiment 2.1 (can be combined with Embodiments 1 and 2): If the normal "LIC_flag" of the current block is true, then the derivation of "LIC_flag_out" can be a function of the previous neighbor's "LIC_flag_out" information. For example, if "most" of the "LIC_flag_out" of the encoded neighboring blocks were true, then the current block's "LIC_flag_out" is set to true. As shown in Figure 10, if the "LIC_flag_out" of neighbors N0 and N2 was true, but the "LIC_flag_out" of another neighbor N1 was false, then the current block's "LIC_flag_out" is derived to be true, which is a large number of values ​​among the neighbors.

[0043] Embodiment 3 (can be combined with Embodiments 1 and 2): In another variant of Embodiment 2, LIC_flag_out[C]=false if the value of the LIC parameter is close to the default. For example, in the case of the LIC model (Equation 1), "close to the default" (default a is 1 (i.e., 32>>5)) can correspond to the following: abs(a-1) <th_a および abs(b)<th_b However, (th_a;th_b) are predetermined values ​​(for example, 0.1 and 10 << (bitdepth_8)).

[0044] Embodiment 4 (can be combined with Embodiments 1, 2, and 3): In another variant, the derivation of "LIC_flag_out[C2]" (570) is a function of the LIC parameters calculated for component C1, where C1 ≠ C2. For example, C1 = luminance, and C2 is one chrominance component (e.g., Cb or Cr).

[0045] Embodiment 5 (can be combined with Embodiments 1, 2, 3, and 4): One approach is to define a single value for "LIC_flag_out" for each component type T. For example, one for luminance and one for chrominance (components Cb and Cr).

[0046] The "LIC_flag" is decoded or inferred based on the type of component.

[0047] "LIC_flag_out" is used for the luminance component as previously used, and for the chrominance component, "LIC_flag_out" is inferred to be false if at least one of the LIC parameters calculated for the chrominance Cb or Cr is default (or close to default). In a variant, "LIC_flag_out" for the chrominance component is inferred to be false if both of the LIC parameters calculated for the chrominance Cb and Cr are default (or close to default, aspect 3).

[0048] Embodiment 6 (Embodiment 1, 2, or 3): This embodiment is depicted in Figure 11.

[0049] "LIC_flag" is associated with "LIC_flag_counter". When combined with aspect 1, there is one "LIC_flag_counter[C]" for each component. The value of "LIC_flag_counter" is initialized when the CU is decoded with the explicitly encoded LIC_flag. If "LIC_flag=true", "LIC_flag_counter" is initialized to "MaxLicCount" (for example, "MaxLicCount=2") (620). If "LIC_flag=false", "LIC_flag_counter" is initialized to 0 (630). In merge mode, "LIC_flag_counter" is inherited (610).

[0050] (a) When combined with aspect 2 or aspect 3, if the LIC parameter is at its default (or close to its default), "LIC_flag_counter" is decremented (640), unless "LIC_flag_counter" is zero.

[0051] In merge mode, the encoded CU inherits "LIC_flag_counter". If "LIC_flag_counter=0", then LIC is invalidated for CU (650). In variant form, if CU inherits "LIC_flag", then "LIC_flag_counter" is decremented (b). One can have only (a), only (b), or (a)+(b) (in this case, "LIC_flag_counter" can be decremented twice).

[0052] In the example in Figure 12, "MaxLicCount = 3". a.CUa has an explicitly encoded "LIC_flag" that is equal to true. The derived LIC parameter is not the default, and furthermore, "LIC_flag_counter" is equal to 3. b.CUb is a merge, inheriting "LIC_flag_counter" from the CU (CUa) above. The derived LIC parameter is not the default, and furthermore, "LIC_flag_counter" remains equal to 3. c.CUc is a merge, inheriting "LIC_flag_counter" from the left CU (CUb). The derived LIC parameter is the default, and furthermore, "LIC_flag_counter" is decremented to equal 2. d.CUd is a merge, inheriting "LIC_flag_counter" from the left CU (CUc). The derived LIC parameter is the default, and furthermore, "LIC_flag_counter" is decremented to equal 1. e.CUe is a merge, inheriting "LIC_flag_counter" from the left CU (CUd). The derived LIC parameter is the default, and furthermore, "LIC_flag_counter" is decremented to equal 0. f.CUf is a merge, inheriting "LIC_flag_counter" from the left CU (CUe). "LIC_flag_counter" is equal to 0, and furthermore, LIC is disabled ("LIC_flag" = false). g.CUg is a merge, inheriting "LIC_flag_counter" from the left CU (CUf). "LIC_flag_counter" is equal to 0, and furthermore, LIC is disabled ("LIC_flag" = false).

[0053] Appearance 7 (Appearance 1): In this embodiment, there is no LIC for the color difference component. The "LIC_flag" and "LIC_flag_out" for the color difference component are always inferred as false.

[0054] Aspect 8: In this embodiment, there is no LIC propagation (inheritance) for the chrominance component. "LIC_flag" is encoded in the bitstream for the luminance and chrominance components, and the LIC is applied as usual. However, "LIC_flag_out" is always inferred as false for the chrominance component, regardless of the LIC parameter value. As a result, the LIC for the chrominance component is always false for the merged block.

[0055] Note that variations, such as a single "LIC_flag", a "LIC_flag" for each component, and a "LIC_flag" for each component type, are possible according to this embodiment. In the case of a single "LIC_flag", one flag is decoded for each CU, and "LIC_flag_out" is always false for the chrominance component and for the luminance component as previously described.

[0056] An embodiment of Method 1500 in the general manner described herein is shown in Figure 15. The Method begins in start block 1501, and the control continues in block 1510 to determine local illumination compensation information to be used for one or more video components of a video block. The control continues from block 1510 to block 1520 to encode the video block using the local illumination compensation information. The control continues from block 1520 to block 1530 to indicate in the bitstream whether the aforementioned illumination compensation information is used for the video block.

[0057] An embodiment of Method 1400 in the general manner described herein is shown in Figure 14. The Method begins in intro block 1401, and the control proceeds to block 1410 to parse the bitstream for syntax information indicating local illumination correction. The control proceeds from block 1410 to block 1420 to decode at least one video block in the aforementioned bitstream based on the aforementioned syntax and conditionally apply local illumination correction to the components of the aforementioned one video block.

[0058] Figure 16 shows one embodiment of a device 1600 for compressing, encoding, or decoding video using an encoding or decoding tool. The device includes a processor 1610, which can be interconnected with memory 1620 through at least one port. Furthermore, both the processor 1610 and memory 1620 may have one or more additional interconnections to external connections.

[0059] Furthermore, the processor 1610 is configured to either insert or receive information in the bitstream, and to compress, encode, or decode it using various encoding tools.

[0060] This application describes various aspects, including tools, features, embodiments, models, and approaches. Many of these aspects are described in a specific manner and, at least to illustrate their individual characteristics, often in a way that may give the impression of limitation. However, this is for the purpose of clarity in the description and does not limit the application or scope of these aspects. Indeed, all the different aspects can be combined and interchangeable to provide further aspects. Furthermore, embodiments can likewise be combined and interchangeable with aspects described in earlier applications.

[0061] The aspects described and envisioned in this application can be implemented in many different ways. Figures 7, 8, and 13 provide several embodiments, but other embodiments are envisioned, and the discussion of Figures 7, 8, and 13 does not limit the scope of implementation. Generally, at least one of the embodiments relates to video encoding and decoding, and generally, at least one other embodiment relates to transmitting the generated or encoded bitstream. The embodiments described above and the other embodiments can be implemented as computer-readable recording media storing methods, apparatus, and instructions for encoding or decoding video data according to any of the described methods, and / or computer-readable recording media storing the bitstream generated according to any of the described methods.

[0062] In this application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image,” “picture,” and “frame” may be used interchangeably. Typically, though not always, the term “reconstructed” is used on the encoder side, while “decoded” is used on the decoder side.

[0063] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a particular order of steps or actions is required for the inherent operation of the method, any particular order and / or use of steps and / or actions may be modified or combined.

[0064] The various methods and other aspects described herein can be used to modify modules, for example, the intra-prediction, entropy coding, and / or decoding modules (160, 360, 145, 330) of the video encoder 100 and decoder 200 shown in Figures 7 and 8. Furthermore, these aspects are not limited to VVC or HEVC, but can be applied to other standards and recommendations, whether existing or to be developed in the future, and to extensions of any of the above standards and recommendations (including VVC and HEVC). Unless otherwise indicated or particularly technically hindered, the aspects described herein can be used individually or in combination.

[0065] Various numerical values ​​are used in this application. Certain values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0066] Figure 7 illustrates encoder 100. While variations of encoder 100 are anticipated, encoder 100 is described below for clarity without explaining all anticipated variations.

[0067] Before encoding, a video sequence may undergo pre-encoding (101), which may involve, for example, applying color conversions to the input color picture (e.g., conversion from RGB4:4:4 to YCbCr4:2:0) or remapping input picture components to make the signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata can be associated with pre-processing and linked to the bitstream.

[0068] In encoder 100, the picture is encoded by the encoder elements described below. The picture to be encoded is divided (102) and processed, for example, by units of CU. Each unit is encoded using either intra or intermode, for example. When a unit is encoded in intramode, intra prediction (160) is performed. In intermode, motion estimation (175) and motion compensation (170) are performed. The encoder determines (105) either intramode or intermode to use to encode the unit, and indicates the intra / inter determination, for example, by a prediction mode flag. The prediction residual is calculated, for example, by subtracting the predicted block from the original image block (110).

[0069] Next, the predicted residual is transformed (125) and quantized (130). The quantized transformed coefficients, as well as the motion vector and other syntax elements, are entropy coded (145) to output the bitstream. The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transformation and quantization, i.e., the residual is coded directly without applying either the transformation or quantization process.

[0070] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transformation coefficients are inversely quantized (140) and inversely transformed (150) to decode the prediction residuals. The decoded prediction residuals and the predicted blocks are combined (155) to reconstruct the image blocks. An in-loop filter (165) is applied to the reconstructed picture to perform deblocking / SAO (Sample Adaptive Offset) filtering, for example, to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0071] Figure 8 illustrates a block diagram of the video decoder 200. In the decoder 200, the bitstream is decoded by the decoder elements described below. Generally, the video decoder 200 performs a decoding path that is the reverse of the encoding path described in Figure 7. Furthermore, generally, the encoder 100 also performs video decoding as part of encoding the video data.

[0072] In particular, the decoder input includes a video bitstream that can be generated by the video encoder 100. First, the bitstream is entropy-decoded (230) to obtain transformation coefficients, motion vectors, and other encoded information. Picture segmentation information indicates how the picture will be segmented. Therefore, the decoder may segment (235) the picture according to the decoded picture segmentation information. The transformation coefficients are inversely quantized (240) and inversely transformed (250) to decode the prediction residuals. The image blocks are reconstructed by combining (255) the decoded prediction residuals with the predicted blocks. The predicted blocks can be obtained (270) from intra-predictions (260) or motion-compensated predictions (i.e., inter-predictions) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0073] Furthermore, the decoded picture can undergo post-decoding (285), such as reverse color conversion (e.g., conversion from YCbCr4:2:0 to RGB4:4:4) or reverse remapping, which is the reverse of the remapping process performed in pre-encoding (101). Post-decoding can use metadata signaled in the bitstream derived in pre-encoding.

[0074] Figure 13 illustrates a block diagram of an example of a system in which various aspects and configurations are implemented. System 1000 can be materialized as a device including various components described below and configured to perform one or more aspects described in this document. Examples of the above device include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home electrical appliances, and servers. The elements of System 1000 can be materialized individually or in combination as a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one configuration, the processing and encoder / decoder elements of System 1000 are distributed across multiple ICs and / or individual components. In various configurations, System 1000 is communicated to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various forms, the system 1000 is configured to implement one or more aspects described in this document.

[0075] System 1000 includes, for example, at least one processor 1010 configured to execute instructions loaded to implement various aspects described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 1000 includes at least one memory 1020 (for example, a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040 which may include non-volatile memory and / or volatile memory, including, but not limited to, EEPROM (Electrically Erasable Programmable Read-Only Memory), ROM (Read-Only Memory), PROM (Programmable Read-Only Memory), RAM (Random Access Memory), DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), flash, magnetic disk drives, and / or optical disk drives. Storage device 1040 may include, as an unspecified example, an internal storage device, an attached storage device (including a detachable storage device and a non-detachable storage device), and / or a network-accessible storage device.

[0076] System 1000 includes, for example, an encoder / decoder module 1030 configured to process data that provides encoded or decoded video, the encoder / decoder module 1030 which may include its own processor and memory. Encoder / decoder module 1030 represents a module(s) which may be included in a device that performs encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of System 1000, or it may be incorporated into the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0077] Program code loaded onto the processor 1010 or encoder / decoder 1030 performing the various aspects described herein can be stored in the storage device 1040 and subsequently loaded into memory 1020 for execution by the processor 1010. Depending on the circumstances, one or more processors 1010, memory 1020, storage device 1040, and encoder / decoder modules 1030 can store one or more items during the execution of the processes described herein. The stored items may include, but are not limited to, input video, decoded video, or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0078] In some embodiments, the internal memory of the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for the processing required during encoding or decoding. However, in other embodiments, external memory of the processing device (for example, the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of the functions described above. The external memory can be memory 1020 and / or storage device 1040, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one aspect, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as MPEG-2 (MPEG refers to the Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, 13818-1 is known as H.222, and 13818-2 is known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, JVET, a standard developed by the Joint Video Experts Team).

[0079] Inputs to the elements of system 1000 can be provided via various input devices, as shown in block 1130. These input devices include, but are not limited to, (i) an RF section for receiving, for example, RF (radio frequency) signals transmitted by a broadcasting station via radio waves, (ii) a COMP (Component) input terminal (or a set of COMP input terminals), (iii) a USB (Universal Serial Bus) input terminal, and / or (iv) an HDMI (High Definition Multimedia Interface) input terminal. Other examples not shown in Figure 13 include composite video. In various embodiments, the input device of block 1130 associates with various input processing elements known in the art. For example, the RF portion can be associated with elements suitable for (i) selecting a desired frequency (also said to be selecting a signal or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting a signal frequency band, which in some embodiments may be called a channel, to a narrower band of the selecting frequency, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream for data packets. Various embodiments of the RF portion include one or more elements that perform the functions described above, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error collectors, and demultiplexers. The RF portion can include tuners that perform various functions, such as down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or to the baseband. In one set-top box configuration, the RF section and associated input processing elements receive, filter, down-convert, and filter again to a desired frequency band to perform frequency selection. Various configurations involve rearranging the order of the elements described above (and others), removing some of the elements mentioned, and / or adding other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various configurations, the RF section includes an antenna.

[0080] Additionally, USB and / or HDMI terminals may include their respective interface processors to connect system 1000 to other electronic devices over USB and / or HDMI connections. It is understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or, if necessary, in processor 1010. Similarly, aspects of USB or HDMI interface processing may be implemented in a separate input processing IC or, if necessary, in processor 1010. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, for example, processor 1010 and encoder / decoder 1030, which works in cooperation with memory and storage elements to process the data stream for submission to output devices as necessary.

[0081] Various elements of system 1000 can be provided within an integrated housing, where the various elements are interconnected and can transmit data between them using an internal bus known in the art, such as an I2C (Inter-IC) bus, wiring, and printed circuit boards.

[0082] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may, but is not limited to, include a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may, but is not limited to, include a modem or a network card, and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0083] Data is provided to system 1000 in various ways, for example, by streaming or otherwise using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals in the manner described above are received via a communication channel 1060 and a communication interface 1050, which are modified to accommodate Wi-Fi communication. Typically, the communication channel 1060 in the manner described above is connected to an access point or router that provides access to an external network, including the Internet, to enable streaming applications and other over-the-top communications. In other ways, the streamed data is provided to system 1000 using a set-top box that distributes the data via the HDMI connection of input block 1130. Still, in other ways, the streamed data is provided to system 1000 using the RF connection of input block 1130. As shown above, various ways provide data in ways other than streaming. Additionally, various ways utilize wireless networks other than Wi-Fi, e.g., cellular networks or Bluetooth networks.

[0084] System 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. Various embodiments of the display 1100 include, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, tablet, laptop, cellular phone (mobile phone), or other device. Furthermore, the display 1100 can be integrated with other components (e.g., in a smartphone) or separate (e.g., with an external monitor for a laptop). In various embodiments, the other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments utilize one or more peripheral devices 1120 that provide functions based on the output of System 1000. For example, a disc player plays the role of playing the output of System 1000.

[0085] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral devices 1120 using signaling such as AV.Link, CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be connected to the system 1000 via dedicated connections through their respective interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to the system 1000 via communication interface 1050 using communication channel 1060. The display 1100 and speaker 1110 can be integrated into a single unit with other components of the system 1000, for example, in an electronic device such as a television. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (TCon) chip.

[0086] Alternatively, the display 1100 and speaker 1110 can be isolated from one or more other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments where the display 1100 and speaker 1110 are external components, the output signals can be submitted via a dedicated output connection, such as an HDMI port, a USB port, or a COMP output.

[0087] The embodiments can be implemented by computer software implemented by processor 1010, by hardware, or by a combination of hardware and software. In a non-limiting example, the embodiments can be implemented by one or more integrated circuits. Memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, for example, in a non-limiting example. Processor 1010 can be of any type suitable for the technical environment and can include, in a non-limiting example, one or more of microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures.

[0088] Various implementations include decoding. As used in this application, “decoding” can include, for example, all or part of processing performed on the receiving encoding sequence to produce a final output suitable for display. In various embodiments, the processing includes one or more of the processing typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, further or alternatively, the processing includes processing performed by the decoders of the various implementations described in this application.

[0089] As further examples, in one aspect, “decoding” refers only to “entropy decoding,” in another aspect, “decoding” refers only to differential decoding, and in yet another aspect, “decoding” refers to a combination of entropy decoding and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or to refer more broadly in general, it is believed that the decoding process will become clear from the context of a particular description and will be well understood by those skilled in the art.

[0090] Various implementations include encoding. In a manner similar to the above discussion of "decoding," "encoding" as used in this application can include, for example, all or part of the processing performed on the input video sequence to generate the encoded bitstream. In various embodiments, the above processing includes one or more of the processing typically performed by an encoder, such as splitting, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, further or alternatively, the above processing includes processing performed by the encoders of the various implementations described in this application.

[0091] As further examples, in one aspect, “encoding” refers only to “entropy decoding,” in another aspect, “encoding” refers only to differential decoding, and in yet another aspect, “encoding” refers to a combination of differential encoding and entropy encoding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to refer more broadly in general, it is believed that encoding processes will become clear from the context of a particular description and will be well understood by those skilled in the art.

[0092] Note that the syntax elements used in this specification are descriptive terms. As mentioned above, they do not preclude the use of other syntax element names.

[0093] When a diagram is given as a flowchart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is given as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0094] Various approaches may cite parametric models or rate distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often given complexity constraints on the computation. This can be measured through the Rate Distortion Optimization (RDO) metric, or through Least Mean Square (LMS), Mean of Absolute Errors (MAE), or other such measurements. Typically, rate distortion optimization is formulated as minimizing a rate distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving rate distortion optimization problems. For example, an approach may be based on extensive testing of all encoding options, including all considered mode or encoding parameter values, along with a complete evaluation of the encoding cost and associated distortion of the reconstructed signal after encoding and decoding. Furthermore, faster approaches may be used, in order to eliminate the complexity of encoding, particularly with the calculation of approximated distortion based on a predicted or predicted residual signal rather than one of the reconstructed ones. A mixture of the two approaches described above can also be used, for example, by using a distortion that approximates only some of the possible encoding options and the full distortion for the other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches employ one of various optimization techniques, but the optimization is not necessarily a full evaluation of both the encoding cost and the distortion associated with it.

[0095] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even when described only in the context of a single form of implementation (e.g., only as a method), the implementation of the described features can also be implemented in other forms (e.g., apparatus or program). Apparatus can be implemented, for example, in appropriate hardware, software, and firmware. Methods can be implemented, for example, in processors, which generally refer to processing devices, including computers, microprocessors, integrated circuits, or programmable logic devices. Furthermore, processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users.

[0096] References to “one aspect,” “aspect,” “one implementation,” or “implementation” mean, as with other variations, that certain features, structures, characteristics, etc., described in relation to an aspect are included in at least one aspect. Therefore, the appearance of the phrase “in one aspect,” “in one aspect,” “in one implementation,” or “in implementation,” which appears in various places throughout this application, does not necessarily refer to all of the same aspects, as with any other variation.

[0097] In addition, this application may use the term “determining” various parts of information. Determining information can include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory.

[0098] Furthermore, this application may use the term “accessing” various parts of information. Accessing information can include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0099] In addition, this application may use the term “receiving” various parts of information. Receiving is intended to be a broad term, as with respect to “accessing.” Receiving information can include, for example, accessing information or retrieving information (for example, from memory). Furthermore, “receiving” usually includes, in some way or another, actions such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0100] It should be understood that the use of any of the following " / ", "and / or", and "at least one of" is intended to include, for example, "A / B", "A and / or B", and "at least one of A and B", which include selecting only the first enumerated option (A), or only the second enumerated option (B), or both options (A and B). As further examples, in the case of "A, B, and / or C" and "at least one of A, B, and C", the above phrases are intended to include selecting only the first enumerated option (A), or only the second enumerated option (B), or only the third enumerated option (C), or only the first and second enumerated options (A and B), or only the first and third enumerated options (A and C), or the second and third enumerated options (B and C), or all three options (A, B, and C). What has been stated can be extended to the matters listed, as will be obvious to those skilled in the art and related businesses.

[0101] Furthermore, as used herein, the word “signaling” primarily refers to indicating something to a corresponding decoder. For example, in a certain manner, an encoder signals one particular one of several transformations, encoding modes, or flags. In this manner, the same transformation, parameter, or mode is used on both the encoder and decoder sides. Thus, for example, an encoder can transmit a particular parameter to a decoder (explicit signaling), and the decoder can use the same particular parameter. Conversely, if the decoder already has the same particular parameter, signaling can be used without transmission (implicit signaling) simply to allow the decoder to know and select the particular parameter. Bit saving is achieved in various ways by avoiding transmission in any real-world function. It is understood that signaling can be performed in various ways. For example, one or more syntax elements, flags, etc., can be used in various ways to signal information to a corresponding decoder. The above concerns the verb form of the word "signaling," but the word "signaling" can also be used as a noun in this specification.

[0102] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be stored or transmitted. For example, the information can include instructions for performing a method or data generated by one of the implementations described. For example, a signal can be formatted to carry a bitstream in the manner described. For example, the above signal can be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. For example, formatting can include encoding a data stream and modulating a carrier wave with the encoded data stream. For example, the information carried by the signal can be analog or digital information. The signal can be transmitted over various separate wired or wireless links, as is known. The signal can be stored in a processor-readable medium.

[0103] We describe several embodiments across various claim categories and types. The features of these embodiments can be provided individually or in any combination. Furthermore, embodiments can include one or more of the following features, devices, or aspects, individually or in any combination, across various claim categories and types. ● A bitstream or signal containing one or more of the described syntax elements, or variations thereof. ● A bitstream or signal containing a syntax that conveys information generated in any of the described embodiments. ● Creating and / or transmitting and / or receiving and / or decoding in any of the described manner. ● A method, process, apparatus, medium for storing instructions, medium for storing data, or signal that conforms to any of the embodiments described. ● Inserting syntax elements into the signaling that enable the decoder to determine the encoding mode in a corresponding manner used by the encoder. ● Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that contains one or more of the described syntax elements or variations thereof. ●A television, set-top box, mobile phone, tablet, or other electronic device that performs the conversion method(s) in any of the described embodiments. ● A television, set-top box, mobile phone, tablet, or other electronic device that performs a conversion method(s) determination according to any of the described embodiments and displays the resulting image (for example, using a monitor, screen, or other type of display). A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image by selecting a channel (for example, using a tuner), band-limiting, or combining channels, and performs a conversion method(s) according to any of the described embodiments. ● A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal (for example, using an antenna) via radio waves containing an encoded image and performs a conversion method(s).

Claims

1. For the current image block, obtain a first local illumination correction (LIC) flag indicating that local illumination correction is applied to the current image block, Obtaining LIC model parameters for at least one component of the current image block, Encoding the current image block using the LIC model parameters, A second LIC flag is derived for at least one component of the current image block based on at least one parameter of the LIC model parameters obtained for the at least one component of the current image block, or based on at least one parameter of the LIC model parameters obtained for another component of the current image block different from the at least one component of the current image block, wherein the second LIC flag, unlike the first LIC flag, is available for use by another image block different from the current image block and indicates whether local illumination correction is applied to the other image block. A method that includes this.

2. The method of claim 1, wherein deriving the second LIC flag includes setting the second LIC flag to false if at least one of the LIC model parameters is within the range of default values.

3. The method of claim 1, wherein deriving the second LIC flag comprises deriving one second LIC flag for the luminance component and another second LIC flag for the chrominance component.

4. The method of claim 1, wherein deriving the second LIC flag includes setting the second LIC flag to false when the LIC model parameters include an LIC scale and the LIC scale is equal to 0.

5. The method of claim 1, wherein deriving a second LIC flag for at least one component of the current image block includes setting the second LIC flag to a multiple value of the second LIC flag of an adjacent block.

6. The method of claim 1, wherein the second LIC flag is presumed to be false with respect to the color difference component.

7. For the current image block, obtain a first local illumination correction (LIC) flag indicating that local illumination correction is applied to the current image block, Obtaining LIC model parameters for at least one component of the current image block, Encoding the current image block using the LIC model parameters, A second LIC flag is derived for at least one component of the current image block based on at least one parameter of the LIC model parameters obtained for the at least one component of the current image block, or based on at least one parameter of the LIC model parameters obtained for another component of the current image block different from the at least one component of the current image block, wherein the second LIC flag, unlike the first LIC flag, is available for use by another image block different from the current image block and indicates whether local illumination correction is applied to the other image block. A device equipped with a processor configured to perform the following actions.

8. The apparatus of claim 7, wherein deriving the second LIC flag includes setting the second LIC flag to false if at least one of the LIC model parameters is within the range of default values.

9. The apparatus of claim 7, wherein deriving the second LIC flag includes deriving one second LIC flag for the luminance component and another second LIC flag for the chrominance component.

10. The apparatus of claim 7, wherein deriving the second LIC flag includes setting the second LIC flag to false when the LIC model parameter includes an LIC scale and the LIC scale is equal to 0.

11. The apparatus of claim 7, wherein deriving a second LIC flag for at least one component of the current image block includes setting the second LIC flag to a multiple value of the second LIC flag of an adjacent block.

12. The apparatus of claim 7, wherein the second LIC flag is presumed to be false with respect to the color difference component.

13. For the current image block, obtain a first local illumination correction (LIC) flag indicating that local illumination correction is applied to the current image block, Obtaining LIC model parameters for at least one component of the current image block, Decoding the current image block using the LIC model parameters, A second LIC flag is derived for at least one component of the current image block based on at least one parameter of the LIC model parameters obtained for the at least one component of the current image block, or based on at least one parameter of the LIC model parameters obtained for another component of the current image block different from the at least one component of the current image block, wherein the second LIC flag, unlike the first LIC flag, is available for use by another image block different from the current image block and indicates whether local illumination correction is applied to the other image block. A method that includes this.

14. The method of claim 13, wherein deriving the second LIC flag includes setting the second LIC flag to false if at least one of the LIC model parameters is within the range of default values.

15. The method of claim 13, wherein deriving the second LIC flag comprises deriving one second LIC flag for the luminance component and another second LIC flag for the chrominance component.

16. The method of claim 13, wherein deriving the second LIC flag includes setting the second LIC flag to false if the LIC model parameters include an LIC scale and the LIC scale is equal to 0.

17. The method of claim 13, wherein deriving a second LIC flag for at least one component of the current image block includes setting the second LIC flag to a multiple value of the second LIC flag of an adjacent block.

18. The method of claim 13, wherein the second LIC flag is presumed to be false with respect to the color difference component.

19. For the current image block, obtain a first local illumination correction (LIC) flag indicating that local illumination correction is applied to the current image block, Obtaining LIC model parameters for at least one component of the current image block, Decoding the current image block using the LIC model parameters, A second LIC flag is derived for at least one component of the current image block based on at least one parameter of the LIC model parameters obtained for the at least one component of the current image block, or based on at least one parameter of the LIC model parameters obtained for another component of the current image block different from the at least one component of the current image block, wherein the second LIC flag, unlike the first LIC flag, is available for use by another image block different from the current image block and indicates whether local illumination correction is applied to the other image block. A device equipped with a processor configured to perform the following actions.

20. The apparatus of claim 19, wherein deriving the second LIC flag includes setting the second LIC flag to false if at least one of the LIC model parameters is within the range of default values.

21. The apparatus of claim 19, wherein deriving the second LIC flag comprises deriving one second LIC flag for the luminance component and another second LIC flag for the chrominance component.

22. The apparatus of claim 19, wherein deriving the second LIC flag includes setting the second LIC flag to false if the LIC model parameter includes an LIC scale and the LIC scale is equal to 0.

23. The apparatus of claim 19, wherein deriving a second LIC flag for at least one component of the current image block includes setting the second LIC flag to a multiple value of the second LIC flag of an adjacent block.

24. The apparatus of claim 19, wherein the second LIC flag is presumed to be false with respect to the color difference component.

25. An apparatus according to any one of claims 19 to 24, (i) an antenna configured to receive a signal, the signal including a video block; (ii) a band limiter configured to limit the received signal to a frequency band including the video block; and (iii) at least one of a display configured to display an output representing the video block. A device equipped with the following features.

26. A computer program that, when executed by a computer, includes instructions causing the computer to perform the method according to any one of claims 1 to 6.

27. A computer program that, when executed by a computer, includes instructions causing the computer to perform the method according to any one of claims 13 to 18.

Images