Method and apparatus for local illumination compensation model inheritance for video coding and decoding

Abstract

The invention discloses a method and device for inheriting LIC parameters. According to the method, a Local Illumination Compensation (LIC) prediction candidate having inherited LIC parameters inherited from LIC parameters of an inherited block is derived. A candidate list including LIC candidates is used to encode or decode a current block, in which a predictor of the current block is generated by applying inherited L IC parameters to a reference block when an LIC candidate is selected for the current block. According to another method, a merge list consisting of LIC parameters of spatially neighboring blocks, non-immediately neighboring blocks, temporally neighboring blocks, and history-based blocks is generated.

Description

Methods and apparatus for inheriting local illumination compensation models in video encoding and decoding

[0001] [Cross-Reference] This invention claims priority to U.S. Provisional Patent Application No. 63 / 589,657, filed October 12, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] [Technical Field] This invention relates to video encoding and decoding systems. Specifically, this invention relates to the inheritance of Local Illumination Compensation (LIC) parameters in video encoding and decoding systems. Background Technology

[0003] [Background Technology] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Video Coding Experts Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Joint Video Experts Team (JVET) of the Moving Picture Experts Group (MPEG) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC). This standard was published as an ISO standard in February 2021: ISO / IEC 23090-3:2021, Information technology – Codec representation of immersive media – Part 3: Versatile video coding. VVC was developed based on its predecessor, High Efficiency Video Coding (HEVC). It improves encoding and decoding efficiency by adding more encoding and decoding tools, and can handle various types of video sources, including three-dimensional (3D) video signals.

[0004] Figure 1A illustrates an exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing. For intra-frame prediction, the prediction data is derived from previously encoded / decoded picture video data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed at the encoder, and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112 and provides the selected prediction data to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118, followed by quantization (Q) 120. The residuals from the transform and quantization are then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as motion and encoding / decoding modes associated with intra-frame and inter-frame prediction) and other information such as parameters associated with loop filters applied to the underlying image regions. As shown in Figure 1A, auxiliary information related to intra-frame prediction 110, inter-frame prediction 112, and loop filter 130 is provided to entropy encoder 122. When inter-frame prediction mode is used, one or more reference images must also be reconstructed at the encoder. Therefore, the residuals from transform and quantization are processed by inverse quantization (IQ) 124 and inverse transformation (IT) 126 to recover the residuals. The residuals are then added back to the prediction data 136, reconstructing the video data at reconstruction (REC) 128. The reconstructed video data can be stored in reference image buffer 134 and used for prediction in other frames.

[0005] As shown in Figure 1A, the incoming video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may be subject to various degradations due to these processes. Therefore, a loop filter 130 is typically applied to the reconstructed video data to improve video quality before storing it in the reference picture buffer 134. For example, a deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF) may be used. Loop filter information may need to be incorporated into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for incorporation into the bitstream. In Figure 1A, the loop filter 130 is applied to the reconstructed video, and then the reconstructed samples are stored in the reference picture buffer 134. The system in Figure 1A is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] As shown in Figure 1B, apart from transform 118 and quantization 120, the decoder can use the same or partially the same function blocks as the encoder, since the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses entropy decoder 140 instead of entropy encoder 122 to decode the video bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 at the decoder end does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140, without motion estimation.

[0007] In the High-Efficient Video Coding standard (HEVC), images are partitioned into a series of coding tree units (CTUs). For an image with three sample arrays, a CTU consists of NxN luma sample blocks and two corresponding chroma sample blocks; or for an image encoded using three separate color planes, a CTU consists of NxN monochromatic plane sample blocks. The CTU concept is broadly similar to the macroblock concept in previous standards, such as Advanced Video Coding (AVC). In the master configuration file, the maximum allowed size of a luma block in a CTU is specified as 64x64. A quadtree structure (called a codec tree) is used to partition the CTU into CUs to accommodate various local features. At the leaf CU level, it is determined whether to use inter-image (temporal) or intra-image (spatial) prediction to encode image regions. Depending on the PU partitioning type, each leaf CU can be further divided into one, two, or four prediction units (PUs). Within a PU, the same prediction processing is applied, and relevant information is transmitted to the decoder based on the PU partitioning type. After obtaining the residual block through prediction processing based on the PU partitioning type, the leaf CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the CU coding tree. A key feature of the HEVC architecture is that it has multiple partitioning concepts, including CU, PU, ​​and TU.

[0008] Versatile Video Coding (VVC) is the successor to HEVC. In VVC, a quadtree using a nested multi-type tree with binary and ternary tree partitioning structures replaces the concept of multiple partition unit types. That is, it eliminates the separation of CU, PU, ​​and TU concepts unless the CU size is too large for the maximum transform length, and supports greater flexibility in CU partition shapes. In the codec tree structure, CUs can have square or rectangular shapes. A coding tree unit (CTU) is first partitioned by a quadtree structure. Then, the leaf nodes of the quadtree can be further partitioned by a multi-type tree structure. As shown in Figure 2, there are four partitioning types in the multi-type tree structure: vertical binary partitioning (SPLIT_BT_VER 210), horizontal binary partitioning (SPLIT_BT_HOR 220), vertical ternary partitioning (SPLIT_TT_VER 230), and horizontal ternary partitioning (SPLIT_TT_HOR 240). Multi-type leaf nodes are called coding units (CUs). Unless the CU is greater than the maximum transform length, this partitioning is used for prediction and transform processing without further subdivision. This means that in most cases, in a quadtree with a nested multi-type tree coding block structure, the CU, PU, ​​and TU have the same block size. An exception occurs when the maximum supported transform length is less than the width or height of the CU color component.

[0009] Figure 3 illustrates a CTU with a quadtree and nested multi-type tree codec block structure divided into multiple CUs, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive codec tree structure composed of CUs. The size of a CU can be the same as the CTU or as small as 4×4 (in luminance samples). For the 4:2:0 chroma format, the maximum chroma CB size is 64×64, and the minimum chroma CB size consists of 16 chroma samples.

[0010] In VVC, the maximum supported luma transformation size is 64×64, and the maximum supported chroma transformation size is 32×32. When the width or height of the CB exceeds the maximum transformation width or height, the CB will automatically be split in the horizontal and / or vertical directions to meet the transformation size limit in that direction.

[0011] In VVC, the codec tree scheme supports the ability for luma and chroma to have separate block tree structures. For P and B segments, the luma and chroma CTBs within a CTU must share the same codec tree structure. However, for I segments, luma and chroma can have separate block tree structures. When the separate block tree mode is applied, the luma CTB is divided into CUs by one codec tree structure, while the chroma CTB is divided into chroma CUs by another codec tree structure. This means that a CU in an I segment can consist of codec blocks for the luma component or codec blocks for the two chroma components, while a CU in a P or B segment always consists of codec blocks for all three color components, unless the video is monochrome.

[0012] For each inter-frame prediction CU, motion parameters consist of motion vectors, reference picture indices, reference picture list usage indices, new encoding / decoding features of the VVC, and additional information required for generating inter-frame prediction samples. Motion parameters can be sent explicitly or implicitly. When encoding / decoding a CU using skip mode, the CU is associated with a PU and has no significant residual coefficients, no encoded / decoded motion vector increments, or reference picture indices. A merge mode is specified, thereby obtaining the motion parameters of the current CU from neighboring CUs (including spatial and temporal candidates) and additional scheduling introduced in the VVC. The merge mode can be applied to any inter-frame prediction CU, not just skip mode. An alternative to the merge mode is to explicitly transmit motion parameters, where motion vectors, corresponding reference picture indices for each reference picture list, reference picture list usage flags, and other required information are explicitly sent for each CU.

[0013] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 5) are investigating the potential need for future video codec standardization, with compression capabilities significantly exceeding the current VVC standard. The Enhanced Compression Model (ECM) reference software serves as a reference implementation of the coding techniques and decoding processes used in JVET's exploration of enhanced compression capabilities beyond VVC. The reference software is accessible at https: / / vcgit.hhi.fraunhofer.de / ecm / ECM.git. ECM is essentially a successor to VVC, and therefore shares many commonalities with it.

[0014] In JVET-AF0080 (Luhang Xu et al., “EE2-2.7: Intra-frame prediction mode based on extrapolation filter,” Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 32nd meeting, Hanover, Germany, October 13-20, 2023, document: JVET-AF0080), intra-frame prediction based on extrapolation filter is disclosed. Intra-frame prediction based on extrapolation filter involves three steps. First, the minimum, maximum, and offset values ​​are obtained from a predetermined reconstruction region. Second, the extrapolation filter coefficients are obtained from adjacent reconstructed pixels of a predetermined template in the current block. Third, the extrapolation process generates predicted values ​​position by position from the top left corner to the bottom right corner of the current block.

[0015] The Multiple Model (MMLM) VVC includes an extended CCLM by adding three Multiple Model LM (MMLM) modes (JVET-D0110, Kai Zhang et al., “Enhanced Intra-Frame Prediction with Cross-Component Linear Models”, Joint Video Exploration Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC1 / SC 29 / WG 11, 4th Meeting: Chengdu, China, October 15-21, 2016). In each MMLM mode, reconstructed neighboring samples are divided into two classes using a threshold that is the average of the neighboring samples in the luminance reconstruction. The linear model for each class is derived using the Least-Mean-Square (LMS) method. For the CCLM mode, the LMS method is also used to derive the linear model. Slope adjustment is applied to the cross-component linear model (CCLM) and the multiple model LM predictions. The adjustment is a skewed linear function that maps luminance values ​​to chrominance values ​​based on a center point determined by the average luminance values ​​of a reference sample.

[0016] CCLM Slope Adjustment: CCLM uses a two-parameter model to map luminance values ​​to chrominance values, as shown in Figure 4A. The slope parameter "a" and the deviation parameter "b" are defined as follows: The adjustment "u" of the slope parameter is sent to update the model to the following form, as shown in Figure 4B: ,in , With this selection, the mapping function revolves around the brightness value y. r The point is tilted or rotated. The average value of the reference brightness sample used in model creation is used as y. r This is so that meaningful modifications can be made to the model.

[0017] Local Illumination Compensation (LIC) is an inter-frame prediction technique used to model the local illumination variation between the current block and its predicted block as a function of the local illumination variation between the current block template and the reference block template. The parameters of this function can be represented by a scaling factor α and an offset β, which form a linear equation, α*p[x]+β, to compensate for the illumination variation, where p[x] is the reference sample that the MV points to, located at position x in the reference image. When surround motion compensation is enabled, the MV should be clipped, taking the surround offset into account. Since α and β can be derived based on the current block template and the reference block template, they require no signaling overhead, except for sending the LIC flag to AMVP mode to indicate the use of LIC.

[0018] The local illumination compensation proposed in JVET-O0066 (Vadim Seregin et al., “CE4-3.1a and CE4-3.1b: Unidirectional Local Illumination Compensation with Affine Prediction,” Joint Video Exploration Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15th meeting: Gothenburg, Sweden, July 3-12, 2019, document: JVET-D0110) is used for unidirectional prediction of inter-frame CUs, with the following modifications.

[0019] • Intra-frame neighbor samples can be used for LIC parameter derivation; • LIC is disabled for blocks with fewer than 32 luma samples; • For non-sub-block and affine modes, LIC parameter derivation is performed based on the template block sample corresponding to the current CU, rather than based on the partial template block sample corresponding to the first top-left 16x16 unit; • The sample of the reference block template is generated using MC and block MV, without needing to round it to integer pixel precision.

[0020] The bidirectional prediction LIC method extends the LIC model to bidirectional prediction CU. Specifically, it applies two different linear models to two prediction blocks and then combines them to generate bidirectional prediction samples for the current CU. as well as , Here, α0 and β0, and α1 and β1, represent the scale and offset in L0 and L1, respectively; ω represents the weight of the weighted combination of L0 and L1 predictions (represented by the CU-level BCW index). The same derivation scheme for the LIC model is repeated and applied iteratively to derive the L0 and L1 LIC parameters. Specifically, the method first derives the L0 parameters by minimizing the difference between the L0 template prediction T0 and the template T, and updates the samples in T by subtracting the corresponding samples from T0. Then, the L1 parameters are calculated to minimize the difference between the L1 template prediction T1 and the updated template. Finally, the L0 parameters are refined again in the same manner.

[0021] According to the current LIC design, a flag is emitted to indicate the LIC mode for AMVP bidirectional prediction CUs, while the flag is inherited for merged inter-frame CUs. Furthermore, LIC is disabled when decoder-sidemotion vector refinement (DMVR) (including multi-pass DMVR, adaptive DMVR, and affine DMVR) and bidirectional optical flow (BDOF) are applied.

[0022] In this method, OBMC with LIC is enabled for inter-frame blocks encoded and decoded using LIC mode. To reduce complexity, OBMC is only applied to the top and left boundaries of the CU, while the boundaries of internal sub-blocks of a LIC CU are always disabled. Furthermore, when a neighboring block is encoded and decoded using LIC, its LIC parameters are applied to generate a corresponding prediction sample for the OBMC of the current block.

[0023] This invention discloses a method and apparatus for inheriting LIC parameters from the block to which the inherited motion vector belongs. Furthermore, it also discloses a method and apparatus for generating a list of LIC candidates. Summary of the Invention

[0024] [Summary of the Invention] A method and apparatus for inheriting LIC parameters are disclosed. According to the method, input data associated with the current block is received, wherein the input data includes pixel data to be encoded on the encoder side or data associated with the current block to be decoded on the decoder side, wherein the current block inherits an inherited motion vector from an inherited block. Local illumination compensation (LIC) prediction candidates with inherited LIC parameters inherited from the inherited block are derived. A candidate list including LIC candidates is used to encode or decode the current block, wherein when an LIC candidate is selected for the current block, the predictor for the current block is generated by applying the inherited LIC parameters to a reference block.

[0025] In one embodiment, inherited LIC parameters are used to guide the derivation of the current LIC parameters for the current block. In another embodiment, TM cost is used to determine whether to use either inherited or derived LIC parameters for the current block, wherein one or more templates of the current block and a reference block are used to derive the derived LIC parameters. In yet another embodiment, a CU-level flag is sent to indicate whether to use either inherited or derived LIC parameters.

[0026] According to another method, input data associated with the current block is received, wherein the input data includes pixel data to be encoded on the encoder side or data associated with the current block to be decoded on the decoder side, wherein the current block is encoded / decoded inter-frame. A Local Illumination Compensation (LIC) candidate list is generated, wherein the LIC candidate list includes first LIC parameters associated with one or more first LIC models of one or more spatially adjacent blocks, second LIC parameters associated with one or more second LIC models of one or more non-immediately adjacent blocks, third LIC parameters associated with one or more third LIC models of one or more temporally adjacent blocks, fourth LIC parameters associated with one or more fourth LIC models of one or more history-based blocks, or combinations thereof. The LIC candidate list is used to encode or decode the current block.

[0027] In one embodiment, a target LIC candidate is selected from the LIC candidate list of the current block, and a predictor for the current block is generated by applying LIC parameters associated with the target LIC candidate to a reference block. In one embodiment, the LIC candidate list also includes one or more predetermined LIC parameters.

[0028] In one embodiment, an index is sent to indicate a target LIC candidate selected from the LIC candidate list. In another embodiment, if the reference block is encoded / decoded by a multi-model LIC, multiple LIC models associated with the reference block are inserted into the LIC candidate list. Attached Figure Description

[0029] [Figure Description] Figure 1A illustrates an exemplary adaptive inter-frame / intra-frame video encoding and decoding system that includes loop processing.

[0030] Figure 1B shows the corresponding decoder for the encoder in Figure 1A.

[0031] Figure 2 shows examples of multiple tree structures corresponding to vertical binary splits (SPLIT_BT_VER), horizontal binary splits (SPLIT_BT_HOR), vertical ternary splits (SPLIT_TT_VER), and horizontal ternary splits (SPLIT_TT_HOR).

[0032] Figure 3 shows an example of dividing a CTU into multiple CUs using a quadtree and a nested multi-type tree coding block structure, where the bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions.

[0033] Figure 4A shows an example of a CCLM model.

[0034] Figure 4B shows an example of the effect of the slope adjustment parameter "u" on model updates.

[0035] Figure 5 illustrates a flowchart of an exemplary video encoding and decoding system according to an embodiment of the present invention, wherein the current block inherits the inherited MV from the target block, and the LIC parameters of the target block are inherited by the current block.

[0036] Figure 6 illustrates a flowchart of an exemplary video encoding and decoding system according to an embodiment of the present invention, wherein the merge list consists of LIC parameters for spatially adjacent blocks, non-immediately adjacent blocks, temporally adjacent blocks, and history-based blocks. Detailed Implementation

[0037] [Detailed Description] The components of the present invention, as generally described and shown in the figures, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as shown in the figures, is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the invention. References to “one embodiment,” “an embodiment,” or similar language in this specification mean that a particular feature, structure, or characteristic described in relation to that embodiment may be included in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in one embodiment” appearing throughout this specification do not necessarily refer to the same embodiment.

[0038] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without using one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. Embodiments of the invention will be best understood by referring to the accompanying drawings, wherein like parts are designated by like numbers throughout. The following description is illustrative only and only illustrates embodiments of specific selected apparatus and methods consistent with the invention declared herein.

[0039] To improve the encoding and decoding performance of video systems that include Local Illumination Compensation (LIC), various schemes related to inheriting the LIC model have been disclosed.

[0040] In one embodiment of the inter-frame merge mode's LIC, the LIC parameters of the reference block to which the inherited motion vector points can be inherited by the current block. In this case, the LIC parameters of the current block do not need to be re-derived based on the templates of the current block and the referenced block.

[0041] In another embodiment, in inter-frame merging mode, the LIC parameters of the inherited motion vector source block can be inherited by the current block. In this case, the LIC parameters of the current block do not need to be re-derived based on the templates of the current block and the referenced block.

[0042] In one embodiment, in inter-frame merging mode, the LIC parameters of the reference block to which the inherited motion vector points can be used to guide the derivation of the LIC parameters of the current block.

[0043] In one embodiment, in inter-frame merging mode, the LIC parameters of the block from which the inherited motion vector originates can be used to guide the derivation of the LIC parameters of the current block.

[0044] For example, a regularization term can be added during the derivation of the LIC parameters of the current block. This regularization term is designed based on the LIC parameters of the reference block to which the inherited motion vector points or the LIC parameters of the block in which the inherited motion vector is located.

[0045] For example, a regularization term multiplied by a lambda value can be added during the derivation of the LIC parameters for the current block. This lambda value is designed based on the prediction mode of the current block, the current block size, the QP value, or the motion compensation stage (i.e., final motion compensation, template derivation for list reordering, template derivation for BCW index derivation, and template derivation for prediction direction derivation).

[0046] In one embodiment, in inter-frame merging mode, self-derived LIC parameters are exported. The self-derived LIC parameters are derived based on the current block template and a reference block template. A first cost is calculated based on the current block template and a first refined reference block template. The first reference block template is derived based on the self-derived LIC parameters. A second cost is calculated based on the current block template and a second refined reference block template. The second refined reference block template is derived based on the LIC parameters of the reference block to which the inherited motion vector points or the LIC parameters of the block from which the inherited motion vector originates. Then, the first cost and the second cost are multiplied by a ratio and compared.

[0047] If the first cost is less than the second cost multiplied by the ratio, then the self-derived LIC parameters are used to refine the current block. Otherwise, if the second cost multiplied by the ratio is less than the first cost, then the LIC parameters of the reference block to which the inherited motion vector points, or the LIC parameters of the block from which the inherited motion vector originates, are used to refine the current block.

[0048] The ratio can be less than 1 to favor inherited LIC parameters. Alternatively, the ratio can be greater than 1 to favor self-derived LIC parameters. Or, the ratio can be equal to 1 for fair comparison.

[0049] In one embodiment, in merge mode, a CU-level flag is sent to indicate whether to use a self-derived LIC parameter or an inherited LIC parameter.

[0050] In one embodiment, in inter-frame merging mode, self-derived LIC parameters are exported. These self-derived LIC parameters are derived based on the templates of the current block and the reference block. Subsequently, the self-derived LIC parameters and inherited LIC parameters are mixed with weight pairs. In this case, the weight pairs can be designed based on the prediction mode of the current block, the current block size, the QP value, or the motion compensation stage (i.e., final motion compensation, template export for list reordering, template export for BCW index export, template export for prediction direction export).

[0051] In one embodiment, if the reference block is a bidirectional predictive codec block and the current block is a unidirectional predictive codec block, then the corresponding LIC parameters will be referenced by the current block.

[0052] For example, if the current block is predicted based on the reference image in list 0, then the current image will reference the LIC parameter of L0 of the bidirectional prediction reference block.

[0053] For example, if the current block is predicted based on the reference image in List 1, then the current image will reference the LIC parameter of L1 of the bidirectional prediction reference block.

[0054] In one embodiment, if the reference block is a bidirectional predictive codec block and the current block is a unidirectional predictive codec block, then the corresponding LIC parameters will be referenced by the current block, and the referenced LIC parameters will be further refined. For example, the templates of the current block and the reference block will be used to refine the inherited LIC parameters.

[0055] In one embodiment, if the reference block is a bidirectional predictive codec block and the current block is a unidirectional predictive codec block, the current block will not reference the LIC parameters of the reference block. The current block will use the self-derived LIC parameters.

[0056] In one embodiment, if the reference block is a unidirectional predictive codec block and the current block is a bidirectional predictive codec block, the current block will not reference the LIC parameters of the reference block. The current block will use the self-derived LIC parameters.

[0057] In one embodiment, if the reference block is a unidirectional predictive codec block and the current block is a bidirectional predictive codec block, the referenced LIC parameters will be used for the corresponding prediction side, while the LIC parameters for the other side will be further derived using the templates of the current block and the reference block.

[0058] In one embodiment, if the reference block is a unidirectional predictive codec block and the current block is a bidirectional predictive codec block, then the referenced LIC parameters are used on the corresponding prediction side (referred to as the first prediction side), and the LIC parameters of the second prediction side are further derived using the current block template and the reference block template, and then the LIC parameters of the first prediction side are further refined.

[0059] In one embodiment, if the reference block is predicted from L0 and the current block is predicted from L1, then the current block can directly reference the LIC parameter of the reference block.

[0060] In one embodiment, if the reference block is predicted from L0 and the current block is predicted from L1, then the self-derived LIC parameters will be used. The current block will not use the LIC parameters of the reference block.

[0061] In one embodiment, if the reference block is predicted from L0 and the current block is predicted from L1, the LIC parameters of the reference block can be referenced by the current block, and they will be further refined using templates of both the reference block and the current block.

[0062] In one embodiment, the above method is used only for motion compensation.

[0063] In one embodiment, the above method is used for list reordering, BCW index refinement, or prediction direction refinement.

[0064] In one embodiment, the above method is used only for motion compensation, and LIC is disabled for list reordering, BCW index refinement, or prediction direction refinement.

[0065] In one embodiment, LIC is disabled when performing list reordering, BCW index refinement, BVD prediction, or prediction direction refinement.

[0066] In one embodiment of the LIC merging mode, an LIC merging list is generated, which consists of the spatially adjacent block LIC parameters, non-immediately adjacent block LIC parameters, temporally adjacent block LIC parameters, and history-based block LIC parameters mentioned above. Additionally, the LIC merging list may also include default LIC parameters.

[0067] For example, the default LIC parameter is generated from some predefined alpha values. Then, the reference template of the reference block and the current template of the current block are used to derive the corresponding beta values.

[0068] For example, the predetermined alpha value is close to 1. As another example, the predetermined alpha value can be negative or positive.

[0069] In one embodiment, if the reference block is encoded and decoded by a multi-model LIC, then multiple LIC models are inserted into the LIC merge list.

[0070] In one embodiment, if a slope-adjusted LIC is used to encode and decode a reference block, the adjusted parameters will be inserted into the LIC merge list.

[0071] In one embodiment, in inter-frame AMVP mode, a CU-level flag is sent to indicate whether LIC merging mode is used. If true, an index is further sent to indicate which LIC model from the LIC merging list is used.

[0072] In one embodiment, in inter-frame merging mode, a CU-level flag is sent to indicate whether LIC merging mode is used. If true, an index is further sent to indicate which LIC model from the LIC merging list is used.

[0073] In one embodiment, LIC is disabled when performing MVP list inference.

[0074] In one embodiment, LIC is disabled when performing a block-level reference image list reordering.

[0075] In one embodiment, LIC is disabled when performing BVD prediction.

[0076] The methods for inheriting LIC parameters or generating LIC candidate lists proposed above can all be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the inter-frame prediction module of the encoder and / or the inter-frame prediction module of the decoder. Alternatively, any of the proposed methods can be implemented as circuitry coupled to the inter-frame prediction module of the encoder and / or the inter-frame prediction module of the decoder to provide the information required by the inter-frame prediction module.

[0077] Figure 5 illustrates a flowchart of an exemplary video encoding / decoding system according to an embodiment of the present invention, wherein the current block inherits an inherited motion vector (MV) from a target block, and the LIC parameters of the target block are inherited by the current block. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 510, input data associated with the current block is received, wherein the input data includes pixel data to be encoded on the encoder side or data associated with the current block to be decoded on the decoder side, wherein the current block inherits an inherited motion vector (MV) from an inherited block. In step 520, local illumination compensation (LIC) prediction candidates with LIC parameters inherited from the inherited block are derived. In step 530, the current block is encoded or decoded using a candidate list including LIC candidates, wherein, when a LIC candidate is selected for use, the predictor of the current block is generated by applying the inherited LIC parameters to a reference block.

[0078] Figure 6 illustrates a flowchart of an exemplary video encoding / decoding system according to an embodiment of the present invention, wherein the merge list consists of LIC parameters for spatially adjacent blocks, non-immediately adjacent blocks, temporally adjacent blocks, and history-based blocks. According to another method, in step 610, input data associated with the current block is received, wherein the input data includes pixel data to be encoded on the encoder side or data associated with the current block to be decoded on the decoder side, wherein the current block is encoded / decoded by inter-frame encoding / decoding. In step 620, a Local Illumination Compensation (LIC) candidate list is generated, wherein the LIC candidate list includes first LIC parameters associated with one or more first LIC models of one or more spatially adjacent blocks, second LIC parameters associated with one or more second LIC models of one or more non-immediately adjacent blocks, third LIC parameters associated with one or more third LIC models of one or more temporally adjacent blocks, fourth LIC parameters associated with one or more fourth LIC models of one or more history-based blocks, or combinations thereof. In step 630, the current block is encoded or decoded using the LIC candidate list.

[0079] The flowchart shown is intended to illustrate examples of video encoding and decoding according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split the steps, or combine the steps to practice the invention without departing from its spirit. In this disclosure, specific syntax and semantics are used to illustrate examples of embodiments of the invention. Those skilled in the art can practice the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0080] The above description is intended to enable those skilled in the art to practice the invention in the context of specific applications and their requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but rather to be given the broadest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details have been set forth to provide a thorough understanding of the invention. However, the invention can be practiced by those skilled in the art.

[0081] As described above, embodiments of the present invention can be implemented through various hardware, software code, or a combination of both. For example, embodiments of the present invention may be program code integrated into one or more circuits in a video compression chip, or integrated into video compression software to perform the processes described herein. Embodiments of the present invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The present invention may also relate to multiple functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the invention, defining the specific methods embodied in the invention by executing machine-readable software code or firmware code. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other means of configuring the code to perform tasks according to the invention, do not depart from the spirit and scope of the invention.

[0082] This invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are to be considered illustrative in all respects, not limiting. Therefore, the scope of the invention should be indicated by the appended claims rather than the foregoing description. All variations within the meaning and equivalence of the claims should be included within its scope.

Claims

1. A video encoding / decoding method, the method comprising: The system receives input data associated with the current block, including pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end, wherein the current block inherits motion vectors from an inherited block; derives local illumination compensation prediction candidates with multiple inherited local illumination compensation parameters, the multiple inherited local illumination compensation parameters being inherited from multiple local illumination compensation parameters of the inherited block; and encodes or decodes the current block using a candidate list including the local illumination compensation, wherein when the local illumination compensation is selected for the current block, a predictor for the current block is generated by applying the multiple inherited local illumination compensation parameters to a reference block.

2. The video encoding / decoding method as described in claim 1, wherein the plurality of inherited local illumination compensation parameters are used to guide the derivation of a plurality of current local illumination compensation parameters for the current block.

3. The video encoding / decoding method as described in claim 1, wherein the template matching cost is used to determine whether the current block uses the plurality of inherited local illumination compensation parameters or the plurality of derived local illumination compensation parameters, wherein, The multiple exported local lighting compensation parameters are exported using one or more templates from the current block and the reference block.

4. The video encoding / decoding method as described in claim 3, wherein, Codec unit-level flags are used to indicate whether the multiple inherited local illumination compensation parameters or the multiple derived local illumination compensation parameters are used.

5. A video encoding / decoding apparatus, the apparatus comprising one or more electronic devices or processors, configured to: receive input data associated with a current block, wherein, The input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end, wherein the current block inherits motion vectors from inherited blocks; derives local illumination compensation prediction candidates with multiple inherited local illumination compensation parameters, the multiple inherited local illumination compensation parameters being inherited from multiple local illumination compensation parameters of the inherited block; and encodes or decodes the current block using a candidate list including the local illumination compensation, wherein, when the local illumination compensation is selected for the current block, a predictor for the current block is generated by applying the multiple inherited local illumination compensation parameters to a reference block.

6. A video encoding / decoding method, the method comprising: The system receives input data associated with the current block, wherein the input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end, wherein the current block is encoded and decoded by inter-frame encoding and decoding; generates a local illumination compensation candidate list, wherein the local illumination compensation candidate list includes multiple first local illumination compensation parameters associated with one or more first local illumination compensation models of one or more spatially adjacent blocks, multiple second local illumination compensation parameters associated with one or more second local illumination compensation models of one or more non-immediately adjacent blocks, multiple third local illumination compensation parameters associated with one or more third local illumination compensation models of one or more temporally adjacent blocks, multiple fourth local illumination compensation parameters associated with one or more fourth local illumination compensation models of one or more history-based blocks, or a combination thereof; and encodes or decodes the current block using the local illumination compensation candidate list.

7. The video encoding / decoding method as described in claim 6, wherein, A target local lighting compensation candidate is selected from the list of local lighting compensation candidates for the current block, and a predictor for the current block is generated by applying multiple local lighting compensation parameters associated with the target local lighting compensation candidate to a reference block.

8. The video encoding / decoding method as described in claim 6, wherein, The list of candidates for local lighting compensation also includes one or more predefined local lighting compensation parameters.

9. The video encoding / decoding method as described in claim 6, wherein, An index is sent to indicate the target local lighting compensation candidate selected from the list of local lighting compensation candidates.

10. The video encoding / decoding method as described in claim 6, wherein, If the reference block uses a multi-model local illumination compensation codec, then multiple local illumination compensation models associated with the reference block are inserted into the local illumination compensation candidate list.

11. A video encoding / decoding apparatus, the apparatus comprising one or more electronic devices or processors, configured to: receive input data associated with a current block, wherein, The input data includes pixel data to be encoded at the encoder end or data associated with the current block to be decoded at the decoder end, wherein the current block is encoded and decoded by inter-frame encoding and decoding; generating a local illumination compensation candidate list, wherein the local illumination compensation candidate list includes multiple first local illumination compensation parameters associated with one or more first local illumination compensation models of one or more spatially adjacent blocks, multiple second local illumination compensation parameters associated with one or more second local illumination compensation models of one or more non-immediately adjacent blocks, multiple third local illumination compensation parameters associated with one or more third local illumination compensation models of one or more temporally adjacent blocks, multiple fourth local illumination compensation parameters associated with one or more fourth local illumination compensation models of one or more history-based blocks, or a combination thereof; and using the local illumination compensation candidate list to encode or decode the current block.

Images