Method and apparatus for picture encoding and decoding
By calculating and storing local illuminance compensation parameters in video encoding, reducing access to additional samples, combined with independent decoding of LIC groups, the delay and storage problems introduced by local illuminance compensation in the prior art are solved, and encoding efficiency and speed are improved.
Patent Information
- Application Number
- CN202411283764.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-09-11
- Filing Date
- 2019-09-19
- Publication Date
- 2024-12-03
AI Technical Summary
When existing video encoding techniques utilize local illumination compensation, there are problems such as accessing additional samples resulting in delay and increasing memory bandwidth, and the latter method increases memory consumption and processing volume.
By calculating and storing LIC parameters after each CU is reconstructed for subsequent CU reuse, access to additional samples is reduced, and LIC groups are introduced during inter-decoding to independently decode different CUs, limiting the inheritance constraints of LIC parameters, and reducing the Pipeline dependency.
It reduces the delay and storage consumption in the video encoding and decoding process, and improves encoding efficiency and processing speed.
Smart Images

Figure HDA0005042486550000011 
Figure HDA0005042486550000012 
Figure HDA0005042486550000013
Abstract
Description
[0001] This divisional application is a divisional application with a filing date of September 19, 2019, application number 201980061694.0, and invention name “Method and device for picture encoding and decoding”. Technical Field
[0002] At least one of the present embodiments generally relates to a method and apparatus for picture encoding and decoding, and more particularly, to a method and apparatus for picture encoding and decoding with local illumination compensation. Background Art
[0003] In order to achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. Generally, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image block and the predicted image block (usually expressed as prediction error or prediction residual) is transformed, quantized and entropy encoded. During encoding, it is usually possible to partition / split the original image block into sub-blocks using quadtree partitioning. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transformation, quantization and entropy coding. Summary of the invention
[0004] One or more of the present embodiments provide a method for encoding / decoding a picture using illumination compensation.
[0005] According to a general aspect of at least one embodiment, a decoding method is provided, comprising:
[0006] - determining at least one illumination compensation parameter for the current block;
[0007] - reconstructing the current block using the determined illumination compensation parameters;
[0008] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0009] According to another general aspect of at least one embodiment, a decoding device is provided, comprising one or more processors configured to implement the following:
[0010] - determining at least one illumination compensation parameter for the current block;
[0011] - reconstructing the current block using the determined illumination compensation parameters;
[0012] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0013] According to another general aspect of at least one embodiment, a coding method is provided, comprising:
[0014] - determining at least one illumination compensation parameter for the current block;
[0015] - reconstructing the current block using the determined illumination compensation parameters;
[0016] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0017] According to another general aspect of at least one embodiment, a coding apparatus is provided, comprising one or more processors configured to implement the following:
[0018] - determining at least one illumination compensation parameter for the current block;
[0019] - encoding the current block using the determined illumination compensation parameters;
[0020] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0021] According to another general aspect of at least one embodiment, a bitstream is formatted to include a signal generated according to the above-described encoding method.
[0022] One or more of the present embodiments also provide a computer-readable storage medium on which instructions for encoding or decoding video data according to at least a portion of any of the above methods are stored. One or more embodiments also provide a computer-readable storage medium on which a bit stream generated according to the above encoding method is stored. One or more embodiments also provide methods and devices for sending or receiving a bit stream generated according to the above encoding method. One or more embodiments also provide a computer program product, including instructions for executing at least a portion of any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1Describes the coding tree unit (CTU) that is partitioned into smaller units;
[0024] Figure 2 Illustrate the principle of local illumination compensation (LIC) parameter derivation for a current block from neighboring reconstructed samples and corresponding co-located reference samples in the case of "JEM-LIC";
[0025] Figure 3 An example of LIC parameter calculation process in the case of "VVC-LIC" is shown;
[0026] Figure 4 LIC parameter derivation from the LIC parameters stored on the upper side in case of "VVC-LIC" is illustrated on the left, and vertical (dark grey) and horizontal (light grey) LIC parameters stored at frame level for 4x4 resolution are illustrated on the right;
[0027] Figure 5 depicts LIC parameters for a CTU of size 128x128 and 32x2 storage in the case of "VVC-LIC";
[0028] Figure 6 Describe the current coding unit (CU) and its left and upper neighboring CUs;
[0029] Figure 7 depicts CTUs partitioned into CUs, and illustrates the principle of unavailability of LIC parameters for some of them in case of "VVC-LIC" or unavailability of neighboring reconstructed samples in case of "JEM-LIC";
[0030] Figure 8 A flowchart illustrating a method for decoding a CU in the case of "VVC-LIC" according to one embodiment;
[0031] Fig. 9 Depicts a CTU partitioned into CUs, and illustrates the principles of LIC parameter access constraints according to various embodiments in the case of “VVC-LIC” or “JEM-LIC”;
[0032] Fig.10 and Fig.11 Depicting a CTU partitioned into CUs and illustrating the principle of LIC-groups according to various embodiments;
[0033] Fig. 12A and 12B A flowchart depicting a decoding method according to various embodiments;
[0034] Fig.13 Describes the current CU and its neighboring CUs, and illustrates the principle of LIC parameter derivation according to one embodiment;
[0035] Fig.14 A block diagram illustrating a video encoder according to an embodiment;
[0036] Fig.15 A block diagram illustrating a video decoder according to an embodiment; and
[0037] Fig.16 Block diagram illustrating an example of a system in which various aspects and embodiments are implemented. DETAILED DESCRIPTION
[0038] In HEVC coding, a picture is partitioned into square-shaped CTUs of configurable size (typically 64x64, 128x128, or 256x256). A CTU is the root of a quadtree that is partitioned into four equally sized square coding units (CUs), i.e. half the size of the parent block in width and height, such as Figure 1 As shown. A quadtree is a tree in which a parent node can be split into four child nodes, each of which can become a parent node that is further split into four child nodes. In HEVC, a coding block (CB) is partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree that is partitioned into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a prediction unit (PU) and a tree-structured transform unit (TU) set, where the PU includes prediction information for all color components and the TU includes a residual coding syntax structure for each color component. The sizes of the CB, PB, and TB of the luminance component apply to the corresponding CU, PU, and TU.
[0039] In more recent coding systems, a CTU is the root of a coding tree partitioned into coding units (CUs). A coding tree is a tree in which a parent node (usually corresponding to a CU) can be partitioned into child nodes (e.g., partitioned into 2, 3, or 4 child nodes), each of which can become a parent node for further partitioning into child nodes. In addition to the quadtree partitioning mode, new partitioning modes (binary tree symmetric partitioning mode, binary tree asymmetric partitioning mode, and ternary tree partitioning mode) are defined, which increase the total number of possible partitioning modes. The coding tree has a unique root node, such as a CTU. The leaves of the coding tree are the terminal nodes of the tree. Each node of the coding tree represents a CU, which can be further partitioned into smaller CUs, also called sub-CUs or more generally sub-blocks. Once the partitioning of a CTU into a CU is determined, the CU corresponding to the leaf of the coding tree is encoded. The partitioning of a CTU into a CU and the coding parameters for encoding each CU (corresponding to the leaf of the coding tree) can be determined at the encoder side by a rate-distortion optimization process. The CB is not partitioned into PBs and TBs, that is, the CU consists of a single PU and a single TU.
[0040] A picture may be partitioned into slices or tiles. Each slice or tile is a group of CUs and / or may contain a portion of a CU.
[0041] In this application, the term "block" or "picture block" may be used to refer to any of CTU, CU, PU, TU, CB, PB and TB. In addition, the term "block" or "picture block" may be used to refer to macroblocks, partitions and sub-blocks specified in H.264 / AVC or other video coding standards, and more generally to sample arrays of various sizes.
[0042] In this application, the terms "reconstructed" and "decoded" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture", "frame", "slice" or "tile" are used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, and "decoded" is used on the decoder side.
[0043] For a coded CU, a prediction block is built from neighboring reconstructed samples (intra-frame prediction) or from a previously reconstructed picture (inter-frame prediction) stored in the decoded picture buffer (DPB). Next, the residual samples calculated as the difference between the original samples and the PU samples are transformed and quantized. In inter-frame prediction, motion compensated temporal prediction is used to exploit the redundancy existing between consecutive pictures of the video. To this end, a motion vector and a reference index 0 (refIdx0) indicating which reference picture of LIST_0 to use are associated with the PU. Possibly, several PUs are combined together to form one final PU (e.g., bidirectional prediction, generalized bidirectional prediction, or multi-hypothesis prediction).
[0044] In the Joint Exploration Model (JEM) developed by the JVET (Joint Video Exploration Team) group, other temporal prediction tools are introduced that make use of associated parameters determined at the decoder side, including Local Illumination Compensation (LIC). Basically, the purpose of LIC is to compensate for the illumination variations that may occur between the prediction block and its reference blocks adopted by motion compensated temporal prediction.
[0045] The use of LIC is typically signaled at the CU level through a flag (LIC flag) associated with each coding unit (CU) of inter-frame coding. According to this tool, the decoder calculates some prediction parameters ( Figure 2 ). In JEM, the use of LIC for a given block depends on a flag associated with that block, called the LIC flag. Throughout this document, this technique is referred to as "JEM-LIC".
[0046] The LIC parameters (weight = a, bias = b) are determined by minimizing the local distortion defined as follows:
[0047] dist=∑ r∈Vcur,s∈Vref (rec_cur(r)-a.rec_ref(s)-b) 2 (Equation 1)
[0048] Where: rec_cur(r) is the neighboring reconstructed sample
[0049] rec_ref is the reference sample
[0050] Vcur is the reconstructed sample adjacent to the current CU.
[0051] Vref is the reconstructed sample adjacent to the reference sample used to establish the PU.
[0052] Once the encoder or decoder obtains the LIC parameters for the current CU, the prediction pred(current_block) of the current CU is determined as follows (unidirectional prediction case):
[0053] pred(current_block)=a×ref_block+b (Equation 1)
[0054] where current_block is the current block to be predicted, pred(current_block) is the prediction of the current block, and ref_block is the reference block used for temporal prediction of the current block.
[0055] A disadvantage of this basic approach is that it requires the encoder / decoder to access reconstructed samples of the already reconstructed top and left neighboring blocks. This aspect is undesirable, mainly because it introduces some latency in the motion compensation process, increases the memory bandwidth (access to additional reference and reconstructed neighboring samples, such as Figure 2 depiction), and potentially break some existing decoder pipeline architectures.
[0056] To address these issues, the LIC parameters can be calculated in a way that avoids accessing these additional samples. To this end, the LIC parameters are calculated with the lower and right edges of the current CU after each CU reconstruction and stored so that they can be reused by subsequent CUs to perform LIC compensation ( Figure 3 ). Throughout this document, this technology is referred to as “VVC-LIC”.
[0057] However, a major disadvantage of this latter approach is that it stores LIC parameters for the entire slice or picture after each CU reconstruction, which results in increased memory consumption ( Figure 4 ) and increase the amount of processing during the decoding process. Figure 4On the left side, the LIC parameters of the current CU are determined from the LIC parameters stored on the upper side. Figure 4 On the right side of the figure, vertical (dark grey) and horizontal (light grey) LIC parameters are stored at the frame level with a 4x4 resolution.
[0058] In the case of "VVC-LIC", the memory requirements can be reduced by storing the LIC parameters in a common LIC buffer used for decoding all CTUs. The LIC buffer consists of a single column of size equal to the CTU height and a single row of size equal to the CTU width, such as Figure 5 At the end of each CU processing, LIC parameters are calculated for each color component and each reference list on the bottom and right boundaries. They are then stored in a common LIC buffer so that they can be accessed by subsequent CUs (to the right or bottom of the current CU). During the CU prediction stage, the LIC parameters can be accessed in the common LIC buffer. The row buffer contains the LIC parameters calculated using the neighboring upper CU, and the column buffer contains the LIC parameters calculated using the neighboring left CU, as shown in Figure 2. Figure 5 Depicted in.
[0059] If the minimum CTU size is 4x4, then for 3 color components ( Figure 5 ) and 2 reference lists, all LIC parameters are stored in one LIC buffer of size (CTU-width / 4) for all top LIC parameters and in one LIC buffer of size (CTU-height / 4) for left LIC parameters.
[0060] Determine the LIC parameters used in PU reconstruction
[0061] In inter-frame unidirectional mode, a PU is established by motion compensating a block of samples in a previously reconstructed reference picture stored in the decoded picture buffer (DPB). The DPB includes two lists of reference pictures (L0 and L1), and each reference picture is associated with an index (refIdx). In bidirectional prediction, two prediction blocks are established from two reference pictures of L0 and L1 using two reference indices (refIdx0, refIdx1). When LIC is enabled for the current CU, the LIC parameters are copied from the LIC parameters calculated by a top (or a left) reconstructed CU (in the case of "VVC-LIC", which are stored in a common row (or column) LIC buffer). In the case of "JEM-LIC", the LIC parameters are determined from the reconstructed top and left samples.
[0062] In the case of "VVC-LIC", since both top and left LIC parameters are available but may be different, pruning is performed to obtain a single LIC model. Intra-coded CU neighbors are discarded in the pruning process. Next, neighbors with trivial models [weight = 1 and offset = 0] are discarded in the pruning process. If no model remains, the trivial model is used. If (multiple) models still remain, the neighbors with the same reference index as the current CU are given higher priority. If multiple neighbors have the same reference index, the upper right (or lower left) neighbor is given higher priority than the upper left (or upper and lower) neighbors ( Figure 6 ).
[0063] In a variation, a neighbor whose MV is closest to the MV of the current CU is given higher priority. Figure 6 Scan the neighbors on the upper and left sides from the upper right to the upper left and from the lower left to the upper left to find a neighbor with the same reference index as the current PU (if any).
[0064] The same pruning principle can be applied to “JEM-LIC.” In that case, the reconstructed samples used to calculate the LIC parameters are the samples corresponding to the CU given a higher priority.
[0065] In the case of "VVC-LIC", before reconstructing a CTU, the LIC buffer storing the top LIC parameters is reset to the default value so that the LIC is disabled for the top CU of the current CTU. When starting to reconstruct a row of CTUs, the LIC buffer storing the left LIC parameters is reset to the default value so that the LIC is disabled only when the CTU is the first CTU in a row ( Figure 7 ).exist Figure 7 In the above example, the upper LIC parameters are not available for the CU at the top of the CTU. In addition, the LIC parameters are not calculated for the CU at the bottom of the CTU because access to the upper LIC parameters is not allowed for the lower CTU ( Figure 7 ).
[0066] The "VVC-LIC" and "JEM-LIC" methods introduce dependencies when decoding inter CUs. In fact, with LIC-flag = true, the current CU needs to wait for the left CU and the upper CU to be decoded before its own decoding starts. In addition, the "VVC-LIC" method increases the amount of calculation by calculating the LIC parameters after each CU reconstruction (bottom and right edges), even if the right or bottom CU does not use LIC. Finally, the cost of encoding the LIC-flag is also high.
[0067] At least one embodiment proposes to reduce the pipeline dependency introduced by LIC when decoding a CU as inter-frame by adding some constraints in the LIC process.
[0068] At least one embodiment proposes increasing signaling for improving coding efficiency and speeding up the decoding process.
[0069] Example 1
[0070] Figure 8 A flowchart representing a method for decoding a current CU in the case of "VVC-LIC" according to a specific and non-limiting embodiment. First, the syntax elements associated with the current CU are parsed from the bitstream (310) and the CU parameters are decoded (320). This includes decoding CU parameters explicitly encoded in the bitstream (e.g.: IC-flag) and other CU parameters inferred from other syntax elements (e.g., in merge mode, where the IC-flag is inferred from the IC-flag of a neighboring CU inherited by the current CU). Next, a predicted PU (motion compensation) is established (321). If IC-flag = true, the LIC parameters are determined from the LIC buffer (322) in the case of "VVC-LIC" and illumination compensation is applied on the prediction (PU), that is, the current CU uses the determined illumination compensation parameters for its reconstruction. Next, the CU is reconstructed by adding the residual to the predicted PU (325).
[0071] The following steps (330-390) deal with the calculation and storage of LIC parameters that may be used by subsequent CUs. If the current CU is located at the bottom of the current CTU (330), the horizontal LIC parameters are set to default values (340), otherwise, the horizontal LIC parameters are calculated using the reconstructed samples of the current CU located at the bottom of the CU. They are then stored in a horizontal LIC buffer (360). If the current CU is located to the right of the current CTU (370), the vertical LIC parameters are set to default values (340), otherwise, the vertical LIC parameters are calculated using the reconstructed samples of the current CU located to the right of the current CU (380), and they are stored in a vertical LIC buffer (390).
[0072] In addition to steps 330-390 specific to VVC-LIC, this embodiment is also applicable to "JEM-LIC". In the case of VVC-LIC, in step 322, LIC parameters for the current CU are derived from the LIC buffer. In the case of JEM-LIC, in step 322, LIC parameters of the current CU are derived from neighboring reconstructed samples.
[0073] Example 2
[0074] At least one embodiment proposes additional conditions for limiting the video pipeline dependencies introduced when decoding of a CU coded in inter-mode in LIC mode is enabled (LIC-flag=true).
[0075] In order to make the decoding of a CU coded in inter mode in one CTU independent of the decoding of a CU in another CTU, the inheritance of the LIC parameters calculated from another CU is constrained to be within the same CTU as the current CU. In other words, a CU can inherit LIC parameters from another CU only if the CU is in the same CTU as the current CU.
[0076] In practice, in the case of "VVC-LIC", this can be accomplished by filling the LIC parameters of the vertical buffer with default LIC values (340) (weight = 1, offset = 0) at the start of decoding the current CTU (see Figure 5 This can also be achieved by disabling the LIC for the top left CU in the CTU. In the case of "JEM-LIC", this can be achieved by disabling the use of reconstructed samples outside the current CTU in Equation 1. Fig. 9 Depicts LIC parameter inheritance ("VVC-LIC" case) or reconstruction sample access ("JEM-LIC" case) constraints corresponding to various embodiments. Fig. 9 In the upper right corner, LIC parameter access or reconstructed sample access is constrained to the current CTU. Fig. 9 , except for the first CU in a picture row, access to LIC parameters or reconstructed samples on the left is allowed, but access to the top LIC parameters or top reconstructed samples is constrained to Fig. 9 The current CTU in the upper left corner.
[0077] Example 3
[0078] In at least one embodiment, a LIC group is defined where LIC parameters can only be inherited or calculated from CUs in the same LIC group. By doing so, the decoding of some CUs encoded in inter-frame mode in one LIC group is independent of the decoding of other CUs in another LIC group. This allows different CUs to be decoded in inter-frame mode by separate threads. When the LIC group is a CTU, embodiment 3 is the same as embodiment 2 ( Fig. 9 ). The LIC group size may be determined by the VPU (Video Processing Unit) or VDPU (Video Data Processing Unit) or HU (Hardware Unit), which is the maximum sample size that the hardware can handle together. In a variant, the LIC group or VPU is encoded in the bitstream in a sequence header (e.g., SPS, SPS stands for Sequence Parameter Set) or a picture / slice header (e.g., PPS, PPS stands for Picture Parameter Set).
[0079] As an example, when the size of CTU is 128x128, the size of "LIC-groups" can be 64x64 or 128x128 or 256x128 or 256x256. Therefore, a LIC group can be a CTU, can be smaller than a CTU, or can be larger than a CTU.
[0080] In a variant, for the top left CU in the LIC group, the LIC-flag is not encoded and is inferred to be false. In other words, if the current block is not located in the top left in the same local illumination compensation group, the flag is decoded to indicate whether the determined illumination compensation parameters are used to reconstruct the current block.
[0081] In another variant, for a CU in a LIC group whose all top neighbors or left neighbors are coded in intra mode, the LIC-flag is not coded and is inferred to be false. In other words, in the case where at least one block located above or to the left of the current block is coded in inter mode, the flag is decoded to indicate whether the determined illumination compensation parameters are used to reconstruct the current block.
[0082] Example 4
[0083] Fig. 12A A flowchart depicting a decoding method according to one embodiment is shown. More precisely, Fig. 12A Decoding of LIC-group-flag-enabled for CUs belonging to a LIC group as a group of CUs is illustrated. In this example, the LIC group corresponds to one CTU. Another LIC group is defined. A flag (LIC-group-flag-enabled) (315) is decoded to indicate whether at least one CU in the LIC group uses LIC for computing prediction. This can be used with "JEM-LIC" or "VVC-LIC".
[0084] Next, the prediction PU (motion compensation) is established (321). If IC-flag=true and LIC-group-flag-enalbed, the LIC parameters are determined from the LIC buffer (322) and illumination compensation is applied to the prediction (PU). Next, the CU is reconstructed by adding the residual to the prediction PU (325).
[0085] Defining such a flag improves coding efficiency because in the case where LIC-group-flag-enabled is equal to false, then:
[0086] a) The LIC-flag is not decoded for the CUs in the LIC group, but is inferred to be false (320).
[0087] b) It speeds up the decoder because for the CU in LIC-group-2, LIC is not applied on the PU (322).
[0088] c) In the case of “VVC-LIC”, it also speeds up the decoder because the LIC parameters are not calculated (335) for CUs within a LIC group unless they are to the right or bottom of the LIC group ( Fig.10 ).
[0089] This embodiment is independent of other embodiments 1-3, but can also be combined with embodiments 1-3. In the first case, "LIC-group-flag-enabled" is encoded and applied: for CUs with IC-flag = true, as in embodiments 1-3, there is no constraint on the derivation of LIC parameters. In the second case, "LIC-group-flag-enabled" is encoded and applied: for CUs with IC-flag = true, the constraints on the derivation of LIC parameters are the same as in embodiments 1-3.
[0090] exist Fig.10 Above, in the case of "VVC-LIC", where LIC-group-flag-enabled is equal to false, LIC parameters are not calculated for CUs within the LIC group (white CUs), unless they are on the right or bottom of the LIC group (gray CUs). Here, a group corresponds to one CTU.
[0091] In a variation, in the case of "VVC-LIC", a flag (LIC-group-flag-enabled) indicates whether LIC parameters should be calculated and stored after reconstructing the CU of the LIC group.
[0092] Example 5
[0093] exist Fig. 12B In the variation of embodiment 4 depicted in , the semantics of LIC-group-flag-enabled is changed. It indicates whether at least one CU has a LIC-flag value, which is true when the LIC-flag (318) is explicitly encoded in the LIC group (not inherited by "merge mode"). In this case, the CU inherits its LIC-flag value from other CUs ("merge mode": 317merge-flag=true) and for which the LIC-flag is not encoded is not considered by LIC-group-flag-enabled. For example, it is possible to have LIC-group-flag-enabled=false in a LIC group, and a CU can inherit LIC-flag=true from a neighboring CU ( Fig.11LIC-group-flag-enabled = true indicates that there is at least one CU in the LIC group for which the value of LIC-flag is explicitly encoded (not inherited by merging) and LIC-flag is equal to true ( Fig.11 LIC-group-flag-enabled = false indicates that all CUs in the LIC group whose LIC-flag value is explicitly coded (not inherited by merging) have LIC-flag equal to false, but when the CU inherits the value of LIC-flag (merge mode) ( Fig.11 ), it is possible to make the CU have LIC-flag equal to true.
[0094] Example 6
[0095] In a variant, "LIC-group-flag-enabled" is encoded with the first CU in a CU group coded in inter mode. In this way, if all CUs are intra coded, the flag "LIC-group-flag-enabled" is not encoded and is inferred to be false.
[0096] In another variant, "LIC-group-flag-enabled" is encoded with the first CU in the group that is coded in inter mode rather than merge mode. In this way, if all CUs are coded in intra or inter merge mode, the flag "LIC-group-flag-enabled" is not encoded and is inferred to be false.
[0097] In a variant, the value of "LIC-group-flag-enabled" does not modify the value of "LIC-flag" of a CU in inter-merge mode. A CU in inter-merge mode infers the value of LIC-flag from the LIC-flag of the reconstructed CU from which it inherits other coding parameters.
[0098] In a variant, if the value of "LIC-group-flag-enabled" is false, the value of "LIC-flag" is inferred to be false for all CUs (in intra and intra merge modes). In this case, the value of LIC-flag is inferred to be false for CUs in intra merge mode.
[0099] Example 7
[0100] In a variation of Embodiment 4, a minimum CU size "minCUsize" is defined, and LIC-group-flag-enabled indicates whether at least one CU in the LIC group whose size is smaller than "minCUsize" uses LIC for calculating prediction. Thus, if LIC-group-flag-enabled is equal to false, LIC-flag is not encoded for CUs whose size is smaller than "minCUsize".
[0101] In a variant, LIC-group-flag-enabled indicates whether at least one CU in the LIC group whose size is greater than "minCUsize" uses LIC for computing prediction. Thus, if LIC-group-flag-enabled is equal to false, LIC-flag is not encoded for CUs with a size greater than "minCUsize".
[0102] Advantageously, the value "minCUsize" is encoded in the bitstream, for example in the SPS, PPS or slice header. Examples of "minCUsize" may typically be 4x4 or 8x8.
[0103] Example 8
[0104] Embodiments 1, 2, 3, 4, 5, 6, and 7 can also be applied to and combined with the cases of "JEM-LIC" and "VVC-LIC" techniques, except that LIC parameters are not calculated for "JEM-LIC" after CU reconstruction. The constraints on access to LIC parameters in the top horizontal LIC buffer or the left vertical LIC buffer of "VVC-LIC" correspond to the constraints on access to reconstructed samples located on the upper side or the left side of the current CU for "JEM-LIC", respectively.
[0105] Example 9
[0106] In the case of "VVC-LIC" or "JEM-LIC", when receiving a signal from a neighboring CU ( Figure 6 ), a list of LIC parameters (or reconstructed samples) associated with the CU is established using the same reference index, and in the case of "VVC-LIC", the LIC parameters for the current CU are the average of these LIC parameters. In the "VVC-LIC" variant, the average is a weighted average, where the weight is proportional to the adjacent CU width (CU on the upper side) or the adjacent CU height (CU on the left). In the variant, the weight is proportional to the length of the common boundary of the current CU, such as Fig.13 Depict, for weights w1 and w7.
[0107] In a variant, the LIC parameters (or reconstructed samples) of the neighbor whose MV is closest to the current MV of the current CU are used for the current CU (to calculate the LIC parameters, respectively).
[0108] Various embodiments enable decoding of a CU (e.g., a CTU) coded in inter mode in one LIC group to be independent of decoding of a CU in another LIC group. The inheritance of LIC parameters calculated from other CUs ("VVC-LIC") or the reconstructed sample sets used to derive LIC parameters ("JEM-LIC") is constrained to be within the same LIC group as the current CU. In other words, a CU can inherit LIC parameters or use reconstructed samples from another CU only if the CU is in the same LIC group as the current CU.
[0109] This article describes a number of aspects including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described, and at least to illustrate the various features, are often described in a manner that may sound restrictive. However, this is for clarity of description and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with aspects described in previous applications.
[0110] The aspects described and contemplated herein may be implemented in many different forms. Fig.14 , Fig.15 and Fig.16 Some embodiments are provided, but other embodiments are contemplated, and Fig.14 , 15 The discussion of 16 does not limit the breadth of implementations. At least one of these aspects generally relates to video encoding and decoding, and at least another aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the methods described, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the methods described.
[0111] Various methods are described above, and each method includes one or more steps or actions for implementing the described method. Unless a specific step or action sequence is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.
[0112] Various methods and other aspects described in this application can be used to modify modules, e.g. Fig.14 and Fig.15The motion compensation and motion estimation modules (170, 175, 275) of the video encoder 100 and decoder 200 shown. In some embodiments, the entropy coding and / or decoding modules (145, 230) are also modified. Moreover, the present aspects are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application can be used alone or in combination.
[0113] Various values are used in this application, for example, the size of a “LIC group” and “minCUsize.” The specific values are for example purposes, and the described aspects are not limited to these specific values.
[0114] Fig.14 An exemplary encoder 100 is illustrated. Variations of the encoder 100 are contemplated, but for clarity, the encoder 100 is described below without describing all contemplated variations.
[0115] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input image components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.
[0116] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (102) and processed, for example in units of CUs. Each CU is encoded, for example, using intra or inter mode. When the unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use for the coding unit, and indicates the intra / inter decision, for example, by a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.
[0117] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients are entropy encoded (145) along with motion vectors and other syntax elements to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process.
[0118] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. The prediction can be established as a combination of several predictions (e.g., bidirectional prediction, generalized bidirectional prediction, or multi-hypothesis prediction). An in-loop filter (165) is applied to the reconstructed picture to, for example, perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).
[0119] Fig.15 200. In the decoder 200, the bitstream is decoded by the decoder elements, as described below. The video decoder 200 generally performs the same Fig.14 The decoding path is the inverse of the encoding path described in . The encoder 100 also typically performs video decoding as part of encoding the video data.
[0120] Specifically, the input to the decoder includes a video bitstream that may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other encoding information. Picture partition information indicates how the picture is partitioned. Thus, the decoder can divide (235) the picture according to the decoded picture partition information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored at a reference picture buffer (280).
[0121] The decoded picture may further undergo post-decoding processing (285), such as inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping to perform the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.
[0122] Fig.16A block diagram illustrating an example of a system in which various aspects and embodiments are implemented. System 1000 may be embodied as a device including various components described below, and is configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances and servers. The elements of system 1000 are embodied in a single integrated circuit, multiple ICs and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or by dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described in this document.
[0123] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 1010 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 1000 may also include at least one memory 1020 (e.g., a volatile memory device, a non-volatile memory device). The system 1000 may include a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, magnetic disk drive, and / or optical disk drive. As non-limiting examples, the storage device 1040 may include an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network accessible storage device.
[0124] The system 1000 includes an encoder / decoder module 1030, which is configured, for example, to process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a (multiple) module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, as known to those skilled in the art, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000, or may be incorporated into the processor 1010 as a combination of hardware and software.
[0125] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to an exemplary embodiment, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during the execution of the processes described herein. Such stored items include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic.
[0126] In some embodiments, memory internal to the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or storage device 1040, such as dynamic volatile memory and / or non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store, for example, an operating system for the television. In at least one embodiment, fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by JVET (Joint Video Experts Group)).
[0127] Input to the elements of system 1000 may be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion for receiving an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal (or a collection of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Fig.10 Other examples not shown include composite video.
[0128] In various embodiments, the input device of block 1130 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) again band-limiting the signal to a narrower band of frequencies to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet. The RF portion of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (e.g., cable) medium, and by filtering, down-convert and filter to the desired frequency band again to perform frequency selection. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF part includes an antenna.
[0129] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices across the USB and / or HDMI connections. It is to be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within the processor 1010 as desired. Similarly, various aspects of USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within the processor 1010 as desired. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 1010 and the encoder / decoder 1030 operating in conjunction with memory and storage elements to process the data streams for presentation on an output device.
[0130] The various elements of system 1000 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data may be transferred between them using a suitable connection arrangement, such as an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and printed circuit boards.
[0131] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.
[0132] In various embodiments, a wireless network Wi-Fi network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers) is used to transmit data streams to or otherwise provide to the system 1000. The Wi-Fi signals of these embodiments are received on a communication channel 1060 and a communication interface 1050 adapted to Wi-Fi communication. The communication channel 1060 of these embodiments is usually connected to an access point or router, which provides access to an external network including the Internet to allow streaming applications and other air communications. Other embodiments use a set-top box to provide streaming data to the system 1000, which is connected to the HDMI of the input block 1130 to pass data. Other embodiments also use the RF connection of the input block 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming mode. In addition, various embodiments use a wireless network other than Wi-Fi, such as a cellular network or a Bluetooth network.
[0133] The system 1000 can provide output signals to various output devices including a display 1100, a speaker 1110, and other peripherals 1120. The display 1100 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 1100 can also be integrated with other components (e.g., as in a smart phone), or standalone (e.g., an external monitor for a laptop computer). Other peripherals 1120 include one or more of a standalone digital video disc (or digital versatile disc) DVR (for both), a disk player, a stereo system, and a lighting system in various examples of embodiments. Various embodiments use one or more peripherals 1120 that provide functions based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.
[0134] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be communicatively coupled to the system 1000 via dedicated connections through various interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and speaker 1110 can be integrated with other components of the system 1000 into a single unit in an electronic device (such as, for example, a television). In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.
[0135] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection (including, for example, an HDMI port, a USB port, or a COMP output).
[0136] The embodiments may be implemented by computer software implemented by the processor 1010 or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. As a non-limiting example, the memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed memory, and removable memory. As a non-limiting example, the processor 1010 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.
[0137] Various implementations involve decoding. "Decoding" as used in this application may encompass, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processing includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processing also or alternatively includes processing performed by a decoder of the various implementations described in this application, such as motion compensation.
[0138] As another example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or broadly to a more extensive decoding process will be clear based on the context of the particular description and is believed to be well understood by those skilled in the art.
[0139] Various implementations involve encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application can encompass, for example, all or part of the processing performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processing includes one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processing also or alternatively includes processing performed by an encoder of the various implementations described in this application, such as motion compensation.
[0140] As another example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of entropy encoding and differential encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or broadly to a more extensive encoding process will be clear based on the context of the particular description and is believed to be well understood by those skilled in the art.
[0141] Note that the syntax elements (e.g., LIC-Flag) used herein are descriptive terms. Therefore, they do not exclude the use of other syntax element names.
[0142] When the figures are presented as flow charts, it should be understood that they also provide block diagrams of corresponding devices. Similarly, when the figures are presented as block diagrams, it should be understood that they also provide flow charts of corresponding methods / processes.
[0143] Various embodiments relate to rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is often considered, often given constraints on computational complexity. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are a variety of different approaches to solving the rate-distortion optimization problem. For example, these approaches can be based on extensive testing of all coding options including all considered modes or coding parameter values, as well as a complete evaluation of their coding costs and associated distortions of the reconstructed signal after encoding and decoding. Faster approaches can also be used to save coding complexity, particularly by calculating approximate distortions based on predictions or prediction residual signals rather than reconstructions. A mixture of these two approaches can also be used, such as by using approximate distortions for only some possible coding options, and full distortions for other coding options. Other approaches only evaluate a subset of possible coding options. More generally, many approaches employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both coding costs and associated distortions.
[0144] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices, such as computers, cellular phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate information communication between end users.
[0145] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places throughout this document are not necessarily all referring to the same embodiment.
[0146] Additionally, this application may refer to "determining" various information. Determining information may include, for example, estimating information, calculating information, predicting information, or retrieving information from a memory. "Determining" and "deriving" may be used interchangeably.
[0147] Additionally, this application may refer to "accessing" various information. Accessing information may include, for example, receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.
[0148] Additionally, the present application may refer to "receiving" various information. As with "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves, in one way or another, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information during an operation.
[0149] It is to be recognized that the use of " / ", "and / or", and "at least one of" below, for example, in the case of "A / B", "A and / or B", and "at least one of A and B", is intended to include selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such language is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). As will be readily apparent to one of ordinary skill in this and related arts, the language may be expanded for many of the items listed.
[0150] In addition, as used herein, the word "signaling" refers to, among other aspects, directing something to a corresponding decoder. For example, in some embodiments, the encoder signals a specific one of multiple parameters for LIC (e.g., LIC-Flag). In this way, in an embodiment, the same parameters are used on both the encoder side and the decoder side. Thus, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters as well as other parameters, signaling can be used without sending (implicit signaling) to simply make the decoder know and select specific parameters. By avoiding the transmission of any actual function, bit savings are achieved in various embodiments. It is to be understood that signaling can be accomplished in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signaling feature" is mentioned above, the word "signal" can also be used as a noun in this article.
[0151] It will be apparent to those skilled in the art that implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.
[0152] We have described a number of embodiments. The features of these embodiments may be provided alone or in any combination. In addition, embodiments may include one or more features across various claim classes and types, alone or in any combination:
[0153] • Determining at least one illumination compensation parameter for the current block from one or more illumination compensation parameters determined for another block or from reconstructed samples of another block, such as a neighboring block.
[0154] The illumination compensation parameters of another block are stored in a buffer;
[0155] The block is a coding unit (CU);
[0156] Determining that at least one illumination compensation parameter for the current CU can only be present in another CU in the same coding tree unit (CTU);
[0157] Determine that at least one illumination compensation parameter for the current CU can only appear in another CU belonging to the same group of CUs;
[0158] The illumination compensation parameter is the local illumination compensation (LIC) parameter;
[0159] Encode (or decode) a flag for a group of CUs, such as LIC-group-flag-enabled,
[0160] Used to indicate whether at least one CU in the group uses illumination compensation for calculating prediction;
[0161] A flag used to encode (or decode) a group of CUs to indicate another flag, such as LIC-
[0162] The flag is true if it is encoded for at least one CU in the group, where another flag indicates for a CU whether illumination compensation is used for predicting this CU.
[0163] • The flag is encoded (or decoded) with the first CU of a group of CUs encoded in inter mode. • The flag is encoded (or decoded) with the first CU of a group of CUs encoded in inter mode but not merge mode.
[0164] Encode (or decode) a flag for a group of CUs to indicate whether at least one CU in the group whose size is less than (or greater than) a value uses illumination compensation for computing prediction;
[0165] The value is encoded in the SPS, PPS or slice header;
[0166] illumination compensation parameters for the current block are determined from illumination compensation parameters determined for at least one other block having the same reference index as the current block;
[0167] illumination compensation parameters for the current block are determined from reconstructed samples for at least one other block having the same reference index as the current block;
[0168] The illumination compensation parameters for the current block are determined as an average of illumination compensation parameters determined for a plurality of other blocks having the same reference index as the reference block;
[0169] A bitstream or signal containing one or more described syntax elements, flags or variations thereof.
[0170] • Inserting a flag in the signaling syntax element that enables the decoder to adapt block prediction using illumination compensation in a manner corresponding to that used by the encoder.
[0171] • Creating and / or sending and / or receiving and / or decoding a bitstream or signal comprising one or more described syntax elements, flags or variations thereof.
[0172] A television, set-top box, mobile phone, tablet or other electronic device that performs illumination compensation according to any of the described embodiments.
[0173] A television, set-top box, cell phone, tablet, or other electronic device that performs illumination compensation according to any of the described embodiments and displays (e.g., using a monitor, screen, or other type of display) the resulting image.
[0174] A television, set-top box, mobile phone, tablet or other electronic device that tunes (e.g., using a tuner) a channel to receive a signal including an encoded image and performs illumination compensation according to any of the described embodiments.
[0175] a television, set-top box, mobile phone, tablet or other electronic device that wirelessly receives (e.g., using an antenna) a signal including an encoded image according to any of the described embodiments,
[0176] And perform illumination compensation.
[0177] According to a general aspect of at least one embodiment, a decoding method is provided, comprising:
[0178] - determining at least one illumination compensation parameter for the current block;
[0179] - reconstructing the current block using the determined illumination compensation parameters;
[0180] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0181] According to a general aspect of at least one embodiment, a decoding device is provided, which includes one or more processors configured to implement:
[0182] - determining at least one illumination compensation parameter for the current block;
[0183] - reconstructing the current block using the determined illumination compensation parameters;
[0184] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0185] According to a general aspect of at least one embodiment, a coding method is proposed, comprising:
[0186] - determining at least one illumination compensation parameter for the current block;
[0187] - reconstructing the current block using the determined illumination compensation parameters;
[0188] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0189] According to a general aspect of at least one embodiment, a coding device is provided, which includes one or more processors configured to implement:
[0190] - determining at least one illumination compensation parameter for the current block;
[0191] - encoding the current block using the determined illumination compensation parameters;
[0192] In which, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters only when at least one spatial neighboring block belongs to the same local illumination compensation group as the current block, called the current local illumination compensation group.
[0193] According to another general aspect of at least one embodiment, a bitstream is formatted to include a signal generated according to the above-described encoding method.
[0194] One or more of the present embodiments also provide a computer-readable storage medium on which instructions for encoding or decoding video data according to at least a portion of any of the above-mentioned encoding / decoding methods are stored. One or more embodiments also provide a computer-readable storage medium on which a bit stream generated according to the above-mentioned encoding method is stored. One or more embodiments also provide a method and apparatus for sending or receiving a bit stream generated according to the above-mentioned encoding method. One or more embodiments also provide a computer program product including instructions for executing at least a portion of any of the above-mentioned methods.
[0195] In one embodiment, the linear illumination compensation group is smaller than, larger than or equal to the coding tree unit.
[0196] In one embodiment, when the current block is not located at the upper left position in the same local illumination compensation group, a flag indicating whether the determined illumination compensation parameter is used to reconstruct the current block is encoded or decoded.
[0197] In one embodiment, when at least one block located above or to the left of the current block is encoded in an inter-frame mode, a flag indicating whether the determined illumination compensation parameter is used to reconstruct the current block is encoded or decoded.
[0198] In one embodiment, a flag is encoded / decoded to indicate whether at least one block of the current local illumination compensation group uses illumination compensation for prediction.
[0199] In one embodiment, a flag is encoded / decoded to indicate whether illumination compensation parameters are to be calculated and stored after reconstructing the blocks of the current local illumination compensation group.
[0200] In one embodiment, a flag is encoded / decoded, which indicates whether at least one block of the current local illumination compensation group has a value of true when another flag is explicitly encoded, wherein the other flag indicates whether the determined illumination compensation parameters are used to reconstruct the current block.
[0201] In one embodiment, the flag is encoded / decoded using the first block of the current local illumination compensation group.
[0202] In one embodiment, the flag is encoded / decoded using the first block of the current local illumination compensation group that is not encoded in merge mode.
[0203] In one embodiment, a flag is encoded / decoded for the current local illumination group, indicating whether at least one block in the group whose size is smaller than a value uses illumination compensation for computing prediction.
[0204] In one embodiment, a flag is encoded / decoded for the current local illumination group, indicating whether at least one block in the group whose size is greater than a value uses illumination compensation for computing prediction.
[0205] In an embodiment, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of at least one spatial neighboring block belonging to the current local illumination compensation group and having the same reference index as the current block or from one or more illumination compensation parameters.
[0206] In an embodiment, at least one illumination compensation parameter for the current block is determined as an average of illumination compensation parameters of two spatial neighboring blocks that belong to the current local illumination compensation group and have the same reference index as the current block.
[0207] In an embodiment, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of at least one spatial neighboring block belonging to the current local illumination compensation group and having the same reference index as the current block.
Claims
1. A method comprising: Obtaining a list of local illumination parameter sets for a first coding unit of a picture, each local illumination parameter set allowing definition of a local illumination linear model and being associated with a second coding unit adjacent to the first coding unit; A local illumination parameter set of the first encoding unit is determined as a function of at least one local illumination parameter set of the list according to a characteristic of the second encoding unit associated with the at least one local illumination parameter set of the list.
2. The method according to claim 1, wherein The local illumination parameter set of the first coding unit depends on the weighted average of the local illumination parameters of the list, wherein each weight associated with a local illumination parameter set of the list is proportional to the width of the second coding unit corresponding to the local illumination parameter set of the list in response to the second coding unit being above the first coding unit, or is proportional to the height of the second coding unit corresponding to the local illumination parameter set of the list in response to the second coding unit being on the left side of the first coding unit.
3. The method according to claim 1, wherein: The local illumination parameter set of the first coding unit depends on a weighted average of the local illumination parameters of the list, wherein each weight associated with a local illumination parameter set of the list is proportional to the length of a common boundary between the second coding unit and the first coding unit corresponding to the local illumination parameter set.
4. The method according to claim 1, wherein: The set of local illumination parameters for the first coding unit is based on local illumination parameters of the list associated with the second coding unit whose motion vector is closest to the motion vector of the first coding unit.
5. The method according to claim 1, wherein: Each local illumination parameter set of the list is associated with the second coding unit belonging to the same local illumination compensation group as the first coding unit, a local illumination compensation group being an area of the current picture consisting of a subset of consecutive samples of the picture, the picture being divided into a plurality of local illumination compensation groups.
6. A device comprising an electronic circuit, the electronic circuit being configured to: Obtaining a list of local illumination parameter sets for a first coding unit of a picture, each local illumination parameter set allowing definition of a local illumination linear model and being associated with a second coding unit adjacent to the first coding unit; A local illumination parameter set of the first encoding unit is determined as a function of at least one local illumination parameter set of the list according to a characteristic of the second encoding unit associated with the at least one local illumination parameter set of the list.
7. The device according to claim 6, wherein: The local illumination parameter set of the first coding unit depends on the weighted average of the local illumination parameters of the list, wherein each weight associated with a local illumination parameter set of the list is proportional to the width of the second coding unit corresponding to the local illumination parameter set of the list in response to the second coding unit being above the first coding unit, or is proportional to the height of the second coding unit corresponding to the local illumination parameter set of the list in response to the second coding unit being on the left side of the first coding unit.
8. The device according to claim 6, wherein: The local illumination parameter set of the first coding unit depends on a weighted average of the local illumination parameters of the list, wherein each weight associated with a local illumination parameter set of the list is proportional to the length of a common boundary between the second coding unit and the first coding unit corresponding to the local illumination parameter set.
9. The device according to claim 6, wherein: The set of local illumination parameters for the first coding unit is based on local illumination parameters of the list associated with the second coding unit whose motion vector is closest to the motion vector of the first coding unit.
10. The device according to claim 6, wherein: Each local illumination parameter set of the list is associated with the second coding unit belonging to the same local illumination compensation group as the first coding unit, a local illumination compensation group being an area of the current picture consisting of a subset of consecutive samples of the picture, the picture being divided into a plurality of local illumination compensation groups.
11. A non-transitory information storage medium storing program code instructions for implementing the method according to claim 1.
12. A decoding method, comprising: - determining at least one illumination compensation parameter for the current block; - reconstructing the current block using the determined illumination compensation parameters; In which, only when at least one spatial neighboring block belongs to the same local illumination compensation group or coding tree unit as the current block, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters, the local illumination group being a group of blocks, wherein the local illumination compensation parameters can only be inherited or calculated from blocks in the group of blocks.
13. A decoding device, comprising: One or more processors configured to perform: - determining at least one illumination compensation parameter for the current block; - reconstructing the current block using the determined illumination compensation parameters; In which, only when at least one spatial neighboring block belongs to the same local illumination compensation group or coding tree unit as the current block, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters, the local illumination group being a group of blocks, wherein the local illumination compensation parameters can only be inherited or calculated from blocks in the group of blocks.
14. A coding method comprising: - determining at least one illumination compensation parameter for the current block; - reconstructing the current block using the determined illumination compensation parameters; In which, only when at least one spatial neighboring block belongs to the same local illumination compensation group or coding tree unit as the current block, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters, the local illumination group being a group of blocks, wherein the local illumination compensation parameters can only be inherited or calculated from blocks in the group of blocks.
15. An encoding device, comprising: One or more processors configured to perform: - determining at least one illumination compensation parameter for the current block; - encoding the current block using the determined illumination compensation parameters; In which, only when at least one spatial neighboring block belongs to the same local illumination compensation group or coding tree unit as the current block, at least one illumination compensation parameter for the current block is determined from one or more reconstructed samples of the at least one spatial neighboring block or from one or more illumination compensation parameters, the local illumination group being a group of blocks, wherein the local illumination compensation parameters can only be inherited or calculated from blocks in the group of blocks.
