Method and device for luma mapping with inter-component scaling - Patents.com
Patent Information
- Application Number
- JP2023531582
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2020-12-23
- Filing Date
- 2021-12-03
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2041-12-03
AI Technical Summary
Existing video coding schemes face inefficiencies in applying luma mapping with inter-component scaling within the prediction loop, which can affect compression efficiency and increase complexity.
A method and device for controlling luma mapping with inter-component scaling processes, allowing for the application of color component-to-color component conversion as post-processing or within the prediction loop, using syntax elements to specify the conversion process.
Improves compression efficiency by optimizing the application of luma mapping and inter-component scaling, reducing complexity and enhancing coding performance.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] At least one of the present embodiments relates generally to methods and devices for image encoding and decoding, and more particularly to methods, apparatus and signals for controlling the application of luma mapping involving an inter-component scaling process. [Background technology]
[0002] To achieve high compression efficiency, video coding schemes typically employ prediction and transformation that exploit spatial and temporal redundancy in the video content. During encoding, an image of the video content is divided into blocks of samples (i.e., pixels), which are then divided into one or more sub-blocks, hereafter referred to as original sub-blocks. Intra- or inter-prediction is then applied to each sub-block to exploit intra- or inter-image correlation. Whatever the prediction method (intra- or inter-) used, a predictor sub-block is determined for each original sub-block. The sub-blocks representing the difference between the original sub-block and the predictor sub-block are then transformed, quantized, and entropy coded to generate an encoded video stream, often denoted as prediction error sub-blocks, prediction residual sub-blocks, or simply residual blocks. To reconstruct the video, the compressed data is decoded by the inverse processes corresponding to the transformation, quantization, and entropy coding.
[0003] An original video signal to be coded consists of a set of original values contained in an original dynamic range of values typically represented by a number of bits (8 bits for a dynamic range of 0 to 255, 10 bits for a dynamic range of 0 to 1023). One effect of quantization is to represent several different values by the same quantized value. Thus, the video signal after quantization has a reduced granularity compared to the original video signal. However, in some video signals, most of the original values are concentrated in a limited subrange (or limited subrange) of the original dynamic range. Applying quantization defined over a dynamic range to a signal occupying only a limited subrange of the dynamic range would strongly affect the granularity of the video signal. Luma mapping (i.e., mapping of luminance values) has been proposed to improve the coding efficiency of video signals by better using the target dynamic range (i.e., luma values are redistributed across the target dynamic range to occupy the entire target dynamic range). The target dynamic range may be equal to or different from the original dynamic range.
[0004] In video signals, luma signals are generally associated with chroma signals. If processing is applied to a luma signal once quantized to improve the granularity of this luma signal, processing should be applied to the associated chroma signals to obtain a similar improvement in granularity. Applying a mapping to the chroma signals, although possible, significantly increases the complexity of the video encoding and decoding process. To reduce the encoding / decoding complexity, some methods propose to apply a mapping only to the luma signal and to apply a scaling process to the chroma signals depending on the mapped luma signal. This scaling process is known by the term inter-component scaling process.
[0005] A new compression tool called Luma Mapping with Chroma Scaling (LMCS), which combines luma mapping with inter-component scaling, has been introduced in the international standard entitled Versatile Video Coding (VVC), which is being developed by a joint collaboration team of ITU-T and ISO / IEC experts known as the Joint Video Experts Team (JVET), hereafter referred to as VVC. LMCS has been added as a new processing block before the loop filter. Thus, LMCS is applied in the prediction loop of the encoding / decoding process.
[0006] Applying luma mapping with inter-component scaling within the prediction loop may not be optimal in terms of compression efficiency. Other configurations, such as applying luma mapping with inter-component scaling as pre-processing or post-processing from the prediction loop, may result in better compression efficiency.
[0007] It is desired to propose a solution that makes it possible to apply luma mapping or, more generally, inter-color component transformation with inter-component scaling inside or outside the prediction loop. Summary of the Invention
[0008] In a first aspect, one or more of the present embodiments provide a method that includes obtaining a syntax element associated with video data, the syntax element specifying that a color component to color component conversion process, in which at least one second color component is converted based on at least one first color component different from the second color component, is to be applied to the video data as a post-processing process following a decoding process to be applied to the video data.
[0009] In one embodiment, the syntax element specifies with a first value that the color component conversion process is applied to the video data as a post-processing process following a decoding process to be applied to the video data, and with a second value that the syntax element specifies that the color component conversion process is applied to the video data in a prediction loop included in the decoding process.
[0010] In one embodiment, the syntax element further specifies in a third value that samples of the first color component before the intra-component conversion must be used for at least one inter-color component conversion process of the second component, and in a fourth value that samples of the first color component after the intra-component conversion must be used for at least one inter-component conversion of the second component.
[0011] In one embodiment, the inter-color component transformation process is specified in a syntax element by parameters representing at least one transformation.
[0012] In one embodiment, a transform intended to be applied to one of the at least one second color components is derived from parameters representing a transform to be applied to another one of the at least one second color components specified in the syntax element, or from parameters representing a transform to be applied to the first color component specified in the syntax element.
[0013] In one embodiment, a syntax element specifies that parameters representing a color component-to-color component conversion process are derived from parameters specifying a luma mapping with a chroma scaling process obtained from another syntax element.
[0014] In one embodiment, the color component to color component conversion process is luma mapping accompanied by a chroma scaling process.
[0015] In one embodiment, the color component conversion process is applied to the video data as a post-processing process following the decoding process to be applied to the video data in response to deactivation of a second color component conversion process to be applied in the prediction loop of the decoding process.
[0016] In a second aspect, one or more of the present embodiments provides a device, the device comprising: A means is provided for obtaining a syntax element associated with the video data, the syntax element specifying that a color component to color component conversion process, in which at least one second color component is converted based on at least one first color component different from the second color component, is to be applied to the video data as a post-processing process following a decoding process to be applied to the video data.
[0017] In one embodiment, the syntax element specifies with a first value that the color component conversion process is applied to the video data as a post-processing process following a decoding process to be applied to the video data, and with a second value that the syntax element specifies that the color component conversion process is applied to the video data in a prediction loop included in the decoding process.
[0018] In one embodiment, the syntax element further specifies in a third value that samples of the first color component before the intra-component conversion must be used for at least one inter-color component conversion process of the second component, and in a fourth value that samples of the first color component after the intra-component conversion must be used for at least one inter-component conversion of the second component.
[0019] In one embodiment, the inter-color component transformation process is specified in a syntax element by parameters representing at least one transformation.
[0020] In one embodiment, a transform intended to be applied to one of the at least one second color components is derived from parameters representing a transform to be applied to another one of the at least one second color components specified in the syntax element, or from parameters representing a transform to be applied to the first color component specified in the syntax element.
[0021] In one embodiment, a syntax element specifies that parameters representing a color component-to-color component conversion process are derived from parameters specifying a luma mapping with a chroma scaling process obtained from another syntax element.
[0022] In one embodiment, the color component to color component conversion process is luma mapping accompanied by a chroma scaling process.
[0023] In one embodiment, the device comprises means for applying the color component conversion process to the video data as a post-processing process following the decoding process to be applied to the video data in response to deactivation of a second color component conversion process to be applied in a prediction loop of the decoding process.
[0024] In a third aspect, one or more of the present embodiments provide a signal including a syntax element associated with video data, the syntax element specifying that a color component to color component conversion process, in which at least one second color component is converted based on at least one first color component different from the second color component, is to be applied to the video data as a post-processing process following a decoding process to be applied to the video data.
[0025] In one embodiment, the syntax element specifies with a first value that the color component conversion process is applied to the video data as a post-processing process following a decoding process to be applied to the video data, and with a second value that the syntax element specifies that the color component conversion process is applied to the video data in a prediction loop included in the decoding process.
[0026] In one embodiment, the syntax element further specifies in a third value that samples of the first color component before the intra-component conversion must be used for at least one inter-color component conversion process of the second component, and further specifies in a fourth value that samples of the first color component after the intra-component conversion must be used for at least one inter-component conversion of the second component.
[0027] In one embodiment, the inter-color component transformation process is specified in a syntax element by parameters representing at least one transformation.
[0028] In one embodiment, a transform intended to be applied to one of the at least one second color components is derived from parameters representing a transform to be applied to another one of the at least one second color components specified in the syntax element, or from parameters representing a transform to be applied to the first color component specified in the syntax element.
[0029] In one embodiment, a syntax element specifies that parameters representing a color component-to-color component conversion process are derived from parameters specifying a luma mapping with a chroma scaling process obtained from another syntax element.
[0030] In one embodiment, the color component to color component conversion process is luma mapping accompanied by a chroma scaling process.
[0031] In a fourth aspect, one or more of the present embodiments provides a method, the method comprising: The method includes signaling a syntax element associated with the video data, the syntax element specifying that an inter-color component conversion process, in which at least one second color component is converted based on at least one first color component different from the second color component, is to be applied to the video data as a post-processing process following a decoding process to be applied to the video data.
[0032] In one embodiment, the syntax element specifies with a first value that the color component conversion process is applied to the video data as a post-processing process following a decoding process to be applied to the video data, and with a second value that the syntax element specifies that the color component conversion process is applied to the video data in a prediction loop included in the decoding process.
[0033] In one embodiment, the syntax element further specifies in a third value that samples of the first color component before the intra-component conversion must be used for at least one inter-color component conversion process of the second component, and further specifies in a fourth value that samples of the first color component after the intra-component conversion must be used for at least one inter-component conversion of the second component.
[0034] In one embodiment, the inter-color component transformation process is specified in a syntax element by parameters representing at least one transformation.
[0035] In one embodiment, a transform intended to be applied to one of the at least one second color components is derived from parameters representing a transform to be applied to another one of the at least one second color components specified in the syntax element, or from parameters representing a transform to be applied to the first color component specified in the syntax element.
[0036] In one embodiment, a syntax element specifies that parameters representing a color component-to-color component conversion process are derived from parameters specifying a luma mapping with a chroma scaling process obtained from another syntax element.
[0037] In one embodiment, the color component to color component conversion process is luma mapping accompanied by a chroma scaling process.
[0038] In a fifth aspect, one or more of the present embodiments provides a device, the device comprising: The method includes signaling a syntax element associated with the video data, the syntax element specifying that an inter-color component conversion process, in which at least one second color component is converted based on at least one first color component different from the second color component, is to be applied to the video data as a post-processing process following a decoding process to be applied to the video data.
[0039] In one embodiment, the syntax element specifies with a first value that the color component conversion process is applied to the video data as a post-processing process following a decoding process to be applied to the video data, and with a second value that the syntax element specifies that the color component conversion process is applied to the video data in a prediction loop included in the decoding process.
[0040] In one embodiment, the syntax element further specifies in a third value that samples of the first color component before the intra-component conversion must be used for at least one inter-color component conversion process of the second component, and further specifies in a fourth value that samples of the first color component after the intra-component conversion must be used for at least one inter-component conversion of the second component.
[0041] In one embodiment, the inter-color component transformation process is specified in a syntax element by parameters representing at least one transformation.
[0042] In one embodiment, a transform intended to be applied to one of the at least one second color components is derived from parameters representing a transform to be applied to another one of the at least one second color components specified in the syntax element, or from parameters representing a transform to be applied to the first color component specified in the syntax element.
[0043] In one embodiment, a syntax element specifies that parameters representing a color component-to-color component conversion process are derived from parameters specifying a luma mapping with a chroma scaling process obtained from another syntax element.
[0044] In one embodiment, the color component to color component conversion process is luma mapping accompanied by a chroma scaling process.
[0045] In a sixth aspect, one or more of the present embodiments provide a computer program comprising program code instructions for implementing a method according to the first or fourth aspect.
[0046] In a seventh aspect, one or more of the present embodiments provide an information storage medium storing program code instructions for implementing a method according to the first or fourth aspect. [Brief description of the drawings]
[0047] [Figure 1] 1 shows an example of an image segmentation of pixels of an original video. [Diagram 2] 2 illustrates generally a method for encoding a video stream executed by an encoding module; [Diagram 3] 1 illustrates a schematic diagram of a method for decoding an encoded video stream (ie, a bitstream) by a decoding module. [Figure 4A] 1 illustrates a schematic example of a hardware architecture of a processing module capable of implementing an encoding module or a decoding module in which various aspects and embodiments are implemented; [Figure 4B] FIG. 1 illustrates a block diagram of an example of a system in which various aspects and embodiments may be implemented. [Diagram 5] 13 illustrates a schematic of an analysis of a modified Color Remapping Information (CRI) SEI message. [Figure 6] 13 illustrates diagrammatically the parsing of a CRI SEI message by a decoding module; [Figure 7] 1 illustrates diagrammatically the parsing of an APS of LMCS_APS type by a decoding module; [Figure 8]3 shows the inter-component chroma scaling function. [Figure 9] An embodiment is presented that allows for improved out-of-loop application of LMCS. [Figure 10A] The construction of an LMCS forward mapping function using eight segments is shown. [Figure 10B] The derivation of the inter-component transformation parameters from the LMCS parameters is shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0048] In the following description, some embodiments use tools developed in the context of VVC or in the context of HEVC (ISO / IEC23008-2-MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265). However, these embodiments are not limited to video coding / decoding methods corresponding to VVC or HEVC, but apply to other video coding / decoding methods such as AVC (H.264 / MPEG-4 part10), EVC (Essential Video Coding / MPEG-5), AV1, and VP9, and also to image coding / decoding methods.
[0049] The video compression method will now be described with reference to FIGS.
[0050] Figure 1 shows an example of the division undergone by an image of a sample 11 of an original video 10. A sample is now considered to consist of three components, namely a luminance (or luma) component and two chrominance (or chroma) components. A sample then corresponds to a pixel. However, the following embodiments are adapted to samples containing another number of components, for example an image in which the sample is composed of grey level samples containing one component, or an image in which the sample is composed of samples containing three colour components and a transparency component and / or a depth component.
[0051] An image is divided into multiple coding entities. First, as shown by reference number 13 in FIG. 1, an image is divided into a grid of blocks called coding tree units (CTUs). A CTU consists of an N×N block of luminance samples and two corresponding blocks of chrominance samples. N is generally a power of two with a maximum value of, for example, "128". Second, an image is divided into one or more groups of CTUs. For example, it can be divided into one or more tile rows and tile columns, where a tile is a sequence of CTUs that covers a rectangular area of the image. In some cases, a tile can be divided into one or more bricks, where each brick consists of at least one CTU row in the tile. On top of the concept of tiles and bricks, there is another coding entity called a slice, where a coding entity can incorporate at least one tile of an image or at least one brick of a tile.
[0052] In the example of FIG. 1, an image 11 is divided equally into three slices S1, S2 and S3, as indicated by reference numeral 12, with each slice including a number of tiles (not shown).
[0053] As shown by reference number 14 in FIG. 1, a CTU may be divided into a form of a hierarchical tree of one or more sub-blocks called coding units (CUs). A CTU is the root (i.e., parent node) of the hierarchical tree and may be divided into multiple CUs (i.e., child nodes). Each CU becomes a leaf of the hierarchical tree if it is not further divided into smaller CUs, and becomes a parent node of smaller CUs (i.e., child nodes) if it is further divided. Several types of hierarchical trees may be applied, including, for example, a quad tree, a binary tree, and a ternary tree. In a quad tree, a CTU (each CU) may be divided into (i.e., may be its parent node) into "4" rectangular CUs of equal size. In a binary tree, a CTU (each CU) may be divided horizontally or vertically into "2" rectangular CUs of equal size. In a ternary tree, a CTU (each CU) may be divided horizontally or vertically into "3" rectangular CUs. For example, a CU of height N and width M is divided vertically (respectively horizontally) into a first CU of height N (respectively N / 4) and width M / 4 (respectively M), a second CU of height N (respectively N / 2) and width M / 2 (respectively M), and a third CU of height N (respectively N / 4) and width M / 4 (respectively M).
[0054] In the example of FIG. 1, CTU 14 is first divided into "4" rectangular CUs using a quadtree type division. The top left CU is not further divided and is therefore a leaf of the hierarchical tree, i.e., it is not a parent node of other CUs. The top right CU is further divided into "4" smaller square CUs, also using a quadtree type division. The bottom right CU is divided vertically into "2" rectangular CUs using a binary tree type division. The bottom left CU is divided vertically into "3" rectangular CUs using a ternary tree type division.
[0055] The combination of a binary tree and a ternary tree is known as a multi-type tree (MTT).
[0056] During image coding, the partitioning is adaptive and each CTU is partitioned to optimize the compression efficiency on a CTU basis.
[0057] In some video compression schemes, the concept of prediction unit (PU) and transform unit (TU) has emerged. In that case, the coding entity used for prediction (i.e., PU) and transformation (i.e., TU) can be a subdivision of CU. For example, as shown in FIG. 1, a CU of size 2N×2N can be divided into PUs 1411 of size N×2N or size 2N×N. The CU can be further divided into 4 TUs 1412 of size N×N or 16 TUs of size (N / 2)×(N / 2).
[0058] In this application, the terms "block" or "image block" or "sub-block" may be used to refer to any one of a CTU, a CU, a PU, and a TU. Furthermore, the terms "block" or "image block" may be used to refer to macroblocks, partitions, and sub-blocks as specified in MPEG-4 / AVC or other video coding standards, and more generally to refer to arrays of samples of multiple sizes.
[0059] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," "subpicture," "slice," and "frame" may be used interchangeably.
[0060] Fig. 2 shows a schematic diagram of a method for encoding a video stream executed by an encoding module. Variations of this method for encoding are contemplated, but for the sake of clarity, the following describes the method for encoding in Fig. 2 without describing all possible variations.
[0061] Encoding the current original image 200 begins with splitting the current original image 200 during step 202, as described in connection with Fig. 1. This splits the current image 200 into CTUs, CUs, PUs, TUs, etc. For each block, the encoding module determines the coding mode between intra prediction and inter prediction.
[0062] Intra prediction, indicated by step 203, consists of predicting samples of the current block from a predictive block derived from samples of reconstructed blocks located in the causal neighborhood of the current block to be coded, according to an intra prediction method. The results of intra prediction are a prediction direction indicating which sample of the neighboring block to use, and a residual block resulting from the calculation of the difference between the current block and the predictive block.
[0063] Inter prediction consists of predicting the samples of the current block from a block of samples, called a reference block, of a previous or subsequent image of the current image, this image being called the reference image.
[0064] During the coding of a current block by the inter prediction method, the block of the reference image which is closest to the current block according to a similarity criterion is determined by a motion estimation step 204. During step 204, a motion vector is determined which indicates the position of the reference block in the reference image. This motion vector is used during a motion compensation step 205, during which a residual block is calculated in the form of the difference between the current block and the reference block.
[0065] In the first video compression standards, the unidirectional inter prediction mode described above was the only inter mode available. As video compression standards have evolved, the family of inter modes has grown significantly and now includes many different inter modes.
[0066] During a selection step 206, the prediction mode that optimizes the compression performance is selected by the encoding module from among the tested prediction modes (intra prediction modes, inter prediction modes) according to a rate / distortion criterion (ie, RDO criterion).
[0067] Once a prediction mode has been selected, in step 207, a residual block corresponding to the selected prediction mode is obtained (e.g. from the residual block calculated in steps 203 and 205), transformed during step 209 and quantized during step 210. During quantization, in the transformed domain, the transformed coefficients are weighted by a scaling matrix in addition to the quantization parameter. The scaling matrix is a coding tool that allows prioritizing some frequencies at the expense of others. Generally, low frequencies are preferred.
[0068] It should be noted that the encoding module may skip the transform and directly apply quantization to the untransformed residual signal.
[0069] If the current block is coded according to an intra prediction mode, the prediction direction and the transformed and quantized residual block are encoded by an entropy encoder during step 212 .
[0070] Once the current block is coded according to an inter prediction mode, the motion data associated with this inter prediction mode is coded in step 211 .
[0071] Generally, two modes can be used to code the motion data, called Adaptive Motion Vector Prediction (AMVP) and Merge, respectively.
[0072] AMVP consists essentially in signaling the reference picture used to predict the current block, the motion vector predictor index, and the motion vector difference (also called the motion vector residual).
[0073] The merge mode consists in signaling the index of some motion data collected in a list of motion data predictors. The list consists of "5" or "7" candidates and is configured the same on the decoder and encoder side. The merge mode therefore aims to derive some motion data taken from the merge list. The merge list typically contains motion data associated with some spatially and temporally adjacent blocks, which are available in a reconstructed state when the current block is being processed.
[0074] Once predicted, the motion information, together with the transformed and quantized residual block, is then coded by an entropy encoder during step 212. Note that the coding module can bypass both the transform and the quantization, i.e., the entropy coding is applied to the residual without applying a transform or quantization process. The result of the entropy coding is inserted into the coded video stream (i.e., bitstream) 213.
[0075] It should be noted that the entropy encoder can be implemented in the form of a context adaptive binary arithmetic coder (CABAC), which encodes binary symbols, keeps the complexity low, and allows probability modeling of the more frequently used bits of any symbol.
[0076] After the quantization step 210, the current block is reconstructed so that the pixels corresponding to it can be used for future prediction. This reconstruction phase is also called a prediction loop. Thus, inverse quantization is applied to the transformed and quantized residual block during step 214 and an inverse transform is applied during step 215. A prediction block of the current block is reconstructed according to the prediction mode used for the current block obtained during step 217. If the current block is coded according to an inter prediction mode, the coding module applies motion compensation to the reference block using the motion information of the current block, when appropriate, during step 218. If the current block is coded according to an intra prediction mode, during step 220, the prediction direction corresponding to the current block is used to reconstruct the reference block of the current block.
[0077] In step 221, the reference block and the reconstructed residual block are added to obtain a reconstructed current block.
[0078] After reconstruction, during step 223, in-loop post-filtering intended to reduce coding artifacts is applied to the reconstructed blocks. This post-filtering is called in-loop post-filtering because it is performed in the prediction loop in order to obtain the same reference image at the encoder as the decoder, thus avoiding drift between the encoding and decoding processes. For example, in-loop post-filtering includes deblocking filtering, sample adaptive offset (SAO) filtering, and Adaptive Loop Filtering (ALF) with block-based filter adaptation.
[0079] Parameters describing the activation or deactivation of an in-loop deblocking filter, and, when activated, the characteristics of said in-loop deblocking filter, are introduced into the encoded video stream 213 during the entropy coding step 212 .
[0080] Once a block is reconstructed, it is inserted into a reconstructed image stored in a decoded picture buffer (DPB) 225 during step 224. The reconstructed image so stored can serve as a reference image for other images to be coded.
[0081] Fig. 3 shows a schematic diagram of a method for decoding an encoded video stream (i.e., a bitstream) 213, encoded according to the method described in relation to Fig. 2. The method for decoding is executed by a decoding module. Variations of this method for decoding are contemplated, but for the sake of clarity, the method for decoding of Fig. 3 is described below without describing all possible variations.
[0082] The decoding is performed block by block: for the current block, the decoding starts with entropy decoding of the current block during step 312. The entropy decoding makes it possible to obtain the prediction mode of the block.
[0083] If the current block is coded according to an intra prediction mode, entropy decoding makes it possible to obtain information representative of the prediction direction and the residual block.
[0084] If the current block is coded according to an inter prediction mode, entropy decoding allows to obtain motion data and data representing the residual block. If appropriate, during step 311, the motion data is reconstructed for the current block according to AMVP or merge mode. In merge mode, the motion data obtained by entropy decoding includes an index in a list of motion vector predictor candidates. The decoding module applies the same process as the encoding module to build the list of candidates for normal merge mode and sub-block merge mode. Using the reconstructed list and index, the decoding module can retrieve the motion vector used to predict the motion vector of the block.
[0085] The method for decoding includes steps 312, 314, 315, 317, 318, 319, 320, 321, and 323, which are identical in all respects to steps 212, 214, 215, 217, 218, 219, 220, 221, and 223, respectively, of the method for encoding. In step 324, the decoded blocks are saved to a decoded image, and the decoded image is stored in the DPB 325. When the decoding module decodes a given image, the image stored in the DPB 325 is identical to the image stored in the DPB 225 by the encoding module during the encoding of the given image. The decoded image may also be output by the decoding module, for example for display.
[0086] As mentioned above, variations of the encoding method of FIG. 2 and the decoding method of FIG. 3 are contemplated, in particular variations involving inter-component scaling processes or more generally luma mapping with inter-color component conversion. In the following, luma mapping with inter-component scaling and inter-color component conversion are used interchangeably. Indeed, in luma mapping with inter-component scaling, chroma components are converted based on luma components. In inter-color component conversion, at least one first color component is converted based on at least one second color component different from the first color component. Therefore, the concept of inter-color component conversion is broader than that of luma mapping with inter-component scaling. However, various embodiments described below apply to inter-color component conversion and luma mapping with inter-component scaling.
[0087] The introduction of luma mapping along with the inter-component scaling process in the encoding and decoding methods can take different forms.
[0088] In a first embodiment, a luma mapping with inter-component scaling process (LMCCS) is implemented in the prediction loop.
[0089] An example implementation of the LMCCS process in the prediction loop (example of the first embodiment) is LMCS LMCS consists of two main components, namely In-loop mapping of the luma component based on an adaptive piecewise linear model; A luma-dependent residual scaling is applied to the chroma components.
[0090] The in-loop mapping of luma uses a forward luma mapping function FwdMap and a corresponding inverse luma mapping function InvMap. The forward luma mapping function FwdMap is signaled using a piecewise linear model with "16" equal parts. The inverse luma mapping function InvMap does not need to be signaled, but is instead derived from the forward luma mapping function FwdMap.
[0091] The luma mapping function is signaled in a container called an adaptation parameter set (APS) using a piecewise linear model. The piecewise linear model divides the dynamic range of the input signal into 16 equal parts, and for each part, its linear mapping parameters are expressed using a number of codewords assigned to that part. Take a 10-bit input signal as an example. Each of the 16 pieces has 64 codewords assigned by default. The signaled number of codewords is used to calculate a scaling factor and adjust the forward luma mapping function for that part accordingly. Each part of the piecewise linear model is defined by two input pivot points and two output (mapped) pivot points.
[0092] On the decoding side, as depicted in Fig. 3, a forward luma mapping function is applied during step 319. In step 322, an inverse luma mapping function is applied. As seen in Fig. 3, for inter-predicted blocks, motion compensation is performed in the original domain. After the motion-compensated prediction block Y_pred is calculated based on a reference image in the DPB 325, a forward mapping function FwdMap is applied to map the luma components of the prediction block Y_pred from the original domain (i.e., the original dynamic range) to the mapped domain (i.e., the target dynamic range): Y'_pred = FwdMap(Y_pred). For intra-coded blocks, the forward mapping function FwdMap is not applied because intra prediction is performed in the mapped domain.
[0093] After a reconstructed block Y_r is calculated from the prediction block Y_pred and the residual block Y_res in step 321, an inverse mapping function InvMap is applied in step 322 to transform the reconstructed luma values from the mapped domain back to the original domain: Y_i=InvMap(Y_r). The inverse mapping function InvMap is applied to both intra-coded and inter-coded luma blocks.
[0094] The luma mapping process (forward and / or reverse mapping) can be implemented using a look-up-table (LUT) or using on-the-fly calculations.
[0095] Luma-dependent chroma residual scaling is designed to compensate for the interaction between a luma signal and its corresponding chroma signal. When luma mapping is enabled, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is enabled or not.
[0096] Luma-dependent chroma residual scaling depends on the average value of the top and / or left reconstructed neighboring luma samples. Denote avgYr as the average of the reconstructed neighboring luma samples. The value of the scaling factor C_ScaleInv is calculated in the following steps: 1. Find the index Y_Idx of the piecewise linear model to which avgYr belongs based on the InvMap function. 2. C_ScaleInv=cScaleInv[Y_Idx], where cScaleInv[] is a 16-LUT pre-computed based on values signaled in the APS.
[0097] Unlike luma mapping and inverse luma mapping, which are performed on a sample basis, the scaling factor C_ScaleInv is a constant value for the entire chroma block.
[0098] On the decoding side, a luma-dependent chroma residual scaling process is applied to the scaled chroma residual block before the reconstruction of the current chroma block in step 316. The chroma residual block C_Res is obtained by scaling the scaled chroma residual block using the scaling factor C_ScaleInv. C_Res=C_ResScale * C_ScaleInv
[0099] Then, in step 321, the residual block C_res is added to the chroma predictor block C_pred to reconstruct the current chroma block C_r. C_r=C_Res+C_pred
[0100] As can be seen, for chroma, neither inverse mapping before in-loop filtering nor forward mapping after motion compensation is performed.
[0101] On the encoding side, the introduction of LMCS is represented by the introduction of steps 201, 208, 216, 219 and 222 in the encoding method of Figure 2. Steps 216, 219 and 222 are identical to steps 316, 319 and 322, respectively. Step 201 is identical to step 219, except that the LMCCS process is applied to the original image, whereas in step 219 it is applied to the reconstructed image.
[0102] In step 208, a luma-dependent chroma residual scaling process is applied to the chroma residual block. A scaled chroma residual block C_ResScale is obtained by scaling the chroma residual block using a scaling factor C_ScaleInv. C_ResScale=C_Res / C_ScaleInv
[0103] As mentioned above, in some implementations, the LMCS parameters are stored within an APS container of type LMCS_APS. The LMCS parameters are stored in a syntax element called lmcs_data.
[0104] [Table 1]
[0105] Table TAB1 represents the syntax element lmcs_data. The semantics of the syntax element lmcs_data can be found in section 7.4.3.19 of document JVET-S2001vH, Versatile Video Coding (Draft10), Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 19th Meeting: Remote Conference, 22 June - 1 July 2020.
[0106] According to some implementations, the forward luma mapping function is a function of the parameter lmcsCW[i](i est (representing the number of codewords in the part), InputPivot[i](i est represents the input pivot point of the part), lmcsPivot[i](i est represents the output pivot point of the part (the equation numbers are equation numbers from the VVC standard, document ITU-T H.266, SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS, Infrastructure of audiovisual services-Coding of moving video, Versatile video coding, 08 / 2020).
[0107] The parameter LmcsMaxBinIdx is set equal to ("15" - lmcs_delta_max_bin_idx). The italicized text below is copied from the VVC standard.
[0108] The variable OrgCW is derived as follows: OrgCW=(1< <BitDepth) / 16 (93)
[0109] The variable lmcsDeltaCW[i] (i = lmcs_min_bin_idx to LmcsMaxBinIdx) is derived as follows. lmcsDeltaCW[i]=(1-2 * lmcs_delta_sign_cw_flag[i]) * lmcs_delta_abs_cw[i] (94)
[0110] The variable lmcsCW[i] is derived as follows. - For i=0~lmcs_min_bin_idx-1, lmcsCW[i] is set equal to 0. - For i = lmcs_min_bin_idx to LmcsMaxBinIdx, the following applies: lmcsCW[i]=OrgCW+lmcsDeltaCW[i] (95)
[0111] The value of lmcsCW[i] must be in the range OrgCW>>3~(OrgCW<<3)-1, inclusive. - For i=LmcsMaxBinIdx+1 to 15, lmcsCW[i] is set equal to 0.
[0112] Bitstream conformance requires that the following conditions are true:
[0113]
number
[0114] The variable InputPivot[i] (i = 0 to 15) is derived as follows. InputPivot[i]=i * OrgCW (97)
[0115] The variables LmcsPivot[i] (i = 0 to 16), ScaleCoeff[i] and InvScaleCoeff[i] (i = 0 to 15) are derived as follows. LmcsPivot[0]=0 for(i=0;i<=15;i++) { LmcsPivot[i+1]=LmcsPivot[i]+lmcsCW[i] ScaleCoeff[i] = (lmcsCW[i] * (1<<11)+(1<< (Log2(OrgCW)-1)))>>(Log2(OrgCW)) if(lmcsCW[i]==0) (98) InvScaleCoeff[i]=0 else InvScaleCoeff[i]=OrgCW * (1<<11) / lmcsCW[i] }
[0116] An illustration of the LMCS forward mapping function configuration using 8 segments (instead of 16) is provided in Figure 10A. As seen above, applying the LMCCS process within the prediction loop is one possible embodiment, but other embodiments exist.
[0117] In a second embodiment, the LMCSS process is applied outside the prediction loop.
[0118] On the decoding side, compared with the first embodiment, steps 316, 319, and 322 are removed from the method for decoding in Fig. 3. A new post-processing step 326 is introduced. When the LMCSS process is LMCS, an inverse mapping function InvMap is applied to transform the reconstructed luma values of the reconstructed image from the mapped domain to the original domain. The scaling factor C_ScaleInv determined using the above process is applied to the chroma values of the reconstructed image.
[0119] On the encoding side, compared to the first embodiment, steps 208, 216, 219 and 222 are removed from the method for encoding in Fig. 2. Step 201 remains the same.
[0120] The first and second embodiments are particularly adapted to control the granularity of the quantized luma and chroma signals, however, the LMCSS process may be used for other purposes.
[0121] In a third embodiment, the LMCSS process is used to map the output of the decoding process from the original dynamic range to a target dynamic range. For example, the LMCSS process is used to enhance the coded video (e.g., with higher quality graded content such as UHDTV Rec.2020) if the display is capable of displaying such enhanced data. It also allows wide gamut graded content (e.g., Rec.709 colorist grade) to be gracefully degraded (e.g., Rec.2020 colorist grade).
[0122] In the third embodiment, compared to the second embodiment, steps 201, 208, 216, 219, 222, 316, 319 and 322 are removed from the method for encoding and the method for decoding. Step 326 is maintained, but consists of applying a forward mapping function instead of an inverse mapping function. If the LMCSS process is LMCS, during step 326 a forward mapping function fwdMap is applied to transform the reconstructed luma values of the reconstructed image from the original domain to the mapped domain. A scaling factor 1 / (C_ScaleInv) is applied to the chroma values of the reconstructed image.
[0123] In the fourth embodiment, the first and third embodiments, or the second and third embodiments, are combined.
[0124] In previous generation video compression standards, two syntax elements are known to allow transporting data related to color mapping. The APS mentioned above for transporting LMCS parameters. An SEI message called Color Remapping Information (CRI).
[0125] For example, a SEI (Supplemental Enhancement Information) message defined in a standard such as AVC, HEVC, or VVC is a data container that is associated with a video stream and includes metadata that provides information about the video stream.
[0126] [Table 2]
[0127] Table TAB2 provides the syntax of the CRI SEI message as defined in Section D.2.23 of the HEVC Standard. The semantics of the syntax elements contained in the CRI SEI message are provided in Section D.3.33 of HEVC. The CRI SEI message provides information that enables mapping of the reconstructed color samples of an image output by a decoder, such as for purposes of transforming the output image into a representation more suitable for an alternative display.
[0128] The CRI SEI defines three transformations: The first transformation (hereafter called "pre_lut") is based on three 1D-LUTs applied to each color component: The second transformation is based on a 3x3 matrix, and The third transformation (hereafter called “post_lut”) is based on three 1D-LUTs applied to each color component.
[0129] The three transformations can be performed in cascade. When the 3x3 matrix transformation is enabled, the color components must be processed in 4:4:4 format, which means that there are three color samples per pixel. When only the 1D-LUT transformation is used, the process can be applied in 4:2:0, 4:2:2, or 4:4:4 color formats, since each color sample is processed independently of the others. The cascade transformation is shown in the following equations for three color components, denoted C0, C1, and C2, respectively. Input samples C0_in(p), C1_in(p), and C2_in(p) at relative position p in the picture are processed as follows to generate output samples C0_out(p), C1_out(p), and C2_out(p):
[0130] First conversion application: C0'=pre_lut[0][C0_in(p)] C1'=pre_lut[1][C1_in(p)] (Formula 1) C2'=pre_lut[2][C2_in(p)] In the formula, pre_lut[0][] represents the first transform applied to the first component, pre_lut[1][] represents the first transform applied to the second component, and pre_lut[2][] represents the first transform applied to the third component.
[0131] 3x3 matrix application (if applicable): (C0”, C1”, C2”) T =M3×3.(C0',C1',C2') T (Formula 2)
[0132] where M3×3 is a 3×3 matrix (syntax element colour_remap_coeffs[c][i]), and ( T is the transposition operator.
[0133] Second conversion application: C0_out(p)=post_lut[0][C0”] C1_out(p)=post_lut[1][C1”] (Formula 3) C2_out(p)=post_lut[2][C2”] where post_lut[0][] represents the second transformation applied to the first component, post_lut[1][] represents the second transformation applied to the second component, and post_lut[2][] represents the second transformation applied to the third component.
[0134] Note that the 1D-LUT is applied independently to each color component, which corresponds to an intra-component transformation.
[0135] As can be seen, the CRI SEI message is not adapted to provide parameters of the LMCCS process, or more generally, of the color component-to-color component conversion. Moreover, the CRI SEI message provides color mapping parameters for post-processing processes. On its side, the APS container of type LMCS_APS is designed to provide parameters for the LMCCS process (here LMCS), which is applied in the prediction loop. Therefore, the CRI SEI message and the APS container of type LMCS_APS do not comply with the embodiments invoked above.
[0136] Therefore, a solution is needed to support the LMCCS process, and more generally to support inter-color component conversion.
[0137] According to a fifth embodiment, a new transformation is added to the CRI SEI message to allow inter-color component transformation. The new transformation is based on a 1D-LUT and is applied as follows (shown below using the first transformation (pre_lut)): In a variant, this can be extended to the second transformation case (post_lut).
[0138] The first color component input sample C0_in is mapped to the first component output sample C0_out for each pixel at relative position p using the intra-component transform pre_lut. C0_out(p)=pre_lut[0][C0_in(p)] (Equation 4)
[0139] The second and third color component input samples C1_in(p) and C2_in(p) are mapped to the second and third component output samples C1_out(p) and C2_out(p), respectively, using an inter-component transformation as follows: C1_out(p)=pre_lut[1][C0coloc(p)] * C1_in(p)-2 * B1-1)+2 * B1-1 C2_out(p)=pre_lut[2][C0coloc(p)] * C2_in(p)-2 * B2-1)+2 * B2-1 (Formula 5) where B1 and B2 are the bit depths of the second and third color component samples, respectively, and C0coloc(p) is the value calculated from the first component sample located at or near relative position p.
[0140] According to the fifth embodiment, the syntax element colour_remap_pre_lut_cross_component_flag is added to the CRI SEI message to indicate whether the 1D LUT representing the first transformation pre_lut for the second and third components is used as an intra-component transformation or as an inter-component transformation.
[0141] In a variation of the first embodiment, a similar syntax element colour_remap_post_lut_cross_component_flag is added to the CRI SEI message for the second transformation post_lut.
[0142] Table TAB3 shows the insertion of the syntax elements colour_remap_pre_lut_cross_component_flag and colour_remap_post_lut_cross_component_flag into the CRI SEI message. The new syntax elements are in bold.
[0143] In a variant, a syntax element can also be inserted to indicate whether samples from a first component used in the inter-component transformation are used before or after being transformed by the first transform.
[0144] Examples of the semantics of the new syntax elements are as follows: colour_remap_pre_lut_cross_component_flag equal to '1' indicates that the syntax elements pre_lut_coded_value[1][i] and pre_lut_coded_value[2][i] (i.e. representing the transformations pre_lut[1][i] and pre_lut[2][i], respectively) define piecewise linear scaling functions indexed by values according to the first colour component sample and applied to the second and third colour component samples, respectively. colour_remap_pre_lut_cross_component_flag equal to '0' indicates that the syntax elements pre_lut_coded_value[1][i] and pre_lut_coded_value[2][i] (i.e. representing the transformations pre_lut[1][i] and pre_lut[2][i], respectively) define piecewise linear functions applied to the second and third colour component samples, respectively. colour_remap_pre_lut_cross_comp_mode indicates that the transformation of the second and third colour component samples, defined by the syntax elements pre_lut_coded_value[1][i] and pre_lut_coded_value[2][i] respectively, depends on the first colour component sample before (colour_remap_pre_lut_cross_comp_mode is equal to '0') or after (colour_remap_pre_lut_cross_comp_mode is equal to '1') they have been processed by the transformation defined by the syntax element pre_lut_coded_value[0][i] (i.e. representing the transformation pre_lut[0][i]). · colour_remap_post_lut_cross_component_flag has the same semantics as colour_remap_pre_lut_cross_component_flag, with "pre" replaced by "post". · colour_remap_post_lut_cross_comp_mode has the same semantics as colour_remap_pre_lut_cross_comp_mode, with "pre" replaced by "post".
[0145] [Table 3]
[0146] Fig. 5 shows a schematic analysis of the modified CRI SEI message, focusing on the new syntax elements introduced in the CRI SEI message, as represented in table TAB3.
[0147] The process of FIG. 5 is performed by processing module 40, described below in conjunction with FIGS. 4A and 4B, during parsing of the CRI SEI message.
[0148] In step 501, the processing module 40 gets the syntax element colour_remap_pre_lut_cross_component_flag and determines whether the syntax element colour_remap_pre_lut_cross_component_flag is equal to zero. If the syntax element colour_remap_pre_lut_cross_component_flag is equal to zero, the processing module 40 in step 502 applies an intra-component transformation to each colour component using equations 1, 2 and 3.
[0149] If not zero, then in step 503, the processing module 40 applies an intra-component transformation to the first color component using equation 4. Thus, the syntax element colour_remap_pre_lut_cross_component_flag equal to "1" specifies that an inter-color component transformation process should be applied. By default (i.e. without further information), this inter-color component transformation process should be applied as a post-processing process following the decoding process.
[0150] In step 504, the processing module 40 determines whether the syntax element colour_remap_pre_lut_cross_comp_mode is equal to zero. If the syntax element colour_remap_pre_lut_cross_comp_mode is equal to zero, the processing module 40 determines that the samples of the first component before the intra-component transformation must be used for the inter-colour-component transformation of the second and third components.
[0151] If not zero, the processing module 40 determines in step 506 that the transformed samples of the first component after the intra-component transformation must be used for the inter-component transformation of the second and third components.
[0152] Steps 505 and 506 are followed by step 507, during which the processing module 40 applies an inter-component transformation to the second and third components using Equation 5 and the transformed or untransformed samples of the first component determined in steps 505 and 506.
[0153] In a variant, the syntax element colour_remap_in_loop_flag is added to the CRI SEI message (as shown in bold in Table TAB4) to indicate whether the colour component-to-colour conversion is achieved within the prediction loop of the video decoding process or after the decoding process as post-processing.
[0154] [Table 4]
[0155] FIG. 6 illustrates diagrammatically the parsing of a CRI SEI message by the decoding module.
[0156] The process in Figure 6 focuses on the analysis of the syntax element colour_remap_in_loop_flag.
[0157] In step 601, the processing module 40 of the decoding module determines whether the syntax element colour_remap_in_loop_flag is equal to "1".
[0158] If the syntax element colour_remap_in_loop_flag is equal to "1", then in step 602 an inter-colour component transformation (e.g. LMCS) is applied within the prediction loop as described in the first embodiment. If the syntax element colour_remap_in_loop_flag is equal to "0", then in step 603 an inter-colour component transformation (e.g. LMCS) is applied as a post-processing step after the decoding process as described in the second and third embodiments.
[0159] In the variant represented in table TAB5, when inter-component transformation is enabled, the syntax element colour_remap_pre_lut_number_minus1 is added to indicate whether one or two LUTs are signaled for the inter-component transformation of the second and third colour components. For example, if colour_remap_pre_lut_number_minus1 is equal to '0', only the transformation pre_lut[1][i] (represented by the syntax element pre_lut_coded_value[1][i]) is signaled and the transformation pre_lut[2][i] (represented by the syntax element pre_lut_coded_value[2][i]) is set equal to the transformation pre_lut[1][i]. If colour_remap_pre_lut_number_minus1 is equal to '1', both transformations are signaled.
[0160] In the variant represented in table TAB5, when inter-component transformation is enabled, the syntax element colour_remap_pre_lut_cross_component_inferred_flag is added to indicate whether the transformations pre_lut[1][i] and pre_lut[2][i] used for the inter-component transformation of the second and third colour components are explicitly signalled or inferred from the transformation associated with the first colour component. For example, if colour_remap_pre_lut_cross_component_inferred_flag is equal to '0', the transformation is signalled. If colour_remap_pre_lut_cross_component_inferred_flag is equal to '1', the transformations pre_lut[1][i] and pre_lut[2][i] are inferred from the transformation pre_lut[0][i]. For example, the transforms pre_lut[1][i] and pre_lut[2][i] are calculated as slopes or as filtered versions of the slope values of the transform pre_lut[0][i].
[0161] [Table 5]
[0162] In a sixth embodiment, instead of using SEI messages to convey metadata, the metadata is conveyed in an adaptation parameter set (APS).
[0163] The parameters for the component-to-component mapping are embedded in the VVC LMCS syntax contained in an APS of type LMCS_APS.
[0164] In a sixth embodiment, a syntax element lmcs_in_loop_flag is added to an APS of type LMCS_APS to indicate whether the LMCS process is achieved within the prediction loop of the video decoding process or as a post-process after the video decoding process, as described in Equation 4 and Equation 5.
[0165] [Table 6]
[0166] Table TAB6 shows (in bold) the insertion of the syntax element lmcs_in_loop_flag into the syntax element lmcs_data for an APS of type LMCS_APS.
[0167] FIG. 7 illustrates diagrammatically the parsing of an APS of LMCS_APS type by the decoding module.
[0168] The process of FIG. 7 focuses on parsing the syntax element lmcs_in_loop_flag.
[0169] In step 701, the processing module 40 of the decoding module determines whether the syntax element lmcs_in_loop_flag is equal to "1".
[0170] If the syntax element lmcs_in_loop_flag is equal to "1", then in step 702, the LMCS process is applied in the prediction loop as described in the first embodiment. If the syntax element lmcs_in_loop_flag is equal to "0", then in step 703, the LMCS process is applied as a post-processing after the decoding process as described in the second and third embodiments.
[0171] In a variant of the sixth embodiment, when inter-component transformation is selected in the CRI SEI message, a flag colour_remap_pre_lut_from_APS_flag is inserted in the CRI SEI message to indicate that the parameters of the inter-component transformation are inherited (i.e. derived) from the LMCS parameters embedded in the APS of the bitstream.
[0172] If the flag colour_remap_pre_lut_from_APS_flag is true the parameter colour_remap_pre_lut_from_APS_id is added indicating the identifier of the APS to be referenced. The identifier of the APS is for example the syntax element aps_adaptation_parameter_set_id. The syntax elements pre_lut_num_val_minus1, pre_lut_coded_value and pre_lut_target_value are inferred from the syntax element lmcs_data signalled in the APS.
[0173] An example of the inference process is described below. pre_lut_num_val_minus1[c] is set to "16" as in LMCS, and "16" segments are used for c=0 to 2 representing the color components. pre_lut_coded_value[0][i] is set to lmcsPivot[i] for i = 0 to "16". pre_lut_target_value[0][i] is set to InputPivot[i] for i = 0 to "16".
[0174] For the transformation of components “1” and “2” represented by LUTs pre_lut[1][i] and pre_lut[2][i], the following can be applied for c=1 and “2”, and i=0 to “15”. pre_lut_coded_value[c][i]=pre_lut_coded_value[0][i] slope=pre_lut_target_value[0][i+1]-pre_lut_target_value[0][i] If slope=0, pre_lut_target_value[c][i]=2048 Otherwise, pre_lut_target_value[c][i]=OrgCW * 2048 / slope
[0175] The value "2048" corresponds to neutral inter-component scaling, i.e., equal to "1.0" in floating-point representation.
[0176] An illustration using the functions illustrated in FIG. 10A is provided in FIG. 10B.
[0177] [Table 7]
[0178] Table TAB7 shows (in bold) the insertion of the syntax elements colour_remap_pre_lut_cross_component_flag, colour_remap_pre_lut_from_APS_flag and colour_remap_pre_lut_from_APS_id into the CRI SEI message.
[0179] The same principle applies to the transformation post_lut (by replacing "pre" with "post").
[0180] Alternatively, the syntax element indicating whether the LMCS process is accomplished within the prediction loop of the video decoding process or after the video decoding process as a post-processing can also be inserted in other high level syntax structures such as SPS (Sequence Parameter Set), PPS (Picture Parameter Set), picture header, slice header, tile header, sub-picture header, etc. Preferably, this syntax element should be inserted in the SPS, PPS, or picture header, since it is expected that this syntax element applies to the whole picture and also to a set of pictures included in one intra period (in other words, inserted together with an IDR picture).
[0181] [Table 8]
[0182] Table TAB8 shows a modification of the SPS syntax (represented in bold) with the addition of the syntax element sps_lmcs_in_loop_flag. If the syntax element sps_lmcs_in_loop_flag is equal to "1", the LMCS process is applied in the prediction loop as described in the first embodiment. If the syntax element sps_lmcs_in_loop_flag is equal to "0", the LMCS process is applied as a post-process after the decoding process as described in the second and third embodiments.
[0183] Alternatively, the constraint flag for gci_no_lmcs_in_loop_constraint_flag may be defined as shown in Table TAB9.
[0184] [Table 9]
[0185] The definition is as follows: gci_no_lmcs_constraint_flag equal to "1" specifies that for all pictures in OlsInScope, sps_lmcs_enabled_flag must be equal to "0". gci_no_lmcs_constraint_flag equal to "0" imposes no such constraint. gci_no_lmcs_in_loop_constraint_flag equal to "1" specifies that for all pictures in OlsInScope sps_lmcs_in_loop_flag must be equal to "0" and, if received, colour_remap_in_loop_flag must be equal to 0. gci_no_lmcs_in_loop_constraint_flag equal to "0" imposes no such constraint.
[0186] Alternatively, the signaling of these parameters can be done at the system level rather than in the SEI message, or in addition to syntax elements in the high-level syntax. Some of the benefits may be that some encoders do not support SEI messages, and that the information used for the entire session can be made easily available at the signaling / payload negotiation level.
[0187] For example, these parameters may be signaled in the following: SDP (session description protocol), a format for describing multimedia communication sessions for the purposes of session announcement and session invitation, e.g. as described in the RFCs and used in conjunction with RTP (Real-time Transport Protocol) transport. For example, it can be defined as follows: a=fmtp:xxx profile-id=xxx;lmcs_inloop_flag=1;Color_Mapping_in_loop_flag=1; Dynamic Adaptive Streaming over HTTP (DASH) Media Presentation Description (MPD) Descriptors: as used, for example, in DASH and transmitted over HTTP, descriptors are associated with a content representation (content being, for example, a video stream) or a set of representations to provide additional characteristics to the representation. RTP header extensions as specified in RFC8285, for example as used during RTP streaming. For example, they can be defined as follows: a=extmap:xxx URI-lmcs_in_loop_flag a=extmap:xxx URI-color_mapping_in_loop_flag m=video a=sendrecv; Or the ISO Base Media File Format, such as that used in OMAF (Omnidirectional MediA Format), which uses boxes, object-oriented building blocks defined by a unique type identifier and a length, also known in some specifications as "atoms".
[0188] In classical LMCS designs, inter-component chroma scaling functions are defined as successive fixed values per interval, resulting in discontinuous functions, as shown in FIG. 8. When applied outside the loop, these discontinuities can lead to coding efficiency losses. In fact, small variations in luma values can lead to large differences in chroma scaling values. These differences can occur frequently because the luma signal is subject to compression errors. An example of such a discontinuous function is shown in FIG. 9, defined for luma component Y in the mapped domain (denoted Ymap in FIG. 8 and FIG. 9).
[0189] Therefore, to avoid this drawback, it is proposed to use a smoothed or more continuous version of the chroma scaling function for the inter-component function. For example, one can use a piecewise linear function defined from the chroma scaling values of the LMCS chroma scaling function, as shown by the dashed line in Figure 9.
[0190] Above, we have seen the fourth embodiment, in which the first (or second) and third embodiments are combined. For example, in this fourth embodiment, LMCS in the prediction loop as in the first embodiment (or outside the prediction loop as in the second embodiment) is combined with the color inter-component transformation defined in the CRI SEI message. For example, CRI is used to perform the inter-component transformation with a static function every intra period, and LMCS is used to adjust or refine the transformation every frame. This can advantageously provide coding gain compared to using only CRI, or using only LMCS, or using neither of them.
[0191] In the seventh embodiment, the in-loop LMCS and the inter-component transformation from the CRI SEI message are exclusive: the inter-component transformation can only be applied if the LMCS is disabled.
[0192] In an eighth embodiment, the in-loop LMCS and the inter-component transformation from the CRI SEI message are non-exclusive. This can be advantageously exploited to enable bitrate reduction and energy consumption control by the end user. For example, the input video is in HDR PQ format (BT.2020PQ). It is mapped to a more SDR-like video by an out-of-loop mapping signaled in the SEI. The mapped video is encoded and decoded using signaling in the VUI (Video Usability Information, as specified in ITU-T Recommendation H.274 ISO / IEC23002-7) indicating that the encoded video is SDR. The in-loop LMCS can be used to complement the out-of-loop mapping to achieve higher coding performance (with a mapping function signaled in the APS that is different from the out-of-loop mapping function). At the decoder, the compressed video is decoded. Since it has a lower picture luminance than the HDR version, it can be displayed as it is, i.e. as an SDR-like video, which may consume less energy by the display. If the end user wants to display the HDR version, the end user can apply the out-of-loop inverse mapping signaled in the SEI, indicating that the target video is in HDR PQ format. The advantages are threefold. 1. Bitrate reduction: Out-of-loop mapping provides better coding compression than in-loop LMCS for HDR PQ (e.g., as specified in Recommendation BT.2100) video, and LMCS can be applied on top of out-of-loop mapping to the mapped video to obtain additional coding gain. In fact, LMCS is also beneficial for coding SDR video. 2. Reduced energy consumption in delivery: Improved compression, and therefore higher bitrates, reduce the energy consumption to deliver video. 3. Reducing energy consumption in rendering: On the rendering side, depending on the energy usage selected by the client / end user, it is possible to either select the decoded and mapped video as is with lower brightness and therefore lower display energy consumption, or to process the decoded and mapped video by out-of-loop inverse mapping with higher brightness and therefore higher display energy consumption.
[0193] In a modified CRI SEI message variant, pre_lut_coded_value[c][i] is not coded but is set to a default value depending on the signal bit depth and the number of coded values (pre_lut_num_val_minus1[c]+1). For example, pre_lut_coded_value[c][0] is set to "0" and pre_lut_coded_value[c][i+1]=pre_lut_coded_value[c][i]+preLutDelta, where preLutDelta=2 bitdepth / (pre_lut_num_val_minus1[c]+1). Moreover, instead of coding pre_lut_target_value[c][i] as an absolute value, it can also be coded as a differential value relative to pre_lut_target_value[c][i-1]. By applying this variant, in fact, it corresponds to a signaling similar to LMCS APS signaling. Similarly, this can be applied to the syntax associated with the transformation post_lut. An example of the corresponding syntax is provided in table TAB10, which concatenates different variants of the modified CRI SEI message described in this document. In addition, the SEI message is a renamed Colour Transform Information SEI message, since it is a generalized version of the CRI SEI message.
[0194] [Table 10-1]
[0195] [Table 10-2]
[0196] When colour_transform_pre_lut_cross_component_flag is equal to '0', the variable colourTransformPreLutNumber is equal to '3' ('3' transforms pre_lut are signalled for components c=0 to 2). When colour_transform_pre_lut_cross_component_flag is equal to '1', if colour_transform_pre_lut_cross_component_inferred_flag is equal to '1' (a single transform pre_lut of '1' is signalled for component c=0 and a transform pre_lut for component c=1 and '2' are inferred from this single signalled transform pre_lut), or if colour_transform_pre_lut_cross_component_inferred_flag is equal to '0' (a transform pre_lut of '1' is signalled for component c=0 and (colour_transform_pre_lut_number_minus1+1) transforms pre_lut for component c=1 and '2' are signalled), then colourTransformPreLutNumber is equal to '1', or (2+colour_transform_pre_lut_number_minus1) if colour_transform_pre_lut_cross_component_inferred_flag is equal to '0' (a transform pre_lut of '1' is signalled for component c=0 and (colour_transform_pre_lut_number_minus1+1) transforms pre_lut for component c=1 and '2' are signalled). The same applies to the transformation post_lut by replacing "pred" with "post".
[0197] The fifth, sixth and seventh embodiments have been described mainly in relation to the decoding module and the decoding method of Fig. 3. However, it is clear that all syntax elements described in relation to the fifth, sixth and seventh embodiments (i.e. the modified CRI SEI message and the modified APS of type LMCS_APS) obtained by the decoding module together with the encoded video stream 211 are associated by the encoding module to the encoded video stream 211.
[0198] FIG. 4A illustrates generally one example of a hardware architecture for a processing module 40 capable of implementing an encoding module or a decoding module capable of implementing the encoding method of FIG. 2 and the decoding method of FIG. 3, respectively, modified according to different aspects and embodiments described above. The processing module 40 includes a processor or central processing unit (CPU) 400, which may include, by way of non-limiting example, one or more microprocessors, general purpose computers, special purpose computers, and processors based on multi-core architectures, connected by a communication bus 405; a random access memory (RAM) 401; a read-only memory (ROM) 402; and a memory array (RAM) 403, which may include, by way of non-limiting example, an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash, a magnetic disk drive, and / or an optical disk drive, or a secure digital (SD) card reader and / or a hard disk drive. The system includes a storage unit 403, which may include non-volatile and / or volatile memory, including but not limited to a storage media reader and / or a network accessible storage device, such as a hard disk drive (HDD), and at least one communication interface 404 for exchanging data with other modules, devices, or equipment. The communication interface 404 may include but is not limited to a transceiver configured to transmit and receive data over a communication channel. The communication interface 404 may include but is not limited to a modem or a network card.
[0199] If processing module 40 implements a decoding module, communication interface 404 may, for example, enable processing module 40 to receive an encoded video stream and provide a decoded video stream. If processing module 40 implements an encoding module, communication interface 404 may, for example, enable processing module 40 to receive and encode original image data and provide an encoded video stream.
[0200] The processor 400 can execute instructions loaded into the RAM 401 from a ROM 402, an external memory (not shown), a storage medium, or a communication network. When the processing module 40 is powered on, the processor 400 can read instructions from the RAM 401 and execute them. These instructions form a computer program for causing the processor 400 to carry out, for example, the decoding method described in relation to Fig. 3 or the encoding method described in relation to Fig. 2, which include various aspects and embodiments described later in this specification.
[0201] All or part of the algorithms and steps of the encoding or decoding methods may be implemented in software form by execution of a set of instructions by a programmable machine such as a DSP (digital signal processor) or a microcontroller, or in hardware form by a machine or dedicated component such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit).
[0202] FIG. 4B illustrates a block diagram of an example of a system 4 in which various aspects and embodiments are implemented. The system 4 may be embodied as a device including various components described below and configured to perform one or more of the aspects and embodiments described herein. Examples of such devices include various electronic devices, such as, but not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected appliances, cameras, and servers. The elements of the system 4, alone or in combination, may be embodied in an integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the system 4 includes one processing module 40 that implements a decoding module or an encoding module. However, in another embodiment, the system 4 may include a first processing module 40 that implements a decoding module and a second processing module 40 that implements a decoding module, or one processing module 40 that implements a decoding module and an encoding module. In various embodiments, system 40 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 40 is configured to implement one or more of the aspects described herein.
[0203] The system 4 includes at least one processing module 40 that may implement one or both of an encoding module or a decoding module.
[0204] Input to processing module 40 may be provided via various input modules, as shown in block 42. Such input modules may include, but are not limited to, (i) a radio frequency (RF) module that receives, for example, a radio frequency (RF) signal transmitted over the air from a broadcast station, (ii) a component (COMP) input module (or a set of COMP input modules), (iii) a Universal Serial Bus (USB) input module, and / or (iv) a High Definition Multimedia Interface (HDMI) input module. Other examples not shown in FIG. 4B include composite video.
[0205] In various embodiments, the input modules of block 42 have associated respective input processing elements as known in the art. For example, the RF module may be associated with appropriate elements to (i) select a desired frequency (also referred to as selecting a signal or band-limiting a signal to a frequency band), (ii) down-convert the selected signal, (iii) band-limit again to a narrower frequency band to select a signal frequency band, which in certain embodiments may be referred to (for example) as a channel, (iv) demodulate the down-converted and band-limited signal, (v) perform error correction, and (vi) demultiplex to select a desired stream of data packets. The RF module of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner that performs various of these functions, including down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to baseband) or to baseband. In one embodiment of a set-top box, the RF module and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and refiltering to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF module includes an antenna.
[0206] Additionally, the USB and / or HDMI modules may include respective interface processors for connecting the system 4 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within the processing module 40, as appropriate. Similarly, aspects of the USB or HDMI interface processing may be implemented in a separate interface IC or within the processing module 40, as appropriate. The demodulated, error corrected, and demultiplexed stream is provided to the processing module 40.
[0207] The various elements of system 4 may be provided within a unitary housing, where the various elements may be interconnected and transmit data between them using any suitable connection arrangement, such as an internal bus known in the art, including an Inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in system 4, processing module 40 is interconnected to the other elements of system 4 by bus 405.
[0208] The communication interface 404 of the processing module 40 enables the system 4 to communicate over a communication channel 41. The communication channel 41 may be implemented, for example, in a wired and / or wireless medium.
[0209] Data is streamed or otherwise provided to system 4 in various embodiments using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received over a communication channel 41 and communication interface 404 that are adapted for Wi-Fi communication. The communication channel 41 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. In other embodiments, streaming data is provided to system 4 using a set-top box that delivers data via an HDMI connection in input block 42. In yet other embodiments, streaming data is provided to system 4 using an RF connection in input block 42. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.
[0210] System 4 can provide output signals to various output devices including a display 46, speakers 47, and other peripheral devices 48. Display 46 in various embodiments includes one or more of a touch screen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display, for example. Display 46 can be for a television, a tablet, a laptop, a mobile phone, or other device. Display 46 can also be integrated with other components (e.g., as in a smartphone) or can be separate (e.g., an external monitor for a laptop). Other peripheral devices 46 include one or more of a stand-alone digital video disc (or digital versatile disc, DVR) (for both terms), a disc player, a stereo system, and / or a lighting system, in various examples of embodiments. Various embodiments use one or more peripheral devices 48 that provide functionality based on the output of system 4. For example, a disc player performs the functionality of playing the output of system 4.
[0211] In various embodiments, control signals are communicated between system 4 and display 46, speakers 47, or other peripheral devices 48 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow control between devices with or without user intervention. The output devices can be communicatively coupled to system 4 via dedicated connections through respective interfaces 43, 44, and 45. Alternatively, the output devices can be connected to system 4 using communication channel 41 via communication interface 404. Display 46 and speakers 47 can be integrated into one unit with other components of system 4 in an electronic device such as, for example, a television. In various embodiments, display interface 43 includes a display driver, such as, for example, a timing controller (T Con) chip.
[0212] Alternatively, display 46 and speakers 47 may be separate from one or more of the other components, for example if the RF module of input 42 is part of a separate set-top box. In various embodiments in which display 46 and speakers 47 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.
[0213] Various implementations include decoding. As used herein, "decoding" may encompass all or part of the processes performed on a received encoded video stream, e.g., to generate a final output suitable for a display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, e.g., entropy decoding, inverse quantization, inverse transform, and prediction. In various embodiments, such processes may also or alternatively include processes performed by the decoder of various implementations or embodiments described herein, e.g., to apply an in-loop or out-of-loop inter-color component conversion process, an LMCCS process, or an LMCS process.
[0214] Whether the phrase "decoding process" is intended to refer specifically to a work subset or to the broader decoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.
[0215] Various implementations involve encoding. Similar to the above discussion regarding "decoding," "encoding," as used herein, may encompass all or part of the processes performed on an input video sequence to generate, for example, an encoded video stream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, such as, for example, segmentation, prediction, transformation, quantization, in-loop post-filtering, and entropy coding. In various embodiments, such processes may also or alternatively include processes performed by the encoder of various implementations or embodiments described herein, for example, to apply in-loop or out-of-loop inter-color component conversion processes, LMCCS processes, or LMCS processes.
[0216] Whether the phrase "encoding process" is intended to refer specifically to a task subset or to the broader encoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.
[0217] It should be noted that the syntax element names, flag names, container names, and coding tool names used herein are descriptive terms, and therefore they do not preclude the use of other syntax element names, flag names, container names, or coding tool names.
[0218] Where a figure is presented as a flow chart, it should be understood that the figure also provides a block diagram of the corresponding apparatus. Similarly, where a figure is presented as a block diagram, it should be understood that the figure also provides a flow chart of the corresponding method / process.
[0219] Various embodiments refer to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered. Rate-distortion optimization is usually formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate-distortion optimization problem. For example, these approaches may be based on an extensive test of all encoding options, including all considered modes or coding parameter values, with a full evaluation of their coding costs and associated distortion of the reconstructed signal after coding and decoding. Also, faster approaches may be used to reduce the encoding complexity, in particular the calculation of approximate distortion based on a prediction or prediction residual signal rather than on the reconstructed signal. A mixture of these two approaches may also be used, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a full evaluation of both the coding cost and the associated distortion.
[0220] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed in the context of only a single implementation form (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., an apparatus or a program). For example, an apparatus may be implemented in suitable hardware, software, and firmware. A method may be implemented, for example, in a processor, where a processor refers to a general processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.
[0221] Reference to "one embodiment" or "an embodiment" or "one implementation" or "an implementation," as well as other variations thereof, means that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "an embodiment" or "in one implementation," as well as other variations thereof, appearing in various places throughout this application are not necessarily all referring to the same embodiment.
[0222] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, inferring information from other information, retrieving information from memory, or obtaining information from another device, module, or user, for example.
[0223] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, obtaining information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, inferring information, or inferring information.
[0224] In addition, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" is typically involved in some manner during an operation such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, inferring information, or estimating information.
[0225] Use of any of " / ", "and / or", "at least one of", "one or more", e.g., "A / B", "A and / or B", "at least one of A and B", "one or more of A and B" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, "A, B, and / or C" and "at least one of A, B, and C", "one or more of A, B, and C" are intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This may be expanded to include as many of the items listed as would be apparent to one of ordinary skill in this and related arts.
[0226] Also, as used herein, the term "signaling" specifically means to indicate something to a corresponding decoder. For example, in some embodiments, the encoder signals information related to inter-color component conversion, LMCCS or LMCS in an APS or SEI message. Thus, in some embodiments, the same parameters are used at both the encoder and decoder sides. Thus, for example, the encoder can transmit a specific parameter to the decoder so that the decoder can use the same specific parameter (explicit signaling). In contrast, if the decoder already has the specific parameter as well as other parameters, it can use signaling without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. By avoiding the transmission of any actual functionality, bit savings are realized in various embodiments. It will be understood that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used in various embodiments to signal information to a corresponding decoder. The above relates to the verb form of the word "signal", which may also be used as a noun in this specification.
[0227] As will be apparent to one of ordinary skill in the art, implementations can result in a variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry an encoded video stream of the described embodiments. For example, such a signal can be formatted as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding the encoded video stream and modulating a carrier wave with the encoded video stream. The information that the signal carries can be, for example, analog information or digital information. As is known, the signal can be transmitted over a variety of different wired or wireless links. The signal can be stored in a processor-readable medium.
[0228] Furthermore, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types. a bitstream or signal including syntax elements carrying information generated by any of the described embodiments; Inserting syntax elements into the signaling that allow the decoder to adapt the decoding process in a manner that corresponds to the method used by the encoder; Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described syntax elements or variations thereof; Creating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments; A method, process, apparatus, instruction storage medium, data storage medium, or signal according to any of the described embodiments; A TV, set-top box, mobile phone, tablet, or other electronic device that performs an adaptation of an encoding or decoding process according to any of the described embodiments; a TV, set-top box, mobile phone, tablet, or other electronic device that performs the adaptation of the encoding or decoding process according to any of the described embodiments and displays the resulting images (e.g., using a monitor, screen, or other type of display); a TV, set-top box, mobile phone, tablet, or other electronic device that selects a channel (e.g., using a tuner) to receive a signal containing encoded images and performs an adaptation of the decoding process according to any of the described embodiments; A TV, set-top box, mobile phone, tablet, or other electronic device that receives an over-the-air signal (e.g., using an antenna) containing the encoded image and performs an adaptation of the encoding process according to any of the described embodiments.
Claims
1. 1. A method comprising: obtaining an information set associated with video data, wherein each sample of each picture represented by the video data is represented by a plurality of color components, the information set including a syntax element indicating with a first value that a first transform that maps values of a first color component from a first dynamic range to a second dynamic range is explicitly signaled in the information set and with a second value that indicates that the first transform is inferred from a second transform signaled in the information set that maps values of a second color component from a third dynamic range to a fourth dynamic range; and decoding the video data based on the syntax elements.
2. The method of claim 1 , wherein the first color component is a chroma component and the second color component is a luma component.
3. 2. The method of claim 1, wherein the first transform is a component-to-component transform that maps values of the first color component from the first dynamic range to the second dynamic range based on the second color component.
4. The method of claim 1 , wherein in response to the syntax element having the second value, the first transform is derived from information representing a gradient of the second transform.
5. The method of claim 1 , wherein the first color component is a first chroma component and the second color component is a second chroma component.
6. 2. The method of claim 1 , wherein, in response to the syntax element having the second value, the first transform is derived from the second transform by adding a difference value signaled in the information set for the first transform to information representing the second transform signaled in the information set.
7. 1. A method comprising:
11. A method comprising: signaling an information set associated with video data, wherein each sample of each picture represented by the video data is represented by a plurality of color components, the information set including syntax elements, the syntax elements indicating with a first value that a first transform that maps values of a first color component from a first dynamic range to a second dynamic range is explicitly signaled in the information set and indicating with a second value that the first transform is inferred from a second transform signaled in the information set that maps values of a second color component from a third dynamic range to a fourth dynamic range.
8. The method of claim 7 , wherein the first color component is a chroma component and the second color component is a luma component.
9. 8. The method of claim 7, wherein the first transform is a component-to-component transform that maps values of the first color component from the first dynamic range to the second dynamic range based on the second color component.
10. The method of claim 7 , wherein in response to the syntax element having the second value, the first transform is derived from information representing a gradient of the second transform.
11. The method of claim 7 , wherein the first color component is a first chroma component and the second color component is a second chroma component.
12. 8. The method of claim 7, wherein, in response to the syntax element having the second value, the first transform is derived from the second transform by adding a difference value signaled in the information set for the first transform to information representing the second transform signaled in the information set.
13. A device comprising an electronic circuit, the electronic circuit comprising: obtaining an information set associated with video data, wherein each sample of each picture represented by the video data is represented by a plurality of color components, the information set including a syntax element indicating with a first value that a first transform that maps values of a first color component from a first dynamic range to a second dynamic range is explicitly signaled in the information set and with a second value that indicates that the first transform is inferred from a second transform signaled in the information set that maps values of a second color component from a third dynamic range to a fourth dynamic range; and decoding the video data based on the syntax elements.
14. The device of claim 13 , wherein the first color component is a chroma component and the second color component is a luma component.
15. 14. The device of claim 13, wherein the first transform is a component-to-component transform that maps values of the first color component from the first dynamic range to the second dynamic range based on the second color component.
16. The device of claim 13 , wherein in response to the syntax element having the second value, the first transform is derived from information representing a gradient of the second transform.
17. The device of claim 13 , wherein the first color component is a first chroma component and the second color component is a second chroma component.
18. 14. The device of claim 13, wherein in response to the syntax element having the second value, the first transform is derived from the second transform by adding a difference value signaled in the information set for the first transform to information representing the second transform signaled in the information set.
19. A device comprising an electronic circuit, the electronic circuit comprising:
11. A device comprising: signaling an information set associated with video data, wherein each sample of each picture represented by the video data is represented by a plurality of color components, the information set including a syntax element, the syntax element indicating with a first value that a first transform that maps values of a first color component from a first dynamic range to a second dynamic range is explicitly signaled in the information set and indicating with a second value that the first transform is inferred from a second transform signaled in the information set that maps values of a second color component from a third dynamic range to a fourth dynamic range.
20. 20. The device of claim 19, wherein the first color component is a chroma component and the second color component is a luma component.
21. 20. The device of claim 19, wherein the first transform is a component-to-component transform that maps values of the first color component from the first dynamic range to the second dynamic range based on the second color component.
22. 20. The device of claim 19, wherein in response to the syntax element having the second value, the first transform is derived from information representing a gradient of the second transform.
23. 20. The device of claim 19, wherein the first color component is a first chroma component and the second color component is a second chroma component.
24. 20. The device of claim 19, wherein, in response to the syntax element having the second value, the first transform is derived from the second transform by adding a difference value signaled in the information set for the first transform to information representing the second transform signaled in the information set.
25. 13. An information storage medium storing program code instructions for implementing the method of claim 1.
26. 8. An information storage medium storing program code instructions for implementing the method of claim 7.
