Method and device for encoding and decoding attenuation maps for energy-conscious images

Abstract

The new metadata associated with visual content relates to the use of a per-pixel attenuation map dedicated to reducing energy consumption when using the visual content, e.g., when rendering it on a display. Information is provided about the type of display compatible with the use of the attenuation map, the type of pre-processing (e.g., upsampling), the type of operation to use for applying the attenuation map, and some metrics about the expected energy reduction and the expected quality impact of using such an attenuation map. These parameters are carried by corresponding syntax elements in the SEI message. The per-pixel attenuation map for one image of the video can be carried over as a specific type of auxiliary image and conventionally encoded. Encoding and decoding methods and devices are described.

Description

[Technical Field]

[0001] The present disclosure is in the field of video compression, and at least one embodiment relates more particularly to encoding and decoding video that includes attenuation map information and corresponding parameters that enable reduced energy consumption when using the video, for example when rendering it on a display. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to European Patent Application No. 22306908.9, filed December 16, 2022, European Patent Application No. 23305521.9, filed April 7, 2022, and European Patent Application No. 23306066.4, filed June 29, 2023, which are hereby incorporated by reference in their entireties.

[0003] Reducing the energy consumption of electronic devices has become a requirement not only for electronic device manufacturers but also for those who want to limit their environmental impact as much as possible and contribute to the emergence of a sustainable display industry. The increase in display resolution from SD to HD, then to 4K, and soon to 8K and beyond, as well as the introduction of high dynamic range imaging, has resulted in a corresponding increase in the energy requirements of display devices. This is not consistent with the global need to reduce energy consumption, despite the fact that a huge number of devices (i.e., TVs, mobile phones, tablets, etc.) have displays. In fact, displays are the most significant source of energy consumption for consumer electronic devices, whether battery-powered (e.g., smartphones, tablets, head-mounted displays, car display screens) or not (e.g., television sets, advertising display panels).

[0004] Various display technologies have been developed in recent years. Modern displays consume energy in a more controllable and efficient manner than older displays, but they remain the most significant energy consumer in the video chain.

[0005] As far as backlit displays are concerned, their energy consumption is primarily determined by the intensity of the backlight.

[0006] Organic light-emitting diodes (OLEDs) are an example of a display technology that is increasingly widely used due to numerous advantages over previous technologies, such as thin-film transistor liquid crystal displays (TFT-LCDs). Rather than using a uniform backlight, OLED displays consist of individual, direct-emitting image pixels, similar to mini-LEDs. OLED power consumption is therefore highly correlated to image content, and power consumption for a given input image can be estimated by considering the values ​​of the image pixels being displayed. While OLED displays consume energy in a more controllable and efficient manner, they remain the most significant energy consumer in the video chain.

[0007] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, in which case the difference between the original and predicted picture blocks, often denoted as prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy coding, quantization, transformation, and prediction. Summary of the Invention [Means for solving the problem]

[0008] Generally, at least one example of an embodiment involves new metadata associated with visual content and related to the use of per-pixel attenuation maps dedicated to reducing energy consumption when using the visual content, e.g., when rendering it on a display. Information is provided about the type of display compatible with the use of the attenuation map, the type of pre-processing (e.g., upsampling), the type of operation to use for applying the attenuation map, and several metrics about the expected energy reduction and the expected quality impact of using such attenuation maps. These parameters are carried by corresponding syntax elements in the SEI message. The per-pixel attenuation map for one image of a video may be carried over as an auxiliary image of a specific type and encoded conventionally. Encoding and decoding methods and devices are described.

[0009] A first aspect is directed to a method including the steps of: obtaining encoded data including at least an image, an attenuation map, and a set of parameters, the set of parameters including at least a first parameter representing an operation of applying the attenuation map to the image and a second parameter representing a mapping between components of the attenuation map and image components affected by the operation; determining an attenuated image having reduced component values ​​by applying the attenuation map to the image through an operation based on the first parameter on components of the image selected based on the second parameter; and providing the attenuated image.

[0010] A second aspect is directed to a method comprising the steps of: obtaining an input image of a video; determining an attenuation map based on the input image according to a selected energy reduction rate, wherein applying the attenuation map to the input image reduces values ​​of components of the input image; and generating an encoded video comprising at least the input image, the attenuation map, and a set of parameters, wherein the set of parameters comprises at least a first parameter representing an operation of applying the attenuation map to the image, and a second parameter representing a mapping between components of the attenuation map and image components affected by the operation.

[0011] A third aspect is directed to a device having a processor, the processor being configured to: obtain encoded data including at least an image, an attenuation map, and a set of parameters, the set of parameters including at least a first parameter representing an operation of applying the attenuation map to the image and a second parameter representing a mapping between components of the attenuation map and image components affected by the operation; determine an attenuated image with reduced component values ​​by applying the attenuation map to the image through an operation based on the first parameter on components of the image selected based on the second parameter; and provide the attenuated image.

[0012] A fourth aspect is directed to a device having a processor configured to: obtain an input image of a video; determine an attenuation map based on the input image according to a selected energy reduction rate, where applying the attenuation map to the input image reduces values ​​of components of the input image; and generate an encoded video including at least the input image, the attenuation map, and a set of parameters, where the set of parameters includes at least a first parameter representing an operation of applying the attenuation map to the image and a second parameter representing a mapping between components of the attenuation map and image components affected by the operation.

[0013] A fifth aspect is directed to a non-transitory computer-readable medium containing data content produced according to the second aspect.

[0014] A sixth aspect is directed to a non-transitory computer-readable medium comprising instructions, the instructions being a program that, when executed by a computer, causes the computer to perform the described embodiments associated with the first and second aspects.

[0015] A seventh aspect is directed to a computer program comprising instructions that, when executed by a computer, cause the computer to perform any of the described embodiments or variations relating to the first and second aspects.

[0016] The foregoing presents a simplified summary of the subject matter to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the subject matter. It is not intended to identify key / critical elements of embodiments or to delineate the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description provided below. [Brief explanation of the drawings]

[0017] The present disclosure may be better understood by considering the following detailed description in conjunction with the accompanying drawings. [Figure 1] FIG. 2 is a block diagram of a video encoder according to one embodiment. [Figure 2] FIG. 2 is a block diagram of a video decoder according to one embodiment. [Figure 3] FIG. 1 is a block diagram of an example system in which various aspects and embodiments may be implemented. [Figure 4] 10A-10C are flowcharts of two examples of video encoding using attenuation map information, according to at least one embodiment. [Figure 5] 1 is a flowchart of an example of video decoding using attenuation map information, according to at least one embodiment. [Figure 6] 1 is a flowchart of video decoding using attenuation map information carried by auxiliary pictures of type AUX_ALPHA, according to at least one embodiment. [Figure 7] 1 is a flowchart of video encoding using attenuation map information carried in auxiliary pictures of type AUX_ALPHA with two SEI messages, a first SEI message for alpha_channel_info and another dedicated SEI message for attenuation_map_info, according to at least one embodiment. [Figure 8] 8 is a flowchart of video decoding using attenuation map information carried by the alpha plane with a dedicated SEI message, according to at least one embodiment, corresponding to the encoding of FIG. 7. [Figure 9] 1 is a flowchart of video decoding using attenuation maps based on multiple components carried in separate auxiliary pictures, according to at least one embodiment.

[0018] It should be understood that the drawings are intended to illustrate examples of various aspects, features, and embodiments according to the present disclosure, and are not necessarily the only possible configuration. Like reference numerals indicate the same or similar features throughout the various views. DETAILED DESCRIPTION OF THE INVENTION

[0019] The present aspects describe principles related to particular drafts of the VVC (Versatile Video Coding) or HEVC (High Efficiency Video Coding) specifications, but are not limited to VVC or HEVC, and may be applied, for example, to other standards and recommendations, whether existing or developed in the future, and to extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically precluded, the aspects described in this application may be used individually or in combination.

[0020] 1 shows a block diagram of a video encoder according to one embodiment. While variations of this encoder 100 are contemplated, the encoder 100 is described below for clarity without describing all possible variations. Before being encoded, a video sequence may undergo encoding pre-processing (101), such as applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0021] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (102) and processed, for example, in units of a CU. Each unit is encoded, for example, using either intra mode or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder determines (105) whether intra mode or inter mode should be used to encode the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (110) the predicted block from the original image block.

[0022] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is coded directly without applying the transform or quantization processes.

[0023] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. Combining the decoded prediction residual with the predicted block (155) reconstructs an image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / sample adaptive offset (SAO), adaptive loop filter (ALF) filtering, and to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0024] FIG. 2 shows a block diagram of a video decoder according to one embodiment. In the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass. The encoder 100 also generally performs video decoding as part of encoding the video data. In particular, the decoder's input includes a video bitstream that may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. Picture partition information (partitioning information) indicates how the picture is partitioned. Thus, the decoder can divide the picture according to the decoded picture partition information (235). The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. Combining the decoded prediction residual with the predicted block (255) reconstructs an image block. The predicted block (270) may be obtained from intra-prediction (260) or motion-compensated prediction (i.e., inter-prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0025] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transformation (e.g., YCbCr 4:2:0 to RGB 4:4:4 conversion) or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding processing (101). The post-decoding processing may use metadata derived in the pre-encoding processing and signaled in the bitstream.

[0026] FIG. 3 shows a block diagram of an example system in which various aspects and embodiments may be implemented. System 1000 may be embodied as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000, singly or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, processing elements and / or encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, e.g., via a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the aspects described herein.

[0027] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein, for example, to implement various aspects described herein. The processor 1010 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes storage device 1040, which may include non-volatile and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drives, and / or optical disk drives. Storage device 1040 may include, by way of non-limiting example, internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0028] System 1000 includes, for example, an encoder / decoder module 1030 configured to process data to provide encoded or decoded video, which may include its own processor and memory. Encoder / decoder module 1030 represents a module or modules that may be included in a device for performing encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated within processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0029] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to implement various aspects described herein may be stored in the storage device 1040 and then loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of a variety of items during performance of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing of equations, expressions, operations, and operational logic.

[0030] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be the memory 1020 and / or the storage device 1040, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store the operating system, for example, of a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations such as MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, JVET, a new standard being developed by the Joint Video Experts Team).

[0031] Input to the elements of system 1000 may be provided through various input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives an RF signal, e.g., transmitted over the air by a broadcaster, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Other examples not shown in FIG. 3 include composite video.

[0032] In various embodiments, the input devices of block 1130 have associated respective input processing elements known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) further band-limiting to a narrower band of frequencies to select a signal frequency band, which may be referred to as a channel in certain embodiments (for example), (iv) demodulating the downconverted, band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner for performing a variety of these functions, including downconverting a received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements perform frequency selection by receiving, filtering, downconverting, and filtering again to the desired frequency band an RF signal transmitted over a wired (e.g., cable) medium. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as, for example, inserting an amplifier and an analog-to-digital converter. In various embodiments, the RF section includes an antenna.

[0033] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 1000 to other electronic devices over USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, as desired, for example, in a separate input processing IC or within processor 1010. Similarly, aspects of the USB or HDMI interface processing may be implemented, as desired, in a separate interface IC or within processor 1010. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including processor 1010 and an encoder / decoder 1030 operating in combination with memory and storage elements, for processing the data stream as desired, for presentation on an output device.

[0034] The various elements of system 1000 may be provided within an integrated housing in which the various elements may be interconnected and transmit data between them using a suitable connection arrangement 1140, e.g., an internal bus as known in the art, including an inter-IC (I2C) bus, wiring, and printed circuit boards.

[0035] System 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented in a wired and / or wireless medium, for example.

[0036] In various embodiments, data is streamed or otherwise provided to system 1000 using a wireless network, such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received over communication channel 1060 and communication interface 1050 adapted for Wi-Fi communication. Communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 1000 using a set-top box that delivers data over the HDMI connection of input block 1130. Still other embodiments provide streamed data to system 1000 using the RF connection of input block 1130. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., cellular networks or Bluetooth networks.

[0037] System 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 may be for a television, a tablet, a laptop, a cell phone, or other device. The display 1100 may also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). The other peripheral devices 1120, in various example embodiments, include one or more of a standalone digital video disc (or digital versatile disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disk player performs the function of playing the output of the system 1000 .

[0038] In various embodiments, control signals are communicated between system 1000 and display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable inter-device control with or without user intervention. Output devices may be communicatively coupled to system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 using communication channel 1060 via communication interface 1050. Display 1100 and speakers 1110 may be integrated into a single unit with other components of system 1000 in an electronic device, such as, for example, a television. In various embodiments, display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0039] The display 1100 and speakers 1110 may alternatively be separate from one or more of the other components, for example, if the RF portion of the input 1130 is part of a separate set-top box. In various embodiments in which the display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0040] The embodiments may be performed by computer software implemented by the processor 1010, by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, by way of non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, by way of non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0041] ISO / IEC 23001-11 specifies metadata (Green Metadata) that facilitates the reduction of energy usage during media consumption, and in particular demonstrates how power consumption on the display side should be reduced by using display adaptation mechanisms. The metadata for display adaptation defined in the Green Metadata specification is designed for specific display technologies, and is particularly well-tuned for transmissive display technologies that incorporate backlight illumination, such as LCD displays. These metadata are designed to achieve display energy reduction by using display adaptation techniques. They consist of metrics made from RGB component statistics and quality indicators of the video content. They can be used to perform RGB picture component rescaling to set the best compromise between backlight / voltage reduction and picture quality. The specified metadata differs depending on whether the use case is point-to-point or point-to-multipoint transmission.

[0042] Such statistics conveyed in these metadata can be useful in the context of OLED screens, where the power consumption of the OLED screen is assumed to be proportional to the luminance. However, this approach is far from optimal because these metadata convey global information. Furthermore, they do not convey information that would guide the use of per-pixel attenuation maps.

[0043] Since the publication of the ISO / IEC 23001-11 document, new emissive technologies have been introduced with the widespread adoption of emissive OLED displays, which allow for more efficient pixel-by-pixel control of their energy consumption, making it possible to reduce energy consumption, for example.

[0044] The User Data Supplemental Enhancement Information (SEI) message defined by ITU-T Recommendation T.35 (ISO / IEC FDIS 23002-7 Information technology - MPEG video techniques - Part 7: Versatile Supplemental Enhancement Information messages for coded video bitstreams), in combination with ISO / IEC 14496-10 for AVC or ISO / IEC DIS 23008-2 for HEVC or ISO / IEC FDIS 23002-7 for VVC, makes it possible to convey additional information that can be received in precise synchronization with the associated audio and video content.

[0045] Some of the metadata conventionally carried on the SEI messages defined by the T.35 recommendation may be related to display adaptation.

[0046] Implementation of the per-pixel dimming process of an image to reduce energy is typically done either on the receiver side (e.g., a post-processing operation) or on the server side (e.g., a pre-processing operation), and as a result has several limitations and drawbacks.

[0047] Part of this processing can be done independently of any information provided at the decoder or display side. Therefore, to better align with the goal of limiting the impact of the video chain on climate change, the embodiments described herein propose to factorize the construction of the attenuation map only once at the encoder side, rather than multiplexing the processing at each device or server in the transmission chain, and to send this map together with the content in the form of some auxiliary picture. In this case, accompanying metadata is also required in the encoded bitstream to provide additional information for its use at the receiver side.

[0048] Furthermore, the method for calculating the attenuation map may be costly. In this case, splitting the whole process into two steps (e.g., first, calculation of the attenuation map on the encoder side, and second, its use on the decoder side) may advantageously enable its use on devices with low computing resources, such as smartphones. Furthermore, information such as the display type may, however, be useful to adapt the application of the attenuation map to the image to be displayed. This suggests a two-step process.

[0049] If performed entirely on the receiver or display side, the resulting image after energy reduction will not necessarily respect the intention of the content creator, since some areas may be affected by the application of a dimming map created without any control from the content creator, and the quality of experience may be degraded. On the other hand, if prepared during content creation, the content creator can evaluate the compliance of the processing with their quality of experience requirements and indicate areas that should not be affected by the processing.

[0050] The embodiments described below are designed with the above in mind and propose to solve these problems by defining new metadata related to the use of per-pixel attenuation maps associated with visual content and dedicated to reducing energy consumption when using the visual content, e.g., when rendering it on a display. For example, information about the type of display compatible with the use of the attenuation map, the type of pre-processing (e.g., upsampling), and the type of operation to use for the application of the attenuation map, as well as some metrics about the expected energy reduction and the expected quality impact of using such attenuation maps, are provided and specified as syntax elements in the SEI message. The per-pixel attenuation map for one image of a video can be carried over as an auxiliary image of a certain type and encoded conventionally.

[0051] The granularity of the signaled metadata may be based on time (i.e., data is signaled per period / duration of the video content), temporal layer (i.e., data is signaled per temporal layer), slice type (intra-slice and inter-slice), or part of the picture (slice, tile). The SEI message is generated at the encoder based on parameters related to the attenuation map. The encoder signals this information in the SEI message, which is also included in the bitstream. The SEI message and attenuation map are decoded by the decoder, and the decoded image is modified to reduce energy consumption.

[0052] The attenuation map is designed so that when applied to an input image, it produces a modified image that requires less energy to display than the input image. One simple implementation is to scale down the luminance according to a selected energy reduction rate. More complex implementations take into account other parameters, such as the similarity between the modified image and the input image, or the contrast sensitivity function of human vision, or smoothness properties that allow the attenuation map to be downscaled without introducing heavy artifacts when upscaling it at the decoder side.

[0053] Applying an attenuation map to an image is based on combining them according to a selected type of operation. The values ​​of the samples of the attenuation map and the type of operation are closely related. In fact, when the operation is addition, the attenuation map contains sample values ​​with negative values, when the operation is subtraction, the attenuation map contains sample values ​​with positive values, and when the operation is multiplication, the attenuation map contains floating-point sample values ​​in the range between 0 (pixel becomes black) and 1 (no attenuation).

[0054] Auxiliary pictures are defined in ISO / IEC 23002-3 Auxiliary Video Data Representation. Table 1 shows the addition of a new specific type of auxiliary picture named "AUX_ATTENUATION" that will be used in conjunction with the new metadata defined above. This new type is in addition to the previous types of auxiliary pictures related to alpha plane and picture depth information.

[0055] [Table 1]

[0056] Table 2 below provides a basic example of metadata, including parameters necessary for the use of attenuation maps carried by SEI messages and sent as auxiliary pictures in the bitstream.

[0057] In this example, it is considered that some metadata are sent globally for the entire bitstream, i.e., they are shared for all auxiliary pictures of type "attenuation map." These metadata may also be shared over a period, where the concept of period can correspond, for example, to a picture, an intra-period, a group of pictures (GOP), a number of pictures, a time duration, and all. In that case, an SEI message is generally inserted per period. It is transmitted at the start of the next period. The next message containing the metadata will be transmitted at the start of the next period. Thus, when the next period is a picture, a message will be transmitted for each picture. However, when the next period is a specified time interval or a specified number of pictures, the associated message will be transmitted with the first picture in the time interval or with the first picture in the specified number of pictures.

[0058] In another embodiment, some of the information related to auxiliary pictures of type attenuation map may be sent globally for the entire bitstream (e.g., information related to the display model compatible with the use of the attenuation map), and other information may be sent, for example, with different periodicity for each picture.

[0059] [Table 2]

[0060] The metadata in Table 2 can be explained as follows:

[0061] - ami_cancel_flag is used to cancel the persistence of any previous Attenuation Map Information SEI message, thus providing a kind of reset information to the decoder. When this flag is equal to 1, it indicates that the SEI message cancels the persistence of any previous Attenuation Map Information SEI message in the output order. When equal to 0, it indicates that Attenuation Map Information parameters follow.

[0062] - ami_display_model indicates for which type of display technology the attenuation map should be applied at the receiver side. This metadata is a bitfield mask that indicates for which display model the attenuation map sample values ​​of the auxiliary picture are for, as shown in Table 3. For example, ami_display_model="0011" means that the attenuation map information can be used for both the "backlit pixel" (i.e., backlit) display model and the "emissive pixel" display model.

[0063] [Table 3]

[0064] - ami_global_flag indicates whether all subsequent information data needs to be redefined for each decoded auxiliary picture of type AUX_ATTENUATION. When this flag is equal to 0, it indicates that for i=0..ami_map_number, ami_attenuation_use_idc[i], ami_attenuation_comp_idc[i], ami_preprocessing_flag[i], ami_preprocessing_type_idc[i], ami_preprocessing_scale_idc[i], ami_backlight_scaling_idc[i] shall be present.

[0065] When this flag is equal to 1, it indicates that only ami_attenuation_use_idc[0], ami_attenuation_comp_idc[0], ami_preprocessing_flag[0], ami_preprocessing_type_idc[0], ami_preprocessing_scale_idc[0], and ami_backlight_scaling_idc[0] shall be present.

[0066] - ami_map_number indicates the number of attenuation maps contained in the metadata. Indeed, if the end user wants to use attenuation maps with a reduction rate different from the provided reduction rate(s), it may be interesting to send more than one map that will serve as control points in the further interpolation process.

[0067] - ami_map_approximation_model indicates which type of interpolation model should be used to infer an attenuation map with a reduction rate different from the reduction rate(s) corresponding to the attenuation map(s) sent in the metadata. When this parameter is equal to 0, linear scaling of the attenuation map sample values ​​of the provided auxiliary picture with its respective ami_energy_reduction_rate is considered to obtain the corresponding attenuation map sample values ​​for another energy reduction rate. If several auxiliary pictures of type attenuation map are provided, the auxiliary picture with the lowest ami_energy_reduction_rate is used for linear scaling. When this parameter is equal to 1, bilinear interpolation between the attenuation map sample values ​​of the provided auxiliary picture(s) with their respective ami_energy_reduction_rate should be considered to obtain the corresponding attenuation map sample values ​​for another energy reduction rate. Similarly, other values ​​of this parameter will specify the use of other models for interpolation, as summarized in Table 4 below.

[0068] [Table 4]

[0069] ami_layer_id[i] specifies the identifier of the decoder layer for the attenuation map with index i.

[0070] - ami_ols_number[i] specifies the number of the output layer set to which the decoder layer for the attenuation map with index i belongs.

[0071] ami_ols_id[i][j] specifies the identifier of the output layer set with index j for the attenuation map with index i. This identifier is used to select the output layer set for outputting both the primary decoded picture and the attenuation map with index i.

[0072] ami_energy_reduction_rate[i] indicates, for a given attenuation map, the corresponding level of energy reduction that can be expected from the use of the attenuation map. The value of this parameter specifies the energy reduction rate expressed as a percentage.

[0073] - ami_video_quality[i] indicates, for a given attenuation map, the corresponding quality that can be expected after using the attenuation map to reduce the energy of the decoded image. ami_video_quality[i] can be, for example, a PSNR, V-MAF, or SSIM value. Such quality metrics can be calculated by the decoder, but to reduce energy consumption, they can also be inferred on the encoder side. In this case, they can correspond to the minimum expected quality value.

[0074] ami_max_value[i] indicates the maximum value of the attenuation map with index i. Such a maximum value may optionally be used in the scaling process to further adjust the dynamics of the encoded attenuation map.

[0075] ami_attenuation_use_idc[i] indicates what type of processing should be used to apply the transmitted attenuation map of index i to the decoded picture to be displayed, as summarized in Table 5 below. For example, the attenuation map may be combined with the decoded picture by subtraction, addition, multiplication, or division, so that it reduces the level of the pixel values ​​of the decoded image. When this parameter is equal to 0, the attenuation map sample value of the decoded auxiliary picture of index i should be added to one or more associated primary picture decoded samples before being displayed on the screen. This implies that the attenuation map value is negative. When this parameter is equal to 1, the attenuation map sample value of the decoded auxiliary picture should be subtracted from one or more associated primary picture decoded samples. When this parameter is equal to 2, the attenuation map sample value of the decoded auxiliary picture should be multiplied by one or more associated primary picture decoded samples before being displayed on the screen. When this parameter is equal to 3, the decoded sample(s) should be divided by the associated attenuation map sample value of the decoded auxiliary picture before being displayed on the screen. When this parameter is equal to 4, the attenuation map sample value of the decoded auxiliary picture should be used in the context of a Contrast Sensitivity Function to determine the attenuation to be applied to one or more associated primary picture decoded samples before being displayed on the screen. When this parameter is equal to 5, the attenuation map sample value of the decoded auxiliary picture should be used according to a proprietary user defined process to modify one or more associated primary picture decoded samples before being displayed on the screen.

[0076] [Table 5]

[0077] The range and type of the attenuation map value depends on the type of combination: a floating point value less than or equal to 1.0 when the combination is multiplication, a floating point value greater than or equal to 1.0 when the combination is division, a positive integer value when the combination is subtraction, or a negative integer value when the combination is addition.

[0078] - ami_attenuation_comp_idc[i] specifies to which color component(s) of the associated primary picture decoded sample(s) the decoded auxiliary picture of type AUX_ATTENUATION with index i should be applied using the process defined by ami_attenuation_use_idc[i]. It also specifies how many components the decoded auxiliary picture with index i should contain.

[0079] When equal to 0, the decoded auxiliary picture of type AUX_ATTENUATION with index i contains only one component, which should be applied to the luma component of the associated primary picture decoded sample(s).

[0080] When equal to 1, the decoded auxiliary picture of type AUX_ATTENUATION at index i contains only one component, which should be applied to the luma and chroma components of the associated primary picture decoded sample(s).

[0081] When equal to 2, the decoded auxiliary picture of type AUX_ATTENUATION at index i contains only one component, which should be applied to the RGB components (after YUV to RGB conversion) of the associated primary picture decoded sample(s).

[0082] When equal to 3, the decoded auxiliary picture of type AUX_ATTENUATION at index i contains two components, the first component of which should be applied to the luma component of the associated primary picture decoded sample(s), and the second component of which should be applied to both chroma components of the associated primary picture decoded sample(s).

[0083] When equal to 4, the decoded auxiliary picture of type AUX_ATTENUATION at index i contains three components, which should be applied to the luma and chroma components, respectively, of the associated primary picture decoded sample(s).

[0084] When equal to 5, the decoded auxiliary picture of type AUX_ATTENUATION at index i contains three components, which should be applied respectively to the RGB components (after YUV to RGB conversion) of the associated primary picture decoded sample(s).

[0085] When equal to 6, the mapping between the component of the decoded auxiliary picture of type AUX_ATTENUATION at index i and its component to which the decoded auxiliary picture of type AUX_ATTENUATION at index i should be applied corresponds to some proprietary user-defined process, which is summarized in Table 6 below.

[0086] [Table 6]

[0087] ami_preprocessing_flag[i] When this flag is true, it specifies the use of preprocessing on the attenuation map sample values ​​of the decoded auxiliary picture with index i, assuming that the preprocessing is an upsampling operation and that the auxiliary coded picture(s) and the primary coded picture have different sizes.

[0088] - ami_preprocessing_type_idc[i] indicates what type of preprocessing should be applied to the attenuation map sample values ​​of the decoded auxiliary picture with index i. When this parameter is equal to 0, the interpolation between the attenuation map sample values ​​of the provided auxiliary picture with index i that are considered to obtain the attenuation map sample values ​​to be applied to the sample values ​​of the decoded picture is a bicubic interpolation to obtain the same resolution as the resolution of the associated decoded picture. When this parameter is equal to 1, the interpolation is a bilinear interpolation, and when it is equal to 2, the interpolation is of Lanczos type. When this parameter is equal to 3, a proprietary user-defined process should be used. This is summarized in Table 7 below.

[0089] [Table 7]

[0090] - ami_preprocessing_scale_idc[i] specifies the scaling that should be applied to the attenuation map sample with index i to obtain the attenuation map sample values ​​(float) before applying them to the decoded picture sample values. When this parameter is equal to 0, a scaling of 1 / 255 should be applied. When this parameter is equal to 1, a proprietary user-defined scaling should be used. This is summarized in Table 8 below.

[0091] [Table 8]

[0092] - ami_backlight_scaling_idc[i] specifies the process for calculating the backlight scaling factor for a transmissive pixel display, derived from the attenuation map sample values ​​of the decoded auxiliary picture with index i.

[0093] When equal to 0, the scaling to apply to the display backlight is calculated as the ratio between the maximum value of the associated primary picture decoded sample after and before applying the attenuation map sample value of the decoded auxiliary picture of index i. The associated primary picture decoded sample(s) to which the attenuation map sample value of the decoded auxiliary picture of index i is applied are further rescaled to their maximum value before application of the attenuation map sample value of the decoded auxiliary picture of index i.

[0094] When greater than 0, the scaling to be applied to the display backlight is determined according to a proprietary user-defined process derived from the attenuation map sample values ​​of the decoded auxiliary picture with index i, as summarized in Table 9 below.

[0095] [Table 9]

[0096] If the periodicity of the SEI message corresponds to a portion of a picture, additional metadata is added that defines to which region of the decoded picture the attenuation map sample values ​​of the auxiliary picture should be applied. In this case, the syntax is modified as shown in Table 10 below.

[0097] [Table 10]

[0098] The new metadata ami_box_xstart, ami_box_ystart, ami_box_width, ami_box_height define the position and size of a bounding box that defines the area of ​​the decoded picture to which the attenuation map should be applied, e.g., the x-coordinate, y-coordinate of the top-left video corner, the width, and the height of the bounding box, respectively. The attenuation map will not be applied outside this bounding box.

[0099] An alternative to this embodiment for region-based attenuation maps is to set the sample values ​​of the attenuation map outside the region where the attenuation map should be applied to 0 (if these samples are added or subtracted) or 1 (if these samples are multiplied), in which case these additional metadata are not required.

[0100] FIG. 4 shows a flowchart of two examples of video encoding using attenuation map information according to at least one embodiment. This encoding process 400 is implemented, for example, by the processor 1010 in the encoder 100 of FIG. 1 or the device 1000 of FIG. 3. The diagram illustrates an SEI message generation and bitstream encapsulation process according to one embodiment, performed, for example, during encoding, according to the syntax introduced above. In this embodiment, the SEI message is inserted for a given picture, i.e., the considered period corresponds to one picture. In step 410, the device conventionally encodes the picture, resulting in a partial bitstream. In step 420, the device performs conventional partial bitstream decoding. In step 430, an attenuation map corresponding to the decoded picture is calculated for a selected energy reduction rate. In step 440, data corresponding to the parameters listed in Table 1 is collected, such as data regarding the use of the attenuation map, the expected energy reduction, the corresponding expected quality, preprocessing operations, etc. In step 450, auxiliary pictures corresponding to the attenuation map are generated and in step 460, encoded and inserted into the partial bitstream. Once all the data are collected, they are inserted into an SEI message in step 470. The SEI message is encoded and inserted into the final bitstream in step 480. In variant embodiments, the order of some of the steps may be changed while still relying on the same principles. For example, all encoding steps may be performed in the same step. In another example, step 440 is performed between steps 460 and 470.

[0101] As previously introduced, this bitstream includes at least the video image and, in relation to this image, a decay map corresponding to a selected energy reduction rate. This enables determining from this bitstream a video image that will enable a decoding device (or display device) to reduce energy consumption when using (e.g., displaying) the video.

[0102] In at least one embodiment, multiple decay maps are calculated respectively for different energy reduction rates. For example, two decay maps with energy reduction rates R1 and R2 can be calculated. This enables interpolating the corresponding decay map on the decoder side for any other reduction rate R, e.g., R1 < R < R2. Process 401 in FIG. 4 shows a flowchart of such an embodiment. Most of the steps are the same as those of process 400. The difference relates to the iteration 425 that is performed for the selected set of energy reduction rates. Thus, a decay map is generated for each of the energy reduction rates and the corresponding set of auxiliary pictures is inserted into the bitstream together with the information data associated with them.

[0103] As a result of this embodiment, the bitstream includes at least the video image and, in relation to this image, a set of different decay maps corresponding to the selected energy reduction rates. This enables determining from this bitstream a video image that will enable a decoding device (or display device) to reduce energy consumption when displaying the video according to an energy reduction rate not in the list of reduction rates, thanks to the multiple decay maps and parameters in the SEI message of the bitstream.

[0104] Again, still based on the same principle, the order of some of the steps can be changed. For example, all auxiliary pictures can be inserted at once, outside the loop on the energy reduction rate. Another example consists of inserting an SEI message for each auxiliary picture, i.e., inside the loop on the energy reduction rate.

[0105] In another embodiment, the calculation of the attenuation map and the collection of the associated metadata are realized outside the encoder, for example in a dedicated device, and these data are stored, for example in a database accessible by the encoder. These additional data are then provided to the encoder along with the input video in a process similar to processes 400 and 401.

[0106] In another embodiment, auxiliary data is sent without any accompanying SEI message and therefore without having parameters in the bitstream to benefit from attenuation maps. This use case targets a specific decoding device with a predetermined behavior. An example of such a device is an advertisement display panel. In this case, the default mode is defined with default values ​​for these parameters. There is persistence of these values ​​for the entire bitstream. The default values ​​are shown in Table 11.

[0107] [Table 11]

[0108] 5 shows a flowchart of an example of video decoding using attenuation map information, according to at least one embodiment. This decoding process 500 may be implemented, for example, by the decoder 200 of FIG. 1, by the processor 1010 in the device 1000 of FIG. 3, or by various electronic devices, such as smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, personal computers, laptop computers, and servers.

[0109] As introduced above, the use of the attenuation map is guided by the accompanying metadata carried by the SEI message. Process 500 can be applied to a picture, or to a group of pictures, or to a part of a picture (slice, tile), according to the level of signaling in the SEI message, as described above. The following description shows the case where the process is applied to a single picture and there is only one attenuation map, but the other cases are similar and based on the same steps.

[0110] In step 510, a picture is decoded from the bitstream to produce a decoded picture 511. In step 520, SEI message data is extracted from the bitstream for the picture, providing different parameters as described according to the syntax set forth in Table 2. In step 525, the end device's display model 521 is checked against the decoded parameter ami_display_model. If the end device's display model 521 does not match the ami_display_model parameter, the decoded picture 511 is sent directly to the display in step 530 without the use of an attenuation map. Otherwise, in step 535, a mapping between auxiliary data of type attenuation map and its corresponding decoded picture is performed by using ami_ols_id[i][j], which gives the identifier of the layer in the bitstream that contains the decoded picture and attenuation map. Within these identifiers, ami_layer_id[i] gives the layer id in which the attenuation map is located. In step 540, auxiliary data of type attenuation map corresponding to the picture is decoded to produce a decoded attenuation map 541. In step 550, ami_preprocessing_flag is checked to obtain information whether upsampling should be applied to the decoded attenuation map 541. In step 560, if upsampling should be applied, the decoded attenuation map 541 is upsampled according to a process given by the parameter ami_preprocessing_idc. In step 565, it is further rescaled according to any scaling factors described by ami_preprocessing_scale_idc and ami_max_value.Then, in step 580, the upsampled, rescaled attenuation map is applied to the decoded picture 511 according to the process described by ami_attenuation_use_idc and ami_attenuation_comp_idc to produce an energy-reduced image 581, which is further sent to the display in step 590. If upsampling of the decoded attenuation map 541 is not required, it is first rescaled according to the scaling factor described by ami_preprocessing_scale_idc and ami_max_value in step 555. The rescaled attenuation map is then applied in step 570 to the decoded picture 511 according to the process described by ami_attenuation_use_idc and ami_attenuation_comp_idc to produce an energy-reduced image 581, which is further sent to the display in step 590. Optionally, if the device display is a backlit display, the ami_backlight_scaling_idc information is further sent to the display, resulting in further scaling of the display backlight and rescaling of the reduced image depending on the attenuation map according to the process described by ami_backlight_scaling_idc.

[0111] In at least one embodiment, a single bit field named ami_flags is used to convey multiple information flags.

[0112] - bit 0: Indicates that the SEI message cancels the persistence of any previous Attenuation Map Information SEI message in the output order. It corresponds to the ami_cancel_flag defined previously.

[0113] - Bit 1: Indicates whether for each decoded auxiliary picture of type AUX_ATTENUATION all subsequent information data in the message need to be redefined. It corresponds to the ami_global_flag defined previously.

[0114] - Bit 2: Indicates whether subsequent attenuation maps in the message can be used to approximate other attenuation maps for other reduction rates. It corresponds to the new ami_approximation_flag.

[0115] - Bit 3: Indicates whether preprocessing is required to use subsequent attenuation maps in the message. It corresponds to the new ami_preprocessing_global_flag.

[0116] - bit 4: Indicates that the attenuation map in the message shall be applied to a region of the primary video defined by the x, y coordinates of the top-left corner and the width and height, respectively, of the region bounding box, which corresponds to the new ami_box_flag.

[0117] - Bit 5: Indicates that the attenuation map in the message shall be used to derive any backlight scaling. It corresponds to the new ami_backlightscaling_flag.

[0118] - Bits 6-7: Reserved for future use.

[0119] The ami_flags bit fields are summarized in Table 12.

[0120] [Table 12]

[0121] The ami_flags bit field allows reducing the size of the SEI message by not sending metadata information that is not required to use and apply the attenuation map. In this case, the syntax in Table 10 is modified as shown in Table 13 below.

[0122] [Table 13]

[0123] In at least one embodiment, rather than grouping metadata for multiple attenuation maps into one SEI, one SEI message is sent per attenuation map. Thus, if three attenuation maps need to be provided, three SEI messages also need to be provided. However, in this embodiment, each SEI message is simpler. In fact, the metadata ami_global_flag and ami_map_number are no longer required. The encoding process remains similar to processes 400 and 401 of FIG. 4. The syntax for such an embodiment is shown in Table 14.

[0124] [Table 14]

[0125] In at least one embodiment, the ami_global_flag is not used. In such an embodiment, all metadata is sent for all attenuation maps in the ami_map_number. The syntax is then modified as shown in Table 15 below.

[0126] [Table 15]

[0127] In at least one embodiment, the attenuation map is carried on top of the existing alpha plane picture. In other words, rather than using an sdi_aux_id of value 3, which corresponds to the new type AUX_ATTENUATION, an sdi_aux_id of value 1 is used to refer to the use of an auxiliary picture of type AUX_ALPHA. Such pictures are traditionally used for alpha blending, i.e., to overlay a second (alpha) image on top of a first image according to transparency level.

[0128] Such an auxiliary picture of type AUX_ALPHA is accompanied by a standardized SEI message containing the following metadata as defined in documents ISO / IEC 23002-3 or ITU-T H.265, ITU-T H.274:

[0129] alpha_channel_cancel_flag equal to 1 indicates that the SEI message cancels the persistence of any previous ACI SEI message in the output order applied to the current layer. alpha_channel_cancel_flag equal to 0 indicates that the ACI continues.

[0130] alpha_channel_use_idc equal to 0 indicates that for alpha blending purposes, the decoded samples of the associated primary picture should be multiplied by the interpretation sample values ​​of the decoded auxiliary picture in the display process after output from the decoding process. alpha_channel_use_idc equal to 1 indicates that the decoded samples of the associated primary picture should not be multiplied by the interpretation sample values ​​of the decoded auxiliary picture in the display process after output from the decoding process. alpha_channel_use_idc equal to 2 indicates that the usage of the auxiliary picture is not specified. Values ​​greater than 2 for alpha_channel_use_idc are reserved for future use by ITU-T|ISO / IEC. When not present, the value of alpha_channel_use_idc is inferred to be equal to 2. Decoders shall ignore Alpha Channel Information SEI messages with alpha_channel_use_idc greater than 2.

[0131] alpha_channel_bit_depth_minus8 plus8 specifies the bit depth of the samples in the luma sample array of the auxiliary picture. alpha_channel_bit_depth_minus8 plus8 shall be equal to the bit depth of the associated primary picture.

[0132] alpha_transparent_value specifies the interpretation sample value of a decoded auxiliary picture luma sample at which the associated luma and chroma samples of the primary coded picture are considered transparent for the purposes of alpha blending. The number of bits used to represent the alpha_transparent_value syntax element is alpha_channel_bit_depth_minus8+9.

[0133] alpha_opaque_value specifies the interpretation sample value of a decoded auxiliary picture luma sample at which the associated luma and chroma samples of the primary coded picture are considered opaque for the purposes of alpha blending. The number of bits used to represent the alpha_opaque_value syntax element is alpha_channel_bit_depth_minus8+9.

[0134] A value of alpha_opaque_value equal to alpha_transparent_value indicates that the auxiliary coded picture is not intended for alpha blending purposes.

[0135] For purposes of alpha blending, the alpha_opaque_value may be greater than the alpha_transparent_value, or it may be less than or equal to the alpha_transparent_value.

[0136] alpha_channel_incr_flag equal to 0 indicates that the interpretation sample value for each decoded auxiliary picture luma sample value is equal to the decoded auxiliary picture sample value for the purposes of alpha blending. sub_channel_incr_flag equal to 1 indicates that, for the purposes of alpha blending, after decoding the auxiliary picture sample, any auxiliary picture luma sample value greater than Min(alpha_opaque_value,alpha_transparent_value) should be incremented by 1 to obtain the interpretation sample value for the auxiliary picture sample, and any auxiliary picture luma sample value less than or equal to Min(alpha_opaque_value,alpha_transparent_value) should be used as the interpretation sample value for the decoded auxiliary picture sample value without modification.

[0137] When alpha_transparent_value is equal to alpha_opaque_value or Log2(Abs(alpha_opaque_value-alpha_transparent_value)) does not have an integer value, alpha_channel_incr_flag shall be equal to 0.

[0138] alpha_channel_clip_flag equal to 0 indicates that no clipping operation is applied to obtain the interpretation sample values ​​of the decoded auxiliary picture. alpha_channel_clip_flag equal to 1 indicates that the interpretation sample values ​​of the decoded auxiliary picture are modified according to the clipping process described by the alpha_channel_clip_type_flag syntax element.

[0139] alpha_channel_clip_type_flag equal to 0 indicates that for the purposes of alpha blending, after decoding the auxiliary picture samples, any auxiliary picture luma sample greater than (alpha_opaque_value+alpha_transparent_value) / 2 is set equal to Max(alpha_transparent_value,alpha_opaque_value) to obtain the interpretation sample value for the auxiliary picture luma sample, and any auxiliary picture luma sample less than or equal to (alpha_opaque_value+alpha_transparent_value) / 2 is set equal to Min(alpha_transparent_value,alpha_opaque_value) to obtain the interpretation sample value for the auxiliary picture luma sample. alpha_channel_clip_type_flag equal to 1 indicates that for the purposes of alpha blending, after decoding the auxiliary picture sample, any auxiliary picture luma sample greater than Max(alpha_transparent_value, alpha_opaque_value) will be set equal to Max(alpha_transparent_value, alpha_opaque_value) to obtain an interpretation sample value for the auxiliary picture luma sample, and any auxiliary picture luma sample less than or equal to Min(alpha_transparent_value, alpha_opaque_value) will be set equal to Min(alpha_transparent_value, alpha_opaque_value) to obtain an interpretation sample value for the auxiliary picture luma sample.

[0140] According to document ITU-T H.265, alpha_channel_use_idc equal to 2 indicates that auxiliary picture usage is not specified. Values ​​greater than 2 for alpha_channel_use_idc are reserved for future use by ITU-T|ISO / IEC. When absent, the value of alpha_channel_use_idc equal to 2 is inferred.

[0141] In the current variant proposed, alpha_channel_use_idc is set equal to 2 and the same additional metadata dedicated to attenuation maps, except for ami_cancel_flag, is added to the SEI message for the alpha map, as shown in Table 16.

[0142] [Table 16]

[0143] The encoding process for such an embodiment is similar to process 400 or 401 of Figure 4, except for some minor modifications. First, the auxiliary picture generated in step 450 is an auxiliary picture of type AUX_ALPHA (i.e., sdi_aux_id equals 1), and the SEI message generated in step 470 initially sets the value of alpha_channel_use_idc equal to 2. Other values ​​associated with auxiliary pictures of type AUX_ALPHA are set as follows:

[0144] - alpha_channel_bit_depth_minus8: any value - alpha_transparent_value: any value - alpha_opaque_value=alpha_transparent_value - alpha_channel_incr_flag=0 - alpha_channel_clip_flag=0 - alpha_channel_clip_type_flag: No need to send It then adds metadata related to the attenuation map to the SEI message.

[0145] In a variant embodiment, the syntax of Table 16 is adapted to support the case where the periodicity of the SEI message corresponds to a portion of a picture. In this case, the syntax also includes information related to the region to which the attenuation map should be applied (xstart, ystart, width height), similar to the syntax of Table 10 or Table 13.

[0146] 6 shows a flowchart of an example of video decoding using attenuation map information carried by an auxiliary picture of type AUX_ALPHA, according to at least one embodiment. This decoding process 600 may be implemented, for example, by the decoder 200 of FIG. 2, by the processor 1010 in the device 1000 of FIG. 3, or by various electronic devices, such as smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, personal computers, laptop computers, and servers. This decoding process 600 is generally similar to the decoding process 500 of FIG. 5. The differences depend on the type of auxiliary data and the SEI message. There are three differences in the decoding steps. First, in step 620, the processor decodes the SEI message corresponding to the auxiliary picture of type AUX_ALPHA (i.e., decodes the alpha_channel_info listed in Table 16 instead of the attenuation_map_info in Table 2). Second, an additional test in step 623 ensures that alpha_channel_use_idc is set to 2 before performing the energy reduction process. If not (the "No" branch), conventional decoding is performed, possibly using conventional alpha blending operations. Third, attenuation map data is decoded from auxiliary pictures of type AUX_ALPHA rather than from auxiliary pictures of type AUX_ATTENUATION. The other steps are based on the same principles as the corresponding steps in Figure 5.

[0147] 7 shows a flowchart of an example of video encoding using attenuation map information carried by an auxiliary picture of type AUX_ALPHA with two SEI messages, namely, a first SEI message for alpha_channel_info and another dedicated SEI message for attenuation_map_info, according to at least one embodiment. In such an embodiment, the attenuation map is encoded using the alpha plane, more specifically in an auxiliary picture of type AUX_ALPHA, but the metadata is transmitted through attenuation_map_info in Table 2 instead of alpha_channel_info as set forth in Table 16. This is done by first setting the alpha map-related SEI messages for this auxiliary picture to the following values:

[0148] - alpha_channel_use_idc=3 - alpha_channel_bit_depth_minus8: any value - alpha_transparent_value: any value - alpha_opaque_value=alpha_transparent_value - alpha_channel_incr_flag=0 - alpha_channel_clip_flag=0 - alpha_channel_clip_type_flag: No need to send The encoding process 700 is generally similar to the encoding process 400 (or 401) of FIG. 4. The differences depend on the type of auxiliary data and the SEI message. There are four differences in the encoding steps. First, the auxiliary picture generated in step 750 is an auxiliary picture of type AUX_ALPHA. Second, in step 765, a first SEI message related to the alpha map is generated, in particular, alpha_channel_use_idc is set to "3", indicating that the alpha map should not be used conventionally. Third, in step 770, a second SEI message is generated containing metadata for using the attenuation map according to the attenuation_map_info syntax described in Table 2. Fourth, both SEI messages are encoded and inserted into the bitstream.

[0149] Figure 8 shows a flowchart of an example of video decoding using attenuation map information carried by an alpha plane with a dedicated SEI message, according to at least one embodiment corresponding to the encoding of Figure 7. This decoding process 800 may be implemented, for example, by the decoder 200 of Figure 2, by the processor 1010 in the device 1000 of Figure 3, or by various electronic devices, such as smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, personal computers, laptop computers, and servers. This decoding process 800 is generally similar to the decoding process 500 of Figure 5. As can be seen in the encoding process of Figure 7, the differences depend on the type of auxiliary data and the SEI message. First, in step 820, after decoding the first SEI message corresponding to an auxiliary picture of type AUX_ALPHA, it is checked that alpha_channel_use_idc is equal to 3. If alpha_channel_use_idc is equal to 3, then the SEI message corresponding to the attenuation map is also decoded in step 824. If alpha_channel_use_idc is not equal to 3 (the "No" branch in step 822), then the energy reduction process is skipped and conventional video decoding is performed, possibly using conventional alpha blending operations. Similarly, if no alpha map is received, then the energy reduction process is also skipped and no attenuation is applied to the video. Another difference is seen in step 840, where the attenuation map is decoded from an auxiliary picture of type AUX_ALPHA.

[0150] Steps 820, 822, 824, and 825 may be reversed, leading to decoding of the SEI message corresponding to the attenuation map interpretation followed by decoding of the SEI message corresponding to AUX_ALPHA. In either case, application of the attenuation process depends at least on the presence of both of these SEI messages, whether alpha_channel_use_idc=3, the type of display model, and the presence of any auxiliary pictures of type AUX_ALPHA that correspond to the SEI-related messages.

[0151] In at least one embodiment, the attenuation map uses multiple components, each stored and transmitted using a separate auxiliary picture of type AUX_ATTENUATION or AUX_ALPHA. For example, three auxiliary pictures of type AUX_ATTENUATION are used, one for each component Y, U, and V of the attenuation map. In another example, two auxiliary pictures may be used, one for Y and one for U and V. Any other combinations may be envisioned. To enable such an embodiment, the information provided by the encoder should include a mechanism for linking multiple auxiliary pictures representing components of the entire attenuation map, so that the entire attenuation map can be reconstructed by the decoder before being applied to the primary video. In the current set of metadata sent in the SEI message, several sets of metadata (corresponding to one attenuation map) are sent, taking into account the total number of ami_map_number, with each set corresponding to an index i in the SEI message. Therefore, it is proposed that each component of the entire attenuation map corresponds to an index in the ami_map_number metadata in the SEI. These indices will be used to link each component of the attenuation map. For this purpose, additional metadata is required that provides information about which auxiliary pictures should be used in combination as individual components of the attenuation map. ami_comp_number[i]: indicates the number of additional auxiliary pictures needed to reconstruct the full attenuation map from the attenuation map of index i, corresponding to one component of the full attenuation map. ami_comp_idc[i][j]: indicates the index of one other set of metadata (i.e. related to the attenuation map with index j) and corresponds to another component of the total attenuation map.

[0152] The list ami_comp_idc[i][j] corresponds to other indices of attenuation maps that, while linked with the attenuation map of index i, will allow the reconstruction of the entire attenuation map.

[0153] In such an embodiment, Table 17 shows some additional values ​​for the definition of ami_attenuation_comp_idc[i] and replaces Table 6.

[0154] [Table 17]

[0155] In addition to the definitions in Table 6, the following definitions are added to Table 17:

[0156] - ami_attenuation_comp_idc[i] equal to 6 specifies that the decoded auxiliary picture of type AUX_ATTENUATION with index i contains one component, which should be applied to the first component of the associated primary picture decoded sample(s).

[0157] - ami_attenuation_comp_idc[i] equal to 7 specifies that the decoded auxiliary picture of type AUX_ATTENUATION with index i contains one component, which should be applied to the second component of the associated primary picture decoded sample(s).

[0158] - ami_attenuation_comp_idc[i] equal to 8 specifies that the decoded auxiliary picture of type AUX_ATTENUATION with index i contains one component, which should be applied to the third component of the associated primary picture decoded sample(s).

[0159] - ami_attenuation_comp_idc[i] equal to 9 specifies that the mapping between the component of the decoded auxiliary picture of type AUX_ATTENUATION at index i and its component to which the decoded auxiliary picture of type AUX_ATTENUATION at index i should be applied corresponds to some proprietary user-defined process.

[0160] This also affects the definition of the Attenuation_map_info syntax compared to the previous syntax described in Table 2. The ami_global_flag cannot be used in this embodiment because it is required to have a separate value for ami_attenuation_comp_idc for each attenuation map of index i. The new syntax is shown in Table 18. The same modifications (with respect to Table 2) can also be made to Tables 14 and 15.

[0161] [Table 18]

[0162] 9 shows a flowchart of an example of video decoding using an attenuation map based on multiple components carried by separate auxiliary pictures, according to at least one embodiment. This decoding process 900 may be implemented, for example, by the decoder 200 of FIG. 2, by the processor 1010 in the device 1000 of FIG. 3, or by various electronic devices, such as smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, personal computers, laptop computers, and servers. This decoding process 900 is generally similar to the decoding process 500 of FIG. 5. In this embodiment, the differences primarily reside in additional iterations to extract separate attenuation map components, apply them to corresponding components of the decoded picture in steps 970 and 980, and combine the resulting components together in step 985.

[0163] This principle of using multiple components carried by separate auxiliary pictures can be applied to the embodiments relating to Tables 12 and 13, with the same modifications.

[0164] In at least one embodiment, if the expected quality for the end device is higher than the quality given by ami_video_quality, the use of the attenuation map is disabled.

[0165] In at least one embodiment, when a single attenuation map is provided, a given energy reduction rate R (i.e., corresponding to the expected energy reduction rate) is checked against the ami_energy_reduction_rate provided in the SEI message. If this rate corresponds to the ami_energy_reduction_rate of the transmitted attenuation map, the corresponding attenuation map is applied to the decoded picture. If the energy reduction rate R is lower than the transmitted ami_energy_reduction_rate, a new attenuation map corresponding to R is inferred from the transmitted attenuation map by extrapolating this new attenuation map according to the process described by the ami_map_approximation_model. An example of such a process can be a simple linear scaling of the attenuation map.

[0166] In at least one embodiment, when two attenuation maps are provided for two different ami_energy_reduction_rates R1 and R2, a given energy reduction rate R is checked against rates R1 and R2. If this rate corresponds to one of rates R1 and R2, the corresponding processed attenuation map (as described in the previous embodiment) is applied to the decoded picture. If the energy reduction rate R is such that R1 < R < R2, a new attenuation map corresponding to R is inferred from the decoded attenuation maps corresponding to R1 and R2 by extrapolating this new attenuation map according to the process described by the ami_map_approximation_model. An example of such a process can be pixel-by-pixel linear interpolation or bicubic interpolation between the two attenuation maps corresponding to R1 and R2. If R is greater than both R1 and R2, some interpolation process for the attenuation map by linear scaling from the maximum energy reduction rate can be envisioned, but there is no guarantee for the resulting quality of the reduced picture. This embodiment can be easily extended to three or more transmitted attenuation maps.

[0167] In at least one embodiment, the use of attenuation maps is disabled for some content pictures depending on the image category (eg, sports images, game images, etc.), display settings (eg, movie mode, etc.), etc.

[0168] In at least one embodiment, the method is disabled for certain content that does not result in significant energy reduction. For example, dark content will result in very low energy reduction regardless of the technique. In this embodiment, the total amount of luminance per picture is calculated, and when it is lower than a given threshold, the energy reduction method is disabled. Alternatively, it can be disabled per GOP, per shot, or per movie.

[0169] In at least one embodiment, an additional check is added to verify that a pixel spatially belongs to a subset of the image to be processed, e.g., belongs to a region where energy reduction is not desired. Such a region or mask can be based, for example, on a spatiotemporal just noticeable difference (JND) map, a motion field, a saliency map, eye-tracking information, or other per-pixel information. When a pixel does not belong to this region or mask, no attenuation map is applied to this pixel. This check can be done before or after upsampling, if there is an attenuation map.

[0170] If any post-processing is applied to the decoded picture, such post-processing should be taken into account before applying the attenuation map. Two embodiments are possible. In at least one embodiment, some post-processing operations are taken into account while creating the attenuation map. For example, this can be done in process 400 by introducing an additional step between steps 420 and 430 to add post-processing operations to the decoded picture before creating the attenuation map. In this case, at the receiver side, the attenuation map should be applied after all post-processing operations. In at least one embodiment, these post-processing operations intervene independently of the encoding-decoding process, and in this case the attenuation map should be adapted to take these post-processing operations into account before applying it to the decoded, post-processed picture.

[0171] If the attenuation map is calculated on an input image in a given color gamut, the decoded picture should be converted to this color gamut before applying the attenuation map. At least one embodiment further includes checking that the color gamut of the decoded picture corresponds to the above color gamut while calculating the attenuation map. This can be done by sending the color gamut of the attenuation map calculation together with the metadata.

[0172] For example, for transmissive pixel displays such as backlit displays, the attenuation map cannot be applied directly. However, it can be used to provide guidance to backlight scaling algorithms. Indeed, for backlit displays, the backlight is a significant contributor to the display's energy consumption.

[0173] In at least one embodiment, the attenuation map is applied to a transmissive pixelated display by determining a minimum, average, or any other global value from the attenuation map and using this information to guide the backlight of the transmissive pixelated display. In the particular case of a local dimming display where the backlight is divided into different regions, the attenuation map may be pre-divided into regions corresponding to the local dimming regions of the display before determining a minimum, average, or any other global value from the attenuation map and using this information to guide the backlight.

[0174] In at least one embodiment, the following steps are implemented: They correspond to ami_backlight_scaling_idc equal to 0. Apply the attenuation map to the image and determine the maximum luminance (or luma) values ​​before and after applying the attenuation map. The ratio between these maximum values ​​determines the percentage by which the backlight can be dimmed. The image with the attenuation map applied is then rescaled to its original value, which is then used to set the pixel values ​​of the LED panel. This latter form of backlight scaling allows for either a stronger reduction in energy or a smaller reduction in visual quality than traditional backlight scaling methods, due to the nonlinearity of the attenuation map. If the backlight consists of an LED array or a tiled backlight, the above method can be applied to sections of the back panel individually.

[0175] At least one example of an embodiment may involve a device including an apparatus described herein and at least one of: (i) an antenna configured to receive a signal including data representing image information; (ii) a band limiter configured to limit the received signal to a band of frequencies including the data representing the image information; and (iii) a display configured to display an image from the image information.

[0176] At least one example of an embodiment may involve a device described herein, where the device includes one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cell phone, a tablet, a computer, a laptop, or other electronic device.

[0177] In general, another example of an embodiment may involve a bitstream or signal formatted to include syntax elements and picture information, where the syntax elements are produced and the picture information is encoded by processing according to any one or more of the example embodiments of the method according to the present disclosure.

[0178] Generally, one or more other example embodiments may also provide a computer-readable storage medium, e.g., a non-volatile computer-readable storage medium, having stored thereon instructions for encoding or decoding picture information, such as video data, in accordance with a method or apparatus described herein. One or more embodiments may also provide a computer-readable storage medium having stored thereon a bitstream generated in accordance with a method or apparatus described herein. One or more embodiments may also provide methods and apparatus for transmitting or receiving a bitstream or signal generated in accordance with a method or apparatus described herein.

[0179] Many of the example embodiments described herein are specifically described and are described in what may often be considered limiting fashion to at least illustrate particular characteristics. However, this is for clarity of description and does not limit the applicability or scope of those aspects. In fact, all of the different aspects can be combined and interchanged to provide further aspects. Furthermore, embodiments, features, and the like can be combined and interchanged with others described in prior applications.

[0180] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of the processes performed on a received encoded sequence to produce a final output suitable for display, for example. In various embodiments, such processes include one or more of processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by decoders of the various implementations described herein.

[0181] As a further example, in one embodiment, "decoding" refers to entropy decoding only, in another embodiment, "decoding" refers to differential decoding only, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process as a whole will be clear based on the context of the detailed description and will be well understood by one of ordinary skill in the art.

[0182] Various implementations involve encoding. Similar to the above description of "decoding," "encoding" as used herein can encompass all or part of the processes performed on an input video sequence to produce, for example, an encoded bitstream. In various embodiments, such processes include one or more of processes typically performed by an encoder, such as partitioning, differential encoding, transforming, quantization, and entropy encoding.

[0183] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process as a whole will be clear based on the context of the detailed description and will be well understood by one of ordinary skill in the art.

[0184] It should be noted that the syntax elements used herein are descriptive terms, and therefore they do not preclude the use of other syntax element names.

[0185] When a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0186] Generally, example embodiments, implementations, features, etc. described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even when discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the discussed feature may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. One or more example methods may be implemented, for example, in a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users. Also, use of the term "processor" herein is intended to broadly encompass various configurations of one processor or two or more processors.

[0187] References to "one embodiment" or "one embodiment" or "one implementation" or "one implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in one embodiment" or "in one implementation" or "in one implementation" appearing in various places throughout this application, as well as any other variations, are not necessarily all referring to the same embodiment.

[0188] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0189] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0190] Additionally, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information, retrieving information (e.g., from memory). Furthermore, "receiving" generally involves in some manner, for example, during an operation such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0191] For example, it should be appreciated that the use of any of " / ," "and / or," and "at least one of" in the cases of "A / B," "A and / or B," and "at least one of A and B" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). This can be expanded as many times as the items listed, as would be apparent to one skilled in the art.

[0192] As will be apparent to those skilled in the art, implementations can produce a variety of signals formatted to carry information, which can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method or data produced by one of the described implementations. For example, a signal can be formatted to carry a bitstream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

[0193] Various embodiments are described herein, the features of which may be provided singly or in any combination across various claim categories and types. [Industrial Applicability]

[0194] This disclosure is in the field of video compression.

Claims

1. obtaining encoded data comprising at least an image, an attenuation map, and a set of parameters, said set of parameters comprising at least a first parameter representing an operation of applying said attenuation map to an image, and a second parameter representing a mapping between components of said attenuation map and image components affected by said operation; determining an attenuated image having reduced component values ​​by applying the attenuation map to the image through operations based on the first parameter on components of the image selected based on the second parameter; providing the attenuated image; A method for providing the above.

2. The method of claim 1 , wherein the set of parameters includes at least a third parameter representing a selected energy reduction rate.

3. 3. The method of claim 1, wherein the set of parameters includes at least a fourth parameter representing the attenuation map, the fourth parameter identifying the attenuation map in a set of auxiliary pictures carried in a bitstream.

4. The method of claim 1 , wherein the first parameter indicates an operation from a set of operations including addition, subtraction, multiplication, a contrast sensitivity function, and division.

5. 5. The method of claim 1, wherein the second parameter is selected from a set including applying a component of the attenuation map to a luma component, applying a component of the attenuation map to a luma component and both chroma components, applying a component of the attenuation map to three RGB components, applying a first component of the attenuation map to a luma component and a second component of the attenuation map to both chroma components, applying three components of the attenuation map to a luma component and both chroma components respectively, applying three components of the attenuation map to three RGB components respectively, applying one component of the attenuation map to the first component, applying one component of the attenuation map to the second component, and applying one component of the attenuation map to a third component.

6. The method of claim 1 , wherein multiple attenuation maps are used, each of the attenuation maps being associated with a different energy reduction rate.

7. 7. The method of claim 6, further comprising: performing interpolation between a first attenuation map associated with a first energy reduction rate and a second attenuation map associated with a second energy reduction rate to obtain an attenuation map corresponding to a third attenuation rate included in a range included between the first energy reduction rate and the second energy reduction rate.

8. 8. The method of claim 1, wherein the set of parameters further comprises a fifth parameter representing a pre-processing operation, the pre-processing operation based on the fifth parameter being performed on the attenuation map prior to its application to components of the image.

9. 9. The method of claim 8, wherein the preprocessing operation is an upsampling operation from a resolution of the attenuation map to a resolution of the image, and the fifth parameter indicates an upsampling function from a set of upsampling functions including at least linear scaling, bilinear interpolation, Lanczos, and bicubic.

10. 10. The method of claim 1, further comprising using multiple attenuation maps for multiple components of the image, and further comprising parameters representing the mapping between the components of the multiple attenuation maps and the image components affected by the operation.

11. The method of claim 1 , wherein the image, the attenuation map and the set of parameters are carried in a bitstream.

12. 12. The method of any one of claims 1 to 11, wherein the set of parameters is conveyed by a supplementary enhancement message according to the format specified in ISO / IEC FDIS 23002-7 and the attenuation map is conveyed by an auxiliary picture according to the format specified in ISO / IEC 23002-3 and ISO / IEC 14496-10 or ISO / IEC DIS 23008-2 or ISO / IEC FDIS 23002-7.

13. The method of claim 12 , wherein the auxiliary picture carrying the attenuation map is an auxiliary picture for alpha blending identified in the format as AUX_ALPHA.

14. obtaining an input image of a video; determining an attenuation map based on the input image according to a selected energy reduction rate, wherein applying the attenuation map to the input image reduces values ​​of components of the input image; generating an encoded video comprising at least the input image, the attenuation map, and a set of parameters, the set of parameters comprising at least a first parameter representing an operation of applying the attenuation map to an image, and a second parameter representing a mapping between components of the attenuation map and image components affected by the operation; A method for providing the above.

15. The method of claim 14 , wherein the set of parameters includes at least a third parameter representing the selected energy reduction rate.

16. 16. The method of claim 14 or 15, wherein the set of parameters includes at least a fourth parameter representing the attenuation map, the fourth parameter identifying the attenuation map in a set of auxiliary pictures carried in a bitstream.

17. 17. The method of claim 14, wherein the first parameter indicates an operation from a set of operations including addition, subtraction, multiplication, a contrast sensitivity function, and division.

18. 18. The method of claim 14, wherein the second parameter is selected from a set including applying a component of the attenuation map to a luma component, applying a component of the attenuation map to a luma component and both chroma components, applying a component of the attenuation map to three RGB components, applying a first component of the attenuation map to a luma component and a second component of the attenuation map to both chroma components, applying three components of the attenuation map to a luma component and both chroma components respectively, applying three components of the attenuation map to three RGB components respectively, applying one component of the attenuation map to the first component, applying one component of the attenuation map to the second component, and applying one component of the attenuation map to a third component.

19. 19. The method of any one of claims 14 to 18, wherein multiple attenuation maps are used, each of the attenuation maps being associated with a different energy reduction rate.

20. performing interpolation between a first attenuation map associated with a first energy reduction rate and a second attenuation map associated with a second energy reduction rate to obtain an attenuation map corresponding to a third attenuation rate included in a range included between the first energy reduction rate and the second energy reduction rate; 20. The method of claim 19, further comprising:

21. 21. The method of claim 14, wherein the set of parameters further comprises a fifth parameter representing a pre-processing operation, the pre-processing operation based on the fifth parameter being performed on the attenuation map prior to its application to components of the image.

22. 22. The method of claim 21, wherein the preprocessing operation is an upsampling operation from a resolution of the attenuation map to a resolution of the image, and the fifth parameter indicates an upsampling function from a set of upsampling functions including at least linear scaling, bilinear interpolation, Lanczos, and bicubic.

23. 23. The method of claim 14, further comprising using multiple attenuation maps for multiple components of the image, and further comprising parameters representing the mapping between the components of the multiple attenuation maps and the image components affected by the operation.

24. 24. The method of any one of claims 14 to 23, wherein the image, the attenuation map and the set of parameters are carried in a bitstream.

25. 25. The method of any one of claims 14 to 24, wherein the set of parameters is conveyed by a supplementary enhancement message according to the format specified in ISO / IEC FDIS 23002-7 and the attenuation map is conveyed by an auxiliary picture according to the format specified in ISO / IEC 23002-3 and ISO / IEC 14496-10 or ISO / IEC DIS 23008-2 or ISO / IEC FDIS 23002-7.

26. 26. The method of claim 25, wherein the auxiliary picture carrying the attenuation map is an auxiliary picture for alpha blending identified in the format as AUX_ALPHA.

27. acquiring encoded data comprising at least an image, an attenuation map, and a set of parameters, said set of parameters comprising at least a first parameter representing an operation of applying said attenuation map to an image, and a second parameter representing a mapping between components of said attenuation map and image components affected by said operation; determining an attenuated image having reduced component values ​​by applying the attenuation map to the image through operations based on the first parameter on components of the image selected based on the second parameter; providing the attenuated image Processor configured to A device comprising:

28. 28. The apparatus of claim 27, wherein the set of parameters includes at least a third parameter representing a selected energy reduction rate.

29. 29. The apparatus of claim 27 or 28, wherein the set of parameters includes at least a fourth parameter representing the attenuation map, the fourth parameter identifying the attenuation map in a set of auxiliary pictures carried in a bitstream.

30. 30. The apparatus of any one of claims 27 to 29, wherein the first parameter indicates an operation from a set of operations including addition, subtraction, multiplication, a contrast sensitivity function, and division.

31. 31. The apparatus of claim 27, wherein the second parameter is selected from a set including applying a component of the attenuation map to a luma component, applying a component of the attenuation map to a luma component and both chroma components, applying a component of the attenuation map to three RGB components, applying a first component of the attenuation map to a luma component and a second component of the attenuation map to both chroma components, applying three components of the attenuation map respectively to a luma component and both chroma components, applying three components of the attenuation map respectively to three RGB components, applying one component of the attenuation map to the first component, applying one component of the attenuation map to the second component, and applying one component of the attenuation map to a third component.

32. 32. Apparatus according to any one of claims 27 to 31, wherein a plurality of attenuation maps are used, each of said attenuation maps being associated with a different energy reduction rate.

33. 33. The apparatus of claim 32, further comprising: performing interpolation between a first attenuation map associated with a first energy reduction rate and a second attenuation map associated with a second energy reduction rate to obtain an attenuation map corresponding to a third attenuation rate that falls within a range included between the first energy reduction rate and the second energy reduction rate.

34. 34. The apparatus of claim 27, wherein the set of parameters further comprises a fifth parameter representing a pre-processing operation, the pre-processing operation based on the fifth parameter being performed on the attenuation map prior to its application to components of the image.

35. 35. The apparatus of claim 34, wherein the preprocessing operation is an upsampling operation from a resolution of the attenuation map to a resolution of the image, and the fifth parameter indicates an upsampling function from a set of upsampling functions including at least linear scaling, bilinear interpolation, Lanczos, and bicubic.

36. 36. The apparatus of claim 27, further comprising using a plurality of attenuation maps for a plurality of components of the image, and further comprising parameters representing the mapping between the components of the plurality of attenuation maps and the image components affected by the operation.

37. 37. The apparatus of any one of claims 27 to 36, wherein the image, the attenuation map and the set of parameters are conveyed in a bitstream.

38. 38. The apparatus of any one of claims 27 to 37, wherein the set of parameters is conveyed by a supplementary enhancement message according to a format specified in ISO / IEC FDIS 23002-7 and the attenuation map is conveyed by an auxiliary picture according to the format specified in ISO / IEC 23002-3 and ISO / IEC 14496-10 or ISO / IEC DIS 23008-2 or ISO / IEC FDIS 23002-7.

39. 39. The apparatus of claim 38, wherein the auxiliary picture carrying the attenuation map is an auxiliary picture for alpha blending identified in the format as AUX_ALPHA.

40. Take the input image of the video, determining an attenuation map based on the input image according to a selected energy reduction rate and applying the attenuation map to the input image reduces values ​​of components of the input image; generating an encoded video comprising at least the input image, the attenuation map, and a set of parameters, the set of parameters comprising at least a first parameter representing an operation of applying the attenuation map to an image, and a second parameter representing a mapping between components of the attenuation map and image components affected by the operation; Processor configured to A device comprising:

41. 41. The apparatus of claim 40, wherein the set of parameters includes at least a third parameter representing the selected energy reduction rate.

42. 42. The apparatus of claim 40 or 41, wherein the set of parameters includes at least a fourth parameter representing the attenuation map, the fourth parameter identifying the attenuation map in a set of auxiliary pictures carried in a bitstream.

43. 43. The apparatus of any one of claims 40 to 42, wherein the first parameter indicates an operation from a set of operations including addition, subtraction, multiplication, a contrast sensitivity function, and division.

44. 44. The apparatus of claim 40, wherein the second parameter is selected from a set including applying a component of the attenuation map to a luma component, applying a component of the attenuation map to a luma component and both chroma components, applying a component of the attenuation map to three RGB components, applying a first component of the attenuation map to a luma component and a second component of the attenuation map to both chroma components, applying three components of the attenuation map respectively to a luma component and both chroma components, applying three components of the attenuation map respectively to three RGB components, applying one component of the attenuation map to the first component, applying one component of the attenuation map to the second component, and applying one component of the attenuation map to a third component.

45. 45. Apparatus according to any one of claims 40 to 44, wherein a plurality of attenuation maps are used, each of the attenuation maps being associated with a different energy reduction rate.

46. 46. ​​The apparatus of claim 45, further comprising: performing interpolation between a first attenuation map associated with a first energy reduction rate and a second attenuation map associated with a second energy reduction rate to obtain an attenuation map corresponding to a third attenuation rate that falls within a range included between the first energy reduction rate and the second energy reduction rate.

47. 47. The apparatus of claim 40, wherein the set of parameters further comprises a fifth parameter representing a pre-processing operation, the pre-processing operation based on the fifth parameter being performed on the attenuation map prior to its application to components of the image.

48. 48. The apparatus of claim 47, wherein the preprocessing operation is an upsampling operation from a resolution of the attenuation map to a resolution of the image, and the fifth parameter indicates an upsampling function from a set of upsampling functions including at least linear scaling, bilinear interpolation, Lanczos, and bicubic.

49. 49. The apparatus of claim 40, further comprising using a plurality of attenuation maps for a plurality of components of the image, and further comprising parameters representing the mapping between the components of the plurality of attenuation maps and the image components affected by the operation.

50. 50. The apparatus of any one of claims 40 to 49, wherein the image, the attenuation map and the set of parameters are conveyed in a bitstream.

51. 51. The apparatus of any one of claims 40 to 50, wherein the set of parameters is conveyed by a supplementary enhancement message according to a format specified in ISO / IEC FDIS 23002-7 and the attenuation map is conveyed by an auxiliary picture according to the format specified in ISO / IEC 23002-3 and ISO / IEC 14496-10 or ISO / IEC DIS 23008-2 or ISO / IEC FDIS 23002-7.

52. 52. The apparatus of claim 51, wherein the auxiliary picture carrying the attenuation map is an auxiliary picture for alpha blending identified in the format as AUX_ALPHA.

53. 53. The apparatus of any one of claims 27 to 52, wherein the device is one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cell phone, a tablet, a computer, a laptop, or other electronic device.

54. A bitstream formatted to include syntax elements according to the method of any one of claims 14 to 26.

55. 27. A signal comprising data generated according to the method of any one of claims 14 to 26.

56. A computer program comprising instructions which, when executed by a computer, cause the computer to perform a method according to any one of claims 1 to 26.

57. 27. A non-transitory computer readable medium storing executable program instructions that cause a computer executing the instructions to perform the method of any one of claims 1 to 26.

Images