Decode the image

Abstract

A method for decoding an image that enables artistic intent to be preserved in the final rendering stage by adjusting the image to the content and the brightness characteristics of a rendering display that will display the image. Values ​​representing the brightness of both the image and the display device are obtained. Parameters of a display adaptation function are adjusted based on these values, and the image is decoded using the adjusted display adaptation function. In at least one embodiment, the adjusted display adaptation function is a tone mapping function. A corresponding apparatus for implementing the method is also described.

Description

1. Technical Field

[0001] At least one embodiment is generally directed to distributing video while preserving high fidelity of the resulting image, and at least one embodiment is particularly directed to decoding an image using a function determined to adjust the image to the content and brightness characteristics of a rendering display on which the image will be displayed. 2. Background technology

[0002] This section is intended to introduce the reader to various aspects of the technology that may be related to various aspects of at least one embodiment described and / or claimed below. It is believed that this discussion will help provide the reader with background information to facilitate a better understanding of at least one embodiment and various aspects of at least one embodiment.

[0003] The advent of the High Efficiency Video Coding (HEVC) standard (ITU-T H.265 Telecommunication Standardization Sector of the ITU (02 / 2018), Series H: Audiovisual and multimedia systems, Infrastructure of audiovisual services - Coding of video in motion, High Efficiency Video Coding, Recommendation ITU-T H.265) enables the deployment of new video services with enhanced viewing experience, such as Ultra HD services. In addition to increased spatial resolution, Ultra HD formats can bring wider color gamut (WCG) and higher dynamic range (HDR) than the standard color gamut (SCG) and standard dynamic range (SDR) of currently deployed high-definition formats, respectively. Different solutions for the representation and decoding of HDR / WCG videos have been proposed, such as the Perceptual Transfer Function Perceptual Quantizer (PQ) (SMPTE ST 2084, "High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays", or Diaz, R., Blinstein, S. and Qu, S. "Integrating HEVC Video Compression with a High Dynamic Range Video Pipeline", SMPTE Motion Imaging Journal, Vol. 125, No. 1, February 2016, pp. 14-21). Typically, SMPTE ST 2084 allows the representation of up to 10000 cd / m with only 10 or 12 bits. 2 The peak brightness of the HDR video signal.

[0004] SDR backward compatibility with decoding and rendering devices is an important feature in some video distribution systems, such as broadcast or multicast systems. Solutions based on a single-layer encoding / decoding process can be backward compatible, such as SDR compatible, and can leverage legacy distribution networks and services that are already in place.

[0005] This single-layer based distribution solution enables high-quality HDR rendering on consumer electronics (CE) devices that support HDR, while also providing high-quality SDR rendering on CE devices that support SDR. This solution is based on an encoded signal (e.g., an SDR signal) and associated metadata (usually using only a few bytes per video frame or scene), which can be used to reconstruct another signal, such as an SDR or HDR signal, from the decoded signal.

[0006] An example of a single-layer based distribution scheme can be found in ETSI Technical Specification TS103433-1 V1.2.1 (August 2017). This single-layer based distribution scheme is denoted SL-HDR1 below.

[0007] In addition, the HDR distribution system (workflow, as well as decoding and rendering equipment) may have been deployed. In fact, there are several global video service providers that include HDR content. However, distributed HDR materials may be represented in a format or characteristics that do not match the characteristics of consumer terminal devices. Typically, consumer terminal devices adapt the decoded materials to their own characteristics. However, due to the differences between the characteristics of consumer terminal devices compared to the master display used to grade the original content in the production environment, the versatility of the technology used in HDR TV has produced important differences in rendering. For content producers, the fidelity of artistic intent and its rendering to consumers are the most important. Therefore, "display adaptive" metadata can be generated at the production stage during the grading process, or under the control of a quality inspection operator before distribution. When the decoded signal is to be adapted to the characteristics of the terminal device, metadata enables the artistic intent to be conveyed to consumers.

[0008] An example of a single-layer based distribution scheme combined with display adaptation can be found in ETSI Technical Specification TS103433-2 V1.1.1 (January 2018). This single-layer based distribution scheme is denoted SL-HDR2 below.

[0009] This single layer based distribution scheme SL-HDR1 or SL-HDR2 generates metadata as parameters for signal reconstruction. The metadata can be static or dynamic.

[0010] Static metadata refers to parameters representing the video content or its format, which remain the same for, eg, a video (group of pictures) and / or a program.

[0011] Static metadata is valid for the entire video content (scene, movie, clip...) and may depend on the image content itself or the representation format of the image content. Static metadata may define, for example, the image format, color space or color gamut. For example, SMPTE ST 2086:2014, "Mastering Display Color Volume Metadata Supporting High Luminance and Wide Color Gamut Images" defines static metadata that describes the mastering display used for grading material in a production environment. The Mastering Display Color Volume (MDCV) SEI (Supplementary Enhancement Information) message corresponds to ST 2086 for H.264 / AVC ("Advanced video coding for generic audiovisual Services", Series H: Audiovisual and Multimedia Systems, ITU-T H.264 Recommendation, Telecommunication Standardization Sector of ITU, April 2017) and HEVC video codec.

[0012] Dynamic metadata is content-dependent information such that the metadata can change with the image / video content, for example for each image or group of images. As an example, SMPTE ST 2094:2016, "Dynamic Metadata for Color Volume Transform" defines dynamic metadata that is typically generated in a production environment. SMPTE ST 2094-30 can be distributed in HEVC and AVC coded video streams using, for example, the Color Remapping Information (CRI) SEI message.

[0013] The display adaptation method described in Annex E of ETSI TS103433-1 v1.2.1 and Clause 7 of ETSI TS103433-2 v1.1.1 uses only the peak brightness of the rendered display to control the tone mapping operation (or its inverse) required in the display adaptation process (see, for example, Clause E.2 from TS103433-1 v1.2.1). Other tone mapping control parameters are determined based on content analysis and the characteristics of the master display used to grade HDR content. However, displays (especially consumer HDR displays) must implement power management strategies for energy saving (https: / / eur-lex.europa.eu / LexUriServ / LexUriServ.do?uri=OJ:L:2009:191:0042:0052:EN:PDF) and their brightness / brightness is adjusted according to the surface, distribution and value of the bright pixels to be displayed to prevent excessive power consumption. The application of such a strategy affects the rendering of the image. Without taking these characteristics of a given display into account, display-adapted image rendering may not match the expected appearance (i.e., display-adapted image rendering is different from what is done in a production facility on professional equipment) because the brightness in reality may be much lower than the peak brightness.

[0014] The unit of brightness is candela per square meter (cd / m 2 The unit replaces the term "nit", which may also be used (although it is deprecated in the International System of Units).

[0015] For example, an image of the night sky may consist mainly of black pixels (nighttime - about a few nits) and a few groups of very bright pixels (starlight - 800 nits). In this case, the peak brightness of the content may be up to 800 nits. If the display can achieve this peak brightness, the highlights of the image (starlight) will likely be displayed correctly at 800 nits. However, in the case of very bright images (e.g. a scene representing a market in the south of France at noon (average brightness over the entire picture is 300 nits), where specular light bounces off a Cadillac chrome (800 nits) or explodes or fades to a white effect...), a consumer display will likely not be able to display the peak brightness at its maximum peak brightness capability (800 nits in this example) while complying with power consumption regulations. In practice, the actual peak brightness of the rendered display depends at least on the surface of the (high) bright area to be represented.

[0016] Therefore, due to the imprecision of the power management strategy of the display device, display adaptation techniques (possibly metadata-guided) that only consider the peak brightness of the rendering display as the only characteristic may not satisfactorily represent the artistic intent as approved at the production stage. 3. Summary of the invention

[0017] A simplified overview of at least one embodiment is presented below to provide a basic understanding of some aspects of at least one embodiment. This overview is not an extensive review of the embodiments. It is not intended to identify the key or important elements of the embodiments. The following overview only presents some aspects of at least one embodiment in a simplified form as a preface to a more detailed description provided elsewhere in this application.

[0018] According to a general aspect of at least one embodiment, a method for decoding an image is provided that achieves artistic intent preservation at a final rendering stage by adjusting the image to the brightness or luminance characteristics of the content and a rendering display that will display the image. Luminance values ​​representing both the content image and the rendering display device are obtained. Parameters of a display adaptation function are adjusted based on these values, thereby determining an adjusted display adaptation function for use in the image decoding process. In at least one embodiment, the adjusted display adaptation function is a tone mapping function.

[0019] In a first aspect, the present disclosure relates to a method for decoding an image, comprising: obtaining a first value representing a brightness parameter of a display device that will display the image, obtaining a second value representing the brightness parameter of the image to be displayed, and decoding the image using an adaptation function whose parameters are adjusted according to the first value and the second value.

[0020] In a second aspect, the present disclosure relates to an apparatus for decoding an image, the apparatus comprising a processor configured to obtain a first value representing a brightness parameter of a display device that will display the image, obtain a second value representing a brightness parameter of the image to be displayed, and decode the image using an adaptation function whose parameters are adjusted according to the first value and the second value.

[0021] In a third aspect, the present disclosure relates to a computer program product comprising program code instructions for performing the steps of the method according to the first aspect when the program is executed on a computer.

[0022] In a fourth aspect, the present disclosure relates to a non-transitory processor-readable medium having program code instructions for performing the steps of the method according to the first aspect when the program is executed on a computer.

[0023] The particular nature of the present principles as well as other objects, advantages, features and uses of the present principles will become apparent from the following description of examples taken in conjunction with the accompanying drawings. 4. Description of the drawings

[0024] In the accompanying drawings, an example of the present principle is shown. It shows:

[0025] - Figure 1A high-level representation of an end-to-end workflow supporting delivery of content for display utilizing improved display adaptation features in accordance with at least one embodiment is shown;

[0026] - Figure 2 An example of an end-to-end processing workflow supporting delivery to HDR and SDR CE displays according to a single-layer based distribution scheme is shown;

[0027] - Figure 3 Shows Figure 2 A specific implementation of the workflow;

[0028] - Figure 4a A diagram showing an example of a perceptual transfer function;

[0029] - Figure 4b An example of a piecewise curve for mapping is shown;

[0030] - Figure 4c An example of a curve for converting a perceptually uniform signal to the linear light domain is shown;

[0031] - Figure 5 A block diagram showing an example of a system in which various aspects and embodiments are implemented;

[0032] - Figure 6 An example of an embodiment for adjusting a display adaptation function based on a content maximum average brightness and a presentation display maximum average brightness is shown; and

[0033] - Figure 7 The adjustment of the tone mapping curve according to the second embodiment is shown, wherein the content maximum average brightness and the display maximum average brightness are matched.

[0034] Similar or identical elements are denoted by the same reference numerals. 5. Specific implementation methods

[0035] At least one embodiment is described more fully below with reference to the accompanying drawings, in which examples of at least one embodiment are shown. However, the embodiments may be embodied in many alternative forms and should not be construed as being limited to the examples set forth herein. Therefore, it should be understood that the embodiments are not intended to be limited to the specific forms disclosed. On the contrary, the present disclosure is intended to cover all modifications, equivalents, and substitutes falling within the spirit and scope of the present application as defined by the claims.

[0036] The terms used herein are only for the purpose of describing a specific example, rather than intended to limit. As used herein, the singular forms "one", "a" and "the / said" are intended to also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include" and / or "comprise" when used in this specification, represent the existence of such as features, integers, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, parts and / or their combinations. In addition, when an element is referred to as "response" or "connection" to another element, it can directly respond or be connected to another element, or there can be an intermediate element. On the contrary, when an element is referred to as "direct response" or "direct connection" to other elements, there is no intermediate element. As used herein, the term "and / or" includes any and all combinations of one or more related listed items, and can be abbreviated as " / ". It should be understood that although the terms first, second, etc. can be used here to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teachings of the present application, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. Although some figures include arrows on the communication path to show the main direction of communication, it should be understood that communication can occur in the opposite direction of the arrows depicted. Some examples are described with respect to block diagrams and operational flow charts, wherein each box represents a circuit element, module or code portion including one or more executable instructions for implementing a specified logical function (one or more). It should also be noted that in other implementations, the function (one or more) marked in the box may not occur in the specified order. For example, depending on the functions involved, the two boxes shown in succession may actually be executed substantially simultaneously, or the boxes may sometimes be executed in the opposite order. Mentioning "according to an example" or "in an example" herein means that a specific feature, structure or characteristic described in conjunction with the example may be included in at least one implementation. The expression "according to an example" or "in an example" appearing in various places in the specification does not necessarily refer to the same example in all, nor is it a separate or alternative example that must be mutually exclusive with other examples. The figure marks appearing in the claims are only for illustration and should not have a limiting effect on the scope of the claims. Although not explicitly described, the present examples and variations may be employed in any combination or sub-combination.

[0037] In the following, image data refers to data, for example, one or several sample arrays (e.g., pixel values) of a particular image / video format, which specifies information related to the pixel values ​​of the image (or video) and / or information that can be used by a display and / or any other device to, for example, visualize and / or decode the image (or video). An image typically includes a first component in the shape of a first sample array, typically representing the brightness (or luma) of the image, and a second component and a third component in the shape of other sample arrays, typically representing the chrominance (or chroma) of the image. Some embodiments use a set of color sample arrays to represent the same information, such as the traditional three-color RGB representation.

[0038] In one or more embodiments, a pixel value is represented by a vector having C values, where C is the number of components. Each value of the vector is typically represented by a number of bits that can define the dynamic range of the pixel value.

[0039] A standard dynamic range image (SDR image) is an image whose brightness values ​​are usually represented by fewer bits (usually 8) than in a high dynamic range image (HDR image). Therefore, the difference between the dynamic range of SDR and HDR images is relative, and an SDR image can have, for example, more than 8 bits. Due to the smaller number of bits, SDR images usually do not allow small signal changes to be rendered correctly, especially in the dark and bright brightness range. In HDR images, signal representation is usually extended to maintain higher precision of the signal over all or part of its range. For example, at least one embodiment uses 10 bits of brightness to represent an HDR image and provides 4 times the value of an 8-bit representation. This additional value allows a larger brightness range to be represented and may also allow finer brightness differences to be represented. In HDR images, pixel values ​​are usually represented in floating point format (usually at least 10 bits per component, i.e., floating point or half floating point), the most popular format being the openEXR half floating point format (e.g., 48 bits per pixel) or in integers with long representation, usually at least 16 bits.

[0040] Usually, two different images have different brightness dynamic ranges. The dynamic range of image brightness is the ratio of the maximum to the minimum brightness value of the image.

[0041] Usually, when the dynamic range of the image brightness is lower than 1000 (e.g. 500: 100 cd / m 2 , than 0.2cd / m 2 ), the image is represented as a standard dynamic range (SDR) image, and when the dynamic range of the brightness of the image is equal to or greater than 1000 (e.g. 10000: for example 1000cd / m 2 than 0.1cd / m 2 ), the image is represented as an HDR image. Brightness is expressed in candela / square meter (cd / m2 ). This unit replaces the term "nit", which may also be used.

[0042] At least one embodiment for preprocessing, encoding, decoding and post-processing an image is described, but it extends to preprocessing, encoding, decoding and post-processing an image (video) sequence in that each image in the sequence is sequentially preprocessed, encoded, decoded and post-processed as described below.

[0043] In the following, the weight represents the components m of image n. These components (where m = 1, 2, 3) represents an image I in a specific image format n Typically, image formats are characterized by color quantities (such as chroma and dynamic range) and color encoding systems (such as RGB, YCbCr, etc.).

[0044] According to at least one embodiment, a method for decoding an image is provided that achieves artistic intent preservation in the final rendering stage by adjusting the image to the content and the brightness characteristics of the rendering display, such as Figure 6 and 7 As detailed in .

[0045] Figure 1 A high-level representation of an end-to-end workflow supporting delivery of content for display with improved display adaptation features in accordance with at least one embodiment is shown. Figure 1 The device A1 is configured to implement a method for preprocessing and encoding an image or a video stream. Figure 1 The device A2 is also included. The device A2 is configured to implement the method for decoding and post-processing an image or video stream as described below. Figure 1 Also included is a device A3. The device A3 is in communication with the device A2 and is configured to display the decoded and post-processed image or video stream.

[0046] Figure 1 Also included is a distributed network NET. Two remote devices A1 and A2 communicate via the distributed network NET, which is configured to provide at least a coded image or video stream from device A1 to device A2.

[0047] The apparatus A1 comprises at least one device configured to implement the preprocessing and / or encoding method as described below. The at least one device belongs to a group of devices including, for example, a mobile device, a communication device, a gaming device, a tablet (or a tablet computer), a computer device such as a laptop computer, a still image camera, a video camera, an encoding chip, a still image server, and a video server (e.g., a broadcast server, a video on demand server, or a web server).

[0048] The apparatus A2 comprises at least one device configured to implement the decoding and / or post-processing method as described below.

[0049] The apparatus A3 comprises at least one device configured to implement the display method.

[0050] At least one device included in apparatus A1, A2 or A3 belongs to a group of devices including, for example, a mobile device, a communication device, a gaming device, a computer device, a set-top box, a TV set (or television), a tablet (or tablet computer), a computer device such as a laptop computer, a display, a head-mounted display and a rendering / display chip.

[0051] According to an example, the network is a broadcast network adapted to broadcast still images or video images from the device A1 to a plurality of devices A2. DVB-based and ATSC-based networks are examples of such broadcast networks.

[0052] According to another example, the network is a broadband network suitable for transmitting still images or video images from the device A1 to the plurality of devices A2. An Internet based network, a GSM network or an IPTV network are examples of such broadband networks.

[0053] In an embodiment, the end-to-end workflow uses a broadcast server for device A1, a set-top box for device A2, a television set for device A3, and a DVB terrestrial broadcast network.

[0054] In an alternative embodiment, devices A2 and A3 are combined in a single device, such as a television with integrated set-top box decoding and post-processing functionality.

[0055] In an alternative embodiment, the distributed network NET is replaced by a physical packaged medium on which the encoded image or video stream is stored.

[0056] Physical packaged media include, for example, optical packaged media such as Blu-ray Disc and Ultra HD Blu-ray, and storage-based packaged media such as used in OTT and VoD services.

[0057] Figure 1 The distributed network NET can support delivery to HDR and SDR CE displays. For example, device A1 provides HDR and / or SDR content to be delivered via the network NET, device A2 receives the delivered HDR and / or SDR content, and device A3 (CE display) displays the HDR or SDR content.

[0058] Figure 2 An example of an end-to-end processing workflow supporting delivery to HDR and SDR CE displays according to a single-layer based distribution scheme is shown.

[0059] This single-layer-based distribution solution can solve SDR direct backward compatibility. That is, the solution makes full use of existing SDR distribution networks and services, and enables high-quality HDR rendering on HDR-capable CE devices, including high-quality SDR rendering on SDR CE devices.

[0060] SL-HDR1 is an example of such a single-layer based distribution scheme.

[0061] Such a single layer based distribution scheme may also involve a scheme for use over a distributed network for which display adaptive dynamic metadata is delivered. This allows, for example, the content to be adapted to the user's display. For example, dynamic metadata may be delivered with a PQ HDR video signal. PQ stands for "Perceptual Quantization" as specified in Rec. ITU-R BT.2100 "Recommendation ITU-R BT.2100-1, Picture parameter values ​​for high dynamic range television for production and international programme exchange".

[0062] Figure 2 The workflow shown involves a single-layer based distribution scheme with associated SL-HDR metadata. This approach shows the use of three components for reconstructing a representation of the input image: The three components This reconstruction is based on three decoded components representing the decoded image and metadata specified, for example, in SL-HDR1 or SL-HDR2.

[0063] The information data ID determines which of the single-layer based distribution schemes (e.g., SL-HDR1 or SL-HDR2) is used. Usually, only one single-layer based distribution scheme is actually used, and the information data ID is a fixed value. If more than one single-layer based distribution scheme can be used, the information data ID indicates which of these single-layer based distribution schemes is used.

[0064] Generally, SL-HDR1 and SL-HDR2 can be used, and the information data ID indicates whether SL-HDR1 or SL-HDR2 must be used.

[0065] As shown in the figure, Figure 2 The single-layer based distribution scheme shown comprises a pre-processing step 20 , an encoding step 23 , decoding steps 25 and 26 and a post-processing step 28 .

[0066] The input and output of the preprocessing step 20 are the three-element quantities and The input and output of post-processing step 28 are the three-element components and

[0067] Figure 2 The single-layer based distribution scheme shown in FIG. 1 may include optional format adaptation steps 21, 22, 27, 29 to make the three components The format of the input is adapted to the further processing to be applied on these components.

[0068] For example, in step 21 (optional), the three components The format of may be adapted to a format suitable for the input format of the preprocessing step 20 or the input format of the encoding step 23. In step 22 (optional), the three components The format of may be adapted to a format suitable for the input format of the encoding step 23 .

[0069] In step 27 (optional), the three components The format of can be adapted to be suitable for the input format of the post-processing step 28, and in step 29, the three components The format of may be adapted to a format that may be defined according to at least one characteristic of the target device 30 (eg, a set-top box, a connected TV, a HDR / SDR enabled CE device, an Ultra HD Blu-ray Disc player).

[0070] The format adaptation steps (21, 22, 27, 29) may include color space conversion and / or gamut mapping (and / or inverse gamut mapping). For example, when the three decoded components of the output image are represented in different color spaces and / or gamuts and three components Or the three components of the input image , inverse gamut mapping can be used.

[0071] The usual format adaptation processes may be used, such as R'G'B' to Y'CbCr or Y'CbCr to R'G'B', BT.709 to BT.2020 or BT.2020 to BT.709, downsampling or upsampling chroma components, etc.

[0072] For example, SL-HDR1 may use the format adaptation process and inverse gamut mapping specified in Annex D of ETSI Technical Specification TS103433-1 V1.2.1 (August 2017).

[0073] The input format adaptation step 21 may also include, for example, Apply a transfer function (such as the PQ or HLG transfer function or its inverse) to convert the three components The bit depth of is adapted to a bit depth such as 10 bits. Recommendation Rec.ITU-R BT.2100 provides an example of such a transfer function.

[0074] In the preprocessing step 20, when the format is not adapted in step 21, the three components are In three components Or when the formats of these components are adapted in step 21, the three components Equal to the adapted version of the three components These three input components are decomposed into three components and a parameter set SP formed by the parameters from steps 21, 200 and / or 203. During step 22, three components may optionally be adapted The format to get three components Switching step 24 determines the three components Is it equal to these three components? Or these three components

[0075] In step 23, the three components can be decoded using any video codec. Encoding is performed, and the output is a signal comprising a bit stream B. The output signal is transported throughout the distribution network.

[0076] According to a variant of step 23, the parameter set SP and / or the information data ID are transmitted as associated static and / or dynamic metadata in the bitstream B, or are transmitted out-of-band.

[0077] According to a variant, the parameter set SP and / or the information data ID are transmitted as associated static and / or dynamic metadata on a specific channel.

[0078] Aims to Figure 1 The at least one signal decoded by the means A2 carries a bitstream B which may include accompanying metadata.

[0079] In a variant, the bitstream B is stored on a storage medium, such as a (Ultra HD) Blu-ray Disc or a hard disk or a memory of a set-top box.

[0080] In a variant, at least some of the accompanying associated metadata are stored on a storage medium, such as a (Ultra HD) Blu-ray Disc or a hard disk or a memory of a set-top box.

[0081] In at least one implementation, in step 23, at least one triplet is quantized using a video codec such as the H.265 / HEVC codec or the H.264 / AVC codec. A sequence of tuples is encoded, with each triplet component representing an image and possibly associated metadata.

[0082] In step 25, a parameter set SP is obtained at least partially from the bitstream B or from another specific channel. At least one parameter of the parameter set SP may also be obtained from a separate storage medium. A processing parameter PP is derived from at least one parameter of the parameter set SP. The processing parameter may be modified in step 25 in order to adapt the image to the content and the brightness characteristics of the presentation display, such as Figure 6 and 7 Described in detail in .

[0083] In step 26, three decoded components are obtained from bitstream B

[0084] The post-processing step 28 is the functional inverse, or substantially the functional inverse, of the pre-processing step 20. In the post-processing step 28, the decoded components are And the obtained parameter set SP reconstructs the three components

[0085] In more detail, the preprocessing step 20 includes steps 200 to 203 .

[0086] In step 200, by The amount in Apply the mapping function to obtain the components Quantity Indicates the brightness of the input image.

[0087] Mathematically speaking,

[0088]

[0089] Where MF is a mapping function that can reduce or increase the dynamic range of image brightness. Note that its inverse, denoted as IMF, can increase or decrease the dynamic range of image brightness, respectively.

[0090] In step 202, by Apply the inverse mapping function to obtain the reconstructed components

[0091]

[0092] Where IMF is the inverse function of the mapping function MF. Therefore, the reconstruction component The value belongs to the component The dynamic range of the values.

[0093] In step 201, by and reconstruction component Corrects the component representing the chrominance of the input image and To export the components and

[0094] This step 201 allows control according to three components The colors obtained allow perceptual matching with the colors of the input image. By tuning the parameters of the chroma correction and inverse mapping steps, the components can be kept under control. and (usually represents the chrominance component). Therefore, according to the three components The color saturation and hue obtained are under control. Normally, such control is not possible when using a non-parametric mapping function (step 200).

[0095] Optionally, in step 203, the component To further control perceived saturation, as follows:

[0096]

[0097] Where a and b are two parameters.

[0098] This step 203 allows controlling the brightness (by component to allow for the three components The perceptual color match between the obtained colors (saturation and hue) and the colors of the input image.

[0099] The parameter set SP may include information data related to the mapping function or its inverse function (steps 200, 202 and 282), information data related to chromaticity correction (steps 201 and 281), information data related to the saturation adjustment function (in particular their parameters a and b) (step 203) and / or information related to optional transformations used in the format adaptation stages 21, 22, 27, 29 (e.g. gamut mapping and / or inverse gamut mapping parameters).

[0100] The parameter set SP may also include information data ID and information characteristics of the output image, such as the three components of the output image Format ( Figure 2 and 3 Step 29, Figure 3 Step 284).

[0101] In more detail, the post-processing step 28 comprises steps 280 - 282 which take as input at least one parameter of the parameter set SP.

[0102] In optional step 280, the three components may be adjusted as follows (i.e., the output of step 27)

[0103]

[0104] Wherein a and b are part of the process parameters PP and are derived from at least one parameter of the parameter set SP.

[0105] For example, step 280 is performed when the information data ID indicates that SL-HDR1 must be considered, and step 280 is not performed when it indicates that SL-HDR2 must be considered.

[0106] In step 282, by or alternatively Apply the mapping function to obtain the three components The amount

[0107]

[0108] Wherein MF1 is a mapping function derived according to at least one parameter of the parameter set SP.

[0109] In step 281, by or alternatively For three components The amount Perform an inverse correction to derive the three components The amount

[0110] According to one embodiment, the component and Multiply by a chromaticity correction function β(.) which is defined by the parameters of parameter set SP and whose value depends on the component or alternatively

[0111] Mathematically speaking, the weight Given by:

[0112]

[0113] Or alternatively,

[0114]

[0115] Figure 3 express Figure 2 A hardware-friendly version of the single-layer based scheme includes two additional steps 283 and 284 and allows reducing the complexity of the hardware implementation by reducing the bus bit width usage.

[0116] In step 283, by considering the parameters of parameter set SP, according to the components and (Output of step S281) Three components represented as (R1, G1, B1) are obtained:

[0117]

[0118] where m0, m1, m2, m3 are part of the processing parameters PP and are derived from at least one parameter of the parameter set SP, and S0 is derived from the component and and other parameters of parameter set SP.

[0119] In step 284, the (Output of step 282) Scale the three components (R1, G1, B1) to obtain the three components

[0120]

[0121] in (Step 282).

[0122] according to Figure 2 or Figure 3 In the first embodiment of the end-to-end workflow, the information data ID indicates that SL-HDR1 must be considered.

[0123] The mapping function MF(.) in equation (1) reduces the dynamic range of the brightness of the input image, and its inverse IMF(.) in equation (2) increases the component The dynamic range of , and the mapping function MF1(.) in equation (5) adds the component dynamic range.

[0124] According to a first variation of the first embodiment, the component is a nonlinear signal, denoted as brightness in the literature, which is obtained by gamma-compressing the RGB components of the input image as follows (step 21):

[0125]

[0126] where γ may be a gamma factor equal to 2.4 in some embodiments.

[0127] According to a first variant, the components are obtained by applying gamma compression to the RGB components of the input image. (Step 21):

[0128]

[0129] Where A=[A1A2A3] Tis a standard 3×3 R'G'B' to Y'CbCr conversion matrix (e.g., according to the color space, Recommendation ITU-R BT.2020-2 or Recommendation ITU-R BT.709-6), A1, A2, A3 are 1×3 matrices, where

[0130] A1=[A 11 A 12 A 13 ]

[0131] A2=[A 21 A 22 A 23 ]

[0132] A3=[A 31 A 32 A 33 ]

[0133] Among them A mn (m=1,..,3,n=1,..3) are the matrix coefficients.

[0134] In step 201, according to a first variant, according to the component and the gamma-compressed reconstructed components and The ratio between the products of is used to correct the components:

[0135]

[0136] in Depends on the quantity The value of, but can also be dependent on three components A constant value for the base color. For example, for BT.2020, it can be equal to 1.2. Possibly, It may also depend on the parameters specified in ETSI TS103433-1 V.1.2.1 clause C.2.3. It can also be a parameter in parameter set SP.

[0137] Further, according to the first variation, the three components Can represent Y'CbCr 4:2:0 gamma transfer characteristic video signal.

[0138] For example, control parameters related to the mapping function MF and / or its inverse IMF and / or the mapping function MF1(.) may be determined as specified in clause C.3.2 (ETSI Technical Specification TS103433-1 V1.2.1). The chromaticity correction function β(.) and its parameters may be determined as specified in clauses C.2.3 and C.3.4 (ETSI Technical Specification TS103433-1 V1.2.1). Information data related to the control parameters, information data related to the mapping function or its inverse function, and information data related to the chromaticity correction function β(.) and its parameters are parameters of the parameter set SP. For example, examples of numerical values ​​of the parameters of the parameter set SP may be found in Appendix F (Table F.1 of ETSI Technical Specification TS103433-1 V1.2.1).

[0139] The parameters m0, m1, m2, m3 and S0 may be determined as specified in clause 6.3.2.6 (matrixCoefficient[i] defines m0, m1, m2, m3) and clause 6.3.2.8 (kCoefficient[i] is used to construct S0) of ETSI technical specification TS103433-1 V1.2.1, and their use for reconstruction may be determined as specified in clause 7.2.4 (ETSI technical specification TS103433-1 V1.2.1). These parameters may be modified in step 25 to adapt the image to the content and the brightness characteristics of the presentation display, such as Figure 6 and 7 Described in detail in .

[0140] According to a second variation of the first embodiment, the component is the linear luminance component L obtained from the RGB components of the input image I1 as follows:

[0141]

[0142] According to a second variant, the components are derived by applying gamma compression to the RGB components of the input image I1 (Step 21):

[0143]

[0144] According to a second variant, by and the gamma-compressed reconstructed components and The ratio between the products of Thus deriving the component (Step 201).

[0145]

[0146] in It depends on the amount The value of and may be obtained according to the parameters specified in ETSI TS103433-1 V.1.2.1 clause C.3.4.2, where in equation (22)

[0147] It can also be a parameter in parameter set SP.

[0148] Further, according to the second variation, the three components Can represent Y'CbCr 4:2:0 gamma transfer characteristic video signal.

[0149] For example, the control parameters associated with the mapping function MF and / or its inverse IMF and / or the mapping function MF1(.) may be determined as specified in clause C.3.2 (ETSI Technical Specification TS103433-1 V1.2.1). The chromaticity correction function β(.) and its parameters may be determined as specified in clause 7.2.3.2 (ETSI Technical Specification TS103433-2 V1.1.1), equation (25), where f sgf (Y n ) = 1. The information data related to the control parameters, the information data related to the mapping function or its inverse function, and the information data related to the chromaticity correction function β(.) and its parameters are parameters of the parameter set SP.

[0150] The parameters m0, m1, m2, m3 and S0 may be determined as specified in clause 6.3.2.6 (matrixCoefficient[i] defines m0, m1, m2, m3) and clause 6.3.2.8 (kCoefficient[i] is used to construct S0) of ETSI technical specification TS103433-1 V1.2.1. The use of the parameters for reconstruction may be determined as specified in clause 7.2.4 (ETSI technical specification TS103433-1 V1.2.1). These parameters may be modified in step 25 to adapt the image to the content and the brightness characteristics of the presentation display, such as Figure 6 and 7 Described in detail in .

[0151] according to Figure 2 or Figure 3 In the second embodiment of the end-to-end workflow, the information data ID indicates that SL-HDR2 must be considered.

[0152] In the second embodiment, the three components It can be represented as a Y'CbCr 4:4:4 full range PQ10 (PQ10 bit) video signal (specified in Rec.ITU-RBT.2100). Three components are provided It represents PQ 10-bit image data and is based on three components The provided components are encoded (step 23) using, for example, a HEVC Main 10 Profile coding scheme. These parameters are a parameter set SP.

[0153] The mapping function MF1(.) in equation (5) can increase or decrease the component according to the variation dynamic range.

[0154] For example, when the peak brightness of the connected HDR CE display is higher than the peak brightness of the content, the mapping function MF1(.) increases the dynamic range. When the peak brightness of the connected HDR or SDR CE display is lower than the peak brightness of the content, the mapping function MF1(.) reduces the dynamic range. For example, the peak brightness can be a parameter of the parameter set SP.

[0155] For example, the control parameters associated with the mapping function MF1 may be determined as specified in clause C.3.2 (ETSI Technical Specification TS103433-1 V1.2.1). The chromaticity correction function β(.) and its parameters may be determined as specified in clause 7.2.3.2 (ETSI Technical Specification TS103433-2 V1.1.1), equation (25), where f sgf (Y n )=1. The information data related to the control parameters, the information data related to the mapping function, and the information data related to the chromaticity correction function β(.) and its parameters are parameters of the parameter set SP. Examples of numerical values ​​of the parameters of the parameter set SP can be found, for example, in Appendix F (Table F.1) (ETSI Technical Specification TS103433-2 V1.1.1).

[0156] The parameters m0, m1, m2, m3 (defined by matrixCoefficient[i] in ETSI Technical Specification TS103433-2 V1.1.1) and S0 (constructed by kCoefficient[i] in ETSI Technical Specification TS103433-2 V1.1.1) may be determined as specified in clause 7.2.4 (ETSI Technical Specification TS103433-2 V1.1.1). These parameters may be modified in step 25 to adjust the image to the brightness characteristics of the content and the presentation display, as Figure 6 and 7 Described in detail in .

[0157] According to a first variant of the second embodiment, the three components representing the output image It is three components

[0158] According to a second variant of the second embodiment, in a post-processing step 28, after decoding, the three components and the parameters of the parameter set SP to reconstruct the three components (Step 25).

[0159] Three Components Can be used for CE displays supporting SDR or HDR. As explained above, it is possible to adapt three components format (step 29).

[0160] The mapping function MF(.) or MF1(.) is based on a perceptual transfer function. The goal of the perceptual transfer function is to transform the components of the input image into components of the output image so as to reduce (or increase) the dynamic range of their brightness values. As a result, the values ​​of the components of the output image belong to a lower (or higher) dynamic range than the values ​​of the components of the input image. The perceptual transfer function uses a limited set of control parameters.

[0161] Figure 4a A diagram showing an example of a perceptual transfer function that can be used to map the luminance component is shown, but similar perceptual transfer functions for mapping the luminance component can be used. The mapping is determined by the master display peak luminance parameter (equal to Figure 4a 5000cd / m 2 ) control. To better control the black and white levels, a signal extending between the content-dependent black and white levels is applied. The converted signal is then mapped using a piecewise curve consisting of three parts, such as Figure 4b As shown. The upper and lower segments are straight lines, with the steepness determined by the shadowGain control and highlightGain control parameters respectively. The middle segment is a parabola, providing a continuous and smooth bridge between the two straight line segments. The width of the crossover is determined by the midToneWidthAdjFactor parameter. All parameters controlling the mapping can be transmitted as metadata, for example by using the SEI message specified in ETSI TS103433-1 Appendix A.2 Metadata.

[0162] Figure 4c The perceptual transfer function TM ( Figure 4a ) to illustrate how to use a target conventional display maximum brightness (e.g., 100 cd / m 2 ) converts the perceptually optimized luminance signal back into the linear light domain.

[0163] In step 25 ( Figure 2 Or 3), obtain the parameter set SP according to the three components Reconstructing three components The processing parameters PP used in the display adaptation process may be derived from metadata including the parameter set SP and obtained from the bitstream (e.g. bitstream B). The processing parameters PP may further be modified in step 25 to adjust the image to the content and the brightness characteristics of the presentation display, such as Figure 6 and 7 Described in detail in .

[0164] ETSI TS103433-1V1.2.1 clause 6 and Appendix A.2 provide examples of syntax for metadata. The syntax of this ETSI Recommendation is described for reconstructing HDR video from SDR video, but the syntax can be extended to reconstruct any image from any decoded component. As an example, TS103433-2V1.1.1 uses the same syntax to reconstruct display-adapted HDR video from HDR video signals (with different dynamic ranges).

[0165] According to ETSI TS103433-1 V1.2.1, dynamic metadata can be transmitted according to the so-called parameter-based mode or table-based mode. The parameter-based mode may be of interest for distribution workflows whose goal is, for example, to provide a direct SDR backward-compatible service with very low additional payload or bandwidth usage for carrying dynamic metadata. The table-based mode may be of interest for workflows equipped with low-end terminals or when a higher level of adaptation is required to properly represent HDR and SDR streams. In the parameter-based mode, the dynamic metadata to be transmitted includes brightness mapping parameters representing the inverse mapping function to be applied in the post-processing step, namely the tmInputSignalBlackLevelOffset; tmInputSignalWhiteLevelOffset; shadowGain; highlightGain; midToneWidthAdjFactor; tmOutputFineTuning parameters.

[0166] In addition, other dynamic metadata to be transmitted include color correction parameters (saturationGainNumVal, saturationGainX(i) and saturationGainY(i)) for fine-tuning the default chrominance correction function β(.), as specified in ETSI TS103433-1 V1.2.1 clauses 6.3.5 and 6.3.6. Parameters a and b can be carried in the saturationGain function parameters as described above, respectively. These dynamic metadata can be transmitted using, for example, the HEVC SL-HDR Information (SL-HDRI) User Data Registration SEI message (see ETSI TS103433-1 V1.2.1 Appendix A.2) or another extended data mechanism such as specified in the AVS2 / IEEE1857.4 specification. Typical dynamic metadata payload sizes are less than 100 bytes per picture or scene.

[0167] Back to Figure 3 In step 25, the SL-HDRI SEI message is parsed to obtain at least one parameter in the parameter set SP. The process parameters PP used in the display adaptation process may be derived from the metadata comprising the parameter set SP. In step 25, the process parameters PP may be further modified to adjust the image to the content and the brightness characteristics of the presentation display, such as Figure 6 and 7 As shown in detail in .

[0168] In steps 282 and 202, an inverse mapping function (so-called lutMapY) is reconstructed (or derived) from the obtained mapping parameters (see ETSI TS103433-1 V1.2.1 clause 7.2.3.1 for more details; the same clause of TS103433-2 V1.1.1).

[0169] In steps 282 and 202, the chromaticity correction function β(.) (so-called lutCC) is also reconstructed (or derived) from the obtained color correction parameters (for more details see ETSI TS103433-1 V1.2.1 clause 7.2.3.2; the same clause of TS103433-2 V1.1.1).

[0170] In table-based mode, the dynamic data to be communicated includes the pivot points of the piecewise linear curve representing the mapping function. For example, the dynamic metadata is luminanceMappingNumVal indicating the number of pivot points, luminanceMappingX indicating the abscissa (x) value of the pivot point, and luminanceMappingY indicating the ordinate (y) value of the pivot point (see ETSI TS103433-1V1.2.1 clauses 6.2.7 and 6.3.7 for more details). In addition, other dynamic metadata to be transmitted may include the pivot points of the piecewise linear curve representing the chromaticity correction function β(.). For example, the dynamic metadata is colorCorrectionNumVal indicating the number of pivot points, colorCorrectionX indicating the x value of the pivot point, and colorCorrectionY indicating the y value of the pivot point (see ETSI TS103433-1V1.2.1 clauses 6.2.8 and 6.3.8 for more details). These dynamic metadata may be transmitted using, for example, the HEVC SL-HDRI SEI message (the mapping between clause 6 parameters and Annex A distribution metadata is provided in Annex A.2.3 of ETSI TS 103433-1 V1.2.1).

[0171] In step 25, the SL-HDRI SEI message is parsed to obtain the pivot point of the piecewise linear curve representing the inverse mapping function and the pivot point of the piecewise linear curve representing the chroma correction function β(.), as well as the chroma to luma injection parameters a and b. These parameters are part of the process parameters PP used in the display adaptation process, and these parameters can be further modified in step 25 to adjust the image to the brightness characteristics of the content and the presentation display, such as Figure 6 and 7 As shown in detail in .

[0172] In steps 282 and 202, the inverse mapping function is derived from those pivot points relative to the piecewise linear curve representing the inverse mapping function ITM (see ETSI TS103433-1 V1.2.1 clause 7.2.3.3 for more details; the same clause of ETSI TS103433-2 V1.1.1).

[0173] In steps 281 and 201, the chromaticity correction function β(.) is also derived from those support points relative to the piecewise linear curve representing the chromaticity correction function β(.) (for more details, see ETSI TS103433-1 V1.2.1 clause 7.2.3.4; the same clause of TS103433-2 V1.1.1).

[0174] Note that static metadata also used by post-processing steps can be transmitted by SEI messages. For example, the selection of parameter-based mode or table-based mode can be performed by the payloadMode information specified by ETSI TS103433-1V1.2.1 (clause A.2.2). Static metadata (e.g., primary colors or maximum master display brightness) is transmitted by the master display volume (MDCCV) SEI message specified in AVC, HEVC, or embedded in the SL-HDRI SEI message specified in ETSI TS103433-1V1.2.1 Appendix A.2.

[0175] According to an embodiment of step 25, the information data ID is explicitly signaled by a syntax element in the bitstream and is thus obtained by parsing the bitstream. For example, the syntax element is part of a SEI message, such as the sl_hdr_mode_value_minus1 syntax element included in the SL-HDRI SEI message.

[0176] According to an embodiment, the information data ID identifies the processing to be applied to the input image to process the parameter set SP. According to this embodiment, the information data ID can then be used to infer how to use the parameters to reconstruct the three components (Step 25).

[0177] For example, when equal to 1, the information data ID indicates that the parameter set SP has been obtained by applying the SL-HDR 1 pre-processing step (step 20) to the input HDR image, and the three components represents an SDR image. When equal to 2, the information data ID indicates that the parameters have been obtained by applying the SL-HDR2 pre-processing step (step 20) to the HDR 10-bit image (the input of step 20), and the three components Indicates HDR 10 images.

[0178] According to an embodiment, the processing parameters PP used in the display adaptation process may be derived from metadata including the parameter set SP and obtained from a bitstream, such as bitstream B. In step 25, the processing parameters PP may be further modified to adjust the image to the content and the brightness characteristics of the presentation display, such as Figure 6 and 7 As shown in detail in .

[0179] exist Figure 1-4C and 6, a module is a functional unit. In various embodiments, all, some or none of these functional units correspond to distinguishable physical units. For example, these modules or some of them can be introduced together into a unique component or circuit, or contribute to the function of the software. As another example, some modules can be composed of separate physical entities. Various embodiments are implemented using pure hardware, for example, using dedicated hardware, such as ASIC or FPGA or VLSI (application-specific integrated circuit, field programmable gate array, very large scale integration, respectively), or according to several integrated electronic components embedded in the device, or according to a mixture of hardware and software components.

[0180] Figure 5 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 5000 can be implemented as a device including various components described below, and is configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 5000 can be implemented individually or in combination in a single integrated circuit, multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 5000 are distributed on multiple ICs and / or discrete components. In various embodiments, system 5000 is coupled to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 5000 is configured to implement one or more aspects described in this document.

[0181] The system 5000 includes at least one processor 5010 configured to execute instructions loaded therein for implementing various aspects described in this document, for example. The processor 5010 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 5000 includes at least one memory 5020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 5000 includes a storage device 5040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an EEPROM, a ROM, a PROM, a RAM, a DRAM, an SRAM, a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 5040 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0182] The system 5000 includes an encoder / decoder module 5030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 5030 may include its own processor and memory. The encoder / decoder module 5030 represents a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 5030 may be implemented as a separate element of the system 5000, or may be incorporated into the processor 5010 as a combination of hardware and software known to those skilled in the art.

[0183] Program code to be loaded onto the processor 5010 or encoder / decoder 5030 to perform various aspects described in this document may be stored in the storage device 5040 and subsequently loaded onto the memory 5020 for execution by the processor 5010. According to various embodiments, one or more of the processor 5010, memory 5020, storage device 5040, and encoder / decoder module 5030 may store one or more of various items during execution of the processes described in this document. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0184] In several embodiments, memory internal to the processor 5010 and / or encoder / decoder module 5030 is used to store instructions and provide working memory for processing required during encoding or decoding.

[0185] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 5010 or the encoder / decoder module 5030) is used for one or more of these functions. The external memory may be a memory 5020 and / or a storage device 5040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the device. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as MPEG-2, HEVC, or VVC (Versatile Video Coding).

[0186] As shown in block 5030, input to the elements of system 5000 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that receives an RF signal transmitted over the air, for example, by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0187] In various embodiments, the input device of block 5030 has associated corresponding input processing elements as known in the art. For example, the RF portion may be associated with the necessary elements for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF portion of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband.

[0188] In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (eg, cable) medium and perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0189] Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0190] Adding components may include inserting components between existing components, such as inserting an amplifier and an analog-to-digital converter.In various embodiments, the RF portion includes an antenna.

[0191] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 5000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, in a separate input processing IC or in the processor 5010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, in a separate interface IC or in the processor 5010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 5010 and the encoder / decoder 5030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0192] The various components of system 5000 may be disposed within an integrated housing. Within the integrated housing, the various components may be interconnected and transmit data between using a suitable connection arrangement, for example, an internal bus known in the art, including an I2C bus, wiring, and a printed circuit board.

[0193] The system 5000 includes a communication interface 5050 that enables communication with other devices via a communication channel 5060. The communication interface 5050 may include, but is not limited to, a transceiver configured to transmit and receive data through the communication channel 5060. The communication interface 5050 may include, but is not limited to, a modem or a network card, and the communication channel 5060 may be implemented, for example, within a wired and / or wireless medium.

[0194] In various embodiments, a Wi-Fi network such as IEEE 802.11 is used to stream data to the system 5000. The Wi-Fi signals of these embodiments are received through a communication channel 5060 suitable for Wi-Fi communications and a communication interface 5050. The communication channel 5060 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0195] Other embodiments provide streaming data to system 5000 using a set top box that delivers data through an HDMI connection of input box 5030 .

[0196] Still other embodiments use the RF connection of input box 5030 to provide streaming data to system 5000.

[0197] In some embodiments, display 5100, speakers 5110, and peripherals 5120 are integrated into system 5000. Televisions, tablet computers, and laptop computers are examples of these embodiments.

[0198] Figure 6 An example of an embodiment for adjusting a display adaptation function according to a content maximum average brightness and a presentation display maximum average brightness is shown. The figure shows both the architecture required to perform the adaptation and the corresponding method steps.

[0199] In step 600, a parameter set SP is obtained from the bitstream B or from another specific channel. In step 605, processing parameters PP are derived from the parameter set. These processing parameters include parameters related to the above-mentioned post-processing stage (e.g. a, b, m0, m1, m2, m3) and are at least Figure 2 or Figure 3In one of blocks 280, 281, 282, 283, 284 or 29 of . In step 610, a display maximum average brightness value is obtained from the display device. Information data representing the brightness characteristics of the consumer presentation display (such as average peak brightness (APL) or full white film brightness) can be carried by the HDR static metadata block specified in CTA-861-G and supported as an option since HDMI 2.0a., in particular, expected content maximum frame average brightness data, as well as expected content maximum brightness data and expected content minimum brightness data (Table 84 in Section 7.5.13 of CTA-861-G, see Table 1 below). This information data may also be used in presentation display registers, and it may be accessed by a metadata-guided display adaptation module located within the presentation display itself. This information is referenced hereinafter as L mdisp .

[0200]

[0201] Table 1

[0202] In step 620, the content maximum average brightness value is obtained from data representing the content, such as data extracted from parameter set SP or directly by analyzing the content (for example by calculating LightnessHDR as described in clause C.3.2.3 and equation (C.46) of ETSI TS103433-1 v1.1.1). In step 630, these values ​​can be compared together to decide whether the display adaptation processing parameters derived from the parameter set (SP) need to be adjusted. Switch 640 is set in an appropriate position so that module 25 provides adjusted PP parameters or unadjusted PP parameters. In the first case, the parameter set is adjusted in step 650 and provided to other modules of the device for post-processing of the content signal ( Figure 2 and Figure 3 In step 28), for example, to decode and display the video. In one embodiment, the parameters are adjusted systematically.

[0203] In an embodiment, the adjustment affects at least one of the following parameters of SL-HDR1 or SL-HDR2 representing the tone mapping curve used in the display adaptation process: the shadowGain control parameter that controls the steepness of the low light map, the midToneWidthAdjFactor parameter that controls the transition between low light and highlight maps, typically in skin tone related areas, or the highlightGain control parameter that controls the highlight (specular) map (see Figure 4b ).

[0204] In one embodiment, the display maximum average brightness is obtained from the expected content maximum frame average brightness value carried in Table 1 above, and the content maximum average brightness is represented by the content maximum frame average brightness level (MaxFALL - specified in Annex P of CTA-861-G). MaxFALL can be present in the content light information SEI message carried along the decoded stream, or in the static metadata type 1 (section 6.9.1) of CTA-861-G carried in the dynamic range and master information frame.

[0205] When MaxFALL information data is not available, another value representing the average brightness of the content can be used, such as the geometric mean of the master display minimum brightness, the master display maximum brightness, and the input HDR image average brightness value (which can be obtained by a content histogram analysis module such as that already implemented in the current TV image processing pipeline). Alternatively, the content maximum average brightness is represented by a reference white level (e.g., reference white or HDR reference white or diffuse white or graphic white, such as described in ITU-R Recommendation BT.2408-1 "Operational Practices in HDR Television Production", 04 / 2018).

[0206] In one embodiment, decision step 630 includes directly comparing the values ​​and determining if the content is brighter than the display can handle, i.e., when MaxFALL>L mdisp , adapt the parameter set.

[0207] In one embodiment, decision step 630 includes a weighting factor in direct comparison to the previous value, and in the case where the content is brighter than the weighted value that the display can handle, i.e., when MaxFALL>α.L mdisp (α is in the range of 0.5 to 2), the adaptation parameter set is determined. In general, OLED displays and LED displays are two different technologies provided by two different power consumption strategies that are inherently dependent on their technology design. The average peak brightness drive can react differently for each display technology, and the α scaling factor accounts for this difference.

[0208] In one embodiment, in the SL-HDRx display adaptation process, the maximum average brightness of the display (L mdisp ) value is combined with the content MaxFALL of the modulation function of the shadow gain control part as the tone mapping part, and the parameters used for display adaptive recalculation can be modulated by the modulation function as follows.

[0209] In a first embodiment, a scaling factor may be determined and used as a modulation function:

[0210]

[0211] Where L mdisp is the maximum frame average brightness of the rendering display. Typically, this value is retrieved from the Expected Content Maximum Frame Average Brightness field of the HDR Static Metadata Block (CTA-861-G), MaxFALL is the maximum frame average brightness level of the input HDR content (this value is typically retrieved from the Content Brightness Level Information SEI message), and the weighting factor α is typically in the range [0.5; 2].

[0212] In a variation of the first embodiment:

[0213]

[0214] In a second variation of the first embodiment, the scaling factor is used to modulate the parameters representing the tone mapping curve and is based on the MaxFALL or L in the tone mapping curve representation. mdisp The coordinates of .

[0215] in as well as

[0216] The scaling factor can be defined by the ratio of these coordinates, and κ and λ are defined in equations (E.1) and (E.2) of TS103433-1v1.2.1, respectively. For example:

[0217] or

[0218] Then, the modulation function is applied (as a factor) to recalculate SGC in equation (E.11) of TS103433-1 v1.2.1 DA :

[0219]

[0220] In an alternative embodiment, equation (C.52) of clause C.3.2.3 is modulated by a modulation function.

[0221] In an alternative embodiment, shadowGain is calculated as described in clause C.3.2.3, except that equation (C.52) is modified as follows:

[0222]

[0223] Alternatively, only LightnessHDRHigh is modified as described above.

[0224] Or, in the above and other documents use to replace.

[0225] In one example of such an embodiment, other recalculated display adaptation parameters (e.g., para DA 、highlightGain DA ,midToneWidthAdjFactor DA ) is saved as described in formulas (E.12) to (E.16).

[0226] In at least one embodiment, only when the maximum average brightness value L of the display is presented mdisp The recalculated display adaptation parameters are modulated by a modulation function as proposed in the present invention only when the maximum frame average brightness of the content is less than the content MaxFALL multiplied by a factor comprised between 0.5 and a positive value greater than 1 (e.g. 2). In practice, in this case, the content may not be displayed as expected, since the rendering display may impose limitations based on the light emitted from the panel, and the present invention allows to then take into account energy limitations when adapting the content to maintain the maximum value of the artistic intent of the content. In practice, the display adaptation parameters are scaled in a perceptually uniform domain, the scaling depending on the actual capabilities of the display relative to its power consumption control. Furthermore, the intention of the present application is not to change the display adaptation parameters of the adaptation function (e.g., the tone mapping curve) when the content maximum frame average brightness is compatible with the rendering display energy strategy (i.e., when MaxFALL is somehow less than the maximum average brightness value of the rendering display). In this case, the equations of Appendix E of SL-HDR1 and Clause 7 of SL-HDR2 remain unchanged (i.e., Figure 6 The intent is that when the content is too bright on average relative to the display capabilities (in terms of an all-white piece), the display adaptive tone mapping compresses or scales the tone mapping curve accordingly (using a more aggressive tone mapping) so that the display adapted and modulated content will be tone mapped in a more controlled manner. In practice, when the display adaptive tone mapping curve is not adapted, the content will be tone mapped by the rendering display so that the energy policy is satisfied, which leads to uncontrolled results.

[0227] In other variant embodiments, the modulation function may be applied to other parameters representing another portion of the tone mapping curve. For example, the modulation function may be applied similarly to the above detailed equations in Appendix E (e.g., para DA 、highlightGain DA 、midToneWidthAdjFactor DA ) with appropriate parameters.

[0228] The proposed principle can be extended to any other analytical tone mapping curves.

[0229] Figure 7 The adjustment of the tone mapping curve according to the second embodiment is shown, where the content maximum average brightness and the display maximum average brightness are matched. Tone mapping is an example of a function used in adaptive processing. Consider equation (C.20) of ETSI TS103433-1:

[0230]

[0231] When considering the first part of the tone mapping curve and When y = SGC × x, we can derive:

[0232]

[0233] in:

[0234] L mdisp is the maximum frame average luminance of the rendering display. Typically, this value is retrieved from the Expected Content Maximum Frame Average Luminance field of the HDR Static Metadata Block (CTA-861-G), and MaxFALL is the maximum frame average light level of the input HDR content. Typically, this value is retrieved from the Content Light Level Information SEI message. The other parameters a, b, c and HGC are documented in clause C.2.2.4 of TS103433-1.

[0235] In a first variation of this second embodiment, the shadowGain parameter is calculated according to equations (C.27) and (C.23):

[0236]

[0237] where expgain is given by (C.24).

[0238] The HGC is fixed (predetermined), and the HGC comes from the input metadata (for SDR to HDR reconstruction for SL-HDR1, or just metadata for SL-HDR2), and the midToneWidthAdjFactor is recalculated from (C.2) and the above SGC equation:

[0239]

[0240] In other variations of the second embodiment, instead of L mdisp Matches MaxFALL, L mdisp Matches a specific point on the tone map curve, for example, a knot between sections of the curve:

[0241] x_SGC as defined in equation (C.22) of TS103433-1;

[0242] x_HGC as defined in equation (C.22) of TS103433-1.

[0243] The system 5000 can provide output signals to various output devices, including a display 5100, speakers 5110, and other peripherals 5120. In various examples of embodiments, the other peripherals 5120 include one or more of: a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 5000.

[0244] In various embodiments, control signals are transmitted between the system 5000 and the display 5100, speakers 5110, or other peripherals 5120 using signaling such as AV.Link, CEC, or other communication protocols, which enables device-to-device control with or without user intervention.

[0245] Output devices may be communicatively coupled to system 5000 through respective interfaces 5070 , 5080 , and 5090 via dedicated connections.

[0246] Alternatively, the output device may be connected to the system 5000 via the communication interface 5050 using the communication channel 5060. The display 5100 and the speaker 5110 may be integrated with the other components of the system 5000 into a single unit in an electronic device such as a television.

[0247] In various embodiments, the display interface 5070 includes a display driver, for example, a timing controller (T Con) chip.

[0248] For example, if the RF portion of input 5130 is part of a separate set-top box, the display 5100 and speaker 5110 may alternatively be separate from one or more of the other components. In various embodiments where the display 5100 and speaker 5110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0249] The embodiments of various processes and features described herein can be embodied in various different devices or applications. The example of such equipment includes encoder, decoder, post-processor for processing the output from the decoder, pre-processor for providing input to the encoder, video encoder, video decoder, video codec, web server, set-top box, laptop computer, personal computer, cellular phone, PDA, any other equipment for processing image or video, and any other communication device. It should be clear that the equipment can be mobile and can even be installed in a mobile vehicle.

[0250] In addition, the method may be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) may be stored on a computer-readable storage medium. A computer-readable storage medium may take the form of a computer-readable program product, which is contained in one or more computer-readable media and has a computer-readable program code contained thereon, which is executable by a computer. As used herein, a computer-readable storage medium is considered to be a non-transitory storage medium that is endowed with an inherent ability to store information therein and to provide an inherent ability to retrieve information therefrom. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any suitable combination of the foregoing. It should be understood that, although more specific examples of computer-readable storage media are provided, the following are merely illustrative and non-exhaustive enumerations, as readily understood by those of ordinary skill in the art: a portable computer; a floppy disk; a hard disk; a read-only memory (ROM); an erasable programmable read-only memory (EPROM or flash memory); a portable compact disk read-only memory (CD-ROM); an optical storage device; a magnetic storage device; or any suitable combination of the foregoing.

[0251] The instructions may form an application program tangibly contained on a processor-readable medium (also referred to as a computer-readable medium or a computer-readable storage medium). The instructions may be, for example, hardware, firmware, software, or a combination thereof. The instructions may be found, for example, in an operating system, a separate application program, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. In addition, the processor-readable medium may store data values ​​generated by the implementation in addition to or in lieu of the instructions.

[0252] As will be appreciated by those skilled in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described embodiments. For example, a signal may be formatted to carry as data the rules for writing or reading the grammar of the described examples, or to carry as data the actual grammar values ​​written by the described examples. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0253] Many embodiments have been described. However, it should be understood that various modifications can be made. For example, elements of different embodiments can be combined, supplemented, modified or removed to produce other embodiments. In addition, it will be understood by those skilled in the art that other structures and processes can replace those disclosed, and the resulting embodiments will perform at least substantially the same function (one or more) in at least substantially the same manner (one or more) to achieve at least substantially the same result (one or more) as the disclosed embodiments. Therefore, the present application contemplates these and other embodiments.

[0254] In an embodiment of the first aspect, the parameter is adjusted only when the second value is greater than the first value. In an embodiment of the first aspect, the first value is the expected content maximum frame average brightness of the display device that will display the image. In an embodiment of the first aspect, the second value is the content maximum frame average brightness of the image to be displayed. In an embodiment of the first aspect, the first value is the expected content maximum frame average brightness of the display device that will display the image, and the second value is the content maximum frame average brightness of the image to be displayed. In an embodiment of the first aspect, the adaptation function is a tone mapping function, and the parameter adjustment of the adaptation function includes applying a first scaling factor equal to the ratio between the first value and the second value. In an embodiment of the first aspect, the adaptation function is a tone mapping function, and adapting the tone mapping function includes defining a new perceptual tone mapping function, the new perceptual tone mapping function at least including a lower linear segment defined by a steepness equal to the ratio between the first value and the second value. In an embodiment of the first aspect, one of the first value or the second value is scaled according to a second scaling factor. In an embodiment of the first aspect, the value of the second scaling factor is included in the range of 0.5 to 2.0.

[0255] In an embodiment of the second aspect, the parameter is adjusted only when the second value is greater than the first value. In an embodiment of the second aspect, the first value is the expected content maximum frame average brightness of the display device that will display the image. In an embodiment of the second aspect, the second value is the content maximum frame average brightness of the image to be displayed. In an embodiment of the second aspect, the first value is the expected content maximum frame average brightness of the display device that will display the image, and the second value is the content maximum frame average brightness of the image to be displayed. In an embodiment of the second aspect, the adaptation function is a tone mapping function, and the parameter adjustment of the adaptation function includes applying a first scaling factor equal to the ratio between the first value and the second value. In an embodiment of the second aspect, the adaptation function is a tone mapping function, and adapting the tone mapping function includes defining a new perceptual tone mapping function, the new perceptual tone mapping function at least including a lower linear segment defined by a steepness equal to the ratio between the first value and the second value. In an embodiment of the second aspect, one of the first value or the second value is scaled according to a second scaling factor. In an embodiment of the second aspect, the value of the second scaling factor is included in the range of 0.5 to 2.0.

Claims

1. A method for decoding an image, comprising: - obtaining parameters of a tone mapping function, said tone mapping function comprising at least a lower linear segment defined by a steepness value, and - decoding the image, wherein the decoding comprises applying the tone mapping function to the image based at least on the obtained parameters, Wherein, when a first value representing a maximum average brightness parameter of a display device that will display the image is less than a second value representing an average brightness parameter of the image to be displayed, the tone mapping parameter is adjusted by scaling the steepness value based on a ratio between a weighted value of the first value and the second value, and wherein the weight is in the range of 0.5 to 2.

2. The method according to claim 1, wherein: The first value is the expected content maximum frame average brightness of the display device according to the HDR static metadata block of CTA-861-G.

3. The method according to claim 1, wherein: The first value is a weighted value of the expected content maximum frame average brightness of the HDR static metadata block according to CTA-861-G.

4. The method according to claim 1, wherein: The second value representing the average brightness of the image to be displayed is: - the maximum frame average luminance level MaxFALL according to Appendix P of CTA-861-G, or - a reference white level or diffuse white according to ITU-R Recommendation BT.2408-1, or -The geometric average of the minimum brightness of the master display, the maximum brightness of the master display and the average brightness of the image.

5. The method according to claim 1, wherein: The first value is obtained from the display device.

6. An apparatus for decoding an image, comprising at least one processor, wherein the at least one processor is configured to: - obtaining parameters of a tone mapping function, said tone mapping function comprising at least a lower linear segment defined by a steepness value, and - decoding the image, wherein: The decoding comprises applying the tone mapping function to the image based at least on the obtained parameters, Wherein, when a first value representing a maximum average brightness parameter of a display device that will display the image is less than a second value representing an average brightness parameter of the image to be displayed, the tone mapping parameter is adjusted by scaling the steepness value based on a ratio between a weighted value of the first value and the second value, and wherein the weight is in the range of 0.5 to 2.

7. The device according to claim 6, wherein: The first value is the expected content maximum frame average brightness of the display device according to the HDR static metadata block of CTA-861-G.

8. The device according to claim 6, wherein: The first value is a weighted value of the expected content maximum frame average brightness of the HDR static metadata block according to CTA-861-G.

9. The device according to claim 6, wherein: The second value representing the average brightness of the image to be displayed is: - the maximum frame average luminance level MaxFALL according to Appendix P of CTA-861-G, or - a reference white level or diffuse white according to ITU-R Recommendation BT.2408-1, or -The geometric average of the minimum brightness of the master display, the maximum brightness of the master display and the average brightness of the image.

10. The device according to claim 6, wherein: The first value is obtained from the display device.

Images